mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-16 10:50:38 +09:00
[codex] Add plugin orchestration host APIs (#4114)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system is the extension path for optional capabilities that should not require core product changes for every integration. > - Plugins need scoped host APIs for issue orchestration, documents, wakeups, summaries, activity attribution, and isolated database state. > - Without those host APIs, richer plugins either cannot coordinate Paperclip work safely or need privileged core-side special cases. > - This pull request adds the plugin orchestration host surface, scoped route dispatch, a database namespace layer, and a smoke plugin that exercises the contract. > - The benefit is a broader plugin API that remains company-scoped, auditable, and covered by tests. ## What Changed - Added plugin orchestration host APIs for issue creation, document access, wakeups, summaries, plugin-origin activity, and scoped API route dispatch. - Added plugin database namespace tables, schema exports, migration checks, and idempotent replay coverage under migration `0059_plugin_database_namespaces`. - Added shared plugin route/API types and validators used by server and SDK boundaries. - Expanded plugin SDK types, protocol helpers, worker RPC host behavior, and testing utilities for orchestration flows. - Added the `plugin-orchestration-smoke-example` package to exercise scoped routes, restricted database namespaces, issue orchestration, documents, wakeups, summaries, and UI status surfaces. - Kept the new orchestration smoke fixture out of the root pnpm workspace importer so this PR preserves the repository policy of not committing `pnpm-lock.yaml`. - Updated plugin docs and database docs for the new orchestration and database namespace surfaces. - Rebased the branch onto `public-gh/master`, resolved conflicts, and removed `pnpm-lock.yaml` from the final PR diff. ## Verification - `pnpm install --frozen-lockfile` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/client.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-scoped-api-routes.test.ts server/src/__tests__/plugin-sdk-orchestration-contract.test.ts` - From `packages/plugins/examples/plugin-orchestration-smoke-example`: `pnpm exec vitest run --config ./vitest.config.ts` - `pnpm --dir packages/plugins/examples/plugin-orchestration-smoke-example run typecheck` - `pnpm --filter @paperclipai/server typecheck` - PR CI on latest head `293fc67c`: `policy`, `verify`, `e2e`, and `security/snyk` all passed. ## Risks - Medium risk: this expands plugin host authority, so route auth, company scoping, and plugin-origin activity attribution need careful review. - Medium risk: database namespace migration behavior must remain idempotent for environments that may have seen earlier branch versions. - Medium risk: the orchestration smoke fixture is intentionally excluded from the root workspace importer to avoid a `pnpm-lock.yaml` PR diff; direct fixture verification remains listed above. - Low operational risk from the PR setup itself: the branch is rebased onto current `master`, the migration is ordered after upstream `0057`/`0058`, and `pnpm-lock.yaml` is not in the final diff. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. Roadmap checked: this work aligns with the completed Plugin system milestone and extends the plugin surface rather than duplicating an unrelated planned core feature. ## Model Used - OpenAI Codex, GPT-5-based coding agent in a tool-enabled CLI environment. Exact hosted model build and context-window size are not exposed by the runtime; reasoning/tool use were enabled for repository inspection, editing, testing, git operations, and PR creation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no core UI screen change; example plugin UI contract is covered by tests) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
16b2b84d84
commit
9c6f551595
53 changed files with 5584 additions and 53 deletions
372
server/src/__tests__/plugin-orchestration-apis.test.ts
Normal file
372
server/src/__tests__/plugin-orchestration-apis.test.ts
Normal file
|
|
@ -0,0 +1,372 @@
|
|||
import { randomUUID } from "node:crypto";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
activityLog,
|
||||
agentWakeupRequests,
|
||||
agents,
|
||||
companies,
|
||||
costEvents,
|
||||
createDb,
|
||||
heartbeatRuns,
|
||||
issueRelations,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { buildHostServices } from "../services/plugin-host-services.js";
|
||||
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||||
|
||||
function createEventBusStub() {
|
||||
return {
|
||||
forPlugin() {
|
||||
return {
|
||||
emit: async () => {},
|
||||
subscribe: () => {},
|
||||
};
|
||||
},
|
||||
} as any;
|
||||
}
|
||||
|
||||
function issuePrefix(id: string) {
|
||||
return `T${id.replace(/-/g, "").slice(0, 6).toUpperCase()}`;
|
||||
}
|
||||
|
||||
if (!embeddedPostgresSupport.supported) {
|
||||
console.warn(
|
||||
`Skipping embedded Postgres plugin orchestration API tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
||||
);
|
||||
}
|
||||
|
||||
describeEmbeddedPostgres("plugin orchestration APIs", () => {
|
||||
let db!: ReturnType<typeof createDb>;
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-plugin-orchestration-");
|
||||
db = createDb(tempDb.connectionString);
|
||||
}, 20_000);
|
||||
|
||||
afterEach(async () => {
|
||||
await db.delete(activityLog);
|
||||
await db.delete(costEvents);
|
||||
await db.delete(heartbeatRuns);
|
||||
await db.delete(agentWakeupRequests);
|
||||
await db.delete(issueRelations);
|
||||
await db.delete(issues);
|
||||
await db.delete(agents);
|
||||
await db.delete(companies);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await tempDb?.cleanup();
|
||||
});
|
||||
|
||||
async function seedCompanyAndAgent() {
|
||||
const companyId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "Paperclip",
|
||||
issuePrefix: issuePrefix(companyId),
|
||||
requireBoardApprovalForNewAgents: false,
|
||||
});
|
||||
await db.insert(agents).values({
|
||||
id: agentId,
|
||||
companyId,
|
||||
name: "Engineer",
|
||||
role: "engineer",
|
||||
status: "idle",
|
||||
adapterType: "process",
|
||||
adapterConfig: { command: "true" },
|
||||
runtimeConfig: {},
|
||||
permissions: {},
|
||||
});
|
||||
return { companyId, agentId };
|
||||
}
|
||||
|
||||
it("creates plugin-origin issues with full orchestration fields and audit activity", async () => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const blockerIssueId = randomUUID();
|
||||
const originRunId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({
|
||||
id: originRunId,
|
||||
companyId,
|
||||
agentId,
|
||||
status: "running",
|
||||
invocationSource: "assignment",
|
||||
contextSnapshot: { issueId: blockerIssueId },
|
||||
});
|
||||
await db.insert(issues).values({
|
||||
id: blockerIssueId,
|
||||
companyId,
|
||||
title: "Blocker",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
identifier: `${issuePrefix(companyId)}-blocker`,
|
||||
});
|
||||
|
||||
const services = buildHostServices(db, "plugin-record-id", "paperclip.missions", createEventBusStub());
|
||||
const issue = await services.issues.create({
|
||||
companyId,
|
||||
title: "Plugin child issue",
|
||||
status: "todo",
|
||||
assigneeAgentId: agentId,
|
||||
billingCode: "mission:alpha",
|
||||
originId: "mission-alpha",
|
||||
blockedByIssueIds: [blockerIssueId],
|
||||
actorAgentId: agentId,
|
||||
actorRunId: originRunId,
|
||||
});
|
||||
|
||||
const [stored] = await db.select().from(issues).where(eq(issues.id, issue.id));
|
||||
expect(stored?.originKind).toBe("plugin:paperclip.missions");
|
||||
expect(stored?.originId).toBe("mission-alpha");
|
||||
expect(stored?.billingCode).toBe("mission:alpha");
|
||||
expect(stored?.assigneeAgentId).toBe(agentId);
|
||||
expect(stored?.createdByAgentId).toBe(agentId);
|
||||
expect(stored?.originRunId).toBe(originRunId);
|
||||
|
||||
const [relation] = await db
|
||||
.select()
|
||||
.from(issueRelations)
|
||||
.where(and(eq(issueRelations.issueId, blockerIssueId), eq(issueRelations.relatedIssueId, issue.id)));
|
||||
expect(relation?.type).toBe("blocks");
|
||||
|
||||
const activities = await db
|
||||
.select()
|
||||
.from(activityLog)
|
||||
.where(and(eq(activityLog.entityType, "issue"), eq(activityLog.entityId, issue.id)));
|
||||
expect(activities).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
actorType: "plugin",
|
||||
actorId: "plugin-record-id",
|
||||
action: "issue.created",
|
||||
agentId,
|
||||
details: expect.objectContaining({
|
||||
sourcePluginId: "plugin-record-id",
|
||||
sourcePluginKey: "paperclip.missions",
|
||||
initiatingActorType: "agent",
|
||||
initiatingActorId: agentId,
|
||||
initiatingRunId: originRunId,
|
||||
}),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it("enforces plugin origin namespaces", async () => {
|
||||
const { companyId } = await seedCompanyAndAgent();
|
||||
const services = buildHostServices(db, "plugin-record-id", "paperclip.missions", createEventBusStub());
|
||||
|
||||
const featureIssue = await services.issues.create({
|
||||
companyId,
|
||||
title: "Feature issue",
|
||||
originKind: "plugin:paperclip.missions:feature",
|
||||
originId: "mission-alpha:feature-1",
|
||||
});
|
||||
expect(featureIssue.originKind).toBe("plugin:paperclip.missions:feature");
|
||||
|
||||
await expect(
|
||||
services.issues.create({
|
||||
companyId,
|
||||
title: "Spoofed issue",
|
||||
originKind: "plugin:other.plugin:feature",
|
||||
}),
|
||||
).rejects.toThrow("Plugin may only use originKind values under plugin:paperclip.missions");
|
||||
|
||||
await expect(
|
||||
services.issues.update({
|
||||
issueId: featureIssue.id,
|
||||
companyId,
|
||||
patch: { originKind: "plugin:other.plugin:feature" },
|
||||
}),
|
||||
).rejects.toThrow("Plugin may only use originKind values under plugin:paperclip.missions");
|
||||
});
|
||||
|
||||
it("asserts checkout ownership for run-scoped plugin actions", async () => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const issueId = randomUUID();
|
||||
const runId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({
|
||||
id: runId,
|
||||
companyId,
|
||||
agentId,
|
||||
status: "running",
|
||||
invocationSource: "assignment",
|
||||
contextSnapshot: { issueId },
|
||||
});
|
||||
await db.insert(issues).values({
|
||||
id: issueId,
|
||||
companyId,
|
||||
title: "Checked out issue",
|
||||
status: "in_progress",
|
||||
priority: "medium",
|
||||
assigneeAgentId: agentId,
|
||||
checkoutRunId: runId,
|
||||
executionRunId: runId,
|
||||
});
|
||||
|
||||
const services = buildHostServices(db, "plugin-record-id", "paperclip.missions", createEventBusStub());
|
||||
await expect(
|
||||
services.issues.assertCheckoutOwner({
|
||||
issueId,
|
||||
companyId,
|
||||
actorAgentId: agentId,
|
||||
actorRunId: runId,
|
||||
}),
|
||||
).resolves.toMatchObject({
|
||||
issueId,
|
||||
status: "in_progress",
|
||||
assigneeAgentId: agentId,
|
||||
checkoutRunId: runId,
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses plugin wakeups for issues with unresolved blockers", async () => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const blockerIssueId = randomUUID();
|
||||
const blockedIssueId = randomUUID();
|
||||
await db.insert(issues).values([
|
||||
{
|
||||
id: blockerIssueId,
|
||||
companyId,
|
||||
title: "Unresolved blocker",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
},
|
||||
{
|
||||
id: blockedIssueId,
|
||||
companyId,
|
||||
title: "Blocked issue",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
assigneeAgentId: agentId,
|
||||
},
|
||||
]);
|
||||
await db.insert(issueRelations).values({
|
||||
companyId,
|
||||
issueId: blockerIssueId,
|
||||
relatedIssueId: blockedIssueId,
|
||||
type: "blocks",
|
||||
});
|
||||
|
||||
const services = buildHostServices(db, "plugin-record-id", "paperclip.missions", createEventBusStub());
|
||||
await expect(
|
||||
services.issues.requestWakeup({
|
||||
issueId: blockedIssueId,
|
||||
companyId,
|
||||
reason: "mission_advance",
|
||||
}),
|
||||
).rejects.toThrow("Issue is blocked by unresolved blockers");
|
||||
});
|
||||
|
||||
it("narrows orchestration cost summaries by subtree and billing code", async () => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const rootIssueId = randomUUID();
|
||||
const childIssueId = randomUUID();
|
||||
const unrelatedIssueId = randomUUID();
|
||||
await db.insert(issues).values([
|
||||
{
|
||||
id: rootIssueId,
|
||||
companyId,
|
||||
title: "Root mission",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
billingCode: "mission:alpha",
|
||||
},
|
||||
{
|
||||
id: childIssueId,
|
||||
companyId,
|
||||
parentId: rootIssueId,
|
||||
title: "Child mission",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
billingCode: "mission:alpha",
|
||||
},
|
||||
{
|
||||
id: unrelatedIssueId,
|
||||
companyId,
|
||||
title: "Different mission",
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
billingCode: "mission:alpha",
|
||||
},
|
||||
]);
|
||||
await db.insert(costEvents).values([
|
||||
{
|
||||
companyId,
|
||||
agentId,
|
||||
issueId: rootIssueId,
|
||||
billingCode: "mission:alpha",
|
||||
provider: "test",
|
||||
model: "unit",
|
||||
inputTokens: 10,
|
||||
cachedInputTokens: 1,
|
||||
outputTokens: 2,
|
||||
costCents: 100,
|
||||
occurredAt: new Date(),
|
||||
},
|
||||
{
|
||||
companyId,
|
||||
agentId,
|
||||
issueId: childIssueId,
|
||||
billingCode: "mission:alpha",
|
||||
provider: "test",
|
||||
model: "unit",
|
||||
inputTokens: 20,
|
||||
cachedInputTokens: 2,
|
||||
outputTokens: 4,
|
||||
costCents: 200,
|
||||
occurredAt: new Date(),
|
||||
},
|
||||
{
|
||||
companyId,
|
||||
agentId,
|
||||
issueId: childIssueId,
|
||||
billingCode: "mission:beta",
|
||||
provider: "test",
|
||||
model: "unit",
|
||||
inputTokens: 30,
|
||||
cachedInputTokens: 3,
|
||||
outputTokens: 6,
|
||||
costCents: 300,
|
||||
occurredAt: new Date(),
|
||||
},
|
||||
{
|
||||
companyId,
|
||||
agentId,
|
||||
issueId: unrelatedIssueId,
|
||||
billingCode: "mission:alpha",
|
||||
provider: "test",
|
||||
model: "unit",
|
||||
inputTokens: 40,
|
||||
cachedInputTokens: 4,
|
||||
outputTokens: 8,
|
||||
costCents: 400,
|
||||
occurredAt: new Date(),
|
||||
},
|
||||
]);
|
||||
|
||||
const services = buildHostServices(db, "plugin-record-id", "paperclip.missions", createEventBusStub());
|
||||
const summary = await services.issues.getOrchestrationSummary({
|
||||
companyId,
|
||||
issueId: rootIssueId,
|
||||
includeSubtree: true,
|
||||
});
|
||||
|
||||
expect(new Set(summary.subtreeIssueIds)).toEqual(new Set([rootIssueId, childIssueId]));
|
||||
expect(summary.costs).toMatchObject({
|
||||
billingCode: "mission:alpha",
|
||||
costCents: 300,
|
||||
inputTokens: 30,
|
||||
cachedInputTokens: 3,
|
||||
outputTokens: 6,
|
||||
});
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue