mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-16 10:50:38 +09:00
[codex] Add plugin orchestration host APIs (#4114)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system is the extension path for optional capabilities that should not require core product changes for every integration. > - Plugins need scoped host APIs for issue orchestration, documents, wakeups, summaries, activity attribution, and isolated database state. > - Without those host APIs, richer plugins either cannot coordinate Paperclip work safely or need privileged core-side special cases. > - This pull request adds the plugin orchestration host surface, scoped route dispatch, a database namespace layer, and a smoke plugin that exercises the contract. > - The benefit is a broader plugin API that remains company-scoped, auditable, and covered by tests. ## What Changed - Added plugin orchestration host APIs for issue creation, document access, wakeups, summaries, plugin-origin activity, and scoped API route dispatch. - Added plugin database namespace tables, schema exports, migration checks, and idempotent replay coverage under migration `0059_plugin_database_namespaces`. - Added shared plugin route/API types and validators used by server and SDK boundaries. - Expanded plugin SDK types, protocol helpers, worker RPC host behavior, and testing utilities for orchestration flows. - Added the `plugin-orchestration-smoke-example` package to exercise scoped routes, restricted database namespaces, issue orchestration, documents, wakeups, summaries, and UI status surfaces. - Kept the new orchestration smoke fixture out of the root pnpm workspace importer so this PR preserves the repository policy of not committing `pnpm-lock.yaml`. - Updated plugin docs and database docs for the new orchestration and database namespace surfaces. - Rebased the branch onto `public-gh/master`, resolved conflicts, and removed `pnpm-lock.yaml` from the final PR diff. ## Verification - `pnpm install --frozen-lockfile` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/client.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-scoped-api-routes.test.ts server/src/__tests__/plugin-sdk-orchestration-contract.test.ts` - From `packages/plugins/examples/plugin-orchestration-smoke-example`: `pnpm exec vitest run --config ./vitest.config.ts` - `pnpm --dir packages/plugins/examples/plugin-orchestration-smoke-example run typecheck` - `pnpm --filter @paperclipai/server typecheck` - PR CI on latest head `293fc67c`: `policy`, `verify`, `e2e`, and `security/snyk` all passed. ## Risks - Medium risk: this expands plugin host authority, so route auth, company scoping, and plugin-origin activity attribution need careful review. - Medium risk: database namespace migration behavior must remain idempotent for environments that may have seen earlier branch versions. - Medium risk: the orchestration smoke fixture is intentionally excluded from the root workspace importer to avoid a `pnpm-lock.yaml` PR diff; direct fixture verification remains listed above. - Low operational risk from the PR setup itself: the branch is rebased onto current `master`, the migration is ordered after upstream `0057`/`0058`, and `pnpm-lock.yaml` is not in the final diff. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. Roadmap checked: this work aligns with the completed Plugin system milestone and extends the plugin surface rather than duplicating an unrelated planned core feature. ## Model Used - OpenAI Codex, GPT-5-based coding agent in a tool-enabled CLI environment. Exact hosted model build and context-window size are not exposed by the runtime; reasoning/tool use were enabled for repository inspection, editing, testing, git operations, and PR creation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no core UI screen change; example plugin UI contract is covered by tests) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
16b2b84d84
commit
9c6f551595
53 changed files with 5584 additions and 53 deletions
427
server/src/__tests__/plugin-scoped-api-routes.test.ts
Normal file
427
server/src/__tests__/plugin-scoped-api-routes.test.ts
Normal file
|
|
@ -0,0 +1,427 @@
|
|||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { pluginManifestV1Schema, type PaperclipPluginManifestV1 } from "@paperclipai/shared";
|
||||
|
||||
const mockRegistry = vi.hoisted(() => ({
|
||||
getById: vi.fn(),
|
||||
getByKey: vi.fn(),
|
||||
}));
|
||||
|
||||
const mockLifecycle = vi.hoisted(() => ({
|
||||
load: vi.fn(),
|
||||
upgrade: vi.fn(),
|
||||
}));
|
||||
|
||||
const mockIssueService = vi.hoisted(() => ({
|
||||
getById: vi.fn(),
|
||||
assertCheckoutOwner: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("../services/plugin-registry.js", () => ({
|
||||
pluginRegistryService: () => mockRegistry,
|
||||
}));
|
||||
|
||||
vi.mock("../services/plugin-lifecycle.js", () => ({
|
||||
pluginLifecycleManager: () => mockLifecycle,
|
||||
}));
|
||||
|
||||
vi.mock("../services/issues.js", () => ({
|
||||
issueService: () => mockIssueService,
|
||||
}));
|
||||
|
||||
vi.mock("../services/activity-log.js", () => ({
|
||||
logActivity: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("../services/live-events.js", () => ({
|
||||
publishGlobalLiveEvent: vi.fn(),
|
||||
}));
|
||||
|
||||
function manifest(apiRoutes: NonNullable<PaperclipPluginManifestV1["apiRoutes"]>): PaperclipPluginManifestV1 {
|
||||
return {
|
||||
id: "paperclip.scoped-api-test",
|
||||
apiVersion: 1,
|
||||
version: "1.0.0",
|
||||
displayName: "Scoped API Test",
|
||||
description: "Test plugin for scoped API routes",
|
||||
author: "Paperclip",
|
||||
categories: ["automation"],
|
||||
capabilities: ["api.routes.register"],
|
||||
entrypoints: { worker: "dist/worker.js" },
|
||||
apiRoutes,
|
||||
};
|
||||
}
|
||||
|
||||
async function createApp(input: {
|
||||
actor: Record<string, unknown>;
|
||||
plugin?: Record<string, unknown> | null;
|
||||
workerRunning?: boolean;
|
||||
workerResult?: unknown;
|
||||
}) {
|
||||
const [{ pluginRoutes }, { errorHandler }] = await Promise.all([
|
||||
import("../routes/plugins.js"),
|
||||
import("../middleware/index.js"),
|
||||
]);
|
||||
|
||||
const workerManager = {
|
||||
isRunning: vi.fn().mockReturnValue(input.workerRunning ?? true),
|
||||
call: vi.fn().mockResolvedValue(input.workerResult ?? { status: 200, body: { ok: true } }),
|
||||
};
|
||||
|
||||
mockRegistry.getById.mockResolvedValue(input.plugin ?? null);
|
||||
mockRegistry.getByKey.mockResolvedValue(input.plugin ?? null);
|
||||
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
app.use((req, _res, next) => {
|
||||
req.actor = input.actor as typeof req.actor;
|
||||
next();
|
||||
});
|
||||
app.use(
|
||||
"/api",
|
||||
pluginRoutes(
|
||||
{} as never,
|
||||
{ installPlugin: vi.fn() } as never,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
{ workerManager } as never,
|
||||
),
|
||||
);
|
||||
app.use(errorHandler);
|
||||
|
||||
return { app, workerManager };
|
||||
}
|
||||
|
||||
describe("plugin scoped API routes", () => {
|
||||
const pluginId = "11111111-1111-4111-8111-111111111111";
|
||||
const companyId = "22222222-2222-4222-8222-222222222222";
|
||||
const agentId = "33333333-3333-4333-8333-333333333333";
|
||||
const runId = "44444444-4444-4444-8444-444444444444";
|
||||
const issueId = "55555555-5555-4555-8555-555555555555";
|
||||
|
||||
beforeEach(() => {
|
||||
vi.resetAllMocks();
|
||||
mockIssueService.getById.mockResolvedValue(null);
|
||||
mockIssueService.assertCheckoutOwner.mockResolvedValue({
|
||||
id: issueId,
|
||||
status: "in_progress",
|
||||
assigneeAgentId: agentId,
|
||||
checkoutRunId: runId,
|
||||
adoptedFromRunId: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("dispatches a board GET route with params, query, actor, and company context", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "summary.get",
|
||||
method: "GET",
|
||||
path: "/companies/:companySlug/summary",
|
||||
auth: "board",
|
||||
capability: "api.routes.register",
|
||||
companyResolution: { from: "query", key: "companyId" },
|
||||
},
|
||||
]);
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "board",
|
||||
userId: "user-1",
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: true,
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
workerResult: { status: 201, body: { handled: true } },
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.get(`/api/plugins/${pluginId}/api/companies/acme/summary?companyId=${companyId}&view=compact`)
|
||||
.set("Authorization", "Bearer should-not-forward");
|
||||
|
||||
expect(res.status).toBe(201);
|
||||
expect(res.body).toEqual({ handled: true });
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "handleApiRequest", expect.objectContaining({
|
||||
routeKey: "summary.get",
|
||||
method: "GET",
|
||||
params: { companySlug: "acme" },
|
||||
query: { companyId, view: "compact" },
|
||||
companyId,
|
||||
actor: expect.objectContaining({ actorType: "user", actorId: "user-1" }),
|
||||
}));
|
||||
expect(workerManager.call.mock.calls[0]?.[2].headers.authorization).toBeUndefined();
|
||||
});
|
||||
|
||||
it("only forwards allowlisted response headers from plugin routes", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "summary.get",
|
||||
method: "GET",
|
||||
path: "/companies/:companySlug/summary",
|
||||
auth: "board",
|
||||
capability: "api.routes.register",
|
||||
companyResolution: { from: "query", key: "companyId" },
|
||||
},
|
||||
]);
|
||||
const { app } = await createApp({
|
||||
actor: {
|
||||
type: "board",
|
||||
userId: "user-1",
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: true,
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
workerResult: {
|
||||
status: 200,
|
||||
body: { handled: true },
|
||||
headers: {
|
||||
"cache-control": "no-store",
|
||||
"content-security-policy": "default-src 'none'",
|
||||
location: "https://example.invalid",
|
||||
"x-request-id": "plugin-request",
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.get(`/api/plugins/${pluginId}/api/companies/acme/summary?companyId=${companyId}`);
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.headers["cache-control"]).toBe("no-store");
|
||||
expect(res.headers["x-request-id"]).toBe("plugin-request");
|
||||
expect(res.headers["content-security-policy"]).toBeUndefined();
|
||||
expect(res.headers.location).toBeUndefined();
|
||||
});
|
||||
|
||||
it("enforces agent checkout ownership before dispatching issue-scoped POST routes", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "issue.advance",
|
||||
method: "POST",
|
||||
path: "/issues/:issueId/advance",
|
||||
auth: "agent",
|
||||
capability: "api.routes.register",
|
||||
checkoutPolicy: "required-for-agent-in-progress",
|
||||
companyResolution: { from: "issue", param: "issueId" },
|
||||
},
|
||||
]);
|
||||
mockIssueService.getById.mockResolvedValue({
|
||||
id: issueId,
|
||||
companyId,
|
||||
status: "in_progress",
|
||||
assigneeAgentId: agentId,
|
||||
});
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "agent",
|
||||
agentId,
|
||||
companyId,
|
||||
runId,
|
||||
source: "agent_key",
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/plugins/${pluginId}/api/issues/${issueId}/advance`)
|
||||
.send({ step: "next" });
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(mockIssueService.assertCheckoutOwner).toHaveBeenCalledWith(issueId, agentId, runId);
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "handleApiRequest", expect.objectContaining({
|
||||
routeKey: "issue.advance",
|
||||
params: { issueId },
|
||||
body: { step: "next" },
|
||||
actor: expect.objectContaining({ actorType: "agent", agentId, runId }),
|
||||
companyId,
|
||||
}));
|
||||
});
|
||||
|
||||
it("rejects checkout-protected agent routes without a run id before worker dispatch", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "issue.advance",
|
||||
method: "POST",
|
||||
path: "/issues/:issueId/advance",
|
||||
auth: "agent",
|
||||
capability: "api.routes.register",
|
||||
checkoutPolicy: "required-for-agent-in-progress",
|
||||
companyResolution: { from: "issue", param: "issueId" },
|
||||
},
|
||||
]);
|
||||
mockIssueService.getById.mockResolvedValue({
|
||||
id: issueId,
|
||||
companyId,
|
||||
status: "in_progress",
|
||||
assigneeAgentId: agentId,
|
||||
});
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "agent",
|
||||
agentId,
|
||||
companyId,
|
||||
source: "agent_key",
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/plugins/${pluginId}/api/issues/${issueId}/advance`)
|
||||
.send({});
|
||||
|
||||
expect(res.status).toBe(401);
|
||||
expect(workerManager.call).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("rejects checkout-protected agent routes when the active checkout belongs to another run", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "issue.advance",
|
||||
method: "POST",
|
||||
path: "/issues/:issueId/advance",
|
||||
auth: "agent",
|
||||
capability: "api.routes.register",
|
||||
checkoutPolicy: "always-for-agent",
|
||||
companyResolution: { from: "issue", param: "issueId" },
|
||||
},
|
||||
]);
|
||||
mockIssueService.getById.mockResolvedValue({
|
||||
id: issueId,
|
||||
companyId,
|
||||
status: "in_progress",
|
||||
assigneeAgentId: agentId,
|
||||
});
|
||||
const conflict = new Error("Issue run ownership conflict") as Error & { status?: number };
|
||||
conflict.status = 409;
|
||||
mockIssueService.assertCheckoutOwner.mockRejectedValue(conflict);
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "agent",
|
||||
agentId,
|
||||
companyId,
|
||||
runId,
|
||||
source: "agent_key",
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/plugins/${pluginId}/api/issues/${issueId}/advance`)
|
||||
.send({});
|
||||
|
||||
expect(res.status).toBe(409);
|
||||
expect(workerManager.call).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("returns a clear error for disabled plugins without worker dispatch", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "summary.get",
|
||||
method: "GET",
|
||||
path: "/summary",
|
||||
auth: "board",
|
||||
capability: "api.routes.register",
|
||||
companyResolution: { from: "query", key: "companyId" },
|
||||
},
|
||||
]);
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "board",
|
||||
userId: "user-1",
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: true,
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "disabled",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.get(`/api/plugins/${pluginId}/api/summary?companyId=${companyId}`);
|
||||
|
||||
expect(res.status).toBe(503);
|
||||
expect(res.body.error).toContain("disabled");
|
||||
expect(workerManager.call).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("returns a clear error when a ready plugin has no running worker", async () => {
|
||||
const apiRoutes = manifest([
|
||||
{
|
||||
routeKey: "summary.get",
|
||||
method: "GET",
|
||||
path: "/summary",
|
||||
auth: "board",
|
||||
capability: "api.routes.register",
|
||||
companyResolution: { from: "query", key: "companyId" },
|
||||
},
|
||||
]);
|
||||
const { app, workerManager } = await createApp({
|
||||
actor: {
|
||||
type: "board",
|
||||
userId: "user-1",
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: true,
|
||||
},
|
||||
plugin: {
|
||||
id: pluginId,
|
||||
pluginKey: apiRoutes.id,
|
||||
status: "ready",
|
||||
manifestJson: apiRoutes,
|
||||
},
|
||||
workerRunning: false,
|
||||
});
|
||||
|
||||
const res = await request(app)
|
||||
.get(`/api/plugins/${pluginId}/api/summary?companyId=${companyId}`);
|
||||
|
||||
expect(res.status).toBe(503);
|
||||
expect(res.body.error).toContain("worker is not running");
|
||||
expect(workerManager.call).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("rejects manifest routes that try to claim core API paths", () => {
|
||||
const result = pluginManifestV1Schema.safeParse(manifest([
|
||||
{
|
||||
routeKey: "bad.shadow",
|
||||
method: "POST",
|
||||
path: "/api/issues/:issueId",
|
||||
auth: "board",
|
||||
capability: "api.routes.register",
|
||||
},
|
||||
]));
|
||||
|
||||
expect(result.success).toBe(false);
|
||||
if (result.success) throw new Error("Expected manifest validation to fail");
|
||||
expect(result.error.issues.map((issue) => issue.message).join("\n")).toContain(
|
||||
"path must stay inside the plugin api namespace",
|
||||
);
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue