mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-17 03:10:38 +09:00
[codex] Add plugin orchestration host APIs (#4114)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system is the extension path for optional capabilities that should not require core product changes for every integration. > - Plugins need scoped host APIs for issue orchestration, documents, wakeups, summaries, activity attribution, and isolated database state. > - Without those host APIs, richer plugins either cannot coordinate Paperclip work safely or need privileged core-side special cases. > - This pull request adds the plugin orchestration host surface, scoped route dispatch, a database namespace layer, and a smoke plugin that exercises the contract. > - The benefit is a broader plugin API that remains company-scoped, auditable, and covered by tests. ## What Changed - Added plugin orchestration host APIs for issue creation, document access, wakeups, summaries, plugin-origin activity, and scoped API route dispatch. - Added plugin database namespace tables, schema exports, migration checks, and idempotent replay coverage under migration `0059_plugin_database_namespaces`. - Added shared plugin route/API types and validators used by server and SDK boundaries. - Expanded plugin SDK types, protocol helpers, worker RPC host behavior, and testing utilities for orchestration flows. - Added the `plugin-orchestration-smoke-example` package to exercise scoped routes, restricted database namespaces, issue orchestration, documents, wakeups, summaries, and UI status surfaces. - Kept the new orchestration smoke fixture out of the root pnpm workspace importer so this PR preserves the repository policy of not committing `pnpm-lock.yaml`. - Updated plugin docs and database docs for the new orchestration and database namespace surfaces. - Rebased the branch onto `public-gh/master`, resolved conflicts, and removed `pnpm-lock.yaml` from the final PR diff. ## Verification - `pnpm install --frozen-lockfile` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/client.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-scoped-api-routes.test.ts server/src/__tests__/plugin-sdk-orchestration-contract.test.ts` - From `packages/plugins/examples/plugin-orchestration-smoke-example`: `pnpm exec vitest run --config ./vitest.config.ts` - `pnpm --dir packages/plugins/examples/plugin-orchestration-smoke-example run typecheck` - `pnpm --filter @paperclipai/server typecheck` - PR CI on latest head `293fc67c`: `policy`, `verify`, `e2e`, and `security/snyk` all passed. ## Risks - Medium risk: this expands plugin host authority, so route auth, company scoping, and plugin-origin activity attribution need careful review. - Medium risk: database namespace migration behavior must remain idempotent for environments that may have seen earlier branch versions. - Medium risk: the orchestration smoke fixture is intentionally excluded from the root workspace importer to avoid a `pnpm-lock.yaml` PR diff; direct fixture verification remains listed above. - Low operational risk from the PR setup itself: the branch is rebased onto current `master`, the migration is ordered after upstream `0057`/`0058`, and `pnpm-lock.yaml` is not in the final diff. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. Roadmap checked: this work aligns with the completed Plugin system milestone and extends the plugin surface rather than duplicating an unrelated planned core feature. ## Model Used - OpenAI Codex, GPT-5-based coding agent in a tool-enabled CLI environment. Exact hosted model build and context-window size are not exposed by the runtime; reasoning/tool use were enabled for repository inspection, editing, testing, git operations, and PR creation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no core UI screen change; example plugin UI contract is covered by tests) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
16b2b84d84
commit
9c6f551595
53 changed files with 5584 additions and 53 deletions
|
|
@ -118,10 +118,13 @@ Subscribe in `setup` with `ctx.events.on(name, handler)` or `ctx.events.on(name,
|
|||
| `project.created`, `project.updated` | project |
|
||||
| `project.workspace_created`, `project.workspace_updated`, `project.workspace_deleted` | project_workspace |
|
||||
| `issue.created`, `issue.updated`, `issue.comment.created` | issue |
|
||||
| `issue.document.created`, `issue.document.updated`, `issue.document.deleted` | issue |
|
||||
| `issue.relations.updated`, `issue.checked_out`, `issue.released`, `issue.assignment_wakeup_requested` | issue |
|
||||
| `agent.created`, `agent.updated`, `agent.status_changed` | agent |
|
||||
| `agent.run.started`, `agent.run.finished`, `agent.run.failed`, `agent.run.cancelled` | run |
|
||||
| `goal.created`, `goal.updated` | goal |
|
||||
| `approval.created`, `approval.decided` | approval |
|
||||
| `budget.incident.opened`, `budget.incident.resolved` | budget_incident |
|
||||
| `cost_event.created` | cost |
|
||||
| `activity.logged` | activity |
|
||||
|
||||
|
|
@ -301,18 +304,29 @@ Declare in `manifest.capabilities`. Grouped by scope:
|
|||
| | `project.workspaces.read` |
|
||||
| | `issues.read` |
|
||||
| | `issue.comments.read` |
|
||||
| | `issue.documents.read` |
|
||||
| | `issue.relations.read` |
|
||||
| | `issue.subtree.read` |
|
||||
| | `agents.read` |
|
||||
| | `goals.read` |
|
||||
| | `goals.create` |
|
||||
| | `goals.update` |
|
||||
| | `activity.read` |
|
||||
| | `costs.read` |
|
||||
| | `issues.orchestration.read` |
|
||||
| | `database.namespace.read` |
|
||||
| | `issues.create` |
|
||||
| | `issues.update` |
|
||||
| | `issues.checkout` |
|
||||
| | `issues.wakeup` |
|
||||
| | `issue.comments.create` |
|
||||
| | `issue.documents.write` |
|
||||
| | `issue.relations.write` |
|
||||
| | `activity.log.write` |
|
||||
| | `metrics.write` |
|
||||
| | `telemetry.track` |
|
||||
| | `database.namespace.migrate` |
|
||||
| | `database.namespace.write` |
|
||||
| **Instance** | `instance.settings.register` |
|
||||
| | `plugin.state.read` |
|
||||
| | `plugin.state.write` |
|
||||
|
|
@ -320,6 +334,7 @@ Declare in `manifest.capabilities`. Grouped by scope:
|
|||
| | `events.emit` |
|
||||
| | `jobs.schedule` |
|
||||
| | `webhooks.receive` |
|
||||
| | `api.routes.register` |
|
||||
| | `http.outbound` |
|
||||
| | `secrets.read-ref` |
|
||||
| **Agent** | `agent.tools.register` |
|
||||
|
|
@ -337,6 +352,144 @@ Declare in `manifest.capabilities`. Grouped by scope:
|
|||
|
||||
Full list in code: import `PLUGIN_CAPABILITIES` from `@paperclipai/plugin-sdk`.
|
||||
|
||||
### Restricted Database Namespace
|
||||
|
||||
Trusted orchestration plugins can declare a host-owned PostgreSQL namespace:
|
||||
|
||||
```ts
|
||||
database: {
|
||||
migrationsDir: "migrations",
|
||||
coreReadTables: ["issues"],
|
||||
}
|
||||
```
|
||||
|
||||
Declare `database.namespace.migrate` and `database.namespace.read`; add
|
||||
`database.namespace.write` when the worker needs runtime writes. Migrations run
|
||||
before worker startup, are checksum-recorded, and may create or alter objects
|
||||
only inside the plugin namespace. Runtime `ctx.db.query()` allows `SELECT` from
|
||||
`ctx.db.namespace` plus manifest-whitelisted `public` core tables. Runtime
|
||||
`ctx.db.execute()` allows `INSERT`, `UPDATE`, and `DELETE` only against the
|
||||
plugin namespace.
|
||||
|
||||
### Scoped API Routes
|
||||
|
||||
Manifest-declared `apiRoutes` expose JSON routes under
|
||||
`/api/plugins/:pluginId/api/*` without letting a plugin claim core paths:
|
||||
|
||||
```ts
|
||||
apiRoutes: [
|
||||
{
|
||||
routeKey: "initialize",
|
||||
method: "POST",
|
||||
path: "/issues/:issueId/smoke",
|
||||
auth: "board-or-agent",
|
||||
capability: "api.routes.register",
|
||||
checkoutPolicy: "required-for-agent-in-progress",
|
||||
companyResolution: { from: "issue", param: "issueId" },
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
Implement `onApiRequest(input)` in the worker to handle the route. The host
|
||||
performs auth, company access, capability, route matching, and checkout policy
|
||||
before dispatch. The worker receives route params, query, parsed JSON body,
|
||||
sanitized headers, actor context, and `companyId`; responses are JSON `{ status?,
|
||||
headers?, body? }`.
|
||||
|
||||
## Issue Orchestration APIs
|
||||
|
||||
Workflow plugins can use `ctx.issues` for orchestration-grade issue operations without importing host server internals.
|
||||
|
||||
Expanded create/update fields include blockers, billing code, board or agent assignees, labels, namespaced plugin origins, request depth, and safe execution workspace fields:
|
||||
|
||||
```ts
|
||||
const child = await ctx.issues.create({
|
||||
companyId,
|
||||
parentId: missionIssueId,
|
||||
inheritExecutionWorkspaceFromIssueId: missionIssueId,
|
||||
title: "Implement feature slice",
|
||||
status: "todo",
|
||||
assigneeAgentId: workerAgentId,
|
||||
billingCode: "mission:alpha",
|
||||
originKind: "plugin:paperclip.missions:feature",
|
||||
originId: "mission-alpha:feature-1",
|
||||
blockedByIssueIds: [planningIssueId],
|
||||
});
|
||||
```
|
||||
|
||||
If `originKind` is omitted, the host stores `plugin:<pluginKey>`. Plugins may use sub-kinds such as `plugin:<pluginKey>:feature`, but the host rejects attempts to set another plugin's namespace.
|
||||
|
||||
Blocker relationships are also exposed as first-class helpers:
|
||||
|
||||
```ts
|
||||
const relations = await ctx.issues.relations.get(child.id, companyId);
|
||||
await ctx.issues.relations.setBlockedBy(child.id, [planningIssueId], companyId);
|
||||
await ctx.issues.relations.addBlockers(child.id, [validationIssueId], companyId);
|
||||
await ctx.issues.relations.removeBlockers(child.id, [planningIssueId], companyId);
|
||||
```
|
||||
|
||||
Subtree reads can include just the issue tree, or compact related data for orchestration dashboards:
|
||||
|
||||
```ts
|
||||
const subtree = await ctx.issues.getSubtree(missionIssueId, companyId, {
|
||||
includeRoot: true,
|
||||
includeRelations: true,
|
||||
includeDocuments: true,
|
||||
includeActiveRuns: true,
|
||||
includeAssignees: true,
|
||||
});
|
||||
```
|
||||
|
||||
Agent-run actions can assert checkout ownership before mutating in-progress work:
|
||||
|
||||
```ts
|
||||
await ctx.issues.assertCheckoutOwner({
|
||||
issueId,
|
||||
companyId,
|
||||
actorAgentId: runCtx.agentId,
|
||||
actorRunId: runCtx.runId,
|
||||
});
|
||||
```
|
||||
|
||||
Plugins can request assignment wakeups through the host so budget stops, execution locks, blocker checks, and heartbeat policy still apply:
|
||||
|
||||
```ts
|
||||
await ctx.issues.requestWakeup(child.id, companyId, {
|
||||
reason: "mission_advance",
|
||||
contextSource: "missions.advance",
|
||||
});
|
||||
|
||||
await ctx.issues.requestWakeups([featureIssueId, validationIssueId], companyId, {
|
||||
reason: "mission_advance",
|
||||
contextSource: "missions.advance",
|
||||
idempotencyKeyPrefix: `mission:${missionIssueId}:advance`,
|
||||
});
|
||||
```
|
||||
|
||||
Use `ctx.issues.summaries.getOrchestration()` when a workflow needs compact reads across a root issue or subtree:
|
||||
|
||||
```ts
|
||||
const summary = await ctx.issues.summaries.getOrchestration({
|
||||
issueId: missionIssueId,
|
||||
companyId,
|
||||
includeSubtree: true,
|
||||
billingCode: "mission:alpha",
|
||||
});
|
||||
```
|
||||
|
||||
Required capabilities:
|
||||
|
||||
| API | Capability |
|
||||
|-----|------------|
|
||||
| `ctx.issues.relations.get` | `issue.relations.read` |
|
||||
| `ctx.issues.relations.setBlockedBy` / `addBlockers` / `removeBlockers` | `issue.relations.write` |
|
||||
| `ctx.issues.getSubtree` | `issue.subtree.read` |
|
||||
| `ctx.issues.assertCheckoutOwner` | `issues.checkout` |
|
||||
| `ctx.issues.requestWakeup` / `requestWakeups` | `issues.wakeup` |
|
||||
| `ctx.issues.summaries.getOrchestration` | `issues.orchestration.read` |
|
||||
|
||||
Plugin-originated mutations are logged with `actorType: "plugin"` and details fields `sourcePluginId`, `sourcePluginKey`, `initiatingActorType`, `initiatingActorId`, and `initiatingRunId` when a user or agent run initiated the plugin work.
|
||||
|
||||
## UI quick start
|
||||
|
||||
```tsx
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue