[codex] Add plugin orchestration host APIs (#4114)

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system is the extension path for optional capabilities
that should not require core product changes for every integration.
> - Plugins need scoped host APIs for issue orchestration, documents,
wakeups, summaries, activity attribution, and isolated database state.
> - Without those host APIs, richer plugins either cannot coordinate
Paperclip work safely or need privileged core-side special cases.
> - This pull request adds the plugin orchestration host surface, scoped
route dispatch, a database namespace layer, and a smoke plugin that
exercises the contract.
> - The benefit is a broader plugin API that remains company-scoped,
auditable, and covered by tests.

## What Changed

- Added plugin orchestration host APIs for issue creation, document
access, wakeups, summaries, plugin-origin activity, and scoped API route
dispatch.
- Added plugin database namespace tables, schema exports, migration
checks, and idempotent replay coverage under migration
`0059_plugin_database_namespaces`.
- Added shared plugin route/API types and validators used by server and
SDK boundaries.
- Expanded plugin SDK types, protocol helpers, worker RPC host behavior,
and testing utilities for orchestration flows.
- Added the `plugin-orchestration-smoke-example` package to exercise
scoped routes, restricted database namespaces, issue orchestration,
documents, wakeups, summaries, and UI status surfaces.
- Kept the new orchestration smoke fixture out of the root pnpm
workspace importer so this PR preserves the repository policy of not
committing `pnpm-lock.yaml`.
- Updated plugin docs and database docs for the new orchestration and
database namespace surfaces.
- Rebased the branch onto `public-gh/master`, resolved conflicts, and
removed `pnpm-lock.yaml` from the final PR diff.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm --filter @paperclipai/db typecheck`
- `pnpm exec vitest run packages/db/src/client.test.ts`
- `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts
server/src/__tests__/plugin-orchestration-apis.test.ts
server/src/__tests__/plugin-routes-authz.test.ts
server/src/__tests__/plugin-scoped-api-routes.test.ts
server/src/__tests__/plugin-sdk-orchestration-contract.test.ts`
- From `packages/plugins/examples/plugin-orchestration-smoke-example`:
`pnpm exec vitest run --config ./vitest.config.ts`
- `pnpm --dir
packages/plugins/examples/plugin-orchestration-smoke-example run
typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- PR CI on latest head `293fc67c`: `policy`, `verify`, `e2e`, and
`security/snyk` all passed.

## Risks

- Medium risk: this expands plugin host authority, so route auth,
company scoping, and plugin-origin activity attribution need careful
review.
- Medium risk: database namespace migration behavior must remain
idempotent for environments that may have seen earlier branch versions.
- Medium risk: the orchestration smoke fixture is intentionally excluded
from the root workspace importer to avoid a `pnpm-lock.yaml` PR diff;
direct fixture verification remains listed above.
- Low operational risk from the PR setup itself: the branch is rebased
onto current `master`, the migration is ordered after upstream
`0057`/`0058`, and `pnpm-lock.yaml` is not in the final diff.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

Roadmap checked: this work aligns with the completed Plugin system
milestone and extends the plugin surface rather than duplicating an
unrelated planned core feature.

## Model Used

- OpenAI Codex, GPT-5-based coding agent in a tool-enabled CLI
environment. Exact hosted model build and context-window size are not
exposed by the runtime; reasoning/tool use were enabled for repository
inspection, editing, testing, git operations, and PR creation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots (N/A: no core UI screen change; example plugin UI contract
is covered by tests)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

doc/DATABASE.md

@@ -94,6 +94,16 @@ Set `DATABASE_URL` in your `.env`:
 DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:6543/postgres
 ```
 
+For hosted deployments that use a pooled runtime URL, set
+`DATABASE_MIGRATION_URL` to the direct connection URL. Paperclip uses it for
+startup schema checks/migrations and plugin namespace migrations, while the app
+continues to use `DATABASE_URL` for runtime queries:
+
+```sh
+DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:6543/postgres
+DATABASE_MIGRATION_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:5432/postgres
+```
+
 If using connection pooling (port 6543), the `postgres` client must disable prepared statements. Update `packages/db/src/client.ts`:
 
 ```ts

doc/plugins/PLUGIN_AUTHORING_GUIDE.md

@@ -10,6 +10,9 @@ It is intentionally narrower than [PLUGIN_SPEC.md](./PLUGIN_SPEC.md). The spec i
 - Plugin UI runs as same-origin JavaScript inside the main Paperclip app.
 - Worker-side host APIs are capability-gated.
 - Plugin UI is not sandboxed by manifest capabilities.
+- Plugin database migrations are restricted to a host-derived plugin namespace.
+- Plugin-owned JSON API routes must be declared in the manifest and are mounted
+  only under `/api/plugins/:pluginId/api/*`.
 - There is no host-provided shared React component kit for plugins yet.
 - `ctx.assets` is not supported in the current runtime.
 
@@ -77,10 +80,12 @@ Worker:
 - secrets
 - activity
 - state
+- database namespace via `ctx.db`
+- scoped JSON API routes declared with `apiRoutes`
 - entities
 - projects and project workspaces
 - companies
-- issues and comments
+- issues, comments, namespaced `plugin:<pluginKey>` origins, blocker relations, checkout assertions, assignment wakeups, and orchestration summaries
 - agents and agent sessions
 - goals
 - data/actions
@@ -89,6 +94,55 @@ Worker:
 - metrics
 - logger
 
+### Plugin database declarations
+
+First-party or otherwise trusted orchestration plugins can declare:
+
+```ts
+database: {
+  migrationsDir: "migrations",
+  coreReadTables: ["issues"],
+}
+```
+
+Required capabilities are `database.namespace.migrate` and
+`database.namespace.read`; add `database.namespace.write` for runtime mutations.
+The host derives `ctx.db.namespace`, runs SQL files in filename order before the
+worker starts, records checksums in `plugin_migrations`, and rejects changed
+already-applied migrations.
+
+Migration SQL may create or alter objects only inside `ctx.db.namespace`. It may
+reference whitelisted `public` core tables for foreign keys or read-only views,
+but may not mutate/alter/drop/truncate public tables, create extensions,
+triggers, untrusted languages, or runtime multi-statement SQL. Runtime
+`ctx.db.query()` is restricted to `SELECT`; runtime `ctx.db.execute()` is
+restricted to namespace-local `INSERT`, `UPDATE`, and `DELETE`.
+
+### Scoped plugin API routes
+
+Plugins can expose JSON-only routes under their own namespace:
+
+```ts
+apiRoutes: [
+  {
+    routeKey: "initialize",
+    method: "POST",
+    path: "/issues/:issueId/smoke",
+    auth: "board-or-agent",
+    capability: "api.routes.register",
+    checkoutPolicy: "required-for-agent-in-progress",
+    companyResolution: { from: "issue", param: "issueId" },
+  },
+]
+```
+
+The host resolves the plugin, checks that it is ready, enforces
+`api.routes.register`, matches the declared method/path, resolves company access,
+and applies checkout policy before dispatching to the worker's `onApiRequest`
+handler. The worker receives sanitized headers, route params, query, parsed JSON
+body, actor context, and company id. Do not use plugin routes to claim core
+paths; they always remain under `/api/plugins/:pluginId/api/*`.
+
 UI:
 
 - `usePluginData`

doc/plugins/PLUGIN_SPEC.md

@@ -28,6 +28,9 @@ Current limitations to keep in mind:
 - The repo example plugins under `packages/plugins/examples/` are development conveniences. They work from a source checkout and should not be assumed to exist in a generic published build unless they are explicitly shipped with that build.
 - Dynamic plugin install is not yet cloud-ready for horizontally scaled or ephemeral deployments. There is no shared artifact store, install coordination, or cross-node distribution layer yet.
 - The current runtime does not yet ship a real host-provided plugin UI component kit, and it does not support plugin asset uploads/reads. Treat those as future-scope ideas in this spec, not current implementation promises.
+- Scoped plugin API routes are JSON-only and must be declared in `apiRoutes`.
+  They mount under `/api/plugins/:pluginId/api/*`; plugins cannot shadow core
+  API routes.
 
 In practice, that means the current implementation is a good fit for local development and self-hosted persistent deployments, but not yet for multi-instance cloud plugin distribution.
 
@@ -624,7 +627,46 @@ Required SDK clients:
 
 Plugins that need filesystem, git, terminal, or process operations handle those directly using standard Node APIs or libraries. The host provides project workspace metadata through `ctx.projects` so plugins can resolve workspace paths, but the host does not proxy low-level OS operations.
 
-## 14.1 Example SDK Shape
+## 14.1 Issue Orchestration APIs
+
+Trusted orchestration plugins can create and update Paperclip issues through `ctx.issues` instead of importing server internals. The public issue contract includes parent/project/goal links, board or agent assignees, blocker IDs, labels, billing code, request depth, execution workspace inheritance, and plugin origin metadata.
+
+Origin rules:
+
+- Built-in core issues keep built-in origins such as `manual` and `routine_execution`.
+- Plugin-managed issues use `plugin:<pluginKey>` or a sub-kind such as `plugin:<pluginKey>:feature`.
+- The host derives the default plugin origin from the installed plugin key and rejects attempts to set `plugin:<otherPluginKey>` origins.
+- `originId` is plugin-defined and should be stable for idempotent generated work.
+
+Relation and read helpers:
+
+- `ctx.issues.relations.get(issueId, companyId)`
+- `ctx.issues.relations.setBlockedBy(issueId, blockerIssueIds, companyId)`
+- `ctx.issues.relations.addBlockers(issueId, blockerIssueIds, companyId)`
+- `ctx.issues.relations.removeBlockers(issueId, blockerIssueIds, companyId)`
+- `ctx.issues.getSubtree(issueId, companyId, options)`
+- `ctx.issues.summaries.getOrchestration({ issueId, companyId, includeSubtree, billingCode })`
+
+Governance helpers:
+
+- `ctx.issues.assertCheckoutOwner({ issueId, companyId, actorAgentId, actorRunId })` lets plugin actions preserve agent-run checkout ownership.
+- `ctx.issues.requestWakeup(issueId, companyId, options)` requests assignment wakeups through host heartbeat semantics, including terminal-status, blocker, assignee, and budget hard-stop checks.
+- `ctx.issues.requestWakeups(issueIds, companyId, options)` applies the same host-owned wakeup semantics to a batch and may use an idempotency key prefix for stable coordinator retries.
+
+Plugin-originated issue, relation, document, comment, and wakeup mutations must write activity entries with `actorType: "plugin"` and details fields for `sourcePluginId`, `sourcePluginKey`, `initiatingActorType`, `initiatingActorId`, and `initiatingRunId` when a user or agent run initiated the plugin work.
+
+Scoped API routes:
+
+- `apiRoutes[]` declares `routeKey`, `method`, plugin-local `path`, `auth`,
+  `capability`, optional checkout policy, and company resolution.
+- The host enforces auth, company access, `api.routes.register`, route matching,
+  and checkout policy before worker dispatch.
+- The worker implements `onApiRequest(input)` and returns a JSON response shape
+  `{ status?, headers?, body? }`.
+- Only safe request headers are forwarded; auth/cookie headers are never passed
+  to the worker.
+
+## 14.2 Example SDK Shape
 
 ```ts
 /** Top-level helper for defining a plugin with type checking */
@@ -696,16 +738,24 @@ The host enforces capabilities in the SDK layer and refuses calls outside the gr
 - `project.workspaces.read`
 - `issues.read`
 - `issue.comments.read`
+- `issue.documents.read`
+- `issue.relations.read`
+- `issue.subtree.read`
 - `agents.read`
 - `goals.read`
 - `activity.read`
 - `costs.read`
+- `issues.orchestration.read`
 
 ### Data Write
 
 - `issues.create`
 - `issues.update`
 - `issue.comments.create`
+- `issue.documents.write`
+- `issue.relations.write`
+- `issues.checkout`
+- `issues.wakeup`
 - `assets.write`
 - `assets.read`
 - `activity.log.write`
@@ -772,6 +822,13 @@ Minimum event set:
 - `issue.created`
 - `issue.updated`
 - `issue.comment.created`
+- `issue.document.created`
+- `issue.document.updated`
+- `issue.document.deleted`
+- `issue.relations.updated`
+- `issue.checked_out`
+- `issue.released`
+- `issue.assignment_wakeup_requested`
 - `agent.created`
 - `agent.updated`
 - `agent.status_changed`
@@ -781,6 +838,8 @@ Minimum event set:
 - `agent.run.cancelled`
 - `approval.created`
 - `approval.decided`
+- `budget.incident.opened`
+- `budget.incident.resolved`
 - `cost_event.created`
 - `activity.logged`
 
@@ -1238,6 +1297,8 @@ Plugin-originated mutations should write:
 
 - `actor_type = plugin`
 - `actor_id = <plugin-id>`
+- details include `sourcePluginId` and `sourcePluginKey`
+- details include `initiatingActorType`, `initiatingActorId`, and `initiatingRunId` when a user or agent run triggered the plugin work
 
 ## 21.5 Plugin Migrations