[codex] Add LLM Wiki plugin host support (#5597)

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system needs host contracts and runtime support before
large plugins can integrate cleanly.
> - The source branch mixed the LLM Wiki package with supporting
host/runtime work, managed plugin skills, root-level storage spaces, and
a bookmarks reference plugin.
> - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be
split by file boundary: plugin package separately from everything else.
> - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have
plugin-local spaces, but Paperclip core should not reorganize top-level
local storage into spaces.
> - Follow-up review clarified that the bookmarks example should not
ship in this PR either.
> - This pull request contains the
non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime
state under the selected Paperclip instance root, and no longer includes
the bookmarks example.

## What Changed

- Added/updated plugin host contracts, SDK types, worker RPC plumbing,
managed plugin skill support, and related server tests.
- Removed the bookmarks example plugin package and its
bundled-example/workspace references.
- Removed the root-level local spaces CLI/migration surface and restored
instance-root runtime defaults for config, db, logs, storage, secrets,
workspaces, projects, and adapter homes.
- Replaced shared root `space-paths` helpers with `home-paths` helpers
for core runtime storage.
- Tightened stranded recovery unique-conflict detection so concurrent
recovery scans reuse the raced recovery issue when Postgres errors are
wrapped.
- Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff;
plugin-local spaces remain in the stacked plugin-only PR.

## Verification

- `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts
cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts
packages/shared/src/home-paths.test.ts
packages/db/src/runtime-config.test.ts
server/src/__tests__/agent-instructions-service.test.ts
server/src/__tests__/claude-local-execute.test.ts
server/src/__tests__/codex-local-execute.test.ts`
- `pnpm exec vitest run packages/db/src/runtime-config.test.ts`
- `pnpm exec vitest run
server/src/__tests__/plugin-routes-authz.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the
raced stranded recovery issue"` skipped locally because embedded
Postgres did not initialize on this macOS temp host; the code path was
typechecked and is covered by Linux CI.
- Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`,
`spaces migrate-default`, `@paperclipai/shared/space-paths`,
`registerSpacesCommands`, or the removed bookmarks example.
- Previous PR head `4f23e034` had green GitHub checks: `verify`, all
four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk,
and `Greptile Review`. Current head `582f466d` is re-running checks
after the bookmarks deletion.

## Risks

- Plugin host changes touch shared runtime paths, so regressions would
most likely appear in adapter startup, plugin loading, or local dev path
defaults.
- Removing the bookmarks example also removes one demonstration of
plugin database namespaces plus local-folder persistence; remaining
plugin examples still cover bundled example discovery and plugin host
flows.
- The plugin package itself is intentionally deferred to the stacked
plugin-only PR, where LLM Wiki plugin-local spaces live.
- Existing installs that tested the transient root-level spaces CLI
should stop using it; this PR intentionally removes that unsupported
migration surface before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution
enabled; context window not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass, except where noted above
for host-specific embedded Postgres initialization
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Stacked follow-up: PR #5592 contains only
`packages/plugins/plugin-llm-wiki/` and targets this branch.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta 2026-05-10 07:34:12 -05:00 committed by GitHub
parent eb12c42009
commit 0096b56a1c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
40 changed files with 1892 additions and 224 deletions

View file

@ -0,0 +1,27 @@
import { describe, expect, it } from "vitest";
import { paperclipConfigSchema } from "./config-schema.js";
describe("paperclip config schema", () => {
it("defaults omitted runtime paths to legacy instance-root locations", () => {
const parsed = paperclipConfigSchema.parse({
$meta: {
version: 1,
updatedAt: "2026-05-10T00:00:00.000Z",
source: "configure",
},
database: {
mode: "embedded-postgres",
},
logging: {
mode: "file",
},
server: {},
});
expect(parsed.database.embeddedPostgresDataDir).toBe("~/.paperclip/instances/default/db");
expect(parsed.database.backup.dir).toBe("~/.paperclip/instances/default/data/backups");
expect(parsed.logging.logDir).toBe("~/.paperclip/instances/default/logs");
expect(parsed.storage.localDisk.baseDir).toBe("~/.paperclip/instances/default/data/storage");
expect(parsed.secrets.localEncrypted.keyFilePath).toBe("~/.paperclip/instances/default/secrets/master.key");
});
});

View file

@ -715,6 +715,7 @@ export const PLUGIN_CAPABILITIES = [
"issue.documents.write",
"projects.managed",
"routines.managed",
"skills.managed",
"agents.pause",
"agents.resume",
"agents.invoke",

View file

@ -0,0 +1,36 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
resolveDefaultBackupDir,
resolveDefaultEmbeddedPostgresDir,
resolveDefaultLogsDir,
resolveDefaultSecretsKeyFilePath,
resolveDefaultStorageDir,
resolvePaperclipConfigPathForInstance,
resolvePaperclipInstanceRoot,
} from "./home-paths.js";
const ORIGINAL_ENV = { ...process.env };
afterEach(() => {
process.env = { ...ORIGINAL_ENV };
});
describe("home path resolution", () => {
it("resolves config and runtime data directly under the instance root", () => {
const home = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-home-paths-"));
process.env.PAPERCLIP_HOME = home;
delete process.env.PAPERCLIP_INSTANCE_ID;
const instanceRoot = path.join(home, "instances", "default");
expect(resolvePaperclipInstanceRoot()).toBe(instanceRoot);
expect(resolvePaperclipConfigPathForInstance()).toBe(path.join(instanceRoot, "config.json"));
expect(resolveDefaultEmbeddedPostgresDir()).toBe(path.join(instanceRoot, "db"));
expect(resolveDefaultBackupDir()).toBe(path.join(instanceRoot, "data", "backups"));
expect(resolveDefaultLogsDir()).toBe(path.join(instanceRoot, "logs"));
expect(resolveDefaultStorageDir()).toBe(path.join(instanceRoot, "data", "storage"));
expect(resolveDefaultSecretsKeyFilePath()).toBe(path.join(instanceRoot, "secrets", "master.key"));
});
});

View file

@ -0,0 +1,92 @@
import os from "node:os";
import path from "node:path";
export const DEFAULT_PAPERCLIP_INSTANCE_ID = "default";
export const PAPERCLIP_CONFIG_BASENAME = "config.json";
export const PAPERCLIP_ENV_FILENAME = ".env";
const PATH_SEGMENT_RE = /^[a-zA-Z0-9_-]+$/;
export function expandHomePrefix(value: string): string {
if (value === "~") return os.homedir();
if (value.startsWith("~/")) return path.resolve(os.homedir(), value.slice(2));
return value;
}
export function resolvePaperclipHomeDir(homeOverride?: string): string {
const raw = homeOverride?.trim() || process.env.PAPERCLIP_HOME?.trim();
if (raw) return path.resolve(expandHomePrefix(raw));
return path.resolve(os.homedir(), ".paperclip");
}
export function resolvePaperclipInstanceId(instanceIdOverride?: string): string {
const raw = instanceIdOverride?.trim() || process.env.PAPERCLIP_INSTANCE_ID?.trim() || DEFAULT_PAPERCLIP_INSTANCE_ID;
if (!PATH_SEGMENT_RE.test(raw)) {
throw new Error(`Invalid PAPERCLIP_INSTANCE_ID '${raw}'.`);
}
return raw;
}
export function resolvePaperclipInstanceRoot(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipHomeDir(input.homeDir), "instances", resolvePaperclipInstanceId(input.instanceId));
}
export function resolvePaperclipInstanceConfigPath(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), PAPERCLIP_CONFIG_BASENAME);
}
export function resolvePaperclipConfigPathForInstance(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return resolvePaperclipInstanceConfigPath(input);
}
export function resolvePaperclipEnvPathForConfig(configPath: string): string {
return path.resolve(path.dirname(configPath), PAPERCLIP_ENV_FILENAME);
}
export function resolveDefaultEmbeddedPostgresDir(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), "db");
}
export function resolveDefaultLogsDir(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), "logs");
}
export function resolveDefaultSecretsKeyFilePath(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), "secrets", "master.key");
}
export function resolveDefaultStorageDir(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), "data", "storage");
}
export function resolveDefaultBackupDir(input: {
homeDir?: string;
instanceId?: string;
} = {}): string {
return path.resolve(resolvePaperclipInstanceRoot(input), "data", "backups");
}
export function resolveHomeAwarePath(value: string): string {
return path.resolve(expandHomePrefix(value));
}

View file

@ -581,10 +581,13 @@ export type {
PluginManagedAgentDeclaration,
PluginManagedProjectDeclaration,
PluginManagedRoutineDeclaration,
PluginManagedSkillDeclaration,
PluginManagedSkillFileDeclaration,
PluginLocalFolderDeclaration,
PluginManagedAgentResolution,
PluginManagedProjectResolution,
PluginManagedRoutineResolution,
PluginManagedSkillResolution,
PluginManagedResourceKind,
PluginManagedResourceRef,
PluginUiSlotDeclaration,

View file

@ -373,10 +373,13 @@ export type {
PluginManagedAgentDeclaration,
PluginManagedProjectDeclaration,
PluginManagedRoutineDeclaration,
PluginManagedSkillDeclaration,
PluginManagedSkillFileDeclaration,
PluginLocalFolderDeclaration,
PluginManagedAgentResolution,
PluginManagedProjectResolution,
PluginManagedRoutineResolution,
PluginManagedSkillResolution,
PluginManagedResourceKind,
PluginManagedResourceRef,
PluginUiSlotDeclaration,

View file

@ -27,6 +27,7 @@ import type {
IssueSurfaceVisibility,
} from "../constants.js";
import type { Agent } from "./agent.js";
import type { CompanySkill } from "./company-skill.js";
import type { Project } from "./project.js";
import type { Routine, RoutineTrigger, RoutineVariable } from "./routine.js";
@ -164,6 +165,7 @@ export interface PluginManagedAgentDeclaration {
instructions?: {
entryFile?: string;
content?: string;
files?: Record<string, string>;
assetPath?: string;
};
}
@ -208,7 +210,33 @@ export interface PluginManagedProjectDeclaration {
settings?: Record<string, unknown>;
}
export type PluginManagedResourceKind = "agent" | "project" | "routine";
export interface PluginManagedSkillFileDeclaration {
/** Relative path inside the skill folder, for example `references/guide.md`. */
path: string;
/** File contents written when the skill is installed or reset. */
content: string;
}
/**
* Declares a company skill that a plugin can install into each company's
* skills library and later resolve by stable key.
*/
export interface PluginManagedSkillDeclaration {
/** Stable identifier for this managed skill, unique within the plugin. */
skillKey: string;
/** Suggested visible skill name. */
displayName: string;
/** Suggested skill slug. Defaults to `skillKey`. */
slug?: string;
/** Suggested skill description. */
description?: string | null;
/** Full `SKILL.md` contents. Defaults to generated markdown from display metadata. */
markdown?: string;
/** Additional files installed with the skill. */
files?: PluginManagedSkillFileDeclaration[];
}
export type PluginManagedResourceKind = "agent" | "project" | "routine" | "skill";
export interface PluginManagedResourceRef {
pluginKey?: string;
@ -258,6 +286,10 @@ export interface PluginManagedAgentResolution {
agent: Agent | null;
status: "missing" | "resolved" | "created" | "relinked" | "reset";
approvalId?: string | null;
defaultDrift?: {
entryFile: string;
changedFiles: string[];
} | null;
}
export interface PluginManagedProjectResolution {
@ -281,6 +313,19 @@ export interface PluginManagedRoutineResolution {
missingRefs?: PluginManagedResourceRef[];
}
export interface PluginManagedSkillResolution {
pluginKey: string;
resourceKind: "skill";
resourceKey: string;
companyId: string;
skillId: string | null;
skill: CompanySkill | null;
status: "missing" | "resolved" | "created" | "relinked" | "reset";
defaultDrift?: {
changedFiles: string[];
} | null;
}
/**
* Declares a UI extension slot the plugin fills with a React component.
*
@ -496,6 +541,8 @@ export interface PaperclipPluginManifestV1 {
projects?: PluginManagedProjectDeclaration[];
/** Suggested company-scoped routines this plugin can provision and resolve by stable key. */
routines?: PluginManagedRoutineDeclaration[];
/** Suggested company skills this plugin can install and resolve by stable key. */
skills?: PluginManagedSkillDeclaration[];
/** Trusted local folders this plugin can configure and access by stable key. */
localFolders?: PluginLocalFolderDeclaration[];
/**

View file

@ -394,6 +394,8 @@ export {
pluginLauncherRenderDeclarationSchema,
pluginLauncherDeclarationSchema,
pluginDatabaseDeclarationSchema,
pluginManagedSkillFileDeclarationSchema,
pluginManagedSkillDeclarationSchema,
pluginApiRouteDeclarationSchema,
pluginManifestV1Schema,
installPluginSchema,
@ -413,6 +415,8 @@ export {
type PluginLauncherRenderDeclarationInput,
type PluginLauncherDeclarationInput,
type PluginDatabaseDeclarationInput,
type PluginManagedSkillFileDeclarationInput,
type PluginManagedSkillDeclarationInput,
type PluginApiRouteDeclarationInput,
type PluginManifestV1Input,
type InstallPlugin,

View file

@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { PLUGIN_CAPABILITIES } from "../constants.js";
import { pluginManagedRoutineDeclarationSchema, pluginUiSlotDeclarationSchema } from "./plugin.js";
import { pluginManagedRoutineDeclarationSchema, pluginManifestV1Schema, pluginUiSlotDeclarationSchema } from "./plugin.js";
describe("plugin capability constants", () => {
it("exposes each capability once", () => {
@ -30,6 +30,41 @@ describe("plugin managed routine validators", () => {
});
});
describe("plugin managed skill validators", () => {
const baseManifest = {
id: "paperclip.test-managed-skills",
apiVersion: 1,
version: "0.1.0",
displayName: "Managed Skills",
description: "Managed skills test plugin.",
author: "Paperclip",
categories: ["automation"],
entrypoints: { worker: "./dist/worker.js" },
} as const;
it("requires skills.managed when managed skills are declared", () => {
const parsed = pluginManifestV1Schema.safeParse({
...baseManifest,
capabilities: [],
skills: [{ skillKey: "wiki-maintainer", displayName: "Wiki Maintainer" }],
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues.some((issue) => issue.message.includes("skills.managed"))).toBe(true);
});
it("accepts managed skills with the skills.managed capability", () => {
const parsed = pluginManifestV1Schema.parse({
...baseManifest,
capabilities: ["skills.managed"],
skills: [{ skillKey: "wiki-maintainer", displayName: "Wiki Maintainer" }],
});
expect(parsed.skills?.[0]?.skillKey).toBe("wiki-maintainer");
});
});
describe("plugin UI slot validators", () => {
it("accepts route-scoped sidebar slots with a routePath", () => {
const parsed = pluginUiSlotDeclarationSchema.parse({

View file

@ -151,6 +151,7 @@ export const pluginManagedAgentDeclarationSchema = z.object({
instructions: z.object({
entryFile: z.string().min(1).max(200).optional(),
content: z.string().max(200_000).optional(),
files: z.record(z.string().max(200_000)).optional(),
assetPath: z.string().min(1).max(500).optional(),
}).optional(),
});
@ -172,7 +173,7 @@ export type PluginManagedProjectDeclarationInput = z.infer<typeof pluginManagedP
const pluginManagedResourceRefSchema = z.object({
pluginKey: z.string().min(1).max(100).optional(),
resourceKind: z.enum(["agent", "project", "routine"]),
resourceKind: z.enum(["agent", "project", "routine", "skill"]),
resourceKey: z.string().min(1).max(100).regex(/^[a-z0-9][a-z0-9._:-]*$/, {
message: "resourceKey must start with a lowercase alphanumeric and contain only lowercase letters, digits, dots, colons, underscores, or hyphens",
}),
@ -232,6 +233,41 @@ export const pluginLocalFolderDeclarationSchema = z.object({
export type PluginLocalFolderDeclarationInput = z.infer<typeof pluginLocalFolderDeclarationSchema>;
export const pluginManagedSkillFileDeclarationSchema = z.object({
path: pluginLocalFolderRelativePathSchema.refine(
(value) => value.toLowerCase() !== "skill.md",
{ message: "managed skill files cannot replace SKILL.md; use markdown for the main skill file" },
),
content: z.string().max(200_000),
});
export type PluginManagedSkillFileDeclarationInput = z.infer<typeof pluginManagedSkillFileDeclarationSchema>;
export const pluginManagedSkillDeclarationSchema = z.object({
skillKey: z.string().min(1).max(100).regex(/^[a-z0-9][a-z0-9._:-]*$/, {
message: "skillKey must start with a lowercase alphanumeric and contain only lowercase letters, digits, dots, colons, underscores, or hyphens",
}),
displayName: z.string().min(1).max(100),
slug: z.string().min(1).max(100).regex(/^[a-z0-9][a-z0-9._:-]*$/, {
message: "slug must start with a lowercase alphanumeric and contain only lowercase letters, digits, dots, colons, underscores, or hyphens",
}).optional(),
description: z.string().max(2000).nullable().optional(),
markdown: z.string().max(200_000).optional(),
files: z.array(pluginManagedSkillFileDeclarationSchema).max(50).optional(),
}).superRefine((value, ctx) => {
const paths = (value.files ?? []).map((file) => file.path);
const duplicates = paths.filter((path, index) => paths.indexOf(path) !== index);
if (duplicates.length > 0) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: `Duplicate managed skill file paths: ${[...new Set(duplicates)].join(", ")}`,
path: ["files"],
});
}
});
export type PluginManagedSkillDeclarationInput = z.infer<typeof pluginManagedSkillDeclarationSchema>;
/**
* Validates a {@link PluginUiSlotDeclaration} a UI extension slot the plugin
* fills with a React component. Includes `superRefine` checks for slot-specific
@ -589,6 +625,7 @@ export const pluginManifestV1Schema = z.object({
agents: z.array(pluginManagedAgentDeclarationSchema).optional(),
projects: z.array(pluginManagedProjectDeclarationSchema).optional(),
routines: z.array(pluginManagedRoutineDeclarationSchema).optional(),
skills: z.array(pluginManagedSkillDeclarationSchema).optional(),
localFolders: z.array(pluginLocalFolderDeclarationSchema).optional(),
launchers: z.array(pluginLauncherDeclarationSchema).optional(),
ui: z.object({
@ -678,6 +715,16 @@ export const pluginManifestV1Schema = z.object({
}
}
if (manifest.skills && manifest.skills.length > 0) {
if (!manifest.capabilities.includes("skills.managed")) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "Capability 'skills.managed' is required when managed skills are declared",
path: ["capabilities"],
});
}
}
if (manifest.localFolders && manifest.localFolders.length > 0) {
if (!manifest.capabilities.includes("local.folders")) {
ctx.addIssue({
@ -871,6 +918,18 @@ export const pluginManifestV1Schema = z.object({
}
}
if (manifest.skills) {
const skillKeys = manifest.skills.map((skill) => skill.skillKey);
const duplicates = skillKeys.filter((key, i) => skillKeys.indexOf(key) !== i);
if (duplicates.length > 0) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: `Duplicate managed skill keys: ${[...new Set(duplicates)].join(", ")}`,
path: ["skills"],
});
}
}
// UI slot ids must be unique within the plugin (namespaced at runtime)
if (manifest.ui) {
if (manifest.ui.slots) {