[codex] Add LLM Wiki plugin package to master (#5716)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system is the extension surface for optional product
capabilities without baking every workflow into core.
> - The LLM Wiki plugin package was reviewed in stacked PR #5592, which
targeted `pap-9173-llm-wiki-rest`.
> - The stack base PR #5597 merged to `master` before #5592 was merged
into that branch, so the plugin package never reached `master`.
> - A direct PR from `pap-9173-llm-wiki-rest` back to `master` would be
noisy because that branch has diverged from current `master`.
> - This pull request reapplies the reviewed
`packages/plugins/plugin-llm-wiki/` package onto current `master` and
updates Docker deps-stage manifest coverage.
> - The branch intentionally no longer changes `pnpm-workspace.yaml`
after maintainer feedback; because the new package is now a root
workspace importer, the remaining integration question is how
maintainers want the root lockfile handled under the current PR policy.
## What Changed
- Added the LLM Wiki plugin package under
`packages/plugins/plugin-llm-wiki/` from the merged PR #5592 head.
- Preserved the post-review cleanup from #5592: generated
design/screenshot artifacts are not committed, and `src/ui/index.tsx` /
`src/wiki.ts` are small public entrypoints.
- Added the new plugin package manifest to the Docker deps stage so
policy can validate package manifest coverage.
- Removed the earlier `pnpm-workspace.yaml` exclusion per maintainer
request, so the plugin is included by the existing `packages/plugins/*`
workspace glob.
## Verification
Current head:
- PGlite migration harness: ran migrations 001-003, verified old
non-space distillation unique constraints were removed, inserted
duplicate cursor and work-item keys in a second space, then reran
migration 003 successfully
- `node ./scripts/check-docker-deps-stage.mjs`
- `git diff --check`
Known current-head install result after removing the workspace
exclusion:
- `pnpm install --frozen-lockfile` fails because `pnpm-lock.yaml` has no
importer for `packages/plugins/plugin-llm-wiki/package.json`.
Previously verified on the same plugin source before the
workspace-exclusion removal:
- `pnpm --filter @paperclipai/plugin-sdk build`
- `cd packages/plugins/plugin-llm-wiki && pnpm install --lockfile=false
&& pnpm test`
## Risks
- The branch now includes `packages/plugins/plugin-llm-wiki` in the root
workspace but does not update `pnpm-lock.yaml`. Root frozen install will
fail until maintainers choose a lockfile path that fits repo policy.
- Committing `pnpm-lock.yaml` directly on this PR conflicts with the
current PR policy check, while excluding the package from
`pnpm-workspace.yaml` was rejected in maintainer feedback.
- The package includes UI code already reviewed in #5592; generated
screenshot/design artifacts were intentionally removed per maintainer
request, so visual review should regenerate screenshots locally if
needed.
- The package depends on plugin host support from #5597, which is
already merged to `master`.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution
enabled; context window not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run the targeted checks listed above
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
Stack context: #5592 was merged into `pap-9173-llm-wiki-rest` after
#5597 had already merged that branch to `master`, so this follow-up PR
is needed to carry the plugin package itself into `master`.
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-11 20:45:41 -05:00
import { readdirSync , readFileSync , statSync } from "node:fs" ;
export const REQUIRED_WIKI_DIRECTORIES = [
"raw" ,
"wiki" ,
"wiki/sources" ,
"wiki/projects" ,
"wiki/entities" ,
"wiki/concepts" ,
"wiki/synthesis" ,
] as const ;
export const REQUIRED_WIKI_FILES = [ "AGENTS.md" , "IDEA.md" , "wiki/index.md" , "wiki/log.md" ] as const ;
export const KARPATHY_LLM_WIKI_GIST_URL = "https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f" ;
function templateFile ( path : string ) : string {
return readFileSync ( new URL ( ` ../templates/ ${ path } ` , import . meta . url ) , "utf8" ) ;
}
function agentInstructionFiles ( agentKey : string ) : Record < string , string > {
const root = new URL ( ` ../agents/ ${ agentKey } / ` , import . meta . url ) ;
const files : Record < string , string > = { } ;
function walk ( relativeDir : string ) {
const dirUrl = new URL ( relativeDir ? ` ${ relativeDir } / ` : "./" , root ) ;
for ( const entry of readdirSync ( dirUrl ) ) {
if ( entry === ".DS_Store" ) continue ;
const relativePath = relativeDir ? ` ${ relativeDir } / ${ entry } ` : entry ;
const entryUrl = new URL ( relativePath , root ) ;
const stat = statSync ( entryUrl ) ;
if ( stat . isDirectory ( ) ) {
walk ( relativePath ) ;
} else if ( stat . isFile ( ) ) {
files [ relativePath ] = readFileSync ( entryUrl , "utf8" ) ;
}
}
}
walk ( "" ) ;
return Object . fromEntries ( Object . entries ( files ) . sort ( ( [ left ] , [ right ] ) = > left . localeCompare ( right ) ) ) ;
}
export const DEFAULT_WIKI_SCHEMA = templateFile ( "AGENTS.md" ) ;
export const DEFAULT_AGENT_INSTRUCTION_FILES = agentInstructionFiles ( "wiki-maintainer" ) ;
export const DEFAULT_AGENT_INSTRUCTIONS = DEFAULT_AGENT_INSTRUCTION_FILES [ "AGENTS.md" ] ? ? "" ;
export const DEFAULT_IDEA = templateFile ( "IDEA.md" ) ;
export const DEFAULT_INDEX = templateFile ( "wiki/index.md" ) ;
export const DEFAULT_LOG = templateFile ( "wiki/log.md" ) ;
Fix LLM Wiki package and migration validation (#6010)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies.
> - Plugins extend the control plane with optional capabilities such as
LLM Wiki.
> - LLM Wiki needs its package assets and plugin-owned database
migrations to work when installed from the packaged plugin.
> - The bundled spaces migration used validation-hostile dynamic SQL,
and the packaged plugin could omit non-dist runtime assets.
> - This pull request makes the LLM Wiki package include its required
assets and cuts the spaces migration over to explicit, idempotent SQL
that passes the production plugin database validator.
> - The benefit is a simpler plugin install path that validates and
applies the bundled LLM Wiki migrations without adding plugin-specific
legacy handling to Paperclip core.
## What Changed
- Added the LLM Wiki package asset allowlist so agents, migrations,
skills, templates, dist output, and README are included when packaged.
- Renamed the bootstrap `.gitignore` template to `gitignore.template`
and updated the runtime lookup so package tooling does not drop the
hidden template file.
- Relaxed plugin migration validation to allow namespace-scoped
`INSERT`/`UPDATE` backfills and `CREATE INDEX` statements while
continuing to reject destructive or cross-namespace SQL.
- Replaced the LLM Wiki spaces migration's dynamic constraint-drop DO
block with explicit `DROP CONSTRAINT IF EXISTS` statements.
- Replaced fragile regex-source dispatch in SQL reference extraction
with explicit capture-group descriptors.
- Added regression coverage that applies the bundled LLM Wiki migrations
through the production validator and checks the expected constraints.
## Verification
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/plugin-database.test.ts --pool=forks
--poolOptions.forks.isolate=true`
- `pnpm --filter @paperclipai/plugin-llm-wiki build`
- `git diff --check`
- Confirmed `pnpm-lock.yaml` is not included in the branch diff.
## Risks
- Low migration risk for current users: LLM Wiki spaces are new, so this
intentionally cuts over the plugin migration instead of adding legacy
handling in core.
- Validator behavior is broader than before, but still requires fully
qualified plugin namespace targets, blocks deletes/destructive DDL, and
keeps public table access read-only and allowlisted.
> Checked [`ROADMAP.md`](ROADMAP.md); this is a targeted plugin
packaging/migration fix and does not duplicate planned core feature
work. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, GPT-5 based coding agent, tool-enabled local repo
access, reasoning mode managed by the Paperclip/Codex runtime. Exact
context window was not surfaced in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-15 10:20:02 -05:00
export const DEFAULT_GITIGNORE = templateFile ( "gitignore.template" ) ;
[codex] Add LLM Wiki plugin package to master (#5716)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system is the extension surface for optional product
capabilities without baking every workflow into core.
> - The LLM Wiki plugin package was reviewed in stacked PR #5592, which
targeted `pap-9173-llm-wiki-rest`.
> - The stack base PR #5597 merged to `master` before #5592 was merged
into that branch, so the plugin package never reached `master`.
> - A direct PR from `pap-9173-llm-wiki-rest` back to `master` would be
noisy because that branch has diverged from current `master`.
> - This pull request reapplies the reviewed
`packages/plugins/plugin-llm-wiki/` package onto current `master` and
updates Docker deps-stage manifest coverage.
> - The branch intentionally no longer changes `pnpm-workspace.yaml`
after maintainer feedback; because the new package is now a root
workspace importer, the remaining integration question is how
maintainers want the root lockfile handled under the current PR policy.
## What Changed
- Added the LLM Wiki plugin package under
`packages/plugins/plugin-llm-wiki/` from the merged PR #5592 head.
- Preserved the post-review cleanup from #5592: generated
design/screenshot artifacts are not committed, and `src/ui/index.tsx` /
`src/wiki.ts` are small public entrypoints.
- Added the new plugin package manifest to the Docker deps stage so
policy can validate package manifest coverage.
- Removed the earlier `pnpm-workspace.yaml` exclusion per maintainer
request, so the plugin is included by the existing `packages/plugins/*`
workspace glob.
## Verification
Current head:
- PGlite migration harness: ran migrations 001-003, verified old
non-space distillation unique constraints were removed, inserted
duplicate cursor and work-item keys in a second space, then reran
migration 003 successfully
- `node ./scripts/check-docker-deps-stage.mjs`
- `git diff --check`
Known current-head install result after removing the workspace
exclusion:
- `pnpm install --frozen-lockfile` fails because `pnpm-lock.yaml` has no
importer for `packages/plugins/plugin-llm-wiki/package.json`.
Previously verified on the same plugin source before the
workspace-exclusion removal:
- `pnpm --filter @paperclipai/plugin-sdk build`
- `cd packages/plugins/plugin-llm-wiki && pnpm install --lockfile=false
&& pnpm test`
## Risks
- The branch now includes `packages/plugins/plugin-llm-wiki` in the root
workspace but does not update `pnpm-lock.yaml`. Root frozen install will
fail until maintainers choose a lockfile path that fits repo policy.
- Committing `pnpm-lock.yaml` directly on this PR conflicts with the
current PR policy check, while excluding the package from
`pnpm-workspace.yaml` was rejected in maintainer feedback.
- The package includes UI code already reviewed in #5592; generated
screenshot/design artifacts were intentionally removed per maintainer
request, so visual review should regenerate screenshots locally if
needed.
- The package depends on plugin host support from #5597, which is
already merged to `master`.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution
enabled; context window not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run the targeted checks listed above
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
Stack context: #5592 was merged into `pap-9173-llm-wiki-rest` after
#5597 had already merged that branch to `master`, so this follow-up PR
is needed to carry the plugin package itself into `master`.
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-11 20:45:41 -05:00
export const QUERY_PROMPT = ` Answer from the LLM Wiki using the installed wiki-query skill.
Read the target space 's wiki/index.md first, inspect relevant pages and raw/source references in that same space, cite the wiki page paths and raw source paths used, and say when the wiki does not contain enough evidence. Useful durable synthesis should be filed back into wiki/synthesis/ inside that same space. Always pass the operation issue' s wikiId and spaceSlug to LLM Wiki tools .
` ;
export const LINT_PROMPT = ` Lint the LLM Wiki using the installed wiki-lint skill.
Audit the target space only for contradictions , stale claims , orphan pages , missing backlinks , weak provenance , and wiki / index . md / wiki / log . md drift . Also look for important concepts mentioned without pages and answers that should have been filed back into wiki / . Return findings grouped by severity with concrete file paths , evidence , and suggested fixes — do not auto - apply edits . Always pass the operation issue ' s wikiId and spaceSlug to LLM Wiki tools .
` ;
export const BOOTSTRAP_FILES : ReadonlyArray < { path : string ; contents : string } > = [
{ path : ".gitignore" , contents : DEFAULT_GITIGNORE } ,
{ path : "AGENTS.md" , contents : DEFAULT_WIKI_SCHEMA } ,
{ path : "IDEA.md" , contents : DEFAULT_IDEA } ,
{ path : "wiki/index.md" , contents : DEFAULT_INDEX } ,
{ path : "wiki/log.md" , contents : DEFAULT_LOG } ,
{ path : "raw/.gitkeep" , contents : "" } ,
{ path : "wiki/sources/.gitkeep" , contents : "" } ,
{ path : "wiki/projects/.gitkeep" , contents : "" } ,
{ path : "wiki/entities/.gitkeep" , contents : "" } ,
{ path : "wiki/concepts/.gitkeep" , contents : "" } ,
{ path : "wiki/synthesis/.gitkeep" , contents : "" } ,
] ;