mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-14 01:50:39 +09:00
feat(adapters): external adapter plugin system with dynamic UI parser
- Plugin loader: install/reload/remove/reinstall external adapters from npm packages or local directories - Plugin store persisted at ~/.paperclip/adapter-plugins.json - Self-healing UI parser resolution with version caching - UI: Adapter Manager page, dynamic loader, display registry with humanized names for unknown adapter types - Dev watch: exclude adapter-plugins dir from tsx watcher to prevent mid-request server restarts during reinstall - All consumer fallbacks use getAdapterLabel() for consistent display - AdapterTypeDropdown uses controlled open state for proper close behavior - Remove hermes-local from built-in UI (externalized to plugin) - Add docs for external adapters and UI parser contract
This commit is contained in:
HenkDz
parent
f8452a4520
commit
14d59da316
72 changed files with 4102 additions and 585 deletions
287
docs/adapters/adapter-ui-parser.md
Normal file
287
docs/adapters/adapter-ui-parser.md
Normal file
|
|
@ -0,0 +1,287 @@
|
|||
---
|
||||
title: Adapter UI Parser Contract
|
||||
summary: Ship a custom run-log parser so the Paperclip UI renders your adapter's output correctly
|
||||
---
|
||||
|
||||
When Paperclip runs an agent, stdout is streamed to the UI in real time. The UI needs a **parser** to convert raw stdout lines into structured transcript entries (tool calls, tool results, assistant messages, system events). Without a custom parser, the UI falls back to a generic shell parser that treats every non-system line as `assistant` output — tool commands leak as plain text, durations are lost, and errors are invisible.
|
||||
|
||||
## The Problem
|
||||
|
||||
Most agent CLIs emit structured stdout with tool calls, progress indicators, and multi-line output. For example:
|
||||
|
||||
```
|
||||
[hermes] Session resumed: abc123
|
||||
┊ 💬 Thinking about how to approach this...
|
||||
┊ $ ls /home/user/project
|
||||
┊ [done] $ ls /home/user/project — /src /README.md 0.3s
|
||||
┊ 💬 I see the project structure. Let me read the README.
|
||||
┊ read /home/user/project/README.md
|
||||
┊ [done] read — Project Overview: A CLI tool for... 1.2s
|
||||
The project is a CLI tool. Here's what I found:
|
||||
- It uses TypeScript
|
||||
- Tests are in /tests
|
||||
```
|
||||
|
||||
Without a parser, the UI shows all of this as raw `assistant` text — the tool calls and results are indistinguishable from the agent's actual response.
|
||||
|
||||
With a parser, the UI renders:
|
||||
|
||||
- `Thinking about how to approach this...` as a collapsible thinking block
|
||||
- `$ ls /home/user/project` as a tool call card (collapsed)
|
||||
- `0.3s` duration as a tool result card
|
||||
- `The project is a CLI tool...` as the assistant's response
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
┌──────────────────┐ package.json ┌──────────────────┐
|
||||
│ Adapter Package │─── exports["./ui-parser"] ──→│ dist/ui-parser.js │
|
||||
│ (npm / local) │ │ (zero imports) │
|
||||
└──────────────────┘ └────────┬─────────┘
|
||||
│ plugin-loader reads at startup
|
||||
▼
|
||||
┌──────────────────┐ GET /api/:type/ui-parser.js ┌──────────────────┐
|
||||
│ Paperclip Server │◄────────────────────────────────│ uiParserCache │
|
||||
│ (in-memory) │ └──────────────────┘
|
||||
└────────┬─────────┘
|
||||
│ serves JS to browser
|
||||
▼
|
||||
┌──────────────────┐ fetch() + eval ┌──────────────────┐
|
||||
│ Paperclip UI │─────────────────────→│ parseStdoutLine │
|
||||
│ (dynamic loader) │ registers parser │ (per-adapter) │
|
||||
└──────────────────┘ └──────────────────┘
|
||||
```
|
||||
|
||||
1. **Build time** — You compile `src/ui-parser.ts` to `dist/ui-parser.js` (zero runtime imports)
|
||||
2. **Server startup** — Plugin loader reads the file and caches it in memory
|
||||
3. **UI load** — When the user opens a run, the UI fetches the parser from `GET /api/:type/ui-parser.js`
|
||||
4. **Runtime** — The fetched module is eval'd and registered. All subsequent lines use the real parser
|
||||
|
||||
## Contract: package.json
|
||||
|
||||
### 1. `paperclip.adapterUiParser` — contract version
|
||||
|
||||
```json
|
||||
{
|
||||
"paperclip": {
|
||||
"adapterUiParser": "1.0.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The Paperclip host checks this field. If the major version is unsupported, the host logs a warning and falls back to the generic parser instead of executing potentially incompatible code.
|
||||
|
||||
| Host expects | Adapter declares | Result |
|
||||
|---|---|---|
|
||||
| `1.x` | `1.0.0` | Parser loaded |
|
||||
| `1.x` | `2.0.0` | Warning logged, generic parser used |
|
||||
| `1.x` | (missing) | Parser loaded (grace period — future versions may require it) |
|
||||
|
||||
### 2. `exports["./ui-parser"]` — file path
|
||||
|
||||
```json
|
||||
{
|
||||
"exports": {
|
||||
".": "./dist/server/index.js",
|
||||
"./ui-parser": "./dist/ui-parser.js"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Contract: Module Exports
|
||||
|
||||
Your `dist/ui-parser.js` must export **at least one** of:
|
||||
|
||||
### `parseStdoutLine(line: string, ts: string): TranscriptEntry[]`
|
||||
|
||||
Static parser. Called for each line of adapter stdout.
|
||||
|
||||
```ts
|
||||
export function parseStdoutLine(line: string, ts: string): TranscriptEntry[] {
|
||||
if (line.startsWith("[my-agent]")) {
|
||||
return [{ kind: "system", ts, text: line }];
|
||||
}
|
||||
return [{ kind: "assistant", ts, text: line }];
|
||||
}
|
||||
```
|
||||
|
||||
### `createStdoutParser(): { parseLine(line, ts): TranscriptEntry[]; reset(): void }`
|
||||
|
||||
Stateful parser factory. Preferred if your parser needs to track multi-line continuation, command nesting, or other cross-call state.
|
||||
|
||||
```ts
|
||||
let counter = 0;
|
||||
|
||||
export function createStdoutParser() {
|
||||
let suppressContinuation = false;
|
||||
|
||||
function parseLine(line: string, ts: string): TranscriptEntry[] {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) return [];
|
||||
|
||||
if (suppressContinuation) {
|
||||
if (/^[\d.]+s$/.test(trimmed)) {
|
||||
suppressContinuation = false;
|
||||
return [];
|
||||
}
|
||||
return []; // swallow continuation lines
|
||||
}
|
||||
|
||||
if (trimmed.startsWith("[tool-done]")) {
|
||||
const id = `tool-${++counter}`;
|
||||
suppressContinuation = true;
|
||||
return [
|
||||
{ kind: "tool_call", ts, name: "shell", input: {}, toolUseId: id },
|
||||
{ kind: "tool_result", ts, toolUseId: id, content: trimmed, isError: false },
|
||||
];
|
||||
}
|
||||
|
||||
return [{ kind: "assistant", ts, text: trimmed }];
|
||||
}
|
||||
|
||||
function reset() {
|
||||
suppressContinuation = false;
|
||||
}
|
||||
|
||||
return { parseLine, reset };
|
||||
}
|
||||
```
|
||||
|
||||
If both are exported, `createStdoutParser` takes priority.
|
||||
|
||||
## Contract: TranscriptEntry
|
||||
|
||||
Each entry must match one of these discriminated union shapes:
|
||||
|
||||
```ts
|
||||
// Assistant message
|
||||
{ kind: "assistant"; ts: string; text: string; delta?: boolean }
|
||||
|
||||
// Thinking / reasoning
|
||||
{ kind: "thinking"; ts: string; text: string; delta?: boolean }
|
||||
|
||||
// User message (rare — usually from agent-initiated prompts)
|
||||
{ kind: "user"; ts: string; text: string }
|
||||
|
||||
// Tool invocation
|
||||
{ kind: "tool_call"; ts: string; name: string; input: unknown; toolUseId?: string }
|
||||
|
||||
// Tool result
|
||||
{ kind: "tool_result"; ts: string; toolUseId: string; content: string; isError: boolean }
|
||||
|
||||
// System / adapter messages
|
||||
{ kind: "system"; ts: string; text: string }
|
||||
|
||||
// Stderr / errors
|
||||
{ kind: "stderr"; ts: string; text: string }
|
||||
|
||||
// Raw stdout (fallback)
|
||||
{ kind: "stdout"; ts: string; text: string }
|
||||
```
|
||||
|
||||
### Linking tool calls to results
|
||||
|
||||
Use `toolUseId` to pair `tool_call` and `tool_result` entries. The UI renders them as collapsible cards.
|
||||
|
||||
```ts
|
||||
const id = `my-tool-${++counter}`;
|
||||
return [
|
||||
{ kind: "tool_call", ts, name: "read", input: { path: "/src/main.ts" }, toolUseId: id },
|
||||
{ kind: "tool_result", ts, toolUseId: id, content: "const main = () => {...}", isError: false },
|
||||
];
|
||||
```
|
||||
|
||||
### Error handling
|
||||
|
||||
Set `isError: true` on tool results to show a red indicator:
|
||||
|
||||
```ts
|
||||
{ kind: "tool_result", ts, toolUseId: id, content: "ENOENT: no such file", isError: true }
|
||||
```
|
||||
|
||||
## Constraints
|
||||
|
||||
1. **Zero runtime imports.** Your file is loaded via `URL.createObjectURL` + dynamic `import()` in the browser. No `import`, no `require`, no top-level `await`.
|
||||
|
||||
2. **No DOM / Node.js APIs.** Runs in a browser sandbox. Use only vanilla JS (ES2020+).
|
||||
|
||||
3. **No side effects.** Module-level code must not modify globals, access `window`, or perform I/O. Only declare and export functions.
|
||||
|
||||
4. **Deterministic.** Given the same `(line, ts)` input, the same output must be produced. This matters for log replay.
|
||||
|
||||
5. **Error-tolerant.** Never throw. Return `[{ kind: "stdout", ts, text: line }]` for any line you can't parse, rather than crashing the transcript.
|
||||
|
||||
6. **File size.** Keep under 50 KB. This is served per-request and eval'd in the browser.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
| Event | What happens |
|
||||
|---|---|
|
||||
| Server starts | Plugin loader reads `exports["./ui-parser"]`, reads the file, caches in memory |
|
||||
| UI opens run | `getUIAdapter(type)` called. If no built-in parser, kicks off async `fetch(/api/:type/ui-parser.js)` |
|
||||
| First lines arrive | Generic process parser handles them immediately (no blocking). Dynamic parser loads in background |
|
||||
| Parser loads | `registerUIAdapter()` called. All subsequent line parsing uses the real parser |
|
||||
| Parser fails (404, eval error) | Warning logged to console. Generic parser continues. Failed type is cached — no retries |
|
||||
| Server restart | In-memory cache is repopulated from adapter packages |
|
||||
|
||||
## Error Behavior
|
||||
|
||||
| Failure | What happens |
|
||||
|---|---|
|
||||
| Module syntax error (import fails) | Caught, logged, falls back to generic parser. No retries. |
|
||||
| Returns wrong shape | Individual entries with missing fields are silently ignored by the transcript builder. |
|
||||
| Throws at runtime | Caught per-line. That line falls back to generic. Parser stays registered for future lines. |
|
||||
| 404 (no ui-parser export) | Type added to failed-loads set. Generic parser from first call onward. |
|
||||
| Contract version mismatch | Server logs warning, skips loading. Generic parser used. |
|
||||
|
||||
## Building
|
||||
|
||||
```sh
|
||||
# Compile TypeScript to JavaScript
|
||||
tsc src/ui-parser.ts --outDir dist --target ES2020 --module ES2020 --declaration false
|
||||
```
|
||||
|
||||
Your `tsconfig.json` can handle this automatically — just make sure `ui-parser.ts` is included in the build and outputs to `dist/ui-parser.js`.
|
||||
|
||||
## Testing
|
||||
|
||||
Test your parser locally by running it against sample stdout:
|
||||
|
||||
```ts
|
||||
// test-parser.ts
|
||||
import { createStdoutParser } from "./dist/ui-parser.js";
|
||||
|
||||
const parser = createStdoutParser();
|
||||
const sampleLines = [
|
||||
"[my-agent] Starting session abc123",
|
||||
"Thinking about the task...",
|
||||
"$ ls /home/user/project",
|
||||
"[done] $ ls — /src /README.md 0.3s",
|
||||
"I'll read the README now.",
|
||||
"Error: file not found",
|
||||
];
|
||||
|
||||
for (const line of sampleLines) {
|
||||
const entries = parser.parseLine(line, new Date().toISOString());
|
||||
for (const entry of entries) {
|
||||
console.log(` ${entry.kind}:`, entry.text ?? entry.name ?? entry.content);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run with: `npx tsx test-parser.ts`
|
||||
|
||||
## Skipping the UI Parser
|
||||
|
||||
If your adapter's stdout is simple (no tool markers, no special formatting), you can skip the UI parser entirely. The generic `process` parser will handle it — every non-system line becomes `assistant` output. This is fine for:
|
||||
|
||||
- Agents that output plain text responses
|
||||
- Custom scripts that just print results
|
||||
- Simple CLIs without structured output
|
||||
|
||||
To skip it, simply don't include `exports["./ui-parser"]` in your `package.json`.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [External Adapters](/adapters/external-adapters) — full guide to building adapter packages
|
||||
- [Creating an Adapter](/adapters/creating-an-adapter) — adapter internals and built-in integration
|
||||
|
|
@ -20,8 +20,8 @@ The `claude_local` adapter runs Anthropic's Claude Code CLI locally. It supports
|
|||
| `env` | object | No | Environment variables (supports secret refs) |
|
||||
| `timeoutSec` | number | No | Process timeout (0 = no timeout) |
|
||||
| `graceSec` | number | No | Grace period before force-kill |
|
||||
| `maxTurnsPerRun` | number | No | Max agentic turns per heartbeat (defaults to `1000`) |
|
||||
| `dangerouslySkipPermissions` | boolean | No | Skip permission prompts (dev only) |
|
||||
| `maxTurnsPerRun` | number | No | Max agentic turns per heartbeat (defaults to `300`) |
|
||||
| `dangerouslySkipPermissions` | boolean | No | Skip permission prompts (default: `true`); required for headless runs where interactive approval is impossible |
|
||||
|
||||
## Prompt Templates
|
||||
|
||||
|
|
|
|||
|
|
@ -9,23 +9,40 @@ Build a custom adapter to connect Paperclip to any agent runtime.
|
|||
If you're using Claude Code, the `.agents/skills/create-agent-adapter` skill can guide you through the full adapter creation process interactively. Just ask Claude to create a new adapter and it will walk you through each step.
|
||||
</Tip>
|
||||
|
||||
## Two Paths
|
||||
|
||||
| | Built-in | External Plugin |
|
||||
|---|---|---|
|
||||
| Source | Inside `paperclip-fork` | Separate npm package |
|
||||
| Distribution | Ships with Paperclip | Independent npm publish |
|
||||
| UI parser | Static import | Dynamic load from API |
|
||||
| Registration | Edit 3 registries | Auto-loaded at startup |
|
||||
| Best for | Core adapters, contributors | Third-party adapters, internal tools |
|
||||
|
||||
For most cases, **build an external adapter plugin**. It's cleaner, independently versioned, and doesn't require modifying Paperclip's source. See [External Adapters](/adapters/external-adapters) for the full guide.
|
||||
|
||||
The rest of this page covers the shared internals that both paths use.
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
packages/adapters/<name>/
|
||||
packages/adapters/<name>/ # built-in
|
||||
── or ──
|
||||
my-adapter/ # external plugin
|
||||
package.json
|
||||
tsconfig.json
|
||||
src/
|
||||
index.ts # Shared metadata
|
||||
server/
|
||||
index.ts # Server exports
|
||||
index.ts # Server exports (createServerAdapter)
|
||||
execute.ts # Core execution logic
|
||||
parse.ts # Output parsing
|
||||
test.ts # Environment diagnostics
|
||||
ui/
|
||||
index.ts # UI exports
|
||||
parse-stdout.ts # Transcript parser
|
||||
index.ts # UI exports (built-in only)
|
||||
parse-stdout.ts # Transcript parser (built-in only)
|
||||
build-config.ts # Config builder
|
||||
ui-parser.ts # Self-contained UI parser (external — see [UI Parser Contract](/adapters/adapter-ui-parser))
|
||||
cli/
|
||||
index.ts # CLI exports
|
||||
format-event.ts # Terminal formatter
|
||||
|
|
@ -46,6 +63,9 @@ Use when: ...
|
|||
Don't use when: ...
|
||||
Core fields: ...
|
||||
`;
|
||||
|
||||
// Required for external adapters (plugin-loader convention)
|
||||
export { createServerAdapter } from "./server/index.js";
|
||||
```
|
||||
|
||||
## Step 2: Server Execute
|
||||
|
|
@ -54,7 +74,7 @@ Core fields: ...
|
|||
|
||||
Key responsibilities:
|
||||
|
||||
1. Read config using safe helpers (`asString`, `asNumber`, etc.)
|
||||
1. Read config using safe helpers (`asString`, `asNumber`, etc.) from `@paperclipai/adapter-utils/server-utils`
|
||||
2. Build environment with `buildPaperclipEnv(agent)` plus context vars
|
||||
3. Resolve session state from `runtime.sessionParams`
|
||||
4. Render prompt with `renderTemplate(template, data)`
|
||||
|
|
@ -62,27 +82,102 @@ Key responsibilities:
|
|||
6. Parse output for usage, costs, session state, errors
|
||||
7. Handle unknown session errors (retry fresh, set `clearSession: true`)
|
||||
|
||||
### Available Helpers
|
||||
|
||||
| Helper | Source | Purpose |
|
||||
|--------|--------|---------|
|
||||
| `runChildProcess(cmd, opts)` | `@paperclipai/adapter-utils/server-utils` | Spawn with timeout, grace, streaming |
|
||||
| `buildPaperclipEnv(agent)` | `@paperclipai/adapter-utils/server-utils` | Inject `PAPERCLIP_*` env vars |
|
||||
| `renderTemplate(tpl, data)` | `@paperclipai/adapter-utils/server-utils` | `{{variable}}` substitution |
|
||||
| `asString(v)` | `@paperclipai/adapter-utils` | Safe config value extraction |
|
||||
| `asNumber(v)` | `@paperclipai/adapter-utils` | Safe number extraction |
|
||||
|
||||
### AdapterExecutionContext
|
||||
|
||||
```ts
|
||||
interface AdapterExecutionContext {
|
||||
runId: string;
|
||||
agent: { id: string; companyId: string; name: string; adapterConfig: unknown };
|
||||
runtime: { sessionId: string | null; sessionParams: Record<string, unknown> | null };
|
||||
config: Record<string, unknown>; // agent's adapterConfig
|
||||
context: Record<string, unknown>; // task, wake reason, etc.
|
||||
onLog: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
|
||||
onMeta?: (meta: AdapterInvocationMeta) => Promise<void>;
|
||||
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
### AdapterExecutionResult
|
||||
|
||||
```ts
|
||||
interface AdapterExecutionResult {
|
||||
exitCode: number | null;
|
||||
signal: string | null;
|
||||
timedOut: boolean;
|
||||
errorMessage?: string | null;
|
||||
usage?: { inputTokens: number; outputTokens: number };
|
||||
sessionParams?: Record<string, unknown> | null; // persist across heartbeats
|
||||
sessionDisplayId?: string | null;
|
||||
provider?: string | null;
|
||||
model?: string | null;
|
||||
costUsd?: number | null;
|
||||
clearSession?: boolean; // set true to force fresh session on next wake
|
||||
}
|
||||
```
|
||||
|
||||
## Step 3: Environment Test
|
||||
|
||||
`src/server/test.ts` validates the adapter config before running.
|
||||
|
||||
Return structured diagnostics:
|
||||
|
||||
- `error` for invalid/unusable setup
|
||||
- `warn` for non-blocking issues
|
||||
- `info` for successful checks
|
||||
| Level | Meaning | Effect |
|
||||
|-------|---------|--------|
|
||||
| `error` | Invalid or unusable setup | Blocks execution |
|
||||
| `warn` | Non-blocking issue | Shown with yellow indicator |
|
||||
| `info` | Successful check | Shown in test results |
|
||||
|
||||
## Step 4: UI Module
|
||||
```ts
|
||||
export async function testEnvironment(
|
||||
ctx: AdapterEnvironmentTestContext,
|
||||
): Promise<AdapterEnvironmentTestResult> {
|
||||
return {
|
||||
adapterType: ctx.adapterType,
|
||||
status: "pass", // "pass" | "warn" | "fail"
|
||||
checks: [
|
||||
{ level: "info", message: "CLI v1.2.0 detected", code: "cli_detected" },
|
||||
{ level: "warn", message: "No API key found", hint: "Set ANTHROPIC_API_KEY", code: "no_key" },
|
||||
],
|
||||
testedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Step 4: UI Module (Built-in Only)
|
||||
|
||||
For built-in adapters registered in Paperclip's source:
|
||||
|
||||
- `parse-stdout.ts` — converts stdout lines to `TranscriptEntry[]` for the run viewer
|
||||
- `build-config.ts` — converts form values to `adapterConfig` JSON
|
||||
- Config fields React component in `ui/src/adapters/<name>/config-fields.tsx`
|
||||
|
||||
For external adapters, use a self-contained `ui-parser.ts` instead. See the [UI Parser Contract](/adapters/adapter-ui-parser).
|
||||
|
||||
## Step 5: CLI Module
|
||||
|
||||
`format-event.ts` — pretty-prints stdout for `paperclipai run --watch` using `picocolors`.
|
||||
|
||||
## Step 6: Register
|
||||
```ts
|
||||
export function formatStdoutEvent(line: string, debug: boolean): void {
|
||||
if (line.startsWith("[tool-done]")) {
|
||||
console.log(chalk.green(` ✓ ${line}`));
|
||||
} else {
|
||||
console.log(` ${line}`);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Step 6: Register (Built-in Only)
|
||||
|
||||
Add the adapter to all three registries:
|
||||
|
||||
|
|
@ -90,6 +185,24 @@ Add the adapter to all three registries:
|
|||
2. `ui/src/adapters/registry.ts`
|
||||
3. `cli/src/adapters/registry.ts`
|
||||
|
||||
For external adapters, registration is automatic — the plugin loader handles it.
|
||||
|
||||
## Session Persistence
|
||||
|
||||
If your agent runtime supports conversation continuity across heartbeats:
|
||||
|
||||
1. Return `sessionParams` from `execute()` (e.g., `{ sessionId: "abc123" }`)
|
||||
2. Read `runtime.sessionParams` on the next wake to resume
|
||||
3. Optionally implement a `sessionCodec` for validation and display
|
||||
|
||||
```ts
|
||||
export const sessionCodec: AdapterSessionCodec = {
|
||||
deserialize(raw) { /* validate raw session data */ },
|
||||
serialize(params) { /* serialize for storage */ },
|
||||
getDisplayId(params) { /* human-readable session label */ },
|
||||
};
|
||||
```
|
||||
|
||||
## Skills Injection
|
||||
|
||||
Make Paperclip skills discoverable to your agent runtime without writing to the agent's working directory:
|
||||
|
|
@ -105,3 +218,10 @@ Make Paperclip skills discoverable to your agent runtime without writing to the
|
|||
- Inject secrets via environment variables, not prompts
|
||||
- Configure network access controls if the runtime supports them
|
||||
- Always enforce timeout and grace period
|
||||
- The UI parser module runs in a browser sandbox — zero runtime imports, no side effects
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [External Adapters](/adapters/external-adapters) — build a standalone adapter plugin
|
||||
- [UI Parser Contract](/adapters/adapter-ui-parser) — ship a custom run-log parser
|
||||
- [How Agents Work](/guides/agent-developer/how-agents-work) — the heartbeat lifecycle
|
||||
|
|
|
|||
392
docs/adapters/external-adapters.md
Normal file
392
docs/adapters/external-adapters.md
Normal file
|
|
@ -0,0 +1,392 @@
|
|||
---
|
||||
title: External Adapters
|
||||
summary: Build, package, and distribute adapters as plugins without modifying Paperclip source
|
||||
---
|
||||
|
||||
Paperclip supports external adapter plugins that can be installed from npm packages or local directories. External adapters work exactly like built-in adapters — they execute agents, parse output, and render transcripts — but they live in their own package and don't require changes to Paperclip's source code.
|
||||
|
||||
## Built-in vs External
|
||||
|
||||
| | Built-in | External |
|
||||
|---|---|---|
|
||||
| Source location | Inside `paperclip-fork/packages/adapters/` | Separate npm package or local directory |
|
||||
| Registration | Hardcoded in three registries | Loaded at startup via plugin system |
|
||||
| UI parser | Static import at build time | Dynamically loaded from API (see [UI Parser](/adapters/adapter-ui-parser)) |
|
||||
| Distribution | Ships with Paperclip | Published to npm or linked via `file:` |
|
||||
| Updates | Requires Paperclip release | Independent versioning |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Minimal Package Structure
|
||||
|
||||
```
|
||||
my-adapter/
|
||||
package.json
|
||||
tsconfig.json
|
||||
src/
|
||||
index.ts # Shared metadata (type, label, models)
|
||||
server/
|
||||
index.ts # createServerAdapter() factory
|
||||
execute.ts # Core execution logic
|
||||
parse.ts # Output parsing
|
||||
test.ts # Environment diagnostics
|
||||
ui-parser.ts # Self-contained UI transcript parser
|
||||
```
|
||||
|
||||
### package.json
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "my-paperclip-adapter",
|
||||
"version": "1.0.0",
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
"paperclip": {
|
||||
"adapterUiParser": "1.0.0"
|
||||
},
|
||||
"exports": {
|
||||
".": "./dist/index.js",
|
||||
"./server": "./dist/server/index.js",
|
||||
"./ui-parser": "./dist/ui-parser.js"
|
||||
},
|
||||
"files": ["dist"],
|
||||
"scripts": {
|
||||
"build": "tsc"
|
||||
},
|
||||
"dependencies": {
|
||||
"@paperclipai/adapter-utils": "^2026.325.0",
|
||||
"picocolors": "^1.1.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"typescript": "^5.7.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key fields:
|
||||
|
||||
| Field | Purpose |
|
||||
|-------|---------|
|
||||
| `exports["."]` | Entry point — must export `createServerAdapter` |
|
||||
| `exports["./ui-parser"]` | Self-contained UI parser module (optional but recommended) |
|
||||
| `paperclip.adapterUiParser` | Contract version for the UI parser (`"1.0.0"`) |
|
||||
| `files` | Limits what gets published — only `dist/` |
|
||||
|
||||
### tsconfig.json
|
||||
|
||||
```json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "Node16",
|
||||
"moduleResolution": "Node16",
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"declaration": true,
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
```
|
||||
|
||||
## Server Module
|
||||
|
||||
The plugin loader calls `createServerAdapter()` from your package root. This function must return a `ServerAdapterModule`.
|
||||
|
||||
### src/index.ts
|
||||
|
||||
```ts
|
||||
export const type = "my_adapter"; // snake_case, globally unique
|
||||
export const label = "My Agent (local)";
|
||||
|
||||
export const models = [
|
||||
{ id: "model-a", label: "Model A" },
|
||||
];
|
||||
|
||||
export const agentConfigurationDoc = `# my_adapter configuration
|
||||
Use when: ...
|
||||
Don't use when: ...
|
||||
`;
|
||||
|
||||
// Required by plugin-loader convention
|
||||
export { createServerAdapter } from "./server/index.js";
|
||||
```
|
||||
|
||||
### src/server/index.ts
|
||||
|
||||
```ts
|
||||
import type { ServerAdapterModule } from "@paperclipai/adapter-utils";
|
||||
import { type, models, agentConfigurationDoc } from "../index.js";
|
||||
import { execute } from "./execute.js";
|
||||
import { testEnvironment } from "./test.js";
|
||||
|
||||
export function createServerAdapter(): ServerAdapterModule {
|
||||
return {
|
||||
type,
|
||||
execute,
|
||||
testEnvironment,
|
||||
models,
|
||||
agentConfigurationDoc,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### src/server/execute.ts
|
||||
|
||||
The core execution function. Receives an `AdapterExecutionContext` and returns an `AdapterExecutionResult`.
|
||||
|
||||
```ts
|
||||
import type {
|
||||
AdapterExecutionContext,
|
||||
AdapterExecutionResult,
|
||||
} from "@paperclipai/adapter-utils";
|
||||
|
||||
import {
|
||||
runChildProcess,
|
||||
buildPaperclipEnv,
|
||||
renderTemplate,
|
||||
} from "@paperclipai/adapter-utils/server-utils";
|
||||
|
||||
export async function execute(
|
||||
ctx: AdapterExecutionContext,
|
||||
): Promise<AdapterExecutionResult> {
|
||||
const { config, agent, runtime, context, onLog, onMeta } = ctx;
|
||||
|
||||
// 1. Read config with safe helpers
|
||||
const cwd = String(config.cwd ?? "/tmp");
|
||||
const command = String(config.command ?? "my-agent");
|
||||
const timeoutSec = Number(config.timeoutSec ?? 300);
|
||||
|
||||
// 2. Build environment with Paperclip vars injected
|
||||
const env = buildPaperclipEnv(agent);
|
||||
|
||||
// 3. Render prompt template
|
||||
const prompt = config.promptTemplate
|
||||
? renderTemplate(String(config.promptTemplate), {
|
||||
agentId: agent.id,
|
||||
agentName: agent.name,
|
||||
companyId: agent.companyId,
|
||||
runId: ctx.runId,
|
||||
taskId: context.taskId ?? "",
|
||||
taskTitle: context.taskTitle ?? "",
|
||||
})
|
||||
: "Continue your work.";
|
||||
|
||||
// 4. Spawn process
|
||||
const result = await runChildProcess(command, {
|
||||
args: [prompt],
|
||||
cwd,
|
||||
env,
|
||||
timeout: timeoutSec * 1000,
|
||||
graceMs: 10_000,
|
||||
onStdout: (chunk) => onLog("stdout", chunk),
|
||||
onStderr: (chunk) => onLog("stderr", chunk),
|
||||
});
|
||||
|
||||
// 5. Return structured result
|
||||
return {
|
||||
exitCode: result.exitCode,
|
||||
timedOut: result.timedOut,
|
||||
// Include session state for persistence
|
||||
sessionParams: { /* ... */ },
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
#### Available Helpers from `@paperclipai/adapter-utils`
|
||||
|
||||
| Helper | Purpose |
|
||||
|--------|---------|
|
||||
| `runChildProcess(command, opts)` | Spawn a child process with timeout, grace period, and streaming callbacks |
|
||||
| `buildPaperclipEnv(agent)` | Inject `PAPERCLIP_*` environment variables |
|
||||
| `renderTemplate(template, data)` | `{{variable}}` substitution in prompt templates |
|
||||
| `asString(v)`, `asNumber(v)`, `asBoolean(v)` | Safe config value extraction |
|
||||
|
||||
### src/server/test.ts
|
||||
|
||||
Validates the adapter configuration before running. Returns structured diagnostics.
|
||||
|
||||
```ts
|
||||
import type {
|
||||
AdapterEnvironmentTestContext,
|
||||
AdapterEnvironmentTestResult,
|
||||
} from "@paperclipai/adapter-utils";
|
||||
|
||||
export async function testEnvironment(
|
||||
ctx: AdapterEnvironmentTestContext,
|
||||
): Promise<AdapterEnvironmentTestResult> {
|
||||
const checks = [];
|
||||
|
||||
// Example: check CLI is installed
|
||||
checks.push({
|
||||
level: "info",
|
||||
message: "My Agent CLI v1.2.0 detected",
|
||||
code: "cli_detected",
|
||||
});
|
||||
|
||||
// Example: check working directory
|
||||
const cwd = String(ctx.config.cwd ?? "");
|
||||
if (!cwd.startsWith("/")) {
|
||||
checks.push({
|
||||
level: "error",
|
||||
message: `Working directory must be absolute: "${cwd}"`,
|
||||
hint: "Use /home/user/project or /workspace",
|
||||
code: "invalid_cwd",
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
adapterType: ctx.adapterType,
|
||||
status: checks.some(c => c.level === "error") ? "fail" : "pass",
|
||||
checks,
|
||||
testedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Check levels:
|
||||
|
||||
| Level | Meaning | Effect |
|
||||
|-------|---------|--------|
|
||||
| `info` | Informational | Shown in test results |
|
||||
| `warn` | Non-blocking issue | Shown with yellow indicator |
|
||||
| `error` | Blocks execution | Prevents agent from running |
|
||||
|
||||
## Installation
|
||||
|
||||
### From npm
|
||||
|
||||
```sh
|
||||
# Via the Paperclip UI
|
||||
# Settings → Adapters → Install from npm → "my-paperclip-adapter"
|
||||
|
||||
# Or via API
|
||||
curl -X POST http://localhost:3102/api/adapters \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"packageName": "my-paperclip-adapter"}'
|
||||
```
|
||||
|
||||
### From local directory
|
||||
|
||||
```sh
|
||||
curl -X POST http://localhost:3102/api/adapters \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"localPath": "/home/user/my-adapter"}'
|
||||
```
|
||||
|
||||
Local adapters are symlinked into Paperclip's adapter directory. Changes to the source are picked up on server restart.
|
||||
|
||||
### Via adapter-plugins.json
|
||||
|
||||
For development, you can also edit `~/.paperclip/adapter-plugins.json` directly:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"packageName": "my-paperclip-adapter",
|
||||
"localPath": "/home/user/my-adapter",
|
||||
"type": "my_adapter",
|
||||
"installedAt": "2026-03-30T12:00:00.000Z"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Optional: Session Persistence
|
||||
|
||||
If your agent runtime supports sessions (conversation continuity across heartbeats), implement a session codec:
|
||||
|
||||
```ts
|
||||
import type { AdapterSessionCodec } from "@paperclipai/adapter-utils";
|
||||
|
||||
export const sessionCodec: AdapterSessionCodec = {
|
||||
deserialize(raw) {
|
||||
if (typeof raw !== "object" || raw === null) return null;
|
||||
const r = raw as Record<string, unknown>;
|
||||
return r.sessionId ? { sessionId: String(r.sessionId) } : null;
|
||||
},
|
||||
serialize(params) {
|
||||
return params?.sessionId ? { sessionId: String(params.sessionId) } : null;
|
||||
},
|
||||
getDisplayId(params) {
|
||||
return params?.sessionId ? String(params.sessionId) : null;
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
Include it in `createServerAdapter()`:
|
||||
|
||||
```ts
|
||||
return { type, execute, testEnvironment, sessionCodec, /* ... */ };
|
||||
```
|
||||
|
||||
## Optional: Skills Sync
|
||||
|
||||
If your agent runtime supports skills/plugins, implement `listSkills` and `syncSkills`:
|
||||
|
||||
```ts
|
||||
return {
|
||||
type,
|
||||
execute,
|
||||
testEnvironment,
|
||||
async listSkills(ctx) {
|
||||
return {
|
||||
adapterType: ctx.adapterType,
|
||||
supported: true,
|
||||
mode: "ephemeral",
|
||||
desiredSkills: [],
|
||||
entries: [],
|
||||
warnings: [],
|
||||
};
|
||||
},
|
||||
async syncSkills(ctx, desiredSkills) {
|
||||
// Install desired skills into the runtime
|
||||
return { /* same shape as listSkills */ };
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
## Optional: Model Detection
|
||||
|
||||
If your runtime has a local config file that specifies the default model:
|
||||
|
||||
```ts
|
||||
async function detectModel() {
|
||||
// Read ~/.my-agent/config.yaml or similar
|
||||
return {
|
||||
model: "anthropic/claude-sonnet-4",
|
||||
provider: "anthropic",
|
||||
source: "~/.my-agent/config.yaml",
|
||||
candidates: ["anthropic/claude-sonnet-4", "openai/gpt-4o"],
|
||||
};
|
||||
}
|
||||
|
||||
return { type, execute, testEnvironment, detectModel: () => detectModel() };
|
||||
```
|
||||
|
||||
## Publishing
|
||||
|
||||
```sh
|
||||
npm run build
|
||||
npm publish
|
||||
```
|
||||
|
||||
Other Paperclip users can then install your adapter by package name from the UI or API.
|
||||
|
||||
## Security
|
||||
|
||||
- Treat agent output as untrusted — parse defensively, never `eval()` agent output
|
||||
- Inject secrets via environment variables, not in prompts
|
||||
- Configure network access controls if the runtime supports them
|
||||
- Always enforce timeout and grace period — don't let agents run forever
|
||||
- The UI parser module runs in a browser sandbox — it must have zero runtime imports and no side effects
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [UI Parser Contract](/adapters/adapter-ui-parser) — add a custom run-log parser so the UI renders your adapter's output correctly
|
||||
- [Creating an Adapter](/adapters/creating-an-adapter) — full walkthrough of adapter internals
|
||||
- [How Agents Work](/guides/agent-developer/how-agents-work) — understand the heartbeat lifecycle your adapter serves
|
||||
|
|
@ -22,43 +22,67 @@ When a heartbeat fires, Paperclip:
|
|||
| [Codex Local](/adapters/codex-local) | `codex_local` | Runs OpenAI Codex CLI locally |
|
||||
| [Gemini Local](/adapters/gemini-local) | `gemini_local` | Runs Gemini CLI locally (experimental — adapter package exists, not yet in stable type enum) |
|
||||
| OpenCode Local | `opencode_local` | Runs OpenCode CLI locally (multi-provider `provider/model`) |
|
||||
| Hermes Local | `hermes_local` | Runs Hermes CLI locally |
|
||||
| Cursor | `cursor` | Runs Cursor in background mode |
|
||||
| Pi Local | `pi_local` | Runs an embedded Pi agent locally |
|
||||
| OpenClaw Gateway | `openclaw_gateway` | Connects to an OpenClaw gateway endpoint |
|
||||
| [Process](/adapters/process) | `process` | Executes arbitrary shell commands |
|
||||
| [HTTP](/adapters/http) | `http` | Sends webhooks to external agents |
|
||||
|
||||
### External (plugin) adapters
|
||||
|
||||
These adapters ship as standalone npm packages and are installed via the plugin system:
|
||||
|
||||
| Adapter | Package | Type Key | Description |
|
||||
|---------|---------|----------|-------------|
|
||||
| Droid Local | `@henkey/droid-paperclip-adapter` | `droid_local` | Runs Factory Droid locally |
|
||||
| Hermes Local | `@henkey/hermes-paperclip-adapter` | `hermes_local` | Runs Hermes CLI locally |
|
||||
|
||||
## External Adapters
|
||||
|
||||
You can build and distribute adapters as standalone packages — no changes to Paperclip's source code required. External adapters are loaded at startup via the plugin system.
|
||||
|
||||
```sh
|
||||
# Install from npm via API
|
||||
curl -X POST http://localhost:3102/api/adapters \
|
||||
-d '{"packageName": "my-paperclip-adapter"}'
|
||||
|
||||
# Or link from a local directory
|
||||
curl -X POST http://localhost:3102/api/adapters \
|
||||
-d '{"localPath": "/home/user/my-adapter"}'
|
||||
```
|
||||
|
||||
See [External Adapters](/adapters/external-adapters) for the full guide.
|
||||
|
||||
## Adapter Architecture
|
||||
|
||||
Each adapter is a package with three modules:
|
||||
Each adapter is a package with modules consumed by three registries:
|
||||
|
||||
```
|
||||
packages/adapters/<name>/
|
||||
my-adapter/
|
||||
src/
|
||||
index.ts # Shared metadata (type, label, models)
|
||||
server/
|
||||
execute.ts # Core execution logic
|
||||
parse.ts # Output parsing
|
||||
test.ts # Environment diagnostics
|
||||
ui/
|
||||
parse-stdout.ts # Stdout -> transcript entries for run viewer
|
||||
build-config.ts # Form values -> adapterConfig JSON
|
||||
ui-parser.ts # Self-contained UI transcript parser (for external adapters)
|
||||
cli/
|
||||
format-event.ts # Terminal output for `paperclipai run --watch`
|
||||
```
|
||||
|
||||
Three registries consume these modules:
|
||||
|
||||
| Registry | What it does |
|
||||
|----------|-------------|
|
||||
| **Server** | Executes agents, captures results |
|
||||
| **UI** | Renders run transcripts, provides config forms |
|
||||
| **CLI** | Formats terminal output for live watching |
|
||||
| Registry | What it does | Source |
|
||||
|----------|-------------|--------|
|
||||
| **Server** | Executes agents, captures results | `createServerAdapter()` from package root |
|
||||
| **UI** | Renders run transcripts, provides config forms | `ui-parser.js` (dynamic) or static import (built-in) |
|
||||
| **CLI** | Formats terminal output for live watching | Static import |
|
||||
|
||||
## Choosing an Adapter
|
||||
|
||||
- **Need a coding agent?** Use `claude_local`, `codex_local`, `opencode_local`, or `hermes_local`
|
||||
- **Need a coding agent?** Use `claude_local`, `codex_local`, `opencode_local`, or install `droid_local` / `hermes_local` as external plugins
|
||||
- **Need to run a script or command?** Use `process`
|
||||
- **Need to call an external service?** Use `http`
|
||||
- **Need something custom?** [Create your own adapter](/adapters/creating-an-adapter)
|
||||
- **Need something custom?** [Create your own adapter](/adapters/creating-an-adapter) or [build an external adapter plugin](/adapters/external-adapters)
|
||||
|
||||
## UI Parser Contract
|
||||
|
||||
External adapters can ship a self-contained UI parser that tells the Paperclip web UI how to render their stdout. Without it, the UI uses a generic shell parser. See the [UI Parser Contract](/adapters/adapter-ui-parser) for details.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue