6 commits

Author	SHA1	Message	Date
Devin Foley	ad0bb57350	Fix exe.dev sandbox installs for gemini/opencode local adapters (#5737) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies, including running adapter CLIs inside remote sandboxes > - The QA matrix in PAPA-316 spins up local-runtime adapters (claude/gemini/opencode) against both SSH and the new exe.dev sandbox provider, and "Test" exercises the same install + probe path the real runtime uses > - On exe.dev the QA matrix failed at three different points: SSH/sandbox secret refs would not resolve, gemini-local could not find npm, and opencode-local installed a binary that was not on the probe-shell PATH > - These are all environment-shape issues the runtime should handle, not regressions in any individual adapter, so they need to be fixed in the shared install/resolve layer before the matrix can pass > - This pull request wires the environment id through to secret-ref resolution, bootstraps npm from a portable Node tarball when the sandbox image lacks Node, and symlinks the opencode binary into a directory that non-login shells see > - The benefit is that the QA matrix passes end-to-end on exe.dev, and any future sandbox provider that ships without Node or relies on rc-file PATH wiring gets the same fixes for free ## What Changed - `server/src/services/environment-execution-target.ts`: pass the environment `id` into `resolveEnvironmentDriverConfigForRuntime` for both the sandbox and SSH branches, so `privateKeySecretRef` / sandbox-provider secret refs (e.g. exe.dev `apiKey`) can resolve against the secret store at runtime instead of throwing `Runtime secret resolution requires an environment id`. - `packages/adapter-utils/src/sandbox-install-command.ts`: extend `buildSandboxNpmInstallCommand` with an `ENSURE_NPM_PREAMBLE` that, when `npm` is missing, downloads a portable Node v22 tarball into `$HOME/.local` and sets `PAPERCLIP_NPM_BOOTSTRAPPED=1` so the install step skips sudo (sudo's `secure_path` would lose the freshly-installed `npm` in `$HOME/.local/bin`). Distro-packaged Node from apt-get is intentionally avoided because it tends to be too old to parse modern JS syntax used by `@google/gemini-cli`. - `packages/adapters/gemini-local/src/index.ts`: switch the hardcoded `npm install -g @google/gemini-cli` to `buildSandboxNpmInstallCommand`, so gemini-local picks up the same sudo-aware + npm-bootstrap behavior as the other local adapters. - `packages/adapters/opencode-local/src/index.ts`: append a step to the install command that symlinks `$HOME/.opencode/bin/opencode` into `$HOME/.local/bin`. The upstream installer only adds `~/.opencode/bin` to PATH via `~/.bashrc`, which non-login `sh -c` probe invocations do not source. - `packages/adapter-utils/src/sandbox-install-command.test.ts`: cover the new preamble plus the unchanged root/sudo/user-prefix branches. ## Verification - `cd packages/adapter-utils && npm test -- sandbox-install-command` (passes; new "bootstraps npm from a portable Node tarball when missing" case is included). - Manual: ran the in-app `Test` action against the QA matrix dev instance for `QA exe.dev Claude`, `QA exe.dev Gemini`, and `QA exe.dev OpenCode` — all three now report `status=pass` including the hello probe. `QA SSH Claude` also passes; without the environment-id fix, SSH resolution threw before the wrapper / install fixes could run. - Suggested reviewer check: re-run the matrix on a fresh exe.dev environment and confirm the install step no longer hits `npm: command not found` for gemini and the opencode probe no longer hits `opencode: command not found`. ## Risks - Low/medium. The npm bootstrap pins Node `v22.11.0` from `nodejs.org/dist`; if that URL becomes unreachable the install will fail with a clear `curl` error rather than corrupting state. The bootstrap path is only taken when `npm` is genuinely missing, so existing sandbox images that ship with Node are unaffected. - The opencode symlink uses `ln -sf` into `$HOME/.local/bin`, which is created with `mkdir -p`; idempotent on re-install. - The `id` change is a strict additive: callers previously got `undefined` and only the secret-ref code paths actually read it. No behavior change for environments without secret refs. ## Model Used - Claude (Anthropic), `claude-opus-4-7`, with extended thinking and tool use enabled. Iterated through the Paperclip QA matrix harness; no other model assisted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a — runtime/install path only) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-05-11 14:28:22 -07:00
Devin Foley	076067865f	Migrate SSH environment callback to bridge (#5116) ... > **Stacked PR (part 3 of 7).** Depends on: - PR #5114 - PR #5115 > Diff against `master` includes commits from earlier PRs in the stack — the new commit in this PR is the topmost one. ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents executing on a remote SSH-backed environment need a way to call back into > the Paperclip control plane (run events, log streaming, signals) > - When the SSH host can't reach the Paperclip host (NAT, firewalls, or simply not > on the same network), the run silently fails or hangs — a recurring class of > failure during SSH testing > - In sandboxed environments we already solved this with a callback bridge that > tunnels back through the existing connection; SSH was the odd one out > - This PR migrates SSH execution to use the same callback bridge, so every > adapter's remote run uses one consistent reverse-channel. Per-adapter SSH glue > is deleted in favour of a shared `CommandManagedRuntimeRunner` built from the > SSH spec > - The benefit is fewer SSH-specific failure modes, a smaller code surface, and > one place to evolve the callback contract going forward ## What Changed - Added `createSshCommandManagedRuntimeRunner` in `packages/adapter-utils/src/ssh.ts` that adapts an SSH spec into a generic command-managed-runtime runner (with cwd, env, and timeout handling) - Removed `paperclipApiUrl` from `SshRemoteExecutionSpec`; the bridge URL now flows through the shared runner - Reworked `execution-target.ts` to use the SSH runner alongside sandbox runners via a unified `CommandManagedRuntimeRunner` interface - Simplified `remote-managed-runtime.ts` and `sandbox-managed-runtime.ts` to consume the shared runner abstraction - Deleted per-adapter SSH callback wiring from claude-local, codex-local, cursor-local, gemini-local, opencode-local, pi-local execute.ts files - Removed `environment-runtime-driver-contract.test.ts` (the contract is now enforced by `environment-execution-target.test.ts`) - Added/updated `execute.remote.test.ts` cases for each adapter to cover the SSH runner path ## Verification - `pnpm --filter @paperclipai/adapter-utils test` - `pnpm test -- execute.remote` (covers all six local adapters' SSH paths) - Manual QA: ran a claude-local agent against an SSH-backed environment, confirmed the agent successfully called back to `/api/agent-callback/*` endpoints during the run ## Risks - Refactor touches all six local adapters. If any adapter had subtle SSH-specific behaviour that wasn't captured in tests, it could regress. Mitigation: each adapter's `execute.remote.test.ts` was extended. - `paperclipApiUrl` removal from `SshRemoteExecutionSpec` is a breaking type change for any internal consumer. Verified no external plugins consume this type. - The new `CommandManagedRuntimeRunner` shape is a public surface in `@paperclipai/adapter-utils`; downstream plugins implementing custom runners may need updates, but no such plugins exist in this repo. ## Model Used - OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI - Provider: OpenAI - Used to author the code changes in this PR ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A - [ ] I have updated relevant documentation to reflect my changes — N/A - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-05-03 12:43:52 -07:00
Devin Foley	a7b45938b7	Let sandbox providers declare shell defaults (#5114) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents execute in sandboxed remote environments served by pluggable sandbox > providers (E2B today, more later) > - Today every sandbox command runs under `sh -lc` regardless of what the > provider's container actually ships > - That misses bash-only shell init on E2B (which ships bash) and prevents > future providers from declaring a different default — there's no way for a > provider to say "I have bash, use it" > - This PR adds a `shellCommand` field to sandbox execution targets so providers > can declare their preferred shell ("bash" for E2B), threads it through the > sandbox-managed-runtime client, callback bridge, and execution-target shell > helper, and validates the value at the lease-metadata boundary > - The benefit is that sandbox commands run under the right shell on the right > provider, and adding new sandbox providers only needs to declare a shell > preference ## What Changed - Added `packages/adapter-utils/src/sandbox-shell.ts` exporting `preferredShellForSandbox(shellCommand)` (returns `"bash"` if input is `"bash"`, else `"sh"`) - Added `shellCommand?: "bash" | "sh" | null` to `AdapterSandboxExecutionTarget` and `CommandManagedRuntimeSpec`; threaded it through `runAdapterExecutionTargetShellCommand`, `prepareAdapterExecutionTargetRuntime`, and `startAdapterExecutionTargetPaperclipBridge` - `createCommandManagedRuntimeClient`, `prepareCommandManagedRuntime`, and `createCommandManagedSandboxCallbackBridgeQueueClient` now take an optional `shellCommand` and use `preferredShellForSandbox` to pick the shell - `startSandboxCallbackBridgeServer` accepts a `shellCommand` for its server startup, readiness probe, and stop hook - E2B sandbox plugin declares `shellCommand: "bash"` in `leaseMetadata` - `resolveEnvironmentExecutionTarget` reads `shellCommand` from lease metadata (validating against `"bash" | "sh" | null`) - `environment-runtime.ts` adds `"shellCommand"` to `INTERNAL_PLUGIN_SANDBOX_CONFIG_KEYS` so the field round-trips through internal plugin config without leaking to external plugin metadata - Updated tests in `command-managed-runtime.test.ts`, `execution-target-sandbox.test.ts`, `sandbox-callback-bridge.test.ts`, `environment-execution-target.test.ts` ## Verification - `pnpm --filter @paperclipai/adapter-utils test` - `pnpm --filter @paperclipai/server test -- environment-execution-target` - `pnpm --filter @paperclipai/sandbox-providers-e2b test` - Manual QA: boot a Paperclip instance, create an E2B-backed environment, run a claude_local agent against it, and confirm the run completes (verifies bash shell semantics flow through the callback bridge end-to-end) ## Risks - E2B sandbox commands now run under `bash -lc` instead of `sh -lc`. Bash is a strict superset for the commands we issue (no busybox-only flags in our shell scripts), so risk is low. The shellCommand field is opt-in via lease metadata — providers that don't declare it stay on `sh`. - New optional field on `CommandManagedRuntimeSpec` and `AdapterSandboxExecutionTarget`. Consumers ignoring the field retain previous behaviour (sh). - Lease metadata now carries an additional field. Existing leases without `shellCommand` resolve to `null` and fall back to sh — backwards compatible. ## Model Used - OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI - Provider: OpenAI - Used to author the code changes in this PR ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A (no UI changes) - [ ] I have updated relevant documentation to reflect my changes — N/A - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-05-03 12:19:35 -07:00
Dotta	4272c1604d	Add ACPX local adapter runtime (#4893) ... ## Thinking Path > - Paperclip orchestrates AI-agent companies through a control plane that can start, supervise, and recover agent runs. > - Local adapters are the bridge between Paperclip issues and concrete agent runtimes such as Claude, Codex, and other ACP-compatible tools. > - The roadmap calls out broader “bring your own agent” and claw-style agent support, and ACPX gives Paperclip one path to normalize multiple ACP agents behind a single adapter. > - The branch needed to become one reviewable PR against current `paperclipai/paperclip:master`, without carrying stale base conflicts or generated lockfile churn. > - This pull request adds an experimental built-in `acpx_local` adapter, integrates it through the server/CLI/UI adapter surfaces, and adds regression coverage for runtime execution, skill sync, stream parsing, diagnostics, and log redaction. > - The benefit is that Paperclip can run Claude/Codex/custom ACP agents through ACPX while keeping operator configuration, skills, logging, and transcript rendering inside the existing adapter model. ## What Changed - Added `@paperclipai/adapter-acpx-local` with server execution, config schema, ACPX session handling, CLI formatting, UI config helpers, and stdout parsing. - Registered `acpx_local` across CLI, server, shared constants, UI adapter metadata, adapter capabilities, and agent creation/editing surfaces. - Added ACPX runtime execution support with persistent sessions, local-agent JWT environment handling, skill snapshots, runtime skill materialization, and isolation/security regressions. - Added ACPX adapter diagnostics and marked the adapter experimental in the UI. - Added command/env secret redaction for resolved command metadata in adapter-utils, server event storage, and the Agent Detail invocation UI. - Added Storybook coverage for ACPX config, transcript rendering, and skill states, plus PR screenshots under `docs/pr-screenshots/pap-2944/`. - Rebased the branch onto current `public-gh/master`; `pnpm-lock.yaml` is intentionally not included and there are no migration/schema changes. ## Verification - `pnpm exec vitest run packages/adapters/acpx-local/src/server/execute.test.ts packages/adapters/acpx-local/src/server/test.test.ts packages/adapters/acpx-local/src/cli/format-event.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/redaction.test.ts server/src/__tests__/acpx-local-execute.test.ts server/src/__tests__/acpx-local-skill-sync.test.ts server/src/__tests__/acpx-local-adapter-environment.test.ts server/src/__tests__/adapter-routes.test.ts server/src/__tests__/agent-skills-routes.test.ts ui/src/adapters/metadata.test.ts` — 12 files, 87 tests passed. - `pnpm --filter @paperclipai/adapter-acpx-local typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Confirmed PR diff does not include `pnpm-lock.yaml`, database schema files, or migrations. Screenshots:    ## Risks - Medium risk: this introduces a new built-in adapter package and touches runtime execution, adapter registration, agent config, skills, and transcript rendering. - ACPX and ACP agent behavior can vary by installed tool versions; the adapter is marked experimental to set operator expectations. - `pnpm-lock.yaml` is excluded per repository PR policy, so dependency lock refresh must be handled by the repo’s automation or maintainers. - No database migration risk: no schema or migration files changed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, with repository tool use, shell execution, git operations, and local verification. Exact hosted context window was not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-04-30 19:57:05 -05:00
Devin Foley	a4ac6ff133	Add sandbox callback bridge for remote environment API access (#4801) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents can run inside sandboxed environments like E2B, which are isolated from the host network > - Sandboxed agents need to call back to the Paperclip API to report progress, post comments, and update issue status > - But sandbox environments cannot reach the Paperclip server directly because they run in isolated network namespaces > - This PR adds a callback bridge that proxies API requests from the sandbox to the Paperclip server, running as a local HTTP server on the host that forwards authenticated requests > - The bridge is started automatically when an adapter launches a sandbox execution, and torn down when the run completes > - The benefit is sandboxed agents can interact with the Paperclip API without requiring network-level access to the host, enabling E2B and similar providers to work end-to-end ## What Changed - Added `sandbox-callback-bridge.ts` in `packages/adapter-utils/` — a lightweight HTTP bridge server that accepts requests from sandbox environments and proxies them to the Paperclip API with authentication - Added request validation and security policy: the bridge only forwards requests to the configured API URL, validates content types, enforces size limits, and rejects non-API paths - Wired the bridge into all remote adapter execute paths (claude, codex, cursor, gemini, pi) — the bridge starts before the agent process and the bridge URL is passed via environment variables - Updated `environment-execution-target.ts` to prefer the explicit API URL from environment lease metadata for sandbox callback routing - Fixed Claude sandbox runtime setup to work with the bridge configuration - Added comprehensive test coverage for bridge request handling, policy enforcement, and sandbox execution integration - Fixed browser bundling — the bridge module is excluded from the frontend bundle via the adapter-utils index export ## Verification - `pnpm test` — all existing and new tests pass, including bridge unit tests and sandbox execution integration tests - `pnpm typecheck` — clean - Manual: configure an E2B environment, run an agent task, verify the agent can post comments and update issue status through the bridge ## Risks - Medium. This is a new network-facing component (HTTP server on localhost). The security policy restricts forwarding to the configured API URL only and validates all requests, but any proxy introduces attack surface. The bridge binds to localhost only and is scoped to the lifetime of a single agent run. ## Model Used Codex GPT 5.4 high via Paperclip. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-04-29 16:37:34 -07:00
Devin Foley	70679a3321	Add sandbox environment support (#4415) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The environment/runtime layer decides where agent work executes and how the control plane reaches those runtimes. > - Today Paperclip can run locally and over SSH, but sandboxed execution needs a first-class environment model instead of one-off adapter behavior. > - We also want sandbox providers to be pluggable so the core does not hardcode every provider implementation. > - This branch adds the Sandbox environment path, the provider contract, and a deterministic fake provider plugin. > - That required synchronized changes across shared contracts, plugin SDK surfaces, server runtime orchestration, and the UI environment/workspace flows. > - The result is that sandbox execution becomes a core control-plane capability while keeping provider implementations extensible and testable. ## What Changed - Added sandbox runtime support to the environment execution path, including runtime URL discovery, sandbox execution targeting, orchestration, and heartbeat integration. - Added plugin-provider support for sandbox environments so providers can be supplied via plugins instead of hardcoded server logic. - Added the fake sandbox provider plugin with deterministic behavior suitable for local and automated testing. - Updated shared types, validators, plugin protocol definitions, and SDK helpers to carry sandbox provider and workspace-runtime contracts across package boundaries. - Updated server routes and services so companies can create sandbox environments, select them for work, and execute work through the sandbox runtime path. - Updated the UI environment and workspace surfaces to expose sandbox environment configuration and selection. - Added test coverage for sandbox runtime behavior, provider seams, environment route guards, orchestration, and the fake provider plugin. ## Verification - Ran locally before the final fixture-only scrub: - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` - Ran locally after the final scrub amend: - `pnpm vitest run server/src/__tests__/runtime-api.test.ts` - Reviewer spot checks: - create a sandbox environment backed by the fake provider plugin - run work through that environment - confirm sandbox provider execution does not inherit host secrets implicitly ## Risks - This touches shared contracts, plugin SDK plumbing, server runtime orchestration, and UI environment/workspace flows, so regressions would likely show up as cross-layer mismatches rather than isolated type errors. - Runtime URL discovery and sandbox callback selection are sensitive to host/bind configuration; if that logic is wrong, sandbox-backed callbacks may fail even when execution succeeds. - The fake provider plugin is intentionally deterministic and test-oriented; future providers may expose capability gaps that this branch does not yet cover. ## Model Used - OpenAI Codex coding agent on a GPT-5-class backend in the Paperclip/Codex harness. Exact backend model ID is not exposed in-session. Tool-assisted workflow with shell execution, file editing, git history inspection, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-04-24 12:15:53 -07:00