Harden remote sandbox runtime probes, timeouts, and installs (#5685)

## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs inside a sandbox environment so its CLI is isolated from the host > - Sandbox-backed adapter runs go through a small set of shared helpers — `ensureAdapterExecutionTargetCommandResolvable`, the sandbox callback bridge runner, and per-adapter `SANDBOX_INSTALL_COMMAND` strings > - When standing up new sandbox provider plugins, the existing helpers timed out, missed install fallbacks, or leaned on assumptions that only held for E2B > - Local adapters (`claude-local`, `codex-local`, `gemini-local`, `opencode-local`) needed slightly hardened probes so they could install themselves and validate inside *any* remote sandbox transport, not just E2B > - This pull request bundles those runtime fixes so future sandbox provider plugins inherit a working baseline > - The benefit is that adding a new sandbox provider plugin no longer requires touching adapter-utils or each local-adapter probe — the supporting infra is already correct ## What Changed - `packages/adapter-utils/src/execution-target.ts`: introduce `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800` and `resolveAdapterExecutionTargetTimeoutSec(...)`. Local and SSH adapters keep the historical "0 means no adapter timeout" behavior; sandbox-backed runs without an explicit `timeoutSec` get an explicit 30-minute default so remote installs and warm-up don't time out at the per-RPC default. Plumbed `timeoutSec` through `ensureAdapterExecutionTargetCommandResolvable` so install probes inside a sandbox honor adapter-level overrides instead of the bridge's 5-minute default. - `packages/adapters/opencode-local/src/index.ts`: switch `SANDBOX_INSTALL_COMMAND` from `npm install -g opencode-ai` to `curl -fsSL https://opencode.ai/install | bash`. The npm package reifies four large prebuilt-binary subpackages in parallel even though only one matches the host arch; on bandwidth-constrained sandboxes that blew through the 240s install budget. The official installer fetches one arch-specific binary and adds `$HOME/.opencode/bin` to PATH via `~/.bashrc`, which the sandbox-callback-bridge login-shell script already sources. - `packages/adapters/{claude,codex,gemini,opencode}-local/`: harden remote-target probes — pass `--skip-git-repo-check` for Codex when probing outside a repo, normalize permission flags for Claude, and add `*.remote.test.ts` coverage that exercises the remote-sandbox path explicitly for each adapter. - `packages/adapter-utils/src/sandbox-install-command.{ts,test.ts}` (new): add `buildSandboxNpmInstallCommand` helper. `server/src/adapters/registry.ts` + new `server/src/__tests__/adapter-registry.test.ts`: wire adapter install commands so they fall back to a writable `$HOME/.local` prefix when global install isn't available. - `server/src/__tests__/plugin-worker-manager.test.ts` + new `server/src/__tests__/fixtures/plugin-worker-delayed.cjs`: pin per-call timeout overrides so plugin worker exec calls honor the caller's timeout instead of the worker's default. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-install-command.test.ts` - `pnpm exec vitest run --no-coverage server/src/__tests__/plugin-worker-manager.test.ts server/src/__tests__/adapter-registry.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/gemini-local-adapter-environment.test.ts` - `pnpm exec vitest run --no-coverage packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/opencode-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts packages/adapters/gemini-local/src/server/execute.remote.test.ts` All passing locally. ## Risks - Touches shared `adapter-utils` and several `*-local` adapters. The 30-minute default applies only when both (a) the target is `remote+sandbox` and (b) no `timeoutSec` is configured — local + SSH paths are unchanged. New test coverage was added alongside each behavior change to pin the contracts. - Switching OpenCode's install command to the official installer is a behavior change for any operator running OpenCode inside a remote sandbox. Local installs are unaffected (the `SANDBOX_INSTALL_COMMAND` only runs when an adapter is being installed inside a sandbox). - Low risk overall — no migrations, no API surface change. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep), no code execution beyond local repo commands ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-14 01:50:39 +09:00 · 2026-05-11 00:31:54 -07:00 · 2026-05-11 00:31:54 -07:00 · b24c6909e8
commit b24c6909e8
parent 6e4fa78d86
30 changed files with 938 additions and 36 deletions
--- a/packages/adapter-utils/src/execution-target-sandbox.test.ts
+++ b/packages/adapter-utils/src/execution-target-sandbox.test.ts
@ -5,9 +5,12 @@ import path from "node:path";
 import { afterEach, describe, expect, it, vi } from "vitest";

 import {
+  DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC,
  adapterExecutionTargetSessionIdentity,
  adapterExecutionTargetToRemoteSpec,
  adapterExecutionTargetUsesPaperclipBridge,
+  ensureAdapterExecutionTargetCommandResolvable,
+  resolveAdapterExecutionTargetTimeoutSec,
  runAdapterExecutionTargetProcess,
  runAdapterExecutionTargetShellCommand,
  startAdapterExecutionTargetPaperclipBridge,
@ -109,6 +112,89 @@ describe("sandbox adapter execution targets", () => {
    });
  });

+  it("applies the remote sandbox fallback when adapter timeoutSec is unset", () => {
+    const sandboxTarget: AdapterSandboxExecutionTarget = {
+      kind: "remote",
+      transport: "sandbox",
+      remoteCwd: "/workspace",
+      runner: createLocalSandboxRunner(),
+    };
+
+    expect(resolveAdapterExecutionTargetTimeoutSec(sandboxTarget, 0)).toBe(
+      DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC,
+    );
+    expect(resolveAdapterExecutionTargetTimeoutSec(sandboxTarget, 90)).toBe(90);
+    expect(resolveAdapterExecutionTargetTimeoutSec({
+      kind: "remote",
+      transport: "ssh",
+      remoteCwd: "/workspace",
+      spec: {
+        host: "127.0.0.1",
+        port: 22,
+        username: "fixture",
+        remoteWorkspacePath: "/workspace",
+        remoteCwd: "/workspace",
+        privateKey: "KEY",
+        knownHosts: "host key",
+        strictHostKeyChecking: true,
+      },
+    }, 0)).toBe(0);
+  });
+
+  it("uses the caller timeout override when installing a missing sandbox command", async () => {
+    const runner = {
+      execute: vi.fn()
+        .mockResolvedValueOnce({
+          exitCode: 1,
+          signal: null,
+          timedOut: false,
+          stdout: "",
+          stderr: "",
+          pid: null,
+          startedAt: new Date().toISOString(),
+        })
+        .mockResolvedValueOnce({
+          exitCode: 0,
+          signal: null,
+          timedOut: false,
+          stdout: "",
+          stderr: "",
+          pid: null,
+          startedAt: new Date().toISOString(),
+        })
+        .mockResolvedValueOnce({
+          exitCode: 0,
+          signal: null,
+          timedOut: false,
+          stdout: "/usr/bin/opencode\n",
+          stderr: "",
+          pid: null,
+          startedAt: new Date().toISOString(),
+        }),
+    };
+    const target: AdapterSandboxExecutionTarget = {
+      kind: "remote",
+      transport: "sandbox",
+      remoteCwd: "/workspace",
+      timeoutMs: 300_000,
+      runner,
+    };
+
+    await ensureAdapterExecutionTargetCommandResolvable(
+      "opencode",
+      target,
+      "/local/workspace",
+      {},
+      { installCommand: "npm install -g opencode", timeoutSec: 1800 },
+    );
+
+    expect(runner.execute).toHaveBeenNthCalledWith(2, expect.objectContaining({
+      command: "sh",
+      args: ["-c", "npm install -g opencode"],
+      timeoutMs: 1_800_000,
+    }));
+  });
+
  it("runs shell commands through the same runner", async () => {
    const runner = {
      execute: vi.fn(async () => ({
@ -363,6 +449,60 @@ describe("sandbox adapter execution targets", () => {
    }
  });

+  it("uses the effective adapter timeout when starting the sandbox callback bridge", async () => {
+    const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-execution-target-bridge-timeout-"));
+    cleanupDirs.push(rootDir);
+    const remoteCwd = path.join(rootDir, "workspace");
+    const runtimeRootDir = path.join(remoteCwd, ".paperclip-runtime", "codex");
+    await mkdir(runtimeRootDir, { recursive: true });
+
+    const delegateRunner = createLocalSandboxRunner();
+    const runner = {
+      execute: vi.fn(async (input: Parameters<typeof delegateRunner.execute>[0]) => delegateRunner.execute(input)),
+    };
+    const apiServer = createServer((req, res) => {
+      res.writeHead(200, { "content-type": "application/json" });
+      res.end(JSON.stringify({ ok: true }));
+    });
+    await new Promise<void>((resolve, reject) => {
+      apiServer.once("error", reject);
+      apiServer.listen(0, "127.0.0.1", () => resolve());
+    });
+    const address = apiServer.address();
+    if (!address || typeof address === "string") {
+      throw new Error("Expected the bridge timeout test API server to listen on a TCP port.");
+    }
+
+    const target: AdapterSandboxExecutionTarget = {
+      kind: "remote",
+      transport: "sandbox",
+      providerKey: "cloudflare",
+      environmentId: "env-1",
+      leaseId: "lease-1",
+      remoteCwd,
+      runner,
+      timeoutMs: 30_000,
+    };
+
+    const bridge = await startAdapterExecutionTargetPaperclipBridge({
+      runId: "run-bridge-timeout",
+      target,
+      runtimeRootDir,
+      adapterKey: "codex",
+      timeoutSec: DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC,
+      hostApiToken: "real-run-jwt",
+      hostApiUrl: `http://127.0.0.1:${address.port}`,
+    });
+    try {
+      expect(bridge).not.toBeNull();
+      expect(runner.execute).toHaveBeenCalled();
+      expect(runner.execute.mock.calls.some(([input]) => input.timeoutMs === 1_800_000)).toBe(true);
+    } finally {
+      await bridge?.stop();
+      await new Promise<void>((resolve) => apiServer.close(() => resolve()));
+    }
+  });
+
  it("fails oversized host responses with a 502 before returning them to the sandbox client", async () => {
    const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-execution-target-bridge-limit-"));
    cleanupDirs.push(rootDir);
--- a/packages/adapter-utils/src/execution-target.ts
+++ b/packages/adapter-utils/src/execution-target.ts
@ -99,6 +99,8 @@ export interface AdapterExecutionTargetPaperclipBridgeHandle {

 export { sanitizeRemoteExecutionEnv } from "./remote-execution-env.js";

+export const DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1_800;
+
 function parseObject(value: unknown): Record<string, unknown> {
  return value && typeof value === "object" && !Array.isArray(value)
    ? (value as Record<string, unknown>)
@ -222,6 +224,26 @@ export function describeAdapterExecutionTarget(
  return `sandbox environment${target.providerKey ? ` (${target.providerKey})` : ""}`;
 }

+export function resolveAdapterExecutionTargetTimeoutSec(
+  target: AdapterExecutionTarget | null | undefined,
+  configuredTimeoutSec: number | null | undefined,
+): number {
+  const normalizedConfiguredTimeoutSec =
+    typeof configuredTimeoutSec === "number" && Number.isFinite(configuredTimeoutSec) && configuredTimeoutSec > 0
+      ? Math.floor(configuredTimeoutSec)
+      : 0;
+  if (normalizedConfiguredTimeoutSec > 0) return normalizedConfiguredTimeoutSec;
+  // Local and SSH adapters preserve the historical "0 means no adapter
+  // timeout" behavior. Sandbox-backed runs execute through provider RPCs
+  // that usually apply their own shorter command defaults, so request an
+  // explicit longer timeout for full adapter runs when the adapter leaves
+  // timeoutSec unset.
+  if (target?.kind === "remote" && target.transport === "sandbox") {
+    return DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC;
+  }
+  return 0;
+}
+
 function requireSandboxRunner(target: AdapterSandboxExecutionTarget): CommandManagedRuntimeRunner {
  if (target.runner) return target.runner;
  throw new Error(
@ -261,10 +283,15 @@ export async function ensureAdapterExecutionTargetCommandResolvable(
  target: AdapterExecutionTarget | null | undefined,
  cwd: string,
  env: NodeJS.ProcessEnv,
-  options: { installCommand?: string | null } = {},
+  options: { installCommand?: string | null; timeoutSec?: number | null } = {},
 ) {
  if (target?.kind === "remote" && target.transport === "sandbox") {
-    await ensureSandboxCommandResolvable(command, target, options.installCommand?.trim() || null);
+    await ensureSandboxCommandResolvable(
+      command,
+      target,
+      options.installCommand?.trim() || null,
+      options.timeoutSec,
+    );
    return;
  }
  await ensureCommandResolvable(command, cwd, env, {
@ -295,6 +322,7 @@ async function ensureSandboxCommandResolvable(
  command: string,
  target: AdapterSandboxExecutionTarget,
  installCommand: string | null,
+  timeoutSec?: number | null,
 ): Promise<void> {
  // Probe whether the binary is resolvable inside the sandbox. We previously
  // short-circuited this for sandbox targets, which let the caller report a
@ -316,12 +344,16 @@ async function ensureSandboxCommandResolvable(
  let installFailureDetail: string | null = null;
  if (installCommand) {
    const runner = requireSandboxRunner(target);
+    const installTimeoutMs =
+      typeof timeoutSec === "number" && Number.isFinite(timeoutSec) && timeoutSec > 0
+        ? Math.floor(timeoutSec * 1000)
+        : target.timeoutMs ?? 300_000;
    try {
      const installResult = await runner.execute({
        command: "sh",
        args: shellCommandArgs(installCommand),
        cwd: target.remoteCwd,
-        timeoutMs: target.timeoutMs ?? 300_000,
+        timeoutMs: installTimeoutMs,
      });
      if (installResult.timedOut) {
        installFailureDetail = `install command timed out: ${installCommand}`;
@ -890,6 +922,7 @@ export async function prepareAdapterExecutionTargetRuntime(input: {
  target: AdapterExecutionTarget | null | undefined;
  adapterKey: string;
  workspaceLocalDir: string;
+  timeoutSec?: number;
  workspaceRemoteDir?: string;
  workspaceExclude?: string[];
  preserveAbsentOnRestore?: string[];
@ -934,7 +967,10 @@ export async function prepareAdapterExecutionTargetRuntime(input: {
      shellCommand: target.shellCommand,
      leaseId: target.leaseId,
      remoteCwd: target.remoteCwd,
-      timeoutMs: target.timeoutMs,
+      timeoutMs:
+        input.timeoutSec && input.timeoutSec > 0
+          ? input.timeoutSec * 1000
+          : target.timeoutMs,
    },
    adapterKey: input.adapterKey,
    workspaceLocalDir: input.workspaceLocalDir,
@ -1017,6 +1053,7 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {
  target: AdapterExecutionTarget | null | undefined;
  runtimeRootDir: string | null | undefined;
  adapterKey: string;
+  timeoutSec?: number | null;
  hostApiToken: string | null | undefined;
  hostApiUrl?: string | null;
  onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
@ -1055,6 +1092,10 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {
    resolveDefaultPaperclipApiUrl();
  const shellCommand = adapterExecutionTargetShellCommand(target);
  const runner = adapterExecutionTargetCommandRunner(target);
+  const bridgeTimeoutMs =
+    typeof input.timeoutSec === "number" && Number.isFinite(input.timeoutSec) && input.timeoutSec > 0
+      ? Math.trunc(input.timeoutSec * 1000)
+      : adapterExecutionTargetTimeoutMs(target);

  await onLog(
    "stdout",
@ -1068,7 +1109,7 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {
    const client = createCommandManagedSandboxCallbackBridgeQueueClient({
      runner,
      remoteCwd: target.remoteCwd,
-      timeoutMs: adapterExecutionTargetTimeoutMs(target),
+      timeoutMs: bridgeTimeoutMs,
      shellCommand,
    });
    // PAPERCLIP_BRIDGE_DEBUG opts into verbose stdout logs of every bridge
@ -1123,7 +1164,7 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {
      queueDir,
      bridgeToken,
      bridgeAsset,
-      timeoutMs: adapterExecutionTargetTimeoutMs(target),
+      timeoutMs: bridgeTimeoutMs,
      maxBodyBytes,
      shellCommand,
    });
--- a/packages/adapter-utils/src/index.ts
+++ b/packages/adapter-utils/src/index.ts
@ -60,6 +60,7 @@ export {
  REDACTED_COMMAND_TEXT_VALUE,
  redactCommandText,
 } from "./command-redaction.js";
+export { buildSandboxNpmInstallCommand } from "./sandbox-install-command.js";
 export { inferOpenAiCompatibleBiller } from "./billing.js";
 // Keep the root adapter-utils entry browser-safe because the UI imports it.
 // The sandbox callback bridge stays available via its dedicated subpath export.
--- a/packages/adapter-utils/src/sandbox-install-command.test.ts
+++ b/packages/adapter-utils/src/sandbox-install-command.test.ts
@ -0,0 +1,14 @@
+import { describe, expect, it } from "vitest";
+import { buildSandboxNpmInstallCommand } from "./sandbox-install-command.js";
+
+describe("buildSandboxNpmInstallCommand", () => {
+  it("installs globally as root, via sudo when available, and under ~/.local otherwise", () => {
+    expect(buildSandboxNpmInstallCommand("@google/gemini-cli")).toBe(
+      'if [ "$(id -u)" -eq 0 ]; then npm install -g \'@google/gemini-cli\'; elif command -v sudo >/dev/null 2>&1 && sudo -n true >/dev/null 2>&1; then sudo -E npm install -g \'@google/gemini-cli\'; else mkdir -p "$HOME/.local" && npm install -g --prefix "$HOME/.local" \'@google/gemini-cli\'; fi',
+    );
+  });
+
+  it("shell-quotes package names", () => {
+    expect(buildSandboxNpmInstallCommand("odd'pkg")).toContain("'odd'\"'\"'pkg'");
+  });
+});
--- a/packages/adapter-utils/src/sandbox-install-command.ts
+++ b/packages/adapter-utils/src/sandbox-install-command.ts
@ -0,0 +1,16 @@
+function shellSingleQuote(value: string): string {
+  return `'${value.replaceAll("'", `'\"'\"'`)}'`;
+}
+
+export function buildSandboxNpmInstallCommand(packageName: string): string {
+  const quotedPackageName = shellSingleQuote(packageName);
+  return [
+    'if [ "$(id -u)" -eq 0 ]; then',
+    `npm install -g ${quotedPackageName};`,
+    'elif command -v sudo >/dev/null 2>&1 && sudo -n true >/dev/null 2>&1; then',
+    `sudo -E npm install -g ${quotedPackageName};`,
+    "else",
+    `mkdir -p "$HOME/.local" && npm install -g --prefix "$HOME/.local" ${quotedPackageName};`,
+    "fi",
+  ].join(" ");
+}