paperclip/packages/adapter-utils/src/workspace-restore-merge.test.ts

85 lines
2.9 KiB
TypeScript
Raw Normal View History

Harden remote workspace sync and restore flows (#5444) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - When an agent runs against a remote target, Paperclip syncs the workspace out to the remote at run start and restores changes back to the local workspace at run end > - The previous restore flow naïvely overwrote local files with whatever the remote returned, so files that the remote run never touched but had timestamp/mode drift could be needlessly rewritten — and a single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH workspace exports race on the same git ref > - This pull request adds a `workspace-restore-merge` module that diffs a pre-run snapshot against the post-run remote state and only writes back files the remote actually changed; SSH workspace exports now use a per-import unique ref so concurrent runs can't trample each other > - Every adapter's execute path threads the snapshot through `prepareAdapterExecutionTargetRuntime` so the merge has the baseline it needs > - The benefit is workspace restores no longer churn untouched files, and concurrent SSH runs no longer collide on the import ref ## What Changed - `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new module — directory snapshot (kind/mode/sha256/symlink target) plus snapshot-aware merge that writes only the files the remote changed - `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`); restore goes through the new merge helper; `ssh-fixture.test.ts` covers the unique-ref + merge paths - `packages/adapter-utils/src/sandbox-managed-runtime.ts` + `remote-managed-runtime.ts`: thread the snapshot/merge through the sandbox and SSH paths - `packages/adapter-utils/src/server-utils.{ts,test.ts}` + `execution-target.ts`: helpers for capturing the pre-run snapshot; `prepareAdapterExecutionTargetRuntime` gains required `runId` and optional `workspaceRemoteDir`, and returns the realized `workspaceRemoteDir` - Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini, opencode, pi) takes the snapshot at run start and passes it through to the runtime restore - Remote execute test mocks updated to match the new `prepareWorkspaceForSshExecution` return shape and the per-run `${managedRemoteWorkspace}` cwd subdirectory ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils --project @paperclipai/adapter-acpx-local --project @paperclipai/adapter-claude-local --project @paperclipai/adapter-codex-local --project @paperclipai/adapter-cursor-local --project @paperclipai/adapter-gemini-local --project @paperclipai/adapter-opencode-local --project @paperclipai/adapter-pi-local` — 196/196 passing - `pnpm typecheck` clean across the workspace ## Risks Medium. The restore path now writes a strict subset of what it previously did — files the remote did not touch are no longer rewritten. If any flow was relying on a touch-without-content-change being copied back (timestamp or permission propagation only), that behavior is now skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at run start; the cost is bounded by the existing exclude list. The `runId` parameter on `prepareAdapterExecutionTargetRuntime` is now required — every in-tree caller is updated; out-of-tree adapter authors need to pass it. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable — new module + every adapter execute path covered - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge
2026-05-07 14:44:45 -07:00
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
fix(remote-sandbox): harden host workspace resumes (#5922) ## Thinking Path > - Paperclip orchestrates AI agents through a control plane while adapters execute work in local, remote, or sandboxed runtimes. > - Remote sandbox execution depends on a strict host-versus-remote workspace boundary: the host prepares/restores files, while the adapter command runs inside the sandbox cwd. > - Jannes' PR #5823 identified host-side failure modes that were not covered by replacement PR #5822. > - Persisting a remote pod cwd in session params could poison the next host heartbeat resume and make Paperclip inspect or upload system temp roots. > - Plugin sandbox providers also need a narrow way to receive model-provider API keys without exposing the full server environment to every plugin worker. > - This pull request ports the host-side fixes from #5823 in the current codebase style, with focused regression coverage. > - The benefit is safer remote sandbox resumes and plugin worker environment handling without broadening core plugin privileges. ## What Changed - Persist host workspace cwd, not remote sandbox cwd, in `claude_local` session params while retaining remote execution identity metadata. - Reject saved session cwds that point at system roots before heartbeat falls back to agent home workspace. - Skip sockets, FIFOs, devices, and other non-file entries during workspace restore snapshot capture/comparison. - Pass a small model-provider API-key allowlist only to plugins declaring `environment.drivers.register`. - Added focused regression tests for remote Claude session params, unsafe session cwd detection, plugin worker env filtering, and non-file snapshot entries. Credits: ports host-side fixes from Jannes' #5823. ## Verification - `pnpm vitest run packages/adapter-utils/src/workspace-restore-merge.test.ts server/src/services/session-workspace-cwd.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/plugin-database.test.ts` (25 passed, 7 skipped by existing embedded-Postgres host guard) - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Low risk: changes are scoped to remote sandbox/session metadata, workspace snapshot filtering, and plugin worker env setup. - Sandbox-provider plugins now receive only the explicit model-provider key allowlist; any provider needing another key name will need a deliberate allowlist update. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-enabled local code execution and repository editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-13 16:23:04 -05:00
import net from "node:net";
Harden remote workspace sync and restore flows (#5444) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - When an agent runs against a remote target, Paperclip syncs the workspace out to the remote at run start and restores changes back to the local workspace at run end > - The previous restore flow naïvely overwrote local files with whatever the remote returned, so files that the remote run never touched but had timestamp/mode drift could be needlessly rewritten — and a single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH workspace exports race on the same git ref > - This pull request adds a `workspace-restore-merge` module that diffs a pre-run snapshot against the post-run remote state and only writes back files the remote actually changed; SSH workspace exports now use a per-import unique ref so concurrent runs can't trample each other > - Every adapter's execute path threads the snapshot through `prepareAdapterExecutionTargetRuntime` so the merge has the baseline it needs > - The benefit is workspace restores no longer churn untouched files, and concurrent SSH runs no longer collide on the import ref ## What Changed - `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new module — directory snapshot (kind/mode/sha256/symlink target) plus snapshot-aware merge that writes only the files the remote changed - `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`); restore goes through the new merge helper; `ssh-fixture.test.ts` covers the unique-ref + merge paths - `packages/adapter-utils/src/sandbox-managed-runtime.ts` + `remote-managed-runtime.ts`: thread the snapshot/merge through the sandbox and SSH paths - `packages/adapter-utils/src/server-utils.{ts,test.ts}` + `execution-target.ts`: helpers for capturing the pre-run snapshot; `prepareAdapterExecutionTargetRuntime` gains required `runId` and optional `workspaceRemoteDir`, and returns the realized `workspaceRemoteDir` - Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini, opencode, pi) takes the snapshot at run start and passes it through to the runtime restore - Remote execute test mocks updated to match the new `prepareWorkspaceForSshExecution` return shape and the per-run `${managedRemoteWorkspace}` cwd subdirectory ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils --project @paperclipai/adapter-acpx-local --project @paperclipai/adapter-claude-local --project @paperclipai/adapter-codex-local --project @paperclipai/adapter-cursor-local --project @paperclipai/adapter-gemini-local --project @paperclipai/adapter-opencode-local --project @paperclipai/adapter-pi-local` — 196/196 passing - `pnpm typecheck` clean across the workspace ## Risks Medium. The restore path now writes a strict subset of what it previously did — files the remote did not touch are no longer rewritten. If any flow was relying on a touch-without-content-change being copied back (timestamp or permission propagation only), that behavior is now skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at run start; the cost is bounded by the existing exclude list. The `runId` parameter on `prepareAdapterExecutionTargetRuntime` is now required — every in-tree caller is updated; out-of-tree adapter authors need to pass it. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable — new module + every adapter execute path covered - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge
2026-05-07 14:44:45 -07:00
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { captureDirectorySnapshot, mergeDirectoryWithBaseline } from "./workspace-restore-merge.js";
describe("workspace restore merge", () => {
const cleanupDirs: string[] = [];
afterEach(async () => {
while (cleanupDirs.length > 0) {
const dir = cleanupDirs.pop();
if (!dir) continue;
await rm(dir, { recursive: true, force: true }).catch(() => undefined);
}
});
it("preserves sibling files when sequential stale-baseline restores create the same nested directory tree", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-restore-merge-"));
cleanupDirs.push(rootDir);
const targetDir = path.join(rootDir, "target");
const sourceADir = path.join(rootDir, "source-a");
const sourceBDir = path.join(rootDir, "source-b");
await mkdir(targetDir, { recursive: true });
await mkdir(path.join(sourceADir, "manual-qa", "environment-matrix", "ssh"), { recursive: true });
await mkdir(path.join(sourceBDir, "manual-qa", "environment-matrix", "ssh"), { recursive: true });
const baseline = await captureDirectorySnapshot(targetDir, { exclude: [] });
await writeFile(
path.join(sourceADir, "manual-qa", "environment-matrix", "ssh", "claude_local.md"),
"ssh claude\n",
"utf8",
);
await writeFile(
path.join(sourceBDir, "manual-qa", "environment-matrix", "ssh", "codex_local.md"),
"ssh codex\n",
"utf8",
);
await mergeDirectoryWithBaseline({
baseline,
sourceDir: sourceADir,
targetDir,
});
await mergeDirectoryWithBaseline({
baseline,
sourceDir: sourceBDir,
targetDir,
});
await expect(
readFile(path.join(targetDir, "manual-qa", "environment-matrix", "ssh", "claude_local.md"), "utf8"),
).resolves.toBe("ssh claude\n");
await expect(
readFile(path.join(targetDir, "manual-qa", "environment-matrix", "ssh", "codex_local.md"), "utf8"),
).resolves.toBe("ssh codex\n");
});
fix(remote-sandbox): harden host workspace resumes (#5922) ## Thinking Path > - Paperclip orchestrates AI agents through a control plane while adapters execute work in local, remote, or sandboxed runtimes. > - Remote sandbox execution depends on a strict host-versus-remote workspace boundary: the host prepares/restores files, while the adapter command runs inside the sandbox cwd. > - Jannes' PR #5823 identified host-side failure modes that were not covered by replacement PR #5822. > - Persisting a remote pod cwd in session params could poison the next host heartbeat resume and make Paperclip inspect or upload system temp roots. > - Plugin sandbox providers also need a narrow way to receive model-provider API keys without exposing the full server environment to every plugin worker. > - This pull request ports the host-side fixes from #5823 in the current codebase style, with focused regression coverage. > - The benefit is safer remote sandbox resumes and plugin worker environment handling without broadening core plugin privileges. ## What Changed - Persist host workspace cwd, not remote sandbox cwd, in `claude_local` session params while retaining remote execution identity metadata. - Reject saved session cwds that point at system roots before heartbeat falls back to agent home workspace. - Skip sockets, FIFOs, devices, and other non-file entries during workspace restore snapshot capture/comparison. - Pass a small model-provider API-key allowlist only to plugins declaring `environment.drivers.register`. - Added focused regression tests for remote Claude session params, unsafe session cwd detection, plugin worker env filtering, and non-file snapshot entries. Credits: ports host-side fixes from Jannes' #5823. ## Verification - `pnpm vitest run packages/adapter-utils/src/workspace-restore-merge.test.ts server/src/services/session-workspace-cwd.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/plugin-database.test.ts` (25 passed, 7 skipped by existing embedded-Postgres host guard) - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Low risk: changes are scoped to remote sandbox/session metadata, workspace snapshot filtering, and plugin worker env setup. - Sandbox-provider plugins now receive only the explicit model-provider key allowlist; any provider needing another key name will need a deliberate allowlist update. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-enabled local code execution and repository editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-13 16:23:04 -05:00
it("ignores non-file entries when capturing snapshots", async () => {
if (process.platform === "win32") return;
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-restore-merge-"));
cleanupDirs.push(rootDir);
const socketPath = path.join(rootDir, "runtime.sock");
const server = net.createServer();
try {
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(socketPath, resolve);
});
const snapshot = await captureDirectorySnapshot(rootDir, { exclude: [] });
expect(snapshot.entries.has("runtime.sock")).toBe(false);
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
Harden remote workspace sync and restore flows (#5444) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - When an agent runs against a remote target, Paperclip syncs the workspace out to the remote at run start and restores changes back to the local workspace at run end > - The previous restore flow naïvely overwrote local files with whatever the remote returned, so files that the remote run never touched but had timestamp/mode drift could be needlessly rewritten — and a single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH workspace exports race on the same git ref > - This pull request adds a `workspace-restore-merge` module that diffs a pre-run snapshot against the post-run remote state and only writes back files the remote actually changed; SSH workspace exports now use a per-import unique ref so concurrent runs can't trample each other > - Every adapter's execute path threads the snapshot through `prepareAdapterExecutionTargetRuntime` so the merge has the baseline it needs > - The benefit is workspace restores no longer churn untouched files, and concurrent SSH runs no longer collide on the import ref ## What Changed - `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new module — directory snapshot (kind/mode/sha256/symlink target) plus snapshot-aware merge that writes only the files the remote changed - `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`); restore goes through the new merge helper; `ssh-fixture.test.ts` covers the unique-ref + merge paths - `packages/adapter-utils/src/sandbox-managed-runtime.ts` + `remote-managed-runtime.ts`: thread the snapshot/merge through the sandbox and SSH paths - `packages/adapter-utils/src/server-utils.{ts,test.ts}` + `execution-target.ts`: helpers for capturing the pre-run snapshot; `prepareAdapterExecutionTargetRuntime` gains required `runId` and optional `workspaceRemoteDir`, and returns the realized `workspaceRemoteDir` - Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini, opencode, pi) takes the snapshot at run start and passes it through to the runtime restore - Remote execute test mocks updated to match the new `prepareWorkspaceForSshExecution` return shape and the per-run `${managedRemoteWorkspace}` cwd subdirectory ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils --project @paperclipai/adapter-acpx-local --project @paperclipai/adapter-claude-local --project @paperclipai/adapter-codex-local --project @paperclipai/adapter-cursor-local --project @paperclipai/adapter-gemini-local --project @paperclipai/adapter-opencode-local --project @paperclipai/adapter-pi-local` — 196/196 passing - `pnpm typecheck` clean across the workspace ## Risks Medium. The restore path now writes a strict subset of what it previously did — files the remote did not touch are no longer rewritten. If any flow was relying on a touch-without-content-change being copied back (timestamp or permission propagation only), that behavior is now skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at run start; the cost is bounded by the existing exclude list. The `runId` parameter on `prepareAdapterExecutionTargetRuntime` is now required — every in-tree caller is updated; out-of-tree adapter authors need to pass it. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable — new module + every adapter execute path covered - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge
2026-05-07 14:44:45 -07:00
});