ake/paperclip
1
0
Fork 0
mirror of https://github.com/alkimake/paperclip.git synced 2026-06-16 10:50:38 +09:00
Code Issues Projects Releases Packages Wiki Activity

Harden remote workspace sync and restore flows (#5444)

Browse source 
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - When an agent runs against a remote target, Paperclip syncs the
workspace out to the remote at run start and restores changes back to
the local workspace at run end
> - The previous restore flow naïvely overwrote local files with
whatever the remote returned, so files that the remote run never touched
but had timestamp/mode drift could be needlessly rewritten — and a
single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH
workspace exports race on the same git ref
> - This pull request adds a `workspace-restore-merge` module that diffs
a pre-run snapshot against the post-run remote state and only writes
back files the remote actually changed; SSH workspace exports now use a
per-import unique ref so concurrent runs can't trample each other
> - Every adapter's execute path threads the snapshot through
`prepareAdapterExecutionTargetRuntime` so the merge has the baseline it
needs
> - The benefit is workspace restores no longer churn untouched files,
and concurrent SSH runs no longer collide on the import ref

## What Changed

- `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new
module — directory snapshot (kind/mode/sha256/symlink target) plus
snapshot-aware merge that writes only the files the remote changed
- `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a
per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`);
restore goes through the new merge helper; `ssh-fixture.test.ts` covers
the unique-ref + merge paths
- `packages/adapter-utils/src/sandbox-managed-runtime.ts` +
`remote-managed-runtime.ts`: thread the snapshot/merge through the
sandbox and SSH paths
- `packages/adapter-utils/src/server-utils.{ts,test.ts}` +
`execution-target.ts`: helpers for capturing the pre-run snapshot;
`prepareAdapterExecutionTargetRuntime` gains required `runId` and
optional `workspaceRemoteDir`, and returns the realized
`workspaceRemoteDir`
- Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini,
opencode, pi) takes the snapshot at run start and passes it through to
the runtime restore
- Remote execute test mocks updated to match the new
`prepareWorkspaceForSshExecution` return shape and the per-run
`${managedRemoteWorkspace}` cwd subdirectory

## Verification

- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-acpx-local --project
@paperclipai/adapter-claude-local --project
@paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project
@paperclipai/adapter-gemini-local --project
@paperclipai/adapter-opencode-local --project
@paperclipai/adapter-pi-local` — 196/196 passing
- `pnpm typecheck` clean across the workspace

## Risks

Medium. The restore path now writes a strict subset of what it
previously did — files the remote did not touch are no longer rewritten.
If any flow was relying on a touch-without-content-change being copied
back (timestamp or permission propagation only), that behavior is now
skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at
run start; the cost is bounded by the existing exclude list. The `runId`
parameter on `prepareAdapterExecutionTargetRuntime` is now required —
every in-tree caller is updated; out-of-tree adapter authors need to
pass it.

## Model Used

Claude Opus 4.7 (1M context)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new module +
every adapter execute path covered
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Devin Foley 2026-05-07 14:44:45 -07:00 • committed by GitHub
parent 824298f414
commit 12cb7b40fd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 1234 additions and 183 deletions
Download patch file Download diff file Expand all files Collapse all files

244
packages/adapter-utils/src/ssh-fixture.test.ts
View file

@ -1,5 +1,5 @@
import { execFile } from "node:child_process";
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
@ -15,6 +15,7 @@ import {
startSshEnvLabFixture,
stopSshEnvLabFixture,
} from "./ssh.js";
import { prepareRemoteManagedRuntime } from "./remote-managed-runtime.js";
async function git(cwd: string, args: string[]): Promise<string> {
return await new Promise((resolve, reject) => {
@ -308,4 +309,245 @@ describe("ssh env-lab fixture", () => {
expect(await git(localRepo, ["status", "--short"])).toContain("M tracked.txt");
expect(await git(localRepo, ["status", "--short"])).not.toContain("._tracked.txt");
});
it("preserves both concurrent SSH restores in a shared git workspace", async () => {
const support = await getSshEnvLabSupport();
if (!support.supported) {
console.warn(
`Skipping concurrent SSH restore test: ${support.reason ?? "unsupported environment"}`,
);
return;
}
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-fixture-"));
cleanupDirs.push(rootDir);
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
await mkdir(localRepo, { recursive: true });
await git(localRepo, ["init", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8");
await git(localRepo, ["add", "tracked.txt"]);
await git(localRepo, ["commit", "-m", "initial"]);
const started = await startSshEnvLabFixture({ statePath });
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const preparedA = await prepareRemoteManagedRuntime({
spec,
runId: "run-a",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
const preparedB = await prepareRemoteManagedRuntime({
spec,
runId: "run-b",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
expect(preparedA.workspaceRemoteDir).not.toBe(preparedB.workspaceRemoteDir);
await runSshCommand(
config,
`sh -lc 'printf "from run a\\n" > ${JSON.stringify(path.posix.join(preparedA.workspaceRemoteDir, "run-a.txt"))}'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await runSshCommand(
config,
`sh -lc 'printf "from run b\\n" > ${JSON.stringify(path.posix.join(preparedB.workspaceRemoteDir, "run-b.txt"))}'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await Promise.all([
preparedA.restoreWorkspace(),
preparedB.restoreWorkspace(),
]);
await expect(readFile(path.join(localRepo, "run-a.txt"), "utf8")).resolves.toBe("from run a\n");
await expect(readFile(path.join(localRepo, "run-b.txt"), "utf8")).resolves.toBe("from run b\n");
});
it("preserves nested per-run files across sequential SSH restores with stale baselines", async () => {
const support = await getSshEnvLabSupport();
if (!support.supported) {
console.warn(
`Skipping sequential nested SSH restore test: ${support.reason ?? "unsupported environment"}`,
);
return;
}
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-fixture-"));
cleanupDirs.push(rootDir);
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
await mkdir(localRepo, { recursive: true });
await git(localRepo, ["init", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8");
await git(localRepo, ["add", "tracked.txt"]);
await git(localRepo, ["commit", "-m", "initial"]);
const started = await startSshEnvLabFixture({ statePath });
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const preparedA = await prepareRemoteManagedRuntime({
spec,
runId: "run-a",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
const preparedB = await prepareRemoteManagedRuntime({
spec,
runId: "run-b",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
await runSshCommand(
config,
`sh -lc 'mkdir -p ${JSON.stringify(path.posix.join(preparedA.workspaceRemoteDir, "manual-qa/environment-matrix/ssh"))} && printf "from run a\\n" > ${JSON.stringify(path.posix.join(preparedA.workspaceRemoteDir, "manual-qa/environment-matrix/ssh/claude_local.md"))}'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await runSshCommand(
config,
`sh -lc 'mkdir -p ${JSON.stringify(path.posix.join(preparedB.workspaceRemoteDir, "manual-qa/environment-matrix/ssh"))} && printf "from run b\\n" > ${JSON.stringify(path.posix.join(preparedB.workspaceRemoteDir, "manual-qa/environment-matrix/ssh/codex_local.md"))}'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await preparedA.restoreWorkspace();
await preparedB.restoreWorkspace();
await expect(readFile(path.join(localRepo, "manual-qa/environment-matrix/ssh/claude_local.md"), "utf8")).resolves
.toBe("from run a\n");
await expect(readFile(path.join(localRepo, "manual-qa/environment-matrix/ssh/codex_local.md"), "utf8")).resolves
.toBe("from run b\n");
});
it("round-trips remote git commits through the managed runtime restore path", async () => {
const support = await getSshEnvLabSupport();
if (!support.supported) {
console.warn(
`Skipping managed-runtime SSH git round-trip test: ${support.reason ?? "unsupported environment"}`,
);
return;
}
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-fixture-"));
cleanupDirs.push(rootDir);
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
await mkdir(localRepo, { recursive: true });
await git(localRepo, ["init", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8");
await git(localRepo, ["add", "tracked.txt"]);
await git(localRepo, ["commit", "-m", "initial"]);
const started = await startSshEnvLabFixture({ statePath });
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const prepared = await prepareRemoteManagedRuntime({
spec,
runId: "run-commit",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
await runSshCommand(
config,
`sh -lc 'cd ${JSON.stringify(prepared.workspaceRemoteDir)} && git config user.name "Paperclip SSH" && git config user.email "ssh@paperclip.dev" && printf "committed\\n" > tracked.txt && git add tracked.txt && git commit -m "remote update" >/dev/null && printf "dirty remote\\n" > tracked.txt'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await prepared.restoreWorkspace();
expect(await git(localRepo, ["log", "-1", "--pretty=%s"])).toBe("remote update");
await expect(readFile(path.join(localRepo, "tracked.txt"), "utf8")).resolves.toBe("dirty remote\n");
});
it("merges concurrent remote commits through the managed runtime restore path", async () => {
const support = await getSshEnvLabSupport();
if (!support.supported) {
console.warn(
`Skipping concurrent managed-runtime SSH git merge test: ${support.reason ?? "unsupported environment"}`,
);
return;
}
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-fixture-"));
cleanupDirs.push(rootDir);
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
await mkdir(localRepo, { recursive: true });
await git(localRepo, ["init", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8");
await git(localRepo, ["add", "tracked.txt"]);
await git(localRepo, ["commit", "-m", "initial"]);
const started = await startSshEnvLabFixture({ statePath });
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const preparedA = await prepareRemoteManagedRuntime({
spec,
runId: "run-commit-a",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
const preparedB = await prepareRemoteManagedRuntime({
spec,
runId: "run-commit-b",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
await runSshCommand(
config,
`sh -lc 'cd ${JSON.stringify(preparedA.workspaceRemoteDir)} && git config user.name "Paperclip SSH" && git config user.email "ssh@paperclip.dev" && printf "from run a\\n" > run-a.txt && git add run-a.txt && git commit -m "remote update a" >/dev/null'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await runSshCommand(
config,
`sh -lc 'cd ${JSON.stringify(preparedB.workspaceRemoteDir)} && git config user.name "Paperclip SSH" && git config user.email "ssh@paperclip.dev" && printf "from run b\\n" > run-b.txt && git add run-b.txt && git commit -m "remote update b" >/dev/null'`,
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await Promise.all([
preparedA.restoreWorkspace(),
preparedB.restoreWorkspace(),
]);
await expect(readFile(path.join(localRepo, "run-a.txt"), "utf8")).resolves.toBe("from run a\n");
await expect(readFile(path.join(localRepo, "run-b.txt"), "utf8")).resolves.toBe("from run b\n");
expect(await git(localRepo, ["log", "-1", "--pretty=%s"])).toContain("Paperclip SSH sync merge");
const recentSubjects = await git(localRepo, ["log", "--pretty=%s", "-3"]);
expect(recentSubjects).toContain("remote update a");
expect(recentSubjects).toContain("remote update b");
});
});