mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-19 20:10:39 +09:00
[codex] Add routine env secrets support (#6212)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Scheduled routines are the control-plane path for recurring agent work. > - Routines already had dispatch/history, but their runtime environment did not carry routine-owned secret bindings through execution. > - Operators need routine-specific secrets that can override project/agent env without exposing secret values in history, logs, or access events. > - This pull request adds the routine env runtime contract, wires it into execution, and makes the routine UI/history surfaces show safe secret metadata. > - The benefit is that routine executions can use scoped secret refs predictably while preserving company boundaries and auditability. ## What Changed - Added routine env persistence/runtime support, including `routines.env`, `routine_runs.routine_revision_id`, revision snapshots, and idempotent migration `0086_routine_env_runtime_contract`. - Resolved routine env during heartbeat adapter config assembly with precedence `agent < project < routine` and secret access events recorded against the routine consumer. - Added secret binding synchronization for routine create/update/restore flows and guarded cross-company, missing, disabled, and deleted secret cases. - Added a Secrets tab to routine detail, env/secret history diff rendering, and Storybook coverage for the new UI states. - Added server/UI regression tests, including an embedded-Postgres QA path for routine secret execution and restore behavior. - Updated implementation/database docs for routine env and secret-binding behavior. ## Verification - `pnpm install --frozen-lockfile` after rebasing onto `public-gh/master` to refresh workspace links for the newly-added upstream Grok adapter package. - `pnpm exec vitest run server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/routines-service.test.ts server/src/__tests__/secrets-service.test.ts server/src/__tests__/qa-routine-secrets-e2e.test.ts ui/src/components/RoutineHistoryTab.test.tsx` passed: 5 files, 92 tests. - `pnpm -r typecheck` passed across the workspace. - `pnpm build` passed. Vite emitted the existing large-chunk/dynamic-import warnings. - UI screenshots were captured locally during QA in `artifacts/pap-9521/` and `artifacts/pap-9522/`; generated screenshots are not committed to avoid adding binary artifacts to the repo. ## Risks - Migration risk is limited by `IF NOT EXISTS` guards for the new columns, FK, and index, and the migration is ordered as `0086` immediately after upstream `0085`. - Runtime behavior changes env precedence for routine executions by adding routine env as the highest-precedence layer; tests cover agent/project/routine precedence. - Secret handling is security-sensitive; tests cover value-free manifests/events/errors, disabled/missing/deleted secrets, and cross-company rejection. - UI history now renders routine env/secret diffs; tests and Storybook stories cover the main rendering paths. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, with shell/tool use and medium reasoning effort. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
3e6610fb93
commit
705c1b8d81
20 changed files with 1736 additions and 50 deletions
|
|
@ -7,7 +7,7 @@ import {
|
|||
} from "../services/heartbeat.ts";
|
||||
|
||||
describe("resolveExecutionRunAdapterConfig", () => {
|
||||
it("overlays project env on top of agent env and unions secret keys", async () => {
|
||||
it("overlays project and routine env on top of agent env and unions secret keys", async () => {
|
||||
const resolveAdapterConfigForRuntime = vi.fn().mockResolvedValue({
|
||||
config: {
|
||||
env: {
|
||||
|
|
@ -29,29 +29,51 @@ describe("resolveExecutionRunAdapterConfig", () => {
|
|||
},
|
||||
],
|
||||
});
|
||||
const resolveEnvBindings = vi.fn().mockResolvedValue({
|
||||
env: {
|
||||
SHARED_KEY: "project",
|
||||
PROJECT_ONLY: "project-only",
|
||||
},
|
||||
secretKeys: new Set(["PROJECT_SECRET"]),
|
||||
manifest: [
|
||||
{
|
||||
configPath: "env.PROJECT_SECRET",
|
||||
envKey: "PROJECT_SECRET",
|
||||
secretId: "secret-project",
|
||||
secretKey: "project-secret",
|
||||
version: 1,
|
||||
provider: "local_encrypted",
|
||||
outcome: "success",
|
||||
const resolveEnvBindings = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
env: {
|
||||
SHARED_KEY: "project",
|
||||
PROJECT_ONLY: "project-only",
|
||||
},
|
||||
],
|
||||
});
|
||||
secretKeys: new Set(["PROJECT_SECRET"]),
|
||||
manifest: [
|
||||
{
|
||||
configPath: "env.PROJECT_SECRET",
|
||||
envKey: "PROJECT_SECRET",
|
||||
secretId: "secret-project",
|
||||
secretKey: "project-secret",
|
||||
version: 1,
|
||||
provider: "local_encrypted",
|
||||
outcome: "success",
|
||||
},
|
||||
],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
env: {
|
||||
SHARED_KEY: "routine",
|
||||
ROUTINE_ONLY: "routine-only",
|
||||
},
|
||||
secretKeys: new Set(["ROUTINE_SECRET"]),
|
||||
manifest: [
|
||||
{
|
||||
configPath: "env.ROUTINE_SECRET",
|
||||
envKey: "ROUTINE_SECRET",
|
||||
secretId: "secret-routine",
|
||||
secretKey: "routine-secret",
|
||||
version: 1,
|
||||
provider: "local_encrypted",
|
||||
outcome: "success",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const result = await resolveExecutionRunAdapterConfig({
|
||||
companyId: "company-1",
|
||||
executionRunConfig: { env: { SHARED_KEY: "agent" } },
|
||||
projectEnv: { SHARED_KEY: "project" },
|
||||
routineEnv: { SHARED_KEY: "routine" },
|
||||
routineId: "routine-1",
|
||||
secretsSvc: {
|
||||
resolveAdapterConfigForRuntime,
|
||||
resolveEnvBindings,
|
||||
|
|
@ -61,18 +83,88 @@ describe("resolveExecutionRunAdapterConfig", () => {
|
|||
expect(result.resolvedConfig).toMatchObject({
|
||||
other: "value",
|
||||
env: {
|
||||
SHARED_KEY: "project",
|
||||
SHARED_KEY: "routine",
|
||||
AGENT_ONLY: "agent-only",
|
||||
PROJECT_ONLY: "project-only",
|
||||
ROUTINE_ONLY: "routine-only",
|
||||
},
|
||||
});
|
||||
expect(Array.from(result.secretKeys).sort()).toEqual(["AGENT_SECRET", "PROJECT_SECRET"]);
|
||||
expect(Array.from(result.secretKeys).sort()).toEqual(["AGENT_SECRET", "PROJECT_SECRET", "ROUTINE_SECRET"]);
|
||||
expect(result.secretManifest.map((entry) => entry.secretId).sort()).toEqual([
|
||||
"secret-agent",
|
||||
"secret-project",
|
||||
"secret-routine",
|
||||
]);
|
||||
expect(JSON.stringify(result.secretManifest)).not.toContain("agent-only");
|
||||
expect(JSON.stringify(result.secretManifest)).not.toContain("project-only");
|
||||
expect(JSON.stringify(result.secretManifest)).not.toContain("routine-only");
|
||||
expect(resolveEnvBindings.mock.calls[1]?.[2]).toMatchObject({
|
||||
consumerType: "routine",
|
||||
consumerId: "routine-1",
|
||||
});
|
||||
});
|
||||
|
||||
it("drops Paperclip runtime-owned env before resolving agent, project, and routine overlays", async () => {
|
||||
const resolveAdapterConfigForRuntime = vi.fn(async (_companyId, config: Record<string, unknown>) => ({
|
||||
config: {
|
||||
...config,
|
||||
env: { ...(config.env as Record<string, unknown>) },
|
||||
},
|
||||
secretKeys: new Set<string>(),
|
||||
manifest: [],
|
||||
}));
|
||||
const resolveEnvBindings = vi.fn(async (_companyId, env: Record<string, unknown>) => ({
|
||||
env: Object.fromEntries(
|
||||
Object.entries(env).filter((entry): entry is [string, string] => typeof entry[1] === "string"),
|
||||
),
|
||||
secretKeys: new Set<string>(),
|
||||
manifest: [],
|
||||
}));
|
||||
|
||||
const result = await resolveExecutionRunAdapterConfig({
|
||||
companyId: "company-1",
|
||||
agentId: "agent-1",
|
||||
executionRunConfig: {
|
||||
env: {
|
||||
PAPERCLIP_API_KEY: { type: "secret_ref", secretId: "secret-api-key", version: "latest" },
|
||||
PAPERCLIP_AGENT_ID: "spoofed-agent",
|
||||
AGENT_ONLY: "agent-only",
|
||||
},
|
||||
},
|
||||
projectEnv: {
|
||||
PAPERCLIP_API_KEY: "project-api-key",
|
||||
PAPERCLIP_COMPANY_ID: "spoofed-company",
|
||||
PROJECT_ONLY: "project-only",
|
||||
},
|
||||
routineEnv: {
|
||||
PAPERCLIP_API_KEY: "routine-api-key",
|
||||
PAPERCLIP_RUN_ID: "spoofed-run",
|
||||
ROUTINE_ONLY: "routine-only",
|
||||
},
|
||||
routineId: "routine-1",
|
||||
secretsSvc: {
|
||||
resolveAdapterConfigForRuntime,
|
||||
resolveEnvBindings,
|
||||
} as any,
|
||||
});
|
||||
|
||||
expect(resolveAdapterConfigForRuntime.mock.calls[0]?.[1]).toEqual({
|
||||
env: {
|
||||
AGENT_ONLY: "agent-only",
|
||||
},
|
||||
});
|
||||
expect(resolveEnvBindings.mock.calls[0]?.[1]).toEqual({
|
||||
PROJECT_ONLY: "project-only",
|
||||
});
|
||||
expect(resolveEnvBindings.mock.calls[1]?.[1]).toEqual({
|
||||
ROUTINE_ONLY: "routine-only",
|
||||
});
|
||||
expect(result.resolvedConfig.env).toEqual({
|
||||
AGENT_ONLY: "agent-only",
|
||||
PROJECT_ONLY: "project-only",
|
||||
ROUTINE_ONLY: "routine-only",
|
||||
});
|
||||
expect(JSON.stringify(result.resolvedConfig.env)).not.toContain("PAPERCLIP_");
|
||||
});
|
||||
|
||||
it("skips project env resolution when the project has no bindings", async () => {
|
||||
|
|
|
|||
458
server/src/__tests__/qa-routine-secrets-e2e.test.ts
Normal file
458
server/src/__tests__/qa-routine-secrets-e2e.test.ts
Normal file
|
|
@ -0,0 +1,458 @@
|
|||
// QA validation for [PAP-9522](/PAP/issues/PAP-9522). Drives the routine-secret
|
||||
// chain end-to-end against a real embedded Postgres:
|
||||
//
|
||||
// 1. Routine env reaches the heartbeat runtime via `resolveExecutionRunAdapterConfig`
|
||||
// using `secretsSvc.resolveEnvBindings` with a `consumerType: "routine"` context,
|
||||
// even when the executing agent has zero direct bindings for that secret.
|
||||
// 2. Precedence: agent < project < routine for a shared key.
|
||||
// 3. `secret_access_events` records routine consumption but NEVER the resolved value.
|
||||
// 4. Restoring an older revision re-syncs `company_secret_bindings` to the snapshot env.
|
||||
// 5. Legacy fallback: a routine_run with null `routine_revision_id` still resolves
|
||||
// the routine's current env (matches the explicit acceptance criterion).
|
||||
// 6. Disabled / missing / cross-company secret bindings fail clearly without
|
||||
// echoing the value.
|
||||
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdirSync, rmSync } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { eq, and } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
agents,
|
||||
companies,
|
||||
companySecretBindings,
|
||||
companySecrets,
|
||||
companySecretVersions,
|
||||
createDb,
|
||||
projects,
|
||||
routineRuns,
|
||||
routines,
|
||||
secretAccessEvents,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { routineService } from "../services/routines.ts";
|
||||
import { secretService } from "../services/secrets.ts";
|
||||
import { resolveExecutionRunAdapterConfig } from "../services/heartbeat.ts";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbedded = support.supported ? describe : describe.skip;
|
||||
if (!support.supported) {
|
||||
console.warn(`Skipping QA e2e on this host: ${support.reason ?? "embedded pg unsupported"}`);
|
||||
}
|
||||
|
||||
describeEmbedded("PAP-9522 QA: routine secrets end-to-end", () => {
|
||||
let db!: ReturnType<typeof createDb>;
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||||
const secretsTmpDir = path.join(os.tmpdir(), `paperclip-qa-routine-secrets-${randomUUID()}`);
|
||||
const previousKeyFile = process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE;
|
||||
|
||||
beforeAll(async () => {
|
||||
mkdirSync(secretsTmpDir, { recursive: true });
|
||||
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = path.join(secretsTmpDir, "master.key");
|
||||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-qa-routine-secrets-");
|
||||
db = createDb(tempDb.connectionString);
|
||||
}, 30_000);
|
||||
|
||||
afterEach(async () => {
|
||||
await db.delete(secretAccessEvents);
|
||||
await db.delete(companySecretBindings);
|
||||
await db.delete(routineRuns);
|
||||
await db.delete(routines);
|
||||
await db.delete(companySecretVersions);
|
||||
await db.delete(companySecrets);
|
||||
await db.delete(projects);
|
||||
await db.delete(agents);
|
||||
await db.delete(companies);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await tempDb?.cleanup();
|
||||
if (previousKeyFile === undefined) delete process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE;
|
||||
else process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = previousKeyFile;
|
||||
rmSync(secretsTmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function seed() {
|
||||
const companyId = randomUUID();
|
||||
const executorAgentId = randomUUID();
|
||||
const issuePrefix = `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`;
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "QA Co",
|
||||
issuePrefix,
|
||||
requireBoardApprovalForNewAgents: false,
|
||||
});
|
||||
// Note: executor agent has NO secret bindings of its own — this is the
|
||||
// whole point of routine env (the secret rides with the routine, not the agent).
|
||||
await db.insert(agents).values({
|
||||
id: executorAgentId,
|
||||
companyId,
|
||||
name: "Executor",
|
||||
role: "engineer",
|
||||
status: "active",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: { env: {} },
|
||||
runtimeConfig: {},
|
||||
permissions: {},
|
||||
});
|
||||
return { companyId, executorAgentId };
|
||||
}
|
||||
|
||||
const ROUTINE_VALUE = "super-sekret-routine-value";
|
||||
const PROJECT_VALUE = "project-overlay-value";
|
||||
const AGENT_VALUE = "agent-base-value";
|
||||
|
||||
it("resolves routine env for an executing agent that has no direct binding, with routine winning precedence and zero value in access events", async () => {
|
||||
const { companyId, executorAgentId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const secret = await secrets.create(companyId, {
|
||||
name: `routine-api-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: ROUTINE_VALUE,
|
||||
});
|
||||
|
||||
const routine = await routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "qa routine",
|
||||
description: null,
|
||||
assigneeAgentId: executorAgentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
SHARED: { type: "plain", value: "routine-overrides" },
|
||||
ROUTINE_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
// Verify binding is owned by the routine, not the executing agent.
|
||||
const bindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(bindings).toMatchObject([
|
||||
{ targetType: "routine", secretId: secret.id, configPath: "env.ROUTINE_API_KEY" },
|
||||
]);
|
||||
|
||||
// Drive the real heartbeat resolution path with the routine env.
|
||||
// issueId/heartbeatRunId left null because secret_access_events has FK
|
||||
// constraints on both — populating them would require seeding issue and
|
||||
// heartbeat_run rows just for FK validity. The routine consumer fields are
|
||||
// what this test cares about.
|
||||
const result = await resolveExecutionRunAdapterConfig({
|
||||
companyId,
|
||||
agentId: executorAgentId,
|
||||
issueId: null,
|
||||
heartbeatRunId: null,
|
||||
projectId: null,
|
||||
routineId: routine.id,
|
||||
executionRunConfig: { env: { SHARED: AGENT_VALUE, AGENT_ONLY: AGENT_VALUE } },
|
||||
projectEnv: { SHARED: { type: "plain", value: PROJECT_VALUE } },
|
||||
routineEnv: routine.env,
|
||||
secretsSvc: secrets,
|
||||
});
|
||||
|
||||
expect(result.resolvedConfig.env).toMatchObject({
|
||||
AGENT_ONLY: AGENT_VALUE,
|
||||
SHARED: "routine-overrides", // routine beats project beats agent
|
||||
ROUTINE_API_KEY: ROUTINE_VALUE,
|
||||
});
|
||||
expect(result.secretKeys.has("ROUTINE_API_KEY")).toBe(true);
|
||||
expect(result.secretManifest.some((m) => m.envKey === "ROUTINE_API_KEY")).toBe(true);
|
||||
// Manifest must not echo the resolved value.
|
||||
expect(JSON.stringify(result.secretManifest)).not.toContain(ROUTINE_VALUE);
|
||||
|
||||
const events = await db
|
||||
.select()
|
||||
.from(secretAccessEvents)
|
||||
.where(eq(secretAccessEvents.secretId, secret.id));
|
||||
expect(events).toHaveLength(1);
|
||||
expect(events[0]).toMatchObject({
|
||||
consumerType: "routine",
|
||||
consumerId: routine.id,
|
||||
actorType: "agent",
|
||||
actorId: executorAgentId,
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
outcome: "success",
|
||||
});
|
||||
// No serialized field of the access event row can contain the secret value.
|
||||
expect(JSON.stringify(events[0])).not.toContain(ROUTINE_VALUE);
|
||||
});
|
||||
|
||||
it("rejects routine env that references a secret from a different company", async () => {
|
||||
const { companyId } = await seed();
|
||||
const { companyId: otherCompanyId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const foreignSecret = await secrets.create(otherCompanyId, {
|
||||
name: `foreign-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "cross-company-leak-bait",
|
||||
});
|
||||
|
||||
await expect(
|
||||
routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "cross company",
|
||||
description: null,
|
||||
assigneeAgentId: null,
|
||||
priority: "medium",
|
||||
status: "paused",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
BAD: { type: "secret_ref", secretId: foreignSecret.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
),
|
||||
).rejects.toThrow(/same company/i);
|
||||
});
|
||||
|
||||
it("surfaces a clear, value-free error when a routine secret is missing/deleted at resolution time", async () => {
|
||||
const { companyId, executorAgentId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const secret = await secrets.create(companyId, {
|
||||
name: `to-be-deleted-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "doomed-secret-value",
|
||||
});
|
||||
|
||||
const routine = await routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "doomed routine",
|
||||
description: null,
|
||||
assigneeAgentId: executorAgentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
DOOMED: { type: "secret_ref", secretId: secret.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
// Hard delete the secret out from under the routine; the routine env now
|
||||
// points at a vanished id.
|
||||
await secrets.remove(secret.id);
|
||||
|
||||
let caught: unknown = null;
|
||||
try {
|
||||
await resolveExecutionRunAdapterConfig({
|
||||
companyId,
|
||||
agentId: executorAgentId,
|
||||
issueId: null,
|
||||
heartbeatRunId: null,
|
||||
projectId: null,
|
||||
routineId: routine.id,
|
||||
executionRunConfig: { env: {} },
|
||||
projectEnv: null,
|
||||
routineEnv: routine.env,
|
||||
secretsSvc: secrets,
|
||||
});
|
||||
} catch (error) {
|
||||
caught = error;
|
||||
}
|
||||
expect(caught).toBeTruthy();
|
||||
const message = String((caught as Error)?.message ?? caught);
|
||||
expect(message).not.toContain("doomed-secret-value");
|
||||
});
|
||||
|
||||
it("restoring an older revision re-syncs company_secret_bindings to the snapshot env", async () => {
|
||||
const { companyId, executorAgentId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const secretA = await secrets.create(companyId, {
|
||||
name: `a-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "val-a",
|
||||
});
|
||||
const secretB = await secrets.create(companyId, {
|
||||
name: `b-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "val-b",
|
||||
});
|
||||
|
||||
const routine = await routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "restore routine",
|
||||
description: null,
|
||||
assigneeAgentId: executorAgentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
ALPHA: { type: "secret_ref", secretId: secretA.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
const rev1Id = routine.latestRevisionId!;
|
||||
|
||||
await routines.update(
|
||||
routine.id,
|
||||
{
|
||||
env: {
|
||||
ALPHA: { type: "secret_ref", secretId: secretA.id, version: "latest" },
|
||||
BETA: { type: "secret_ref", secretId: secretB.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
let bindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(bindings.map((b) => b.configPath).sort()).toEqual(["env.ALPHA", "env.BETA"]);
|
||||
|
||||
await routines.restoreRevision(routine.id, rev1Id, {});
|
||||
|
||||
bindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(bindings.map((b) => b.configPath)).toEqual(["env.ALPHA"]);
|
||||
expect(bindings[0]?.secretId).toBe(secretA.id);
|
||||
});
|
||||
|
||||
it("legacy run with null routine_revision_id falls back to the routine's current env (still resolves)", async () => {
|
||||
const { companyId, executorAgentId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const secret = await secrets.create(companyId, {
|
||||
name: `legacy-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "legacy-value",
|
||||
});
|
||||
|
||||
const routine = await routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "legacy routine",
|
||||
description: null,
|
||||
assigneeAgentId: executorAgentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
LEGACY: { type: "secret_ref", secretId: secret.id, version: "latest" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
// Simulate an old routine_run row (predating the migration) with no
|
||||
// routine_revision_id. The fallback path in `getRoutineEnvForExecutionIssue`
|
||||
// should still resolve to the routine's current env. Here we exercise the
|
||||
// resolution layer directly with routine.env to mirror that behavior.
|
||||
await db.insert(routineRuns).values({
|
||||
id: randomUUID(),
|
||||
companyId,
|
||||
routineId: routine.id,
|
||||
triggerId: null,
|
||||
source: "manual",
|
||||
status: "issue_created",
|
||||
triggeredAt: new Date(),
|
||||
completedAt: new Date(),
|
||||
routineRevisionId: null,
|
||||
});
|
||||
|
||||
const result = await resolveExecutionRunAdapterConfig({
|
||||
companyId,
|
||||
agentId: executorAgentId,
|
||||
issueId: null,
|
||||
heartbeatRunId: null,
|
||||
projectId: null,
|
||||
routineId: routine.id,
|
||||
executionRunConfig: { env: {} },
|
||||
projectEnv: null,
|
||||
routineEnv: routine.env,
|
||||
secretsSvc: secrets,
|
||||
});
|
||||
expect(result.resolvedConfig.env).toMatchObject({ LEGACY: "legacy-value" });
|
||||
});
|
||||
|
||||
it("routines created with null env (no Secrets tab interaction) still resolve normally with empty env", async () => {
|
||||
const { companyId, executorAgentId } = await seed();
|
||||
const secrets = secretService(db);
|
||||
const routines = routineService(db, { heartbeat: { wakeup: async () => null } });
|
||||
|
||||
const routine = await routines.create(
|
||||
companyId,
|
||||
{
|
||||
projectId: null,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "null env routine",
|
||||
description: null,
|
||||
assigneeAgentId: executorAgentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "coalesce_if_active",
|
||||
catchUpPolicy: "skip_missed",
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
expect(routine.env ?? null).toBeNull();
|
||||
|
||||
const bindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(bindings).toHaveLength(0);
|
||||
|
||||
const result = await resolveExecutionRunAdapterConfig({
|
||||
companyId,
|
||||
agentId: executorAgentId,
|
||||
issueId: null,
|
||||
heartbeatRunId: null,
|
||||
projectId: null,
|
||||
routineId: routine.id,
|
||||
executionRunConfig: { env: { AGENT_ONLY: "agent" } },
|
||||
projectEnv: null,
|
||||
routineEnv: null,
|
||||
secretsSvc: secrets,
|
||||
});
|
||||
expect(result.resolvedConfig.env).toEqual({ AGENT_ONLY: "agent" });
|
||||
expect(result.secretKeys.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
|
@ -5,6 +5,7 @@ import {
|
|||
activityLog,
|
||||
agents,
|
||||
companies,
|
||||
companySecretBindings,
|
||||
companySecrets,
|
||||
companySecretVersions,
|
||||
createDb,
|
||||
|
|
@ -19,6 +20,7 @@ import {
|
|||
routineRuns,
|
||||
routines,
|
||||
routineTriggers,
|
||||
secretAccessEvents,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
|
|
@ -28,6 +30,7 @@ import { issueService } from "../services/issues.ts";
|
|||
import { instanceSettingsService } from "../services/instance-settings.ts";
|
||||
import * as providerRegistry from "../secrets/provider-registry.ts";
|
||||
import { routineService } from "../services/routines.ts";
|
||||
import { secretService } from "../services/secrets.ts";
|
||||
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||||
|
|
@ -57,6 +60,8 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
|
|||
await db.delete(activityLog);
|
||||
await db.delete(issueInboxArchives);
|
||||
await db.delete(issueReadStates);
|
||||
await db.delete(secretAccessEvents);
|
||||
await db.delete(companySecretBindings);
|
||||
await db.delete(routineRuns);
|
||||
await db.delete(routineTriggers);
|
||||
await db.delete(routines);
|
||||
|
|
@ -331,6 +336,89 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
|
|||
expect(revisions[1]?.snapshot.routine.description).toBe("Run the frog routine");
|
||||
});
|
||||
|
||||
it("stores routine env in revisions, syncs routine secret bindings, and stamps runs with the dispatch revision", async () => {
|
||||
const { agentId, companyId, projectId, svc } = await seedFixture();
|
||||
const secrets = secretService(db);
|
||||
const secret = await secrets.create(companyId, {
|
||||
name: `routine-api-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "secret-value",
|
||||
});
|
||||
|
||||
const routine = await svc.create(
|
||||
companyId,
|
||||
{
|
||||
projectId,
|
||||
goalId: null,
|
||||
parentIssueId: null,
|
||||
title: "secret routine",
|
||||
description: null,
|
||||
assigneeAgentId: agentId,
|
||||
priority: "medium",
|
||||
status: "active",
|
||||
concurrencyPolicy: "always_enqueue",
|
||||
catchUpPolicy: "skip_missed",
|
||||
env: {
|
||||
ROUTINE_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
|
||||
ROUTINE_PLAIN: { type: "plain", value: "plain-value" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
const bindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(bindings).toMatchObject([
|
||||
{
|
||||
companyId,
|
||||
secretId: secret.id,
|
||||
targetType: "routine",
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
},
|
||||
]);
|
||||
|
||||
const [initialRevision] = await svc.listRevisions(routine.id);
|
||||
expect(initialRevision?.snapshot.routine.env).toEqual(routine.env);
|
||||
|
||||
await db.delete(companySecretBindings).where(eq(companySecretBindings.targetId, routine.id));
|
||||
const repaired = await svc.update(routine.id, { env: routine.env }, {});
|
||||
expect(repaired).not.toBeNull();
|
||||
const repairedBindings = await db
|
||||
.select()
|
||||
.from(companySecretBindings)
|
||||
.where(eq(companySecretBindings.targetId, routine.id));
|
||||
expect(repairedBindings).toMatchObject([
|
||||
{
|
||||
companyId,
|
||||
secretId: secret.id,
|
||||
targetType: "routine",
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
},
|
||||
]);
|
||||
|
||||
const currentRoutine = repaired ?? routine;
|
||||
const runBefore = await svc.runRoutine(routine.id, { source: "manual" });
|
||||
expect(runBefore.routineRevisionId).toBe(currentRoutine.latestRevisionId);
|
||||
|
||||
const updated = await svc.update(
|
||||
routine.id,
|
||||
{
|
||||
env: {
|
||||
ROUTINE_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
|
||||
ROUTINE_PLAIN: { type: "plain", value: "changed" },
|
||||
},
|
||||
},
|
||||
{},
|
||||
);
|
||||
expect(updated?.latestRevisionNumber).toBe(currentRoutine.latestRevisionNumber + 1);
|
||||
|
||||
const runAfter = await svc.runRoutine(routine.id, { source: "manual" });
|
||||
expect(runAfter.routineRevisionId).toBe(updated?.latestRevisionId);
|
||||
expect(runAfter.dispatchFingerprint).not.toBe(runBefore.dispatchFingerprint);
|
||||
});
|
||||
|
||||
it("rejects stale routine baseRevisionId updates", async () => {
|
||||
const { routine, svc } = await seedFixture();
|
||||
const updated = await svc.update(routine.id, { description: "new description" }, {});
|
||||
|
|
|
|||
|
|
@ -205,6 +205,116 @@ describeEmbeddedPostgres("secretService", () => {
|
|||
expect(JSON.stringify(events)).not.toContain("runtime-secret");
|
||||
});
|
||||
|
||||
it("resolves routine env secret refs through routine bindings and records value-free access metadata", async () => {
|
||||
const companyId = await seedCompany();
|
||||
const svc = secretService(db);
|
||||
const secret = await svc.create(companyId, {
|
||||
name: `routine-secret-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "routine-super-secret",
|
||||
});
|
||||
const env = {
|
||||
ROUTINE_API_KEY: { type: "secret_ref" as const, secretId: secret.id, version: "latest" as const },
|
||||
};
|
||||
await svc.syncEnvBindingsForTarget(companyId, { targetType: "routine", targetId: "routine-1" }, env);
|
||||
|
||||
const resolved = await svc.resolveEnvBindings(companyId, env, {
|
||||
consumerType: "routine",
|
||||
consumerId: "routine-1",
|
||||
actorType: "agent",
|
||||
actorId: "agent-1",
|
||||
});
|
||||
|
||||
expect(resolved.env.ROUTINE_API_KEY).toBe("routine-super-secret");
|
||||
expect(resolved.manifest).toEqual([
|
||||
expect.objectContaining({
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
envKey: "ROUTINE_API_KEY",
|
||||
secretId: secret.id,
|
||||
outcome: "success",
|
||||
}),
|
||||
]);
|
||||
|
||||
const events = await svc.listAccessEvents(companyId, secret.id);
|
||||
expect(events).toHaveLength(1);
|
||||
expect(events[0]).toMatchObject({
|
||||
companyId,
|
||||
secretId: secret.id,
|
||||
consumerType: "routine",
|
||||
consumerId: "routine-1",
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
actorType: "agent",
|
||||
actorId: "agent-1",
|
||||
outcome: "success",
|
||||
});
|
||||
expect(JSON.stringify(events)).not.toContain("routine-super-secret");
|
||||
});
|
||||
|
||||
it("records stable redacted failure codes for routine env secret resolution", async () => {
|
||||
const companyId = await seedCompany();
|
||||
const svc = secretService(db);
|
||||
const secret = await svc.create(companyId, {
|
||||
name: `routine-failure-codes-${randomUUID()}`,
|
||||
provider: "local_encrypted",
|
||||
value: "routine-super-secret",
|
||||
});
|
||||
const env = {
|
||||
ROUTINE_API_KEY: { type: "secret_ref" as const, secretId: secret.id, version: "latest" as const },
|
||||
};
|
||||
const context = {
|
||||
consumerType: "routine" as const,
|
||||
consumerId: "routine-1",
|
||||
actorType: "agent" as const,
|
||||
actorId: "agent-1",
|
||||
};
|
||||
await svc.syncEnvBindingsForTarget(companyId, { targetType: "routine", targetId: "routine-1" }, env);
|
||||
|
||||
await expect(
|
||||
svc.resolveEnvBindings(companyId, env, { ...context, consumerId: "routine-2" }),
|
||||
).rejects.toThrow(/not bound/i);
|
||||
|
||||
await db.update(companySecrets).set({ status: "disabled" }).where(eq(companySecrets.id, secret.id));
|
||||
await expect(svc.resolveEnvBindings(companyId, env, context)).rejects.toThrow(/not active/i);
|
||||
|
||||
await db.update(companySecrets).set({ status: "active" }).where(eq(companySecrets.id, secret.id));
|
||||
await expect(
|
||||
svc.resolveSecretValue(companyId, secret.id, 999, {
|
||||
...context,
|
||||
configPath: "env.ROUTINE_API_KEY",
|
||||
}),
|
||||
).rejects.toThrow(/version not found/i);
|
||||
|
||||
await db
|
||||
.update(companySecretVersions)
|
||||
.set({ status: "disabled" })
|
||||
.where(eq(companySecretVersions.secretId, secret.id));
|
||||
await expect(svc.resolveEnvBindings(companyId, env, context)).rejects.toThrow(/version is not active/i);
|
||||
|
||||
await db
|
||||
.update(companySecretVersions)
|
||||
.set({ status: "current" })
|
||||
.where(eq(companySecretVersions.secretId, secret.id));
|
||||
vi.spyOn(localEncryptedProvider, "resolveVersion").mockRejectedValueOnce(
|
||||
new Error("provider leaked value routine-super-secret"),
|
||||
);
|
||||
await expect(svc.resolveEnvBindings(companyId, env, context)).rejects.toThrow(/provider leaked value/i);
|
||||
|
||||
await db.update(companySecrets).set({ status: "deleted" }).where(eq(companySecrets.id, secret.id));
|
||||
await expect(svc.resolveEnvBindings(companyId, env, context)).rejects.toThrow(/not found/i);
|
||||
|
||||
const events = await svc.listAccessEvents(companyId, secret.id);
|
||||
expect(events.map((event) => event.errorCode).sort()).toEqual([
|
||||
"binding_missing",
|
||||
"provider_error",
|
||||
"secret_deleted",
|
||||
"secret_inactive",
|
||||
"version_inactive",
|
||||
"version_missing",
|
||||
]);
|
||||
expect(JSON.stringify(events)).not.toContain("routine-super-secret");
|
||||
expect(JSON.stringify(events)).not.toContain("provider leaked value");
|
||||
});
|
||||
|
||||
it("scopes env binding sync deletes to the env path prefix", async () => {
|
||||
const companyId = await seedCompany();
|
||||
const svc = secretService(db);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue