[codex] Add routine env secrets support (#6212)

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - Scheduled routines are the control-plane path for recurring agent
work.
> - Routines already had dispatch/history, but their runtime environment
did not carry routine-owned secret bindings through execution.
> - Operators need routine-specific secrets that can override
project/agent env without exposing secret values in history, logs, or
access events.
> - This pull request adds the routine env runtime contract, wires it
into execution, and makes the routine UI/history surfaces show safe
secret metadata.
> - The benefit is that routine executions can use scoped secret refs
predictably while preserving company boundaries and auditability.

## What Changed

- Added routine env persistence/runtime support, including
`routines.env`, `routine_runs.routine_revision_id`, revision snapshots,
and idempotent migration `0086_routine_env_runtime_contract`.
- Resolved routine env during heartbeat adapter config assembly with
precedence `agent < project < routine` and secret access events recorded
against the routine consumer.
- Added secret binding synchronization for routine create/update/restore
flows and guarded cross-company, missing, disabled, and deleted secret
cases.
- Added a Secrets tab to routine detail, env/secret history diff
rendering, and Storybook coverage for the new UI states.
- Added server/UI regression tests, including an embedded-Postgres QA
path for routine secret execution and restore behavior.
- Updated implementation/database docs for routine env and
secret-binding behavior.

## Verification

- `pnpm install --frozen-lockfile` after rebasing onto
`public-gh/master` to refresh workspace links for the newly-added
upstream Grok adapter package.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/routines-service.test.ts
server/src/__tests__/secrets-service.test.ts
server/src/__tests__/qa-routine-secrets-e2e.test.ts
ui/src/components/RoutineHistoryTab.test.tsx` passed: 5 files, 92 tests.
- `pnpm -r typecheck` passed across the workspace.
- `pnpm build` passed. Vite emitted the existing
large-chunk/dynamic-import warnings.
- UI screenshots were captured locally during QA in
`artifacts/pap-9521/` and `artifacts/pap-9522/`; generated screenshots
are not committed to avoid adding binary artifacts to the repo.

## Risks

- Migration risk is limited by `IF NOT EXISTS` guards for the new
columns, FK, and index, and the migration is ordered as `0086`
immediately after upstream `0085`.
- Runtime behavior changes env precedence for routine executions by
adding routine env as the highest-precedence layer; tests cover
agent/project/routine precedence.
- Secret handling is security-sensitive; tests cover value-free
manifests/events/errors, disabled/missing/deleted secrets, and
cross-company rejection.
- UI history now renders routine env/secret diffs; tests and Storybook
stories cover the main rendering paths.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agent based on GPT-5, with shell/tool use and
medium reasoning effort.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta 2026-05-17 16:30:34 -05:00 committed by GitHub
parent 3e6610fb93
commit 705c1b8d81
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 1736 additions and 50 deletions

View file

@ -5,6 +5,7 @@ import {
activityLog,
agents,
companies,
companySecretBindings,
companySecrets,
companySecretVersions,
createDb,
@ -19,6 +20,7 @@ import {
routineRuns,
routines,
routineTriggers,
secretAccessEvents,
} from "@paperclipai/db";
import {
getEmbeddedPostgresTestSupport,
@ -28,6 +30,7 @@ import { issueService } from "../services/issues.ts";
import { instanceSettingsService } from "../services/instance-settings.ts";
import * as providerRegistry from "../secrets/provider-registry.ts";
import { routineService } from "../services/routines.ts";
import { secretService } from "../services/secrets.ts";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
@ -57,6 +60,8 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
await db.delete(activityLog);
await db.delete(issueInboxArchives);
await db.delete(issueReadStates);
await db.delete(secretAccessEvents);
await db.delete(companySecretBindings);
await db.delete(routineRuns);
await db.delete(routineTriggers);
await db.delete(routines);
@ -331,6 +336,89 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
expect(revisions[1]?.snapshot.routine.description).toBe("Run the frog routine");
});
it("stores routine env in revisions, syncs routine secret bindings, and stamps runs with the dispatch revision", async () => {
const { agentId, companyId, projectId, svc } = await seedFixture();
const secrets = secretService(db);
const secret = await secrets.create(companyId, {
name: `routine-api-${randomUUID()}`,
provider: "local_encrypted",
value: "secret-value",
});
const routine = await svc.create(
companyId,
{
projectId,
goalId: null,
parentIssueId: null,
title: "secret routine",
description: null,
assigneeAgentId: agentId,
priority: "medium",
status: "active",
concurrencyPolicy: "always_enqueue",
catchUpPolicy: "skip_missed",
env: {
ROUTINE_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
ROUTINE_PLAIN: { type: "plain", value: "plain-value" },
},
},
{},
);
const bindings = await db
.select()
.from(companySecretBindings)
.where(eq(companySecretBindings.targetId, routine.id));
expect(bindings).toMatchObject([
{
companyId,
secretId: secret.id,
targetType: "routine",
configPath: "env.ROUTINE_API_KEY",
},
]);
const [initialRevision] = await svc.listRevisions(routine.id);
expect(initialRevision?.snapshot.routine.env).toEqual(routine.env);
await db.delete(companySecretBindings).where(eq(companySecretBindings.targetId, routine.id));
const repaired = await svc.update(routine.id, { env: routine.env }, {});
expect(repaired).not.toBeNull();
const repairedBindings = await db
.select()
.from(companySecretBindings)
.where(eq(companySecretBindings.targetId, routine.id));
expect(repairedBindings).toMatchObject([
{
companyId,
secretId: secret.id,
targetType: "routine",
configPath: "env.ROUTINE_API_KEY",
},
]);
const currentRoutine = repaired ?? routine;
const runBefore = await svc.runRoutine(routine.id, { source: "manual" });
expect(runBefore.routineRevisionId).toBe(currentRoutine.latestRevisionId);
const updated = await svc.update(
routine.id,
{
env: {
ROUTINE_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
ROUTINE_PLAIN: { type: "plain", value: "changed" },
},
},
{},
);
expect(updated?.latestRevisionNumber).toBe(currentRoutine.latestRevisionNumber + 1);
const runAfter = await svc.runRoutine(routine.id, { source: "manual" });
expect(runAfter.routineRevisionId).toBe(updated?.latestRevisionId);
expect(runAfter.dispatchFingerprint).not.toBe(runBefore.dispatchFingerprint);
});
it("rejects stale routine baseRevisionId updates", async () => {
const { routine, svc } = await seedFixture();
const updated = await svc.update(routine.id, { description: "new description" }, {});