mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-14 01:50:39 +09:00
Add secrets provider vaults and remote import (#5429)
## Thinking Path
> - Paperclip orchestrates AI-agent companies and needs secrets handling
to work across local development, hosted operators, and governed agent
execution.
> - The affected subsystem is the company-scoped secrets control plane:
database schema, server services/routes, CLI workflows, and the Secrets
settings UI.
> - The gap was that secrets were local-only and operators could not
manage provider vaults or import existing remote references without
exposing plaintext.
> - This branch adds provider vault configuration plus an AWS Secrets
Manager remote-import path while preserving company boundaries, binding
context, and audit trails.
> - I kept the PR to a single branch PR, removed unrelated
lockfile/package drift, rebased the full branch onto the current
`public-gh/master`, and addressed fresh Greptile findings.
> - The benefit is a reviewable implementation of provider-backed
secrets with focused tests covering provider selection, import
conflicts, deleted secret reuse, rotation guards, and AWS signing
behavior.
## What Changed
- Added provider vault support for company secrets, including provider
config storage, default vault handling, health checks, binding usage,
access events, and remote import preview/commit.
- Added an AWS Secrets Manager provider using SigV4 request signing,
bounded request timeouts, namespace guardrails, cached runtime
credential resolution, and external-reference linking without plaintext
reads.
- Added Secrets UI surfaces for vault management and remote import, plus
CLI/API documentation for setup and operations.
- Stabilized routine webhook secret binding paths and SSH
environment-driver fixture bindings discovered during verification.
- Addressed Greptile and CI findings: no lockfile/package drift,
monotonic migration metadata, disabled-vault default races, soft-deleted
secret hiding/recreate behavior, remove behavior with disabled vaults,
soft-deleted external-reference re-import, non-active rotation guards,
managed-secret soft deletion through PATCH, and per-call AWS SDK
credential client churn.
- Rebased this branch onto `public-gh/master` at `0e1a5828` and
force-pushed with lease to keep this as the single PR for the branch.
## Verification
- `git fetch public-gh master`
- `git rebase public-gh/master`
- `git diff --name-only public-gh/master...HEAD | grep
'^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR
diff.
- Confirmed migration ordering: master ends at `0081_optimal_dormammu`;
this PR adds `0082_dry_vision` and
`0083_company_secret_provider_configs`.
- Inspected migrations for repeat safety: new tables/indexes use `IF NOT
EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column
additions use `ADD COLUMN IF NOT EXISTS`.
- `pnpm -r typecheck` passed before the Greptile follow-up commits.
- `pnpm test:run` ran the full stable Vitest path before the Greptile
follow-up commits; it completed with 3 timing-related failures under
parallel load: `codex-local-execute.test.ts`,
`cursor-local-execute.test.ts`, and `environment-service.test.ts`.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/codex-local-execute.test.ts
src/__tests__/cursor-local-execute.test.ts
src/__tests__/environment-service.test.ts` passed on targeted rerun
(`24/24`).
- `pnpm build` passed before the Greptile follow-up commits. Vite
reported existing chunk-size/dynamic-import warnings.
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts
src/__tests__/secrets-service.test.ts` passed (`39/39`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
typecheck` passed.
- Captured Storybook screenshots from `ui/storybook-static` for visual
review.
- Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites
1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review`
pass; aggregate `verify` is still registering the completed child
checks.
- Greptile review loop continued through the latest requested pass; all
Greptile review threads are resolved and the latest `Greptile Review`
check on `5ca3a5cf` passed with 0 comments added.
## Screenshots
Before: the provider-vault and remote-import surfaces did not exist on
`master`; these are after-state screenshots from the Storybook fixtures.
## Risks
- Migration risk: this adds new secret provider tables and extends
existing secret rows. The migrations were checked for monotonic ordering
and idempotent guards, but reviewers should still inspect upgrade
behavior carefully.
- Provider risk: AWS support uses direct SigV4 requests. Automated tests
cover signing, request timeouts, vault-config selection, namespace
guardrails, pending-version archival, sanitized provider errors, and
service-level cleanup paths. A real-vault AWS smoke test remains
deployment validation for an operator with AWS credentials rather than
an unverified merge blocker in this local branch.
- UI risk: the Secrets page and import dialog are large new surfaces;
screenshots are included above for reviewer inspection.
- Verification risk: the full local stable test command hit
parallel-load timing failures, although the exact failed files passed
when rerun directly.
- Operational risk: remote import intentionally avoids plaintext reads;
operators must understand that imported external references resolve at
runtime and may fail if AWS permissions change.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, GPT-5 coding agent with local shell/tool use in the
Paperclip worktree. Exact context-window size was not exposed by the
runtime.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dotta
GitHub
parent
06e6ee25cd
commit
778e775c35
103 changed files with 16971 additions and 509 deletions
|
|
@ -11,6 +11,7 @@ export const API = {
|
|||
goals: `${API_PREFIX}/goals`,
|
||||
approvals: `${API_PREFIX}/approvals`,
|
||||
secrets: `${API_PREFIX}/secrets`,
|
||||
secretProviderConfigs: `${API_PREFIX}/secret-provider-configs`,
|
||||
costs: `${API_PREFIX}/costs`,
|
||||
activity: `${API_PREFIX}/activity`,
|
||||
dashboard: `${API_PREFIX}/dashboard`,
|
||||
|
|
|
|||
|
|
@ -395,6 +395,54 @@ export const SECRET_PROVIDERS = [
|
|||
] as const;
|
||||
export type SecretProvider = (typeof SECRET_PROVIDERS)[number];
|
||||
|
||||
export const SECRET_PROVIDER_CONFIG_STATUSES = [
|
||||
"ready",
|
||||
"warning",
|
||||
"coming_soon",
|
||||
"disabled",
|
||||
] as const;
|
||||
export type SecretProviderConfigStatus = (typeof SECRET_PROVIDER_CONFIG_STATUSES)[number];
|
||||
|
||||
export const SECRET_PROVIDER_CONFIG_HEALTH_STATUSES = [
|
||||
"ready",
|
||||
"warning",
|
||||
"error",
|
||||
"coming_soon",
|
||||
"disabled",
|
||||
] as const;
|
||||
export type SecretProviderConfigHealthStatus =
|
||||
(typeof SECRET_PROVIDER_CONFIG_HEALTH_STATUSES)[number];
|
||||
|
||||
export const SECRET_STATUSES = ["active", "disabled", "archived", "deleted"] as const;
|
||||
export type SecretStatus = (typeof SECRET_STATUSES)[number];
|
||||
|
||||
export const SECRET_MANAGED_MODES = ["paperclip_managed", "external_reference"] as const;
|
||||
export type SecretManagedMode = (typeof SECRET_MANAGED_MODES)[number];
|
||||
|
||||
export const SECRET_VERSION_STATUSES = [
|
||||
"current",
|
||||
"previous",
|
||||
"disabled",
|
||||
"destroyed",
|
||||
"failed",
|
||||
] as const;
|
||||
export type SecretVersionStatus = (typeof SECRET_VERSION_STATUSES)[number];
|
||||
|
||||
export const SECRET_BINDING_TARGET_TYPES = [
|
||||
"agent",
|
||||
"project",
|
||||
"environment",
|
||||
"routine",
|
||||
"plugin",
|
||||
"issue",
|
||||
"run",
|
||||
"system",
|
||||
] as const;
|
||||
export type SecretBindingTargetType = (typeof SECRET_BINDING_TARGET_TYPES)[number];
|
||||
|
||||
export const SECRET_ACCESS_OUTCOMES = ["success", "failure"] as const;
|
||||
export type SecretAccessOutcome = (typeof SECRET_ACCESS_OUTCOMES)[number];
|
||||
|
||||
export const STORAGE_PROVIDERS = ["local_disk", "s3"] as const;
|
||||
export type StorageProvider = (typeof STORAGE_PROVIDERS)[number];
|
||||
|
||||
|
|
|
|||
|
|
@ -71,6 +71,8 @@ export {
|
|||
APPROVAL_TYPES,
|
||||
APPROVAL_STATUSES,
|
||||
SECRET_PROVIDERS,
|
||||
SECRET_PROVIDER_CONFIG_STATUSES,
|
||||
SECRET_PROVIDER_CONFIG_HEALTH_STATUSES,
|
||||
STORAGE_PROVIDERS,
|
||||
BILLING_TYPES,
|
||||
FINANCE_EVENT_KINDS,
|
||||
|
|
@ -182,6 +184,8 @@ export {
|
|||
type ApprovalType,
|
||||
type ApprovalStatus,
|
||||
type SecretProvider,
|
||||
type SecretProviderConfigStatus,
|
||||
type SecretProviderConfigHealthStatus,
|
||||
type StorageProvider,
|
||||
type BillingType,
|
||||
type FinanceEventKind,
|
||||
|
|
@ -530,7 +534,29 @@ export type {
|
|||
EnvBinding,
|
||||
AgentEnvConfig,
|
||||
CompanySecret,
|
||||
CompanySecretProviderConfig,
|
||||
SecretProviderConfigPayload,
|
||||
SecretProviderConfigHealthDetails,
|
||||
SecretProviderConfigHealthResponse,
|
||||
CompanySecretBinding,
|
||||
CompanySecretBindingTarget,
|
||||
CompanySecretUsageBinding,
|
||||
CompanySecretVersion,
|
||||
SecretAccessEvent,
|
||||
RemoteSecretImportCandidate,
|
||||
RemoteSecretImportCandidateStatus,
|
||||
RemoteSecretImportConflict,
|
||||
RemoteSecretImportPreviewResult,
|
||||
RemoteSecretImportResult,
|
||||
RemoteSecretImportRowResult,
|
||||
RemoteSecretImportRowStatus,
|
||||
SecretAccessOutcome,
|
||||
SecretBindingTargetType,
|
||||
SecretManagedMode,
|
||||
SecretProviderDescriptor,
|
||||
SecretStatus,
|
||||
SecretVersionSelector,
|
||||
SecretVersionStatus,
|
||||
Routine,
|
||||
RoutineManagedByPlugin,
|
||||
RoutineVariable,
|
||||
|
|
@ -826,7 +852,19 @@ export {
|
|||
envBindingSchema,
|
||||
envConfigSchema,
|
||||
createSecretSchema,
|
||||
createSecretProviderConfigSchema,
|
||||
updateSecretProviderConfigSchema,
|
||||
remoteSecretImportPreviewSchema,
|
||||
remoteSecretImportSchema,
|
||||
remoteSecretImportSelectionSchema,
|
||||
localEncryptedProviderConfigSchema,
|
||||
awsSecretsManagerProviderConfigSchema,
|
||||
gcpSecretManagerProviderConfigSchema,
|
||||
vaultProviderConfigSchema,
|
||||
secretProviderConfigPayloadSchema,
|
||||
createSecretBindingSchema,
|
||||
rotateSecretSchema,
|
||||
secretBindingTargetSchema,
|
||||
updateSecretSchema,
|
||||
createRoutineSchema,
|
||||
updateRoutineSchema,
|
||||
|
|
@ -840,6 +878,11 @@ export {
|
|||
routineRevisionSnapshotV1Schema,
|
||||
routineRevisionSnapshotSchema,
|
||||
type CreateSecret,
|
||||
type CreateSecretProviderConfig,
|
||||
type UpdateSecretProviderConfig,
|
||||
type RemoteSecretImportPreview,
|
||||
type RemoteSecretImport,
|
||||
type RemoteSecretImportSelection,
|
||||
type RotateSecret,
|
||||
type UpdateSecret,
|
||||
type CreateRoutine,
|
||||
|
|
|
|||
|
|
@ -244,7 +244,28 @@ export type {
|
|||
EnvBinding,
|
||||
AgentEnvConfig,
|
||||
CompanySecret,
|
||||
CompanySecretProviderConfig,
|
||||
SecretProviderConfigPayload,
|
||||
SecretProviderConfigHealthDetails,
|
||||
SecretProviderConfigHealthResponse,
|
||||
CompanySecretBinding,
|
||||
CompanySecretBindingTarget,
|
||||
CompanySecretUsageBinding,
|
||||
CompanySecretVersion,
|
||||
SecretAccessEvent,
|
||||
RemoteSecretImportCandidate,
|
||||
RemoteSecretImportCandidateStatus,
|
||||
RemoteSecretImportConflict,
|
||||
RemoteSecretImportPreviewResult,
|
||||
RemoteSecretImportResult,
|
||||
RemoteSecretImportRowResult,
|
||||
RemoteSecretImportRowStatus,
|
||||
SecretAccessOutcome,
|
||||
SecretBindingTargetType,
|
||||
SecretManagedMode,
|
||||
SecretProviderDescriptor,
|
||||
SecretStatus,
|
||||
SecretVersionStatus,
|
||||
} from "./secrets.js";
|
||||
export type {
|
||||
Routine,
|
||||
|
|
|
|||
|
|
@ -1,8 +1,24 @@
|
|||
export type SecretProvider =
|
||||
| "local_encrypted"
|
||||
| "aws_secrets_manager"
|
||||
| "gcp_secret_manager"
|
||||
| "vault";
|
||||
import type {
|
||||
SecretAccessOutcome,
|
||||
SecretBindingTargetType,
|
||||
SecretManagedMode,
|
||||
SecretProvider,
|
||||
SecretProviderConfigHealthStatus,
|
||||
SecretProviderConfigStatus,
|
||||
SecretStatus,
|
||||
SecretVersionStatus,
|
||||
} from "../constants.js";
|
||||
|
||||
export type {
|
||||
SecretAccessOutcome,
|
||||
SecretBindingTargetType,
|
||||
SecretManagedMode,
|
||||
SecretProvider,
|
||||
SecretProviderConfigHealthStatus,
|
||||
SecretProviderConfigStatus,
|
||||
SecretStatus,
|
||||
SecretVersionStatus,
|
||||
};
|
||||
|
||||
export type SecretVersionSelector = number | "latest";
|
||||
|
||||
|
|
@ -25,13 +41,22 @@ export type AgentEnvConfig = Record<string, EnvBinding>;
|
|||
export interface CompanySecret {
|
||||
id: string;
|
||||
companyId: string;
|
||||
key: string;
|
||||
name: string;
|
||||
provider: SecretProvider;
|
||||
status: SecretStatus;
|
||||
managedMode: SecretManagedMode;
|
||||
externalRef: string | null;
|
||||
providerConfigId: string | null;
|
||||
providerMetadata: Record<string, unknown> | null;
|
||||
latestVersion: number;
|
||||
description: string | null;
|
||||
lastResolvedAt: Date | null;
|
||||
lastRotatedAt: Date | null;
|
||||
deletedAt: Date | null;
|
||||
createdByAgentId: string | null;
|
||||
createdByUserId: string | null;
|
||||
referenceCount?: number;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
|
@ -40,4 +65,180 @@ export interface SecretProviderDescriptor {
|
|||
id: SecretProvider;
|
||||
label: string;
|
||||
requiresExternalRef: boolean;
|
||||
supportsManagedValues?: boolean;
|
||||
supportsExternalReferences?: boolean;
|
||||
configured?: boolean;
|
||||
}
|
||||
|
||||
export interface LocalEncryptedProviderConfig {
|
||||
backupReminderAcknowledged?: boolean;
|
||||
}
|
||||
|
||||
export interface AwsSecretsManagerProviderConfig {
|
||||
region: string;
|
||||
namespace?: string | null;
|
||||
secretNamePrefix?: string | null;
|
||||
kmsKeyId?: string | null;
|
||||
ownerTag?: string | null;
|
||||
environmentTag?: string | null;
|
||||
}
|
||||
|
||||
export interface GcpSecretManagerProviderConfig {
|
||||
projectId?: string | null;
|
||||
location?: string | null;
|
||||
namespace?: string | null;
|
||||
secretNamePrefix?: string | null;
|
||||
}
|
||||
|
||||
export interface VaultProviderConfig {
|
||||
address?: string | null;
|
||||
namespace?: string | null;
|
||||
mountPath?: string | null;
|
||||
secretPathPrefix?: string | null;
|
||||
}
|
||||
|
||||
export type SecretProviderConfigPayload =
|
||||
| LocalEncryptedProviderConfig
|
||||
| AwsSecretsManagerProviderConfig
|
||||
| GcpSecretManagerProviderConfig
|
||||
| VaultProviderConfig;
|
||||
|
||||
export interface SecretProviderConfigHealthDetails {
|
||||
code: string;
|
||||
message: string;
|
||||
missingFields?: string[];
|
||||
guidance?: string[];
|
||||
}
|
||||
|
||||
export interface CompanySecretProviderConfig {
|
||||
id: string;
|
||||
companyId: string;
|
||||
provider: SecretProvider;
|
||||
displayName: string;
|
||||
status: SecretProviderConfigStatus;
|
||||
isDefault: boolean;
|
||||
config: SecretProviderConfigPayload;
|
||||
healthStatus: SecretProviderConfigHealthStatus | null;
|
||||
healthCheckedAt: Date | null;
|
||||
healthMessage: string | null;
|
||||
healthDetails: SecretProviderConfigHealthDetails | null;
|
||||
disabledAt: Date | null;
|
||||
createdByAgentId: string | null;
|
||||
createdByUserId: string | null;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
export interface SecretProviderConfigHealthResponse {
|
||||
configId: string;
|
||||
provider: SecretProvider;
|
||||
status: SecretProviderConfigHealthStatus;
|
||||
message: string;
|
||||
details: SecretProviderConfigHealthDetails;
|
||||
checkedAt: Date;
|
||||
}
|
||||
|
||||
export interface CompanySecretVersion {
|
||||
id: string;
|
||||
secretId: string;
|
||||
version: number;
|
||||
providerVersionRef: string | null;
|
||||
status: SecretVersionStatus;
|
||||
fingerprintSha256: string;
|
||||
rotationJobId: string | null;
|
||||
createdAt: Date;
|
||||
revokedAt: Date | null;
|
||||
}
|
||||
|
||||
export interface CompanySecretBinding {
|
||||
id: string;
|
||||
companyId: string;
|
||||
secretId: string;
|
||||
targetType: SecretBindingTargetType;
|
||||
targetId: string;
|
||||
configPath: string;
|
||||
versionSelector: SecretVersionSelector;
|
||||
required: boolean;
|
||||
label: string | null;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
export interface CompanySecretBindingTarget {
|
||||
type: SecretBindingTargetType;
|
||||
id: string;
|
||||
label: string;
|
||||
href: string | null;
|
||||
status: string | null;
|
||||
}
|
||||
|
||||
export interface CompanySecretUsageBinding extends CompanySecretBinding {
|
||||
target: CompanySecretBindingTarget;
|
||||
}
|
||||
|
||||
export interface SecretAccessEvent {
|
||||
id: string;
|
||||
companyId: string;
|
||||
secretId: string;
|
||||
version: number | null;
|
||||
provider: SecretProvider;
|
||||
actorType: "agent" | "user" | "system" | "plugin";
|
||||
actorId: string | null;
|
||||
consumerType: SecretBindingTargetType;
|
||||
consumerId: string;
|
||||
configPath: string | null;
|
||||
issueId: string | null;
|
||||
heartbeatRunId: string | null;
|
||||
pluginId: string | null;
|
||||
outcome: SecretAccessOutcome;
|
||||
errorCode: string | null;
|
||||
createdAt: Date;
|
||||
}
|
||||
|
||||
export type RemoteSecretImportCandidateStatus = "ready" | "duplicate" | "conflict";
|
||||
|
||||
export interface RemoteSecretImportConflict {
|
||||
type: "exact_reference" | "name" | "key" | "provider_guardrail";
|
||||
message: string;
|
||||
existingSecretId?: string;
|
||||
}
|
||||
|
||||
export interface RemoteSecretImportCandidate {
|
||||
externalRef: string;
|
||||
remoteName: string;
|
||||
name: string;
|
||||
key: string;
|
||||
providerVersionRef: string | null;
|
||||
providerMetadata: Record<string, unknown> | null;
|
||||
status: RemoteSecretImportCandidateStatus;
|
||||
importable: boolean;
|
||||
conflicts: RemoteSecretImportConflict[];
|
||||
}
|
||||
|
||||
export interface RemoteSecretImportPreviewResult {
|
||||
providerConfigId: string;
|
||||
provider: SecretProvider;
|
||||
nextToken: string | null;
|
||||
candidates: RemoteSecretImportCandidate[];
|
||||
}
|
||||
|
||||
export type RemoteSecretImportRowStatus = "imported" | "skipped" | "error";
|
||||
|
||||
export interface RemoteSecretImportRowResult {
|
||||
externalRef: string;
|
||||
name: string;
|
||||
key: string;
|
||||
status: RemoteSecretImportRowStatus;
|
||||
reason: string | null;
|
||||
secretId: string | null;
|
||||
conflicts: RemoteSecretImportConflict[];
|
||||
}
|
||||
|
||||
export interface RemoteSecretImportResult {
|
||||
providerConfigId: string;
|
||||
provider: SecretProvider;
|
||||
importedCount: number;
|
||||
skippedCount: number;
|
||||
errorCount: number;
|
||||
results: RemoteSecretImportRowResult[];
|
||||
}
|
||||
|
|
|
|||
|
|
@ -282,9 +282,27 @@ export {
|
|||
envBindingSchema,
|
||||
envConfigSchema,
|
||||
createSecretSchema,
|
||||
createSecretProviderConfigSchema,
|
||||
updateSecretProviderConfigSchema,
|
||||
remoteSecretImportPreviewSchema,
|
||||
remoteSecretImportSchema,
|
||||
remoteSecretImportSelectionSchema,
|
||||
localEncryptedProviderConfigSchema,
|
||||
awsSecretsManagerProviderConfigSchema,
|
||||
gcpSecretManagerProviderConfigSchema,
|
||||
vaultProviderConfigSchema,
|
||||
secretProviderConfigPayloadSchema,
|
||||
createSecretBindingSchema,
|
||||
rotateSecretSchema,
|
||||
secretBindingTargetSchema,
|
||||
updateSecretSchema,
|
||||
type CreateSecretBinding,
|
||||
type CreateSecret,
|
||||
type CreateSecretProviderConfig,
|
||||
type UpdateSecretProviderConfig,
|
||||
type RemoteSecretImportPreview,
|
||||
type RemoteSecretImport,
|
||||
type RemoteSecretImportSelection,
|
||||
type RotateSecret,
|
||||
type UpdateSecret,
|
||||
} from "./secret.js";
|
||||
|
|
|
|||
157
packages/shared/src/validators/secret.test.ts
Normal file
157
packages/shared/src/validators/secret.test.ts
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
createSecretProviderConfigSchema,
|
||||
createSecretSchema,
|
||||
remoteSecretImportPreviewSchema,
|
||||
remoteSecretImportSchema,
|
||||
secretProviderConfigPayloadSchema,
|
||||
updateSecretProviderConfigSchema,
|
||||
} from "./secret.js";
|
||||
|
||||
describe("secret validators", () => {
|
||||
it("rejects externalRef on managed secrets", () => {
|
||||
expect(() =>
|
||||
createSecretSchema.parse({
|
||||
name: "OpenAI API Key",
|
||||
managedMode: "paperclip_managed",
|
||||
value: "secret-value",
|
||||
externalRef: "arn:aws:secretsmanager:us-east-1:123456789012:secret:shared/other",
|
||||
}),
|
||||
).toThrow(/Managed secrets cannot set externalRef/);
|
||||
});
|
||||
|
||||
it("allows externalRef on external reference secrets", () => {
|
||||
const parsed = createSecretSchema.parse({
|
||||
name: "Shared Secret",
|
||||
managedMode: "external_reference",
|
||||
externalRef: "arn:aws:secretsmanager:us-east-1:123456789012:secret:shared/other",
|
||||
});
|
||||
|
||||
expect(parsed.externalRef).toContain(":secret:shared/other");
|
||||
});
|
||||
|
||||
it("accepts non-sensitive local and AWS provider vault metadata", () => {
|
||||
expect(() =>
|
||||
createSecretProviderConfigSchema.parse({
|
||||
provider: "local_encrypted",
|
||||
displayName: "Local",
|
||||
config: { backupReminderAcknowledged: true },
|
||||
}),
|
||||
).not.toThrow();
|
||||
|
||||
expect(() =>
|
||||
createSecretProviderConfigSchema.parse({
|
||||
provider: "aws_secrets_manager",
|
||||
displayName: "AWS",
|
||||
config: {
|
||||
region: "us-east-1",
|
||||
namespace: "production",
|
||||
secretNamePrefix: "paperclip",
|
||||
},
|
||||
}),
|
||||
).not.toThrow();
|
||||
});
|
||||
|
||||
it("accepts origin-only Vault provider vault addresses", () => {
|
||||
expect(() =>
|
||||
createSecretProviderConfigSchema.parse({
|
||||
provider: "vault",
|
||||
displayName: "Vault draft",
|
||||
config: { address: " https://vault.example.com/ " },
|
||||
}),
|
||||
).not.toThrow();
|
||||
|
||||
const parsed = secretProviderConfigPayloadSchema.parse({
|
||||
provider: "vault",
|
||||
config: { address: " https://vault.example.com/ " },
|
||||
});
|
||||
|
||||
expect(parsed.provider).toBe("vault");
|
||||
if (parsed.provider !== "vault") throw new Error("Expected vault provider payload");
|
||||
expect(parsed.config.address).toBe("https://vault.example.com");
|
||||
});
|
||||
|
||||
it.each([
|
||||
"https://user:pass@vault.example.com",
|
||||
"https://vault.example.com?token=hvs.x",
|
||||
"https://vault.example.com#token=hvs.x",
|
||||
"https://vault.example.com/v1/secret",
|
||||
])("rejects credential-bearing or non-origin Vault addresses: %s", (address) => {
|
||||
expect(() =>
|
||||
createSecretProviderConfigSchema.parse({
|
||||
provider: "vault",
|
||||
displayName: "Vault draft",
|
||||
config: { address },
|
||||
}),
|
||||
).toThrow(/origin-only HTTP\(S\) URL/i);
|
||||
});
|
||||
|
||||
it("rejects unsafe Vault addresses in provider payload validation used by updates", () => {
|
||||
expect(() =>
|
||||
secretProviderConfigPayloadSchema.parse({
|
||||
provider: "vault",
|
||||
config: { address: "https://vault.example.com?client_token=hvs.x" },
|
||||
}),
|
||||
).toThrow(/origin-only HTTP\(S\) URL/i);
|
||||
});
|
||||
|
||||
it("rejects unsafe Vault addresses in provider vault update payloads", () => {
|
||||
expect(() =>
|
||||
updateSecretProviderConfigSchema.parse({
|
||||
config: { address: "https://vault.example.com#token=hvs.x" },
|
||||
}),
|
||||
).toThrow(/origin-only HTTP\(S\) URL/i);
|
||||
});
|
||||
|
||||
it("validates AWS remote import preview and import payloads", () => {
|
||||
expect(
|
||||
remoteSecretImportPreviewSchema.parse({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
query: "openai",
|
||||
pageSize: 50,
|
||||
}),
|
||||
).toEqual({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
query: "openai",
|
||||
pageSize: 50,
|
||||
});
|
||||
|
||||
expect(
|
||||
remoteSecretImportSchema.parse({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
secrets: [
|
||||
{
|
||||
externalRef: "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/openai",
|
||||
name: "OpenAI API key",
|
||||
key: "OPENAI_API_KEY",
|
||||
description: " Operator-entered Paperclip description ",
|
||||
providerMetadata: { name: "prod/openai" },
|
||||
},
|
||||
],
|
||||
}),
|
||||
).toMatchObject({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
secrets: [
|
||||
expect.objectContaining({
|
||||
key: "OPENAI_API_KEY",
|
||||
description: "Operator-entered Paperclip description",
|
||||
}),
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("caps AWS remote import paging and row counts", () => {
|
||||
expect(() =>
|
||||
remoteSecretImportPreviewSchema.parse({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
pageSize: 101,
|
||||
}),
|
||||
).toThrow();
|
||||
expect(() =>
|
||||
remoteSecretImportSchema.parse({
|
||||
providerConfigId: "11111111-1111-4111-8111-111111111111",
|
||||
secrets: [],
|
||||
}),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
|
@ -1,5 +1,11 @@
|
|||
import { z } from "zod";
|
||||
import { SECRET_PROVIDERS } from "../constants.js";
|
||||
import {
|
||||
SECRET_BINDING_TARGET_TYPES,
|
||||
SECRET_MANAGED_MODES,
|
||||
SECRET_PROVIDER_CONFIG_STATUSES,
|
||||
SECRET_PROVIDERS,
|
||||
SECRET_STATUSES,
|
||||
} from "../constants.js";
|
||||
|
||||
export const envBindingPlainSchema = z.object({
|
||||
type: z.literal("plain"),
|
||||
|
|
@ -23,25 +29,252 @@ export const envConfigSchema = z.record(envBindingSchema);
|
|||
|
||||
export const createSecretSchema = z.object({
|
||||
name: z.string().min(1),
|
||||
key: z.string().min(1).regex(/^[a-zA-Z0-9_.-]+$/).optional(),
|
||||
provider: z.enum(SECRET_PROVIDERS).optional(),
|
||||
value: z.string().min(1),
|
||||
providerConfigId: z.string().uuid().optional().nullable(),
|
||||
managedMode: z.enum(SECRET_MANAGED_MODES).optional(),
|
||||
value: z.string().min(1).optional().nullable(),
|
||||
description: z.string().optional().nullable(),
|
||||
externalRef: z.string().optional().nullable(),
|
||||
providerMetadata: z.record(z.unknown()).optional().nullable(),
|
||||
providerVersionRef: z.string().optional().nullable(),
|
||||
}).superRefine((value, ctx) => {
|
||||
if ((value.managedMode ?? "paperclip_managed") === "external_reference") {
|
||||
if (!value.externalRef?.trim()) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["externalRef"],
|
||||
message: "External reference secrets require externalRef",
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (value.externalRef?.trim()) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["externalRef"],
|
||||
message: "Managed secrets cannot set externalRef",
|
||||
});
|
||||
}
|
||||
if (!value.value?.trim()) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["value"],
|
||||
message: "Managed secrets require value",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
export type CreateSecret = z.infer<typeof createSecretSchema>;
|
||||
|
||||
export const rotateSecretSchema = z.object({
|
||||
value: z.string().min(1),
|
||||
value: z.string().min(1).optional().nullable(),
|
||||
externalRef: z.string().optional().nullable(),
|
||||
providerVersionRef: z.string().optional().nullable(),
|
||||
providerConfigId: z.string().uuid().optional().nullable(),
|
||||
});
|
||||
|
||||
export type RotateSecret = z.infer<typeof rotateSecretSchema>;
|
||||
|
||||
export const updateSecretSchema = z.object({
|
||||
name: z.string().min(1).optional(),
|
||||
key: z.string().min(1).regex(/^[a-zA-Z0-9_.-]+$/).optional(),
|
||||
status: z.enum(SECRET_STATUSES).optional(),
|
||||
providerConfigId: z.string().uuid().optional().nullable(),
|
||||
description: z.string().optional().nullable(),
|
||||
externalRef: z.string().optional().nullable(),
|
||||
providerMetadata: z.record(z.unknown()).optional().nullable(),
|
||||
});
|
||||
|
||||
export type UpdateSecret = z.infer<typeof updateSecretSchema>;
|
||||
|
||||
export const secretBindingTargetSchema = z.object({
|
||||
targetType: z.enum(SECRET_BINDING_TARGET_TYPES),
|
||||
targetId: z.string().min(1),
|
||||
configPath: z.string().min(1),
|
||||
});
|
||||
|
||||
export const createSecretBindingSchema = secretBindingTargetSchema.extend({
|
||||
secretId: z.string().uuid(),
|
||||
versionSelector: z.union([z.literal("latest"), z.number().int().positive()]).default("latest"),
|
||||
required: z.boolean().default(true),
|
||||
label: z.string().optional().nullable(),
|
||||
});
|
||||
|
||||
export type CreateSecretBinding = z.infer<typeof createSecretBindingSchema>;
|
||||
|
||||
const safeShortText = z.string().trim().min(1).max(160);
|
||||
const optionalSafeShortText = safeShortText.optional().nullable();
|
||||
|
||||
const deniedProviderConfigKeyPattern =
|
||||
/^(access[-_]?key([-_]?id)?|secret[-_]?access[-_]?key|secret[-_]?key|token|password|passwd|credential|credentials|private[-_]?key|pem|jwt|session[-_]?token|service[-_]?account([-_]?json)?|client[-_]?secret|secret[-_]?id|unseal[-_]?key|recovery[-_]?key|key[-_]?file([-_]?path)?|token[-_]?file([-_]?path)?)$/i;
|
||||
|
||||
function rejectSensitiveProviderConfigKeys(value: unknown, ctx: z.RefinementCtx) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return;
|
||||
for (const key of Object.keys(value)) {
|
||||
if (!deniedProviderConfigKeyPattern.test(key)) continue;
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["config", key],
|
||||
message: `Provider vault config cannot persist sensitive field: ${key}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export const localEncryptedProviderConfigSchema = z.object({
|
||||
backupReminderAcknowledged: z.boolean().optional(),
|
||||
}).strict();
|
||||
|
||||
export const awsSecretsManagerProviderConfigSchema = z.object({
|
||||
region: z.string().trim().regex(/^[a-z]{2}(?:-gov)?-[a-z]+-\d+$/, "Invalid AWS region"),
|
||||
namespace: optionalSafeShortText,
|
||||
secretNamePrefix: optionalSafeShortText,
|
||||
kmsKeyId: z.string().trim().min(1).max(512).optional().nullable(),
|
||||
ownerTag: optionalSafeShortText,
|
||||
environmentTag: optionalSafeShortText,
|
||||
}).strict();
|
||||
|
||||
export const gcpSecretManagerProviderConfigSchema = z.object({
|
||||
projectId: z.string().trim().min(1).max(128).regex(/^[a-z][a-z0-9-]{4,127}$/).optional().nullable(),
|
||||
location: optionalSafeShortText,
|
||||
namespace: optionalSafeShortText,
|
||||
secretNamePrefix: optionalSafeShortText,
|
||||
}).strict();
|
||||
|
||||
const vaultAddressSchema = z.preprocess(
|
||||
(value) => typeof value === "string" ? value.trim() : value,
|
||||
z.string().url().superRefine((value, ctx) => {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(value);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const hasPath = url.pathname !== "" && url.pathname !== "/";
|
||||
if (
|
||||
(url.protocol !== "http:" && url.protocol !== "https:") ||
|
||||
url.username ||
|
||||
url.password ||
|
||||
url.search ||
|
||||
url.hash ||
|
||||
hasPath
|
||||
) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
message: "Vault address must be an origin-only HTTP(S) URL without credentials, path, query, or fragment",
|
||||
});
|
||||
}
|
||||
}).transform((value) => new URL(value).origin),
|
||||
);
|
||||
|
||||
function rejectUnsafeVaultAddress(value: unknown, ctx: z.RefinementCtx) {
|
||||
if (value === undefined || value === null) return;
|
||||
const parsed = vaultAddressSchema.safeParse(value);
|
||||
if (parsed.success) return;
|
||||
for (const issue of parsed.error.issues) {
|
||||
ctx.addIssue({
|
||||
...issue,
|
||||
path: ["config", "address", ...issue.path],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export const vaultProviderConfigSchema = z.object({
|
||||
address: vaultAddressSchema.optional().nullable(),
|
||||
namespace: optionalSafeShortText,
|
||||
mountPath: optionalSafeShortText,
|
||||
secretPathPrefix: optionalSafeShortText,
|
||||
}).strict();
|
||||
|
||||
export const secretProviderConfigPayloadSchema = z.discriminatedUnion("provider", [
|
||||
z.object({ provider: z.literal("local_encrypted"), config: localEncryptedProviderConfigSchema }),
|
||||
z.object({ provider: z.literal("aws_secrets_manager"), config: awsSecretsManagerProviderConfigSchema }),
|
||||
z.object({ provider: z.literal("gcp_secret_manager"), config: gcpSecretManagerProviderConfigSchema }),
|
||||
z.object({ provider: z.literal("vault"), config: vaultProviderConfigSchema }),
|
||||
]);
|
||||
|
||||
export const createSecretProviderConfigSchema = z.object({
|
||||
provider: z.enum(SECRET_PROVIDERS),
|
||||
displayName: z.string().trim().min(1).max(120),
|
||||
status: z.enum(SECRET_PROVIDER_CONFIG_STATUSES).optional(),
|
||||
isDefault: z.boolean().optional(),
|
||||
config: z.record(z.unknown()).default({}),
|
||||
}).superRefine((value, ctx) => {
|
||||
rejectSensitiveProviderConfigKeys(value.config, ctx);
|
||||
const parsed = secretProviderConfigPayloadSchema.safeParse({
|
||||
provider: value.provider,
|
||||
config: value.config,
|
||||
});
|
||||
if (!parsed.success) {
|
||||
for (const issue of parsed.error.issues) {
|
||||
ctx.addIssue({
|
||||
...issue,
|
||||
path: issue.path[0] === "config" ? issue.path : ["config", ...issue.path],
|
||||
});
|
||||
}
|
||||
}
|
||||
const status = value.status ?? (["gcp_secret_manager", "vault"].includes(value.provider) ? "coming_soon" : "ready");
|
||||
if ((value.provider === "gcp_secret_manager" || value.provider === "vault") && status !== "coming_soon" && status !== "disabled") {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["status"],
|
||||
message: `${value.provider} provider vaults are locked while coming soon`,
|
||||
});
|
||||
}
|
||||
if ((status === "coming_soon" || status === "disabled") && value.isDefault) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["isDefault"],
|
||||
message: "Only ready or warning provider vaults can be default",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
export type CreateSecretProviderConfig = z.infer<typeof createSecretProviderConfigSchema>;
|
||||
|
||||
export const updateSecretProviderConfigSchema = z.object({
|
||||
displayName: z.string().trim().min(1).max(120).optional(),
|
||||
status: z.enum(SECRET_PROVIDER_CONFIG_STATUSES).optional(),
|
||||
isDefault: z.boolean().optional(),
|
||||
config: z.record(z.unknown()).optional(),
|
||||
}).superRefine((value, ctx) => {
|
||||
if (value.config !== undefined) {
|
||||
rejectSensitiveProviderConfigKeys(value.config, ctx);
|
||||
rejectUnsafeVaultAddress(value.config.address, ctx);
|
||||
}
|
||||
if ((value.status === "coming_soon" || value.status === "disabled") && value.isDefault) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["isDefault"],
|
||||
message: "Only ready or warning provider vaults can be default",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
export type UpdateSecretProviderConfig = z.infer<typeof updateSecretProviderConfigSchema>;
|
||||
|
||||
export const remoteSecretImportPreviewSchema = z.object({
|
||||
providerConfigId: z.string().uuid(),
|
||||
query: z.string().trim().max(200).optional().nullable(),
|
||||
nextToken: z.string().trim().min(1).max(4096).optional().nullable(),
|
||||
pageSize: z.number().int().min(1).max(100).optional(),
|
||||
});
|
||||
|
||||
export type RemoteSecretImportPreview = z.infer<typeof remoteSecretImportPreviewSchema>;
|
||||
|
||||
export const remoteSecretImportSelectionSchema = z.object({
|
||||
externalRef: z.string().trim().min(1).max(2048),
|
||||
name: z.string().trim().min(1).max(160).optional().nullable(),
|
||||
key: z.string().trim().min(1).max(120).regex(/^[a-zA-Z0-9_.-]+$/).optional().nullable(),
|
||||
description: z.string().trim().max(500).optional().nullable(),
|
||||
providerVersionRef: z.string().trim().min(1).max(512).optional().nullable(),
|
||||
providerMetadata: z.record(z.unknown()).optional().nullable(),
|
||||
});
|
||||
|
||||
export const remoteSecretImportSchema = z.object({
|
||||
providerConfigId: z.string().uuid(),
|
||||
secrets: z.array(remoteSecretImportSelectionSchema).min(1).max(100),
|
||||
});
|
||||
|
||||
export type RemoteSecretImportSelection = z.infer<typeof remoteSecretImportSelectionSchema>;
|
||||
export type RemoteSecretImport = z.infer<typeof remoteSecretImportSchema>;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue