mirror of
https://github.com/alkimake/paperclip.git
synced 2026-06-19 04:00:38 +09:00
[codex] Split backend control-plane QoL slice (#4700)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, so backend task ownership, recovery, review visibility, and company-scoped limits need to stay enforceable without UI-only coupling. > - Closed PR #4692 bundled those backend changes with UI workflow, docs, skills, workflow, and lockfile churn. > - PAP-2694 asks for a clean backend/control-plane slice from that closed branch. > - This branch starts from current `master` and mines only the `cli`, `packages/db`, `packages/shared`, and `server` contracts/tests needed for the backend behavior. > - It explicitly excludes UI workflow/performance work, `.github/workflows/pr.yml`, `pnpm-lock.yaml`, docs, skills, package-script, adapter UI build-config, and perf fixture script changes; the only UI files are fixture/test updates required by the tightened shared `Company` contract. > - The benefit is a smaller reviewable PR that preserves the control-plane fixes while staying under Greptile s 100-file review limit. ## What Changed - Added company-scoped attachment-size limits through DB schema/migrations, shared company portability contracts, CLI import/export coverage, and server attachment upload enforcement. - Added productivity review service/API behavior for no-comment streak, long-active, and high-churn review issues, including request-depth clamping and issue summary exposure. - Hardened issue ownership and recovery/control-plane paths: peer-agent mutation denial, issue tree pause/resume behavior, stranded recovery origins, and related activity/test coverage. - Preserved related backend contract updates for routine timestamp variables and managed agent instruction bundles because they live in shared/server contracts from the source branch. - Addressed Greptile feedback by making `Company.attachmentMaxBytes` non-optional, simplifying review request-depth clamping, fixing the migration final newline, and enforcing the process-level attachment cap as the final ceiling for uploads. - Added minimal company fixtures needed for repo-wide typecheck/build and kept the PR to 66 changed files with forbidden/non-slice paths excluded. ## Verification - `pnpm install --frozen-lockfile` - `git diff --check origin/master..HEAD` - `git diff --name-only origin/master..HEAD | wc -l` -> 66 files - `git diff --name-only origin/master..HEAD -- .github/workflows/pr.yml pnpm-lock.yaml package.json doc skills .agents scripts packages/adapters` -> no output - `pnpm exec vitest run --config vitest.config.ts packages/shared/src/validators/issue.test.ts packages/shared/src/routine-variables.test.ts packages/shared/src/adapter-types.test.ts cli/src/__tests__/company-import-export-e2e.test.ts cli/src/__tests__/company.test.ts server/src/__tests__/productivity-review-service.test.ts server/src/__tests__/issue-tree-control-service.test.ts server/src/__tests__/issue-tree-control-routes.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/issue-attachment-routes.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/issues-service.test.ts` -> 12 files, 147 tests passed - `pnpm exec vitest run --config vitest.config.ts cli/src/__tests__/company-delete.test.ts cli/src/__tests__/company-import-export-e2e.test.ts server/src/__tests__/productivity-review-service.test.ts` -> 3 files, 18 tests passed - `pnpm exec vitest run --config vitest.config.ts server/src/__tests__/issue-attachment-routes.test.ts` -> 1 file, 6 tests passed - `pnpm --filter @paperclipai/db typecheck && pnpm --filter @paperclipai/shared typecheck && pnpm --filter @paperclipai/server typecheck && pnpm --filter paperclipai typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck && pnpm --filter @paperclipai/ui build` ## Risks - Includes migrations `0073_shiny_salo.sql` and `0074_striped_genesis.sql`; merge ordering matters if another PR adds migrations first. - This is intentionally backend-only apart from fixture/test updates forced by shared type correctness; UI affordances from PR #4692 are not present here and should land in separate UI slices. - The worktree install emitted plugin SDK bin-link warnings for unbuilt plugin packages, but the targeted tests and package typechecks completed successfully. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected; check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled terminal/GitHub workflow. Exact runtime context window was not exposed by the harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
d9f540c331
commit
1991ec9d6f
66 changed files with 34186 additions and 148 deletions
427
server/src/__tests__/productivity-review-service.test.ts
Normal file
427
server/src/__tests__/productivity-review-service.test.ts
Normal file
|
|
@ -0,0 +1,427 @@
|
|||
import { randomUUID } from "node:crypto";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
activityLog,
|
||||
agents,
|
||||
companies,
|
||||
createDb,
|
||||
heartbeatRuns,
|
||||
issueComments,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { MAX_ISSUE_REQUEST_DEPTH } from "@paperclipai/shared";
|
||||
import {
|
||||
DEFAULT_PRODUCTIVITY_REVIEW_NO_COMMENT_STREAK_RUNS,
|
||||
PRODUCTIVITY_REVIEW_ORIGIN_KIND,
|
||||
productivityReviewService,
|
||||
} from "../services/productivity-review.ts";
|
||||
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||||
|
||||
if (!embeddedPostgresSupport.supported) {
|
||||
console.warn(
|
||||
`Skipping embedded Postgres productivity review tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
||||
);
|
||||
}
|
||||
|
||||
describeEmbeddedPostgres("productivity review service", () => {
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-productivity-review-");
|
||||
db = createDb(tempDb.connectionString);
|
||||
}, 30_000);
|
||||
|
||||
afterEach(async () => {
|
||||
await db.execute(sql.raw(`TRUNCATE TABLE "companies" CASCADE`));
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await tempDb?.cleanup();
|
||||
});
|
||||
|
||||
async function seedAssignedIssue(opts?: {
|
||||
status?: "todo" | "in_progress";
|
||||
startedAt?: Date;
|
||||
parentId?: string | null;
|
||||
originKind?: string;
|
||||
}) {
|
||||
const companyId = randomUUID();
|
||||
const managerId = randomUUID();
|
||||
const coderId = randomUUID();
|
||||
const issueId = randomUUID();
|
||||
const issuePrefix = `PR${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`;
|
||||
const createdAt = new Date("2026-04-28T10:00:00.000Z");
|
||||
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "Productivity Review Co",
|
||||
issuePrefix,
|
||||
requireBoardApprovalForNewAgents: false,
|
||||
});
|
||||
await db.insert(agents).values([
|
||||
{
|
||||
id: managerId,
|
||||
companyId,
|
||||
name: "CTO",
|
||||
role: "cto",
|
||||
status: "idle",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {},
|
||||
permissions: {},
|
||||
},
|
||||
{
|
||||
id: coderId,
|
||||
companyId,
|
||||
name: "Coder",
|
||||
role: "engineer",
|
||||
status: "idle",
|
||||
reportsTo: managerId,
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {},
|
||||
permissions: {},
|
||||
},
|
||||
]);
|
||||
await db.insert(issues).values({
|
||||
id: issueId,
|
||||
companyId,
|
||||
title: "Implement data import",
|
||||
status: opts?.status ?? "in_progress",
|
||||
priority: "medium",
|
||||
assigneeAgentId: coderId,
|
||||
parentId: opts?.parentId ?? null,
|
||||
originKind: opts?.originKind ?? "manual",
|
||||
issueNumber: 1,
|
||||
identifier: `${issuePrefix}-1`,
|
||||
startedAt: opts?.startedAt ?? createdAt,
|
||||
createdAt,
|
||||
updatedAt: createdAt,
|
||||
});
|
||||
|
||||
return { companyId, managerId, coderId, issueId, issuePrefix, createdAt };
|
||||
}
|
||||
|
||||
async function insertRuns(input: {
|
||||
companyId: string;
|
||||
agentId: string;
|
||||
issueId: string;
|
||||
count: number;
|
||||
now: Date;
|
||||
withRunComments?: boolean;
|
||||
}) {
|
||||
const runs: Array<typeof heartbeatRuns.$inferInsert> = [];
|
||||
for (let index = 0; index < input.count; index += 1) {
|
||||
const runId = randomUUID();
|
||||
const createdAt = new Date(input.now.getTime() - index * 60_000);
|
||||
runs.push({
|
||||
id: runId,
|
||||
companyId: input.companyId,
|
||||
agentId: input.agentId,
|
||||
status: "succeeded",
|
||||
invocationSource: "assignment",
|
||||
triggerDetail: "system",
|
||||
startedAt: createdAt,
|
||||
finishedAt: new Date(createdAt.getTime() + 30_000),
|
||||
contextSnapshot: { issueId: input.issueId, taskId: input.issueId },
|
||||
livenessState: "advanced",
|
||||
nextAction: "Continue processing the next batch.",
|
||||
createdAt,
|
||||
updatedAt: createdAt,
|
||||
});
|
||||
}
|
||||
await db.insert(heartbeatRuns).values(runs);
|
||||
|
||||
if (input.withRunComments) {
|
||||
await db.insert(issueComments).values(
|
||||
runs.map((run, index) => ({
|
||||
companyId: input.companyId,
|
||||
issueId: input.issueId,
|
||||
authorAgentId: input.agentId,
|
||||
createdByRunId: run.id,
|
||||
body: `Progress update ${index}`,
|
||||
createdAt: run.createdAt as Date,
|
||||
updatedAt: run.createdAt as Date,
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
return runs;
|
||||
}
|
||||
|
||||
async function listProductivityReviews(companyId: string) {
|
||||
return db
|
||||
.select()
|
||||
.from(issues)
|
||||
.where(and(eq(issues.companyId, companyId), eq(issues.originKind, PRODUCTIVITY_REVIEW_ORIGIN_KIND)))
|
||||
.orderBy(issues.createdAt);
|
||||
}
|
||||
|
||||
it("creates exactly one manager-assigned review for a no-comment run streak and refreshes it idempotently", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: DEFAULT_PRODUCTIVITY_REVIEW_NO_COMMENT_STREAK_RUNS,
|
||||
now,
|
||||
});
|
||||
|
||||
const service = productivityReviewService(db);
|
||||
const first = await service.reconcileProductivityReviews({ now, companyId: seeded.companyId });
|
||||
const second = await service.reconcileProductivityReviews({ now, companyId: seeded.companyId });
|
||||
|
||||
expect(first.created).toBe(1);
|
||||
expect(second.updated).toBe(1);
|
||||
const reviews = await listProductivityReviews(seeded.companyId);
|
||||
expect(reviews).toHaveLength(1);
|
||||
expect(reviews[0]?.parentId).toBe(seeded.issueId);
|
||||
expect(reviews[0]?.assigneeAgentId).toBe(seeded.managerId);
|
||||
expect(reviews[0]?.originId).toBe(seeded.issueId);
|
||||
expect(reviews[0]?.originFingerprint).toBe(`productivity-review:${seeded.issueId}`);
|
||||
expect(reviews[0]?.description).toContain("Primary trigger: `no_comment_streak`");
|
||||
expect(reviews[0]?.description).toContain("No-comment completed-run streak: 10");
|
||||
|
||||
const comments = await db
|
||||
.select()
|
||||
.from(issueComments)
|
||||
.where(eq(issueComments.issueId, reviews[0]!.id));
|
||||
expect(comments.some((comment) => comment.body.includes("Productivity review evidence refreshed"))).toBe(true);
|
||||
});
|
||||
|
||||
it("creates a long-active review without enabling a continuation hold", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue({
|
||||
status: "in_progress",
|
||||
startedAt: new Date(now.getTime() - 7 * 60 * 60 * 1000),
|
||||
});
|
||||
const service = productivityReviewService(db);
|
||||
|
||||
const result = await service.reconcileProductivityReviews({ now, companyId: seeded.companyId });
|
||||
const hold = await service.isProductivityReviewContinuationHoldActive({
|
||||
companyId: seeded.companyId,
|
||||
issueId: seeded.issueId,
|
||||
agentId: seeded.coderId,
|
||||
now,
|
||||
});
|
||||
|
||||
expect(result.created).toBe(1);
|
||||
const [review] = await listProductivityReviews(seeded.companyId);
|
||||
expect(review?.description).toContain("Primary trigger: `long_active_duration`");
|
||||
expect(review?.priority).toBe("medium");
|
||||
expect(hold.held).toBe(false);
|
||||
});
|
||||
|
||||
it("creates a high-churn review even when every sampled run has a progress comment", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: 10,
|
||||
now,
|
||||
withRunComments: true,
|
||||
});
|
||||
|
||||
const result = await productivityReviewService(db).reconcileProductivityReviews({
|
||||
now,
|
||||
companyId: seeded.companyId,
|
||||
});
|
||||
|
||||
expect(result.created).toBe(1);
|
||||
const [review] = await listProductivityReviews(seeded.companyId);
|
||||
expect(review?.description).toContain("Primary trigger: `high_churn`");
|
||||
expect(review?.description).toContain("Runs in rolling windows: 10/1h");
|
||||
});
|
||||
|
||||
it("ignores non-assignee comments when evaluating high-churn productivity reviews", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: 9,
|
||||
now,
|
||||
});
|
||||
const managerRuns = await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.managerId,
|
||||
issueId: seeded.issueId,
|
||||
count: 10,
|
||||
now,
|
||||
});
|
||||
await db.insert(issueComments).values(
|
||||
managerRuns.map((run, index) => ({
|
||||
companyId: seeded.companyId,
|
||||
issueId: seeded.issueId,
|
||||
authorAgentId: seeded.managerId,
|
||||
createdByRunId: run.id,
|
||||
body: `Manager note ${index}`,
|
||||
createdAt: run.createdAt as Date,
|
||||
updatedAt: run.createdAt as Date,
|
||||
})),
|
||||
);
|
||||
|
||||
const result = await productivityReviewService(db).reconcileProductivityReviews({
|
||||
now,
|
||||
companyId: seeded.companyId,
|
||||
});
|
||||
|
||||
expect(result.created).toBe(0);
|
||||
expect(await listProductivityReviews(seeded.companyId)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("skips productivity-review descendants so reviews cannot recursively spawn reviews", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
const reviewId = randomUUID();
|
||||
const childId = randomUUID();
|
||||
await db.insert(issues).values({
|
||||
id: reviewId,
|
||||
companyId: seeded.companyId,
|
||||
title: "Existing productivity review",
|
||||
status: "todo",
|
||||
priority: "high",
|
||||
originKind: PRODUCTIVITY_REVIEW_ORIGIN_KIND,
|
||||
originId: seeded.issueId,
|
||||
originFingerprint: `productivity-review:${seeded.issueId}`,
|
||||
parentId: seeded.issueId,
|
||||
issueNumber: 2,
|
||||
identifier: `${seeded.issuePrefix}-2`,
|
||||
});
|
||||
await db.insert(issues).values({
|
||||
id: childId,
|
||||
companyId: seeded.companyId,
|
||||
title: "Review follow-up child",
|
||||
status: "in_progress",
|
||||
priority: "medium",
|
||||
assigneeAgentId: seeded.coderId,
|
||||
parentId: reviewId,
|
||||
issueNumber: 3,
|
||||
identifier: `${seeded.issuePrefix}-3`,
|
||||
startedAt: new Date(now.getTime() - 7 * 60 * 60 * 1000),
|
||||
});
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: childId,
|
||||
count: 10,
|
||||
now,
|
||||
});
|
||||
|
||||
const result = await productivityReviewService(db).reconcileProductivityReviews({
|
||||
now,
|
||||
companyId: seeded.companyId,
|
||||
});
|
||||
const reviews = await listProductivityReviews(seeded.companyId);
|
||||
|
||||
expect(result.created).toBe(0);
|
||||
expect(reviews).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("treats a recently completed review as a snooze window", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: 10,
|
||||
now,
|
||||
});
|
||||
const service = productivityReviewService(db);
|
||||
await service.reconcileProductivityReviews({ now, companyId: seeded.companyId });
|
||||
const [review] = await listProductivityReviews(seeded.companyId);
|
||||
await db
|
||||
.update(issues)
|
||||
.set({ status: "done", updatedAt: now })
|
||||
.where(eq(issues.id, review!.id));
|
||||
|
||||
const result = await service.reconcileProductivityReviews({
|
||||
now: new Date(now.getTime() + 30 * 60 * 1000),
|
||||
companyId: seeded.companyId,
|
||||
});
|
||||
const reviews = await listProductivityReviews(seeded.companyId);
|
||||
|
||||
expect(result.snoozed).toBe(1);
|
||||
expect(reviews).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("reports and logs soft-stop holds for open no-comment reviews", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
const [latestRun] = await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: 10,
|
||||
now,
|
||||
});
|
||||
const service = productivityReviewService(db);
|
||||
await service.reconcileProductivityReviews({ now, companyId: seeded.companyId });
|
||||
const [review] = await listProductivityReviews(seeded.companyId);
|
||||
|
||||
const hold = await service.isProductivityReviewContinuationHoldActive({
|
||||
companyId: seeded.companyId,
|
||||
issueId: seeded.issueId,
|
||||
agentId: seeded.coderId,
|
||||
now,
|
||||
});
|
||||
expect(hold.held).toBe(true);
|
||||
if (!hold.held) return;
|
||||
|
||||
await service.recordContinuationHold({
|
||||
companyId: seeded.companyId,
|
||||
issueId: seeded.issueId,
|
||||
runId: latestRun!.id as string,
|
||||
agentId: seeded.coderId,
|
||||
reviewIssueId: review!.id,
|
||||
trigger: hold.trigger,
|
||||
reason: hold.reason,
|
||||
});
|
||||
const activities = await db
|
||||
.select()
|
||||
.from(activityLog)
|
||||
.where(eq(activityLog.action, "issue.productivity_review_continuation_held"));
|
||||
expect(activities).toHaveLength(1);
|
||||
expect(activities[0]?.entityId).toBe(seeded.issueId);
|
||||
});
|
||||
|
||||
it("clamps poisoned requestDepth metadata instead of aborting productivity reconciliation", async () => {
|
||||
const now = new Date("2026-04-28T12:00:00.000Z");
|
||||
const seeded = await seedAssignedIssue();
|
||||
|
||||
await db
|
||||
.update(issues)
|
||||
.set({ requestDepth: 2_147_483_647 })
|
||||
.where(eq(issues.id, seeded.issueId));
|
||||
|
||||
await insertRuns({
|
||||
companyId: seeded.companyId,
|
||||
agentId: seeded.coderId,
|
||||
issueId: seeded.issueId,
|
||||
count: DEFAULT_PRODUCTIVITY_REVIEW_NO_COMMENT_STREAK_RUNS,
|
||||
now,
|
||||
});
|
||||
|
||||
const result = await productivityReviewService(db).reconcileProductivityReviews({
|
||||
now,
|
||||
companyId: seeded.companyId,
|
||||
});
|
||||
|
||||
expect(result.failed).toBe(0);
|
||||
const [review] = await listProductivityReviews(seeded.companyId);
|
||||
expect(review?.requestDepth).toBe(MAX_ISSUE_REQUEST_DEPTH);
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue