[codex] Add run liveness continuations (#4083)

## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Heartbeat runs are the control-plane record of each agent execution window. > - Long-running local agents can exhaust context or stop while still holding useful next-step state. > - Operators need that stop reason, next action, and continuation path to be durable and visible. > - This pull request adds run liveness metadata, continuation summaries, and UI surfaces for issue run ledgers. > - The benefit is that interrupted or long-running work can resume with clearer context instead of losing the agent's last useful handoff. ## What Changed - Added heartbeat-run liveness fields, continuation attempt tracking, and an idempotent `0058` migration. - Added server services and tests for run liveness, continuation summaries, stop metadata, and activity backfill. - Wired local and HTTP adapters to surface continuation/liveness context through shared adapter utilities. - Added shared constants, validators, and heartbeat types for liveness continuation state. - Added issue-detail UI surfaces for continuation handoffs and the run ledger, with component tests. - Updated agent runtime docs, heartbeat protocol docs, prompt guidance, onboarding assets, and skills instructions to explain continuation behavior. - Addressed Greptile feedback by scoping document evidence by run, excluding system continuation-summary documents from liveness evidence, importing shared liveness types, surfacing hidden ledger run counts, documenting bounded retry behavior, and moving run-ledger liveness backfill off the request path. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/run-continuations.test.ts server/src/__tests__/run-liveness.test.ts server/src/__tests__/activity-service.test.ts server/src/__tests__/documents-service.test.ts server/src/__tests__/issue-continuation-summary.test.ts server/src/services/heartbeat-stop-metadata.test.ts ui/src/components/IssueRunLedger.test.tsx ui/src/components/IssueContinuationHandoff.test.tsx ui/src/components/IssueDocumentsSection.test.tsx` - `pnpm --filter @paperclipai/db build` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts ui/src/components/IssueRunLedger.test.tsx` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts server/src/__tests__/run-continuations.test.ts ui/src/components/IssueRunLedger.test.tsx` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "treats a plan document update"` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts -t "activity service|treats a plan document update"` - Remote PR checks on head `e53b1a1d`: `verify`, `e2e`, `policy`, and Snyk all passed. - Confirmed `public-gh/master` is an ancestor of this branch after fetching `public-gh master`. - Confirmed `pnpm-lock.yaml` is not included in the branch diff. - Confirmed migration `0058_wealthy_starbolt.sql` is ordered after `0057` and uses `IF NOT EXISTS` guards for repeat application. - Greptile inline review threads are resolved. ## Risks - Medium risk: this touches heartbeat execution, liveness recovery, activity rendering, issue routes, shared contracts, docs, and UI. - Migration risk is mitigated by additive columns/indexes and idempotent guards. - Run-ledger liveness backfill is now asynchronous, so the first ledger response can briefly show historical missing liveness until the background backfill completes. - UI screenshot coverage is not included in this packaging pass; validation is currently through focused component tests. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.4, local tool-use coding agent with terminal, git, GitHub connector, GitHub CLI, and Paperclip API access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Screenshot note: no before/after screenshots were captured in this PR packaging pass; the UI changes are covered by focused component tests listed above. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-17 03:10:38 +09:00 · 2026-04-20 06:01:49 -05:00 · 2026-04-20 06:01:49 -05:00 · 236d11d36f
commit 236d11d36f
parent b9a80dcf22
71 changed files with 18254 additions and 85 deletions
--- a/server/src/services/run-liveness.ts
+++ b/server/src/services/run-liveness.ts
@ -0,0 +1,227 @@
+import type { HeartbeatRunStatus, IssueStatus, RunLivenessState } from "@paperclipai/shared";
+
+export interface RunLivenessIssueInput {
+  status: IssueStatus | string;
+  title: string;
+  description: string | null;
+}
+
+export interface RunLivenessEvidenceInput {
+  issueCommentsCreated: number;
+  documentRevisionsCreated: number;
+  planDocumentRevisionsCreated: number;
+  workProductsCreated: number;
+  workspaceOperationsCreated: number;
+  activityEventsCreated: number;
+  toolOrActionEventsCreated: number;
+  latestEvidenceAt: Date | null;
+}
+
+export interface RunLivenessClassificationInput {
+  runStatus: HeartbeatRunStatus | string;
+  issue: RunLivenessIssueInput | null;
+  resultJson?: Record<string, unknown> | null;
+  stdoutExcerpt?: string | null;
+  stderrExcerpt?: string | null;
+  error?: string | null;
+  errorCode?: string | null;
+  continuationAttempt?: number | null;
+  evidence?: Partial<RunLivenessEvidenceInput> | null;
+}
+
+export interface RunLivenessClassification {
+  livenessState: RunLivenessState;
+  livenessReason: string;
+  continuationAttempt: number;
+  lastUsefulActionAt: Date | null;
+  nextAction: string | null;
+}
+
+const DEFAULT_EVIDENCE: RunLivenessEvidenceInput = {
+  issueCommentsCreated: 0,
+  documentRevisionsCreated: 0,
+  planDocumentRevisionsCreated: 0,
+  workProductsCreated: 0,
+  workspaceOperationsCreated: 0,
+  activityEventsCreated: 0,
+  toolOrActionEventsCreated: 0,
+  latestEvidenceAt: null,
+};
+
+const PLANNING_ONLY_RE =
+  /\b(?:i(?:'ll| will| am going to|'m going to)|let me|i need to|next(?:,| i will| i'll)?|my next step is|the next step is)\s+(?:first\s+)?(?:inspect|check|review|look|investigate|analy[sz]e|open|read|start|begin|work on|implement|fix|test|update|create|add)\b/i;
+const NEXT_STEPS_RE = /^\s*(?:next steps?|plan)\s*:/im;
+const BLOCKER_RE =
+  /\b(?:blocked|can't proceed|cannot proceed|unable to proceed|waiting on|need(?:s|ed)? .{0,80}\b(?:approval|access|credential|credentials|secret|api key|token|input|clarification)|requires? .{0,80}\b(?:approval|access|credential|credentials|secret|api key|token|input|clarification))\b/i;
+const NEGATED_BLOCKER_RE = /\b(?:not blocked|no blocker|no blockers|unblocked)\b/i;
+const PLAN_TASK_TITLE_RE = /\b(?:plan|planning|analysis|investigation|research|report|proposal|design doc|write-?up)\b/i;
+const PLAN_TASK_DESCRIPTION_RE =
+  /\b(?:create|write|produce|draft|update|revise|prepare)\s+(?:a\s+|the\s+)?(?:plan|analysis|investigation|research report|report|proposal|design doc|write-?up)\b/i;
+
+function compactReason(reason: string) {
+  return reason.length <= 500 ? reason : `${reason.slice(0, 497)}...`;
+}
+
+function normalizeCount(value: unknown) {
+  return typeof value === "number" && Number.isFinite(value) ? Math.max(0, Math.floor(value)) : 0;
+}
+
+function normalizeContinuationAttempt(value: unknown) {
+  return typeof value === "number" && Number.isFinite(value) ? Math.max(0, Math.floor(value)) : 0;
+}
+
+function readText(value: unknown): string | null {
+  if (typeof value !== "string") return null;
+  const trimmed = value.trim();
+  return trimmed.length > 0 ? trimmed : null;
+}
+
+function resultText(resultJson: Record<string, unknown> | null | undefined) {
+  if (!resultJson) return "";
+  return [
+    readText(resultJson.summary),
+    readText(resultJson.result),
+    readText(resultJson.message),
+    readText(resultJson.stdout),
+    readText(resultJson.stderr),
+  ]
+    .filter((value): value is string => Boolean(value))
+    .join("\n");
+}
+
+function combinedOutput(input: RunLivenessClassificationInput) {
+  return [
+    resultText(input.resultJson),
+    readText(input.stdoutExcerpt),
+    readText(input.stderrExcerpt),
+    readText(input.error),
+  ]
+    .filter((value): value is string => Boolean(value))
+    .join("\n")
+    .trim();
+}
+
+export function hasUsefulOutput(input: RunLivenessClassificationInput) {
+  return combinedOutput(input).length > 0;
+}
+
+export function declaredBlocker(input: RunLivenessClassificationInput) {
+  if (input.issue?.status === "blocked") return true;
+  const text = combinedOutput(input);
+  if (!text || NEGATED_BLOCKER_RE.test(text)) return false;
+  return BLOCKER_RE.test(text);
+}
+
+export function looksLikePlanningOnly(input: RunLivenessClassificationInput) {
+  const text = combinedOutput(input);
+  if (!text) return false;
+  return PLANNING_ONLY_RE.test(text) || NEXT_STEPS_RE.test(text);
+}
+
+export function isPlanningOrDocumentTask(issue: RunLivenessIssueInput | null | undefined) {
+  if (!issue) return false;
+  if (PLAN_TASK_TITLE_RE.test(issue.title)) return true;
+  return PLAN_TASK_DESCRIPTION_RE.test(issue.description ?? "");
+}
+
+function normalizeEvidence(evidence: Partial<RunLivenessEvidenceInput> | null | undefined): RunLivenessEvidenceInput {
+  return {
+    issueCommentsCreated: normalizeCount(evidence?.issueCommentsCreated),
+    documentRevisionsCreated: normalizeCount(evidence?.documentRevisionsCreated),
+    planDocumentRevisionsCreated: normalizeCount(evidence?.planDocumentRevisionsCreated),
+    workProductsCreated: normalizeCount(evidence?.workProductsCreated),
+    workspaceOperationsCreated: normalizeCount(evidence?.workspaceOperationsCreated),
+    activityEventsCreated: normalizeCount(evidence?.activityEventsCreated),
+    toolOrActionEventsCreated: normalizeCount(evidence?.toolOrActionEventsCreated),
+    latestEvidenceAt: evidence?.latestEvidenceAt instanceof Date ? evidence.latestEvidenceAt : null,
+  };
+}
+
+export function hasConcreteActionEvidence(evidence: Partial<RunLivenessEvidenceInput> | null | undefined) {
+  const normalized = normalizeEvidence(evidence);
+  // Workspace creation is setup evidence, not task progress by itself. It can
+  // appear in reasons alongside durable activity, but it must not prevent a
+  // planning-only or empty run from receiving a bounded continuation.
+  return (
+    normalized.issueCommentsCreated +
+      normalized.documentRevisionsCreated +
+      normalized.workProductsCreated +
+      normalized.activityEventsCreated +
+      normalized.toolOrActionEventsCreated >
+    0
+  );
+}
+
+function evidenceReason(evidence: RunLivenessEvidenceInput) {
+  const parts: string[] = [];
+  if (evidence.issueCommentsCreated > 0) parts.push(`${evidence.issueCommentsCreated} issue comment(s)`);
+  if (evidence.documentRevisionsCreated > 0) parts.push(`${evidence.documentRevisionsCreated} document revision(s)`);
+  if (evidence.workProductsCreated > 0) parts.push(`${evidence.workProductsCreated} work product(s)`);
+  if (evidence.workspaceOperationsCreated > 0) parts.push(`${evidence.workspaceOperationsCreated} workspace operation(s)`);
+  if (evidence.activityEventsCreated > 0) parts.push(`${evidence.activityEventsCreated} activity event(s)`);
+  if (evidence.toolOrActionEventsCreated > 0) parts.push(`${evidence.toolOrActionEventsCreated} tool/action event(s)`);
+  return parts.join(", ");
+}
+
+function extractNextAction(input: RunLivenessClassificationInput) {
+  const text = combinedOutput(input);
+  if (!text) return null;
+  const line = text
+    .split(/\r?\n/)
+    .map((entry) => entry.trim())
+    .find((entry) => PLANNING_ONLY_RE.test(entry) || /^next(?: steps?| action)?\s*:/i.test(entry));
+  if (!line) return null;
+  return line.length <= 500 ? line : `${line.slice(0, 497)}...`;
+}
+
+export function classifyRunLiveness(input: RunLivenessClassificationInput): RunLivenessClassification {
+  const evidence = normalizeEvidence(input.evidence);
+  const continuationAttempt = normalizeContinuationAttempt(input.continuationAttempt);
+  const issueStatus = input.issue?.status ?? null;
+  const usefulOutput = hasUsefulOutput(input);
+  const concreteEvidence = hasConcreteActionEvidence(evidence);
+  const planExempt = isPlanningOrDocumentTask(input.issue) || evidence.planDocumentRevisionsCreated > 0;
+  const lastUsefulActionAt = concreteEvidence ? evidence.latestEvidenceAt : null;
+
+  const output = (state: RunLivenessState, reason: string, nextAction: string | null = null): RunLivenessClassification => ({
+    livenessState: state,
+    livenessReason: compactReason(reason),
+    continuationAttempt,
+    lastUsefulActionAt: state === "advanced" || state === "completed" || state === "blocked" ? lastUsefulActionAt : null,
+    nextAction,
+  });
+
+  if (input.runStatus !== "succeeded") {
+    return output("failed", input.errorCode ? `Run ended with ${input.runStatus} (${input.errorCode})` : `Run ended with ${input.runStatus}`);
+  }
+
+  if (issueStatus === "done" || issueStatus === "cancelled") {
+    return output("completed", `Issue is ${issueStatus}`);
+  }
+
+  if (declaredBlocker(input)) {
+    return output("blocked", issueStatus === "blocked" ? "Issue status is blocked" : "Run output declared a concrete blocker", extractNextAction(input));
+  }
+
+  if (!usefulOutput && !concreteEvidence) {
+    return output("empty_response", "Run succeeded without useful output or concrete action evidence");
+  }
+
+  if (concreteEvidence) {
+    return output("advanced", `Run produced concrete action evidence: ${evidenceReason(evidence)}`);
+  }
+
+  if (planExempt && usefulOutput) {
+    return output("advanced", "Planning/document task produced useful output and is exempt from plan-only classification");
+  }
+
+  if (looksLikePlanningOnly(input)) {
+    return output("plan_only", "Run described future work without concrete action evidence", extractNextAction(input));
+  }
+
+  if (usefulOutput) {
+    return output("needs_followup", "Run produced useful output but no concrete action evidence", extractNextAction(input));
+  }
+
+  return output("empty_response", "Run succeeded without useful output");
+}