[codex] Harden execution reliability and heartbeat tooling (#3679)

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Reliable execution depends on heartbeat routing, issue lifecycle
semantics, telemetry, and a fast enough local verification loop to keep
regressions visible
> - The remaining commits on this branch were mostly server/runtime
correctness fixes plus test and documentation follow-ups in that area
> - Those changes are logically separate from the UI-focused
issue-detail and workspace/navigation branches even when they touch
overlapping issue APIs
> - This pull request groups the execution reliability, heartbeat,
telemetry, and tooling changes into one standalone branch
> - The benefit is a focused review of the control-plane correctness
work, including the follow-up fix that restored the implicit
comment-reopen helpers after branch splitting

## What Changed

- Hardened issue/heartbeat execution behavior, including self-review
stage skipping, deferred mention wakes during active execution, stranded
execution recovery, active-run scoping, assignee resolution, and
blocked-to-todo wake resumption
- Reduced noisy polling/logging overhead by trimming issue run payloads,
compacting persisted run logs, silencing high-volume request logs, and
capping heartbeat-run queries in dashboard/inbox surfaces
- Expanded telemetry and status semantics with adapter/model fields on
task completion plus clearer status guidance in docs/onboarding material
- Updated test infrastructure and verification defaults with faster
route-test module isolation, cheaper default `pnpm test`, e2e isolation
from local state, and repo verification follow-ups
- Included docs/release housekeeping from the branch and added a small
follow-up commit restoring the implicit comment-reopen helpers that were
dropped during branch reconstruction

## Verification

- `pnpm vitest run
server/src/__tests__/issue-comment-reopen-routes.test.ts
server/src/__tests__/issue-telemetry-routes.test.ts`
- `pnpm vitest run server/src/__tests__/http-log-policy.test.ts
server/src/__tests__/heartbeat-run-log.test.ts
server/src/__tests__/health.test.ts`
- `server/src/__tests__/activity-service.test.ts`,
`server/src/__tests__/heartbeat-comment-wake-batching.test.ts`, and
`server/src/__tests__/heartbeat-process-recovery.test.ts` were attempted
on this host but the embedded Postgres harness reported
init-script/data-dir problems and skipped or failed to start, so they
are noted as environment-limited

## Risks

- Medium: this branch changes core issue/heartbeat routing and
reopen/wakeup behavior, so regressions would affect agent execution flow
rather than isolated UI polish
- Because it also updates verification infrastructure, reviewers should
pay attention to whether the new tests are asserting the right failure
modes and not just reshaping harness behavior

## Model Used

- OpenAI Codex coding agent (GPT-5-class runtime in Codex CLI; exact
deployed model ID is not exposed in this environment), reasoning
enabled, tool use and local code execution enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta 2026-04-14 13:34:52 -05:00 committed by GitHub
parent e89076148a
commit 7f893ac4ec
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
106 changed files with 4682 additions and 713 deletions

View file

@ -1,8 +1,6 @@
import express from "express";
import request from "supertest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { errorHandler } from "../middleware/index.js";
import { agentRoutes } from "../routes/agents.js";
const agentId = "11111111-1111-4111-8111-111111111111";
const companyId = "22222222-2222-4222-8222-222222222222";
@ -34,6 +32,7 @@ const baseAgent = {
const mockAgentService = vi.hoisted(() => ({
getById: vi.fn(),
list: vi.fn(),
create: vi.fn(),
updatePermissions: vi.fn(),
getChainOfCommand: vi.fn(),
@ -89,31 +88,34 @@ const mockWorkspaceOperationService = vi.hoisted(() => ({}));
const mockLogActivity = vi.hoisted(() => vi.fn());
const mockTrackAgentCreated = vi.hoisted(() => vi.fn());
const mockGetTelemetryClient = vi.hoisted(() => vi.fn());
const mockSyncInstructionsBundleConfigFromFilePath = vi.hoisted(() => vi.fn());
vi.mock("@paperclipai/shared/telemetry", () => ({
trackAgentCreated: mockTrackAgentCreated,
trackErrorHandlerCrash: vi.fn(),
}));
function registerModuleMocks() {
vi.doMock("@paperclipai/shared/telemetry", () => ({
trackAgentCreated: mockTrackAgentCreated,
trackErrorHandlerCrash: vi.fn(),
}));
vi.mock("../telemetry.js", () => ({
getTelemetryClient: mockGetTelemetryClient,
}));
vi.doMock("../telemetry.js", () => ({
getTelemetryClient: mockGetTelemetryClient,
}));
vi.mock("../services/index.js", () => ({
agentService: () => mockAgentService,
agentInstructionsService: () => mockAgentInstructionsService,
accessService: () => mockAccessService,
approvalService: () => mockApprovalService,
companySkillService: () => mockCompanySkillService,
budgetService: () => mockBudgetService,
heartbeatService: () => mockHeartbeatService,
issueApprovalService: () => mockIssueApprovalService,
issueService: () => mockIssueService,
logActivity: mockLogActivity,
secretService: () => mockSecretService,
syncInstructionsBundleConfigFromFilePath: vi.fn((_agent, config) => config),
workspaceOperationService: () => mockWorkspaceOperationService,
}));
vi.doMock("../services/index.js", () => ({
agentService: () => mockAgentService,
agentInstructionsService: () => mockAgentInstructionsService,
accessService: () => mockAccessService,
approvalService: () => mockApprovalService,
companySkillService: () => mockCompanySkillService,
budgetService: () => mockBudgetService,
heartbeatService: () => mockHeartbeatService,
issueApprovalService: () => mockIssueApprovalService,
issueService: () => mockIssueService,
logActivity: mockLogActivity,
secretService: () => mockSecretService,
syncInstructionsBundleConfigFromFilePath: mockSyncInstructionsBundleConfigFromFilePath,
workspaceOperationService: () => mockWorkspaceOperationService,
}));
}
function createDbStub() {
return {
@ -131,7 +133,11 @@ function createDbStub() {
};
}
function createApp(actor: Record<string, unknown>) {
async function createApp(actor: Record<string, unknown>) {
const [{ errorHandler }, { agentRoutes }] = await Promise.all([
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
vi.importActual<typeof import("../routes/agents.js")>("../routes/agents.js"),
]);
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
@ -145,9 +151,18 @@ function createApp(actor: Record<string, unknown>) {
describe("agent permission routes", () => {
beforeEach(() => {
vi.resetAllMocks();
vi.resetModules();
vi.doUnmock("@paperclipai/shared/telemetry");
vi.doUnmock("../telemetry.js");
vi.doUnmock("../services/index.js");
vi.doUnmock("../routes/agents.js");
vi.doUnmock("../middleware/index.js");
registerModuleMocks();
vi.clearAllMocks();
mockSyncInstructionsBundleConfigFromFilePath.mockImplementation((_agent, config) => config);
mockGetTelemetryClient.mockReturnValue({ track: vi.fn() });
mockAgentService.getById.mockResolvedValue(baseAgent);
mockAgentService.list.mockResolvedValue([baseAgent]);
mockAgentService.getChainOfCommand.mockResolvedValue([]);
mockAgentService.resolveByReference.mockResolvedValue({ ambiguous: false, agent: baseAgent });
mockAgentService.create.mockResolvedValue(baseAgent);
@ -191,7 +206,7 @@ describe("agent permission routes", () => {
});
it("grants tasks:assign by default when board creates a new agent", async () => {
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
@ -226,8 +241,26 @@ describe("agent permission routes", () => {
);
});
it("rejects unsupported query parameters on the agent list route", async () => {
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
isInstanceAdmin: true,
companyIds: [companyId],
});
const res = await request(app)
.get(`/api/companies/${companyId}/agents`)
.query({ urlKey: "builder" });
expect(res.status).toBe(400);
expect(res.body.error).toContain("urlKey");
expect(mockAgentService.list).not.toHaveBeenCalled();
});
it("normalizes direct agent creation to disable timer heartbeats by default", async () => {
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
@ -264,7 +297,7 @@ describe("agent permission routes", () => {
});
it("normalizes hire requests to disable timer heartbeats by default", async () => {
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
@ -315,7 +348,7 @@ describe("agent permission routes", () => {
},
]);
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
@ -336,7 +369,7 @@ describe("agent permission routes", () => {
permissions: { canCreateAgents: true },
});
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "local_implicit",
@ -371,7 +404,7 @@ describe("agent permission routes", () => {
},
]);
const app = createApp({
const app = await createApp({
type: "agent",
agentId,
companyId,
@ -402,7 +435,7 @@ describe("agent permission routes", () => {
status: "running",
});
const app = createApp({
const app = await createApp({
type: "board",
userId: "board-user",
source: "session",