paperclip/server/src/services/dashboard.ts

159 lines
5.5 KiB
TypeScript
Raw Normal View History

import { and, eq, gte, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
[codex] Harden dashboard run activity charts (#4126) ## Thinking Path > - Paperclip gives operators a live view of agent work across dashboards, transcripts, and run activity charts > - Those views consume live run updates and aggregate run activity from backend dashboard data > - Missing or partial run data could make charts brittle, and live transcript updates were heavier than needed > - Operators need dashboard data to stay stable even when recent run payloads are incomplete > - This pull request hardens dashboard run aggregation, guards chart rendering, and lightens live run update handling > - The benefit is a more reliable dashboard during active agent execution ## What Changed - Added dashboard run activity types and backend aggregation coverage. - Guarded activity chart rendering when run data is missing or partial. - Reduced live transcript update churn in active agent and run chat surfaces. - Fixed issue chat avatar alignment in the thread renderer. - Added focused dashboard, activity chart, and live transcript tests. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm exec vitest run server/src/__tests__/dashboard-service.test.ts ui/src/components/ActivityCharts.test.tsx ui/src/components/transcript/useLiveRunTranscripts.test.tsx` - Result: 8 tests passed, 1 skipped. The embedded Postgres dashboard service test skipped on this host with the existing PGlite/Postgres init warning; UI chart and transcript tests passed. ## Risks - Medium-low risk: aggregation semantics changed, but the UI remains guarded around incomplete data. - The dashboard service test is host-skipped here, so CI should confirm the embedded database path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local shell and GitHub workflow, exact runtime context window not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots, or documented why targeted component tests are sufficient here - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 10:34:21 -05:00
import { agents, approvals, companies, costEvents, heartbeatRuns, issues } from "@paperclipai/db";
import { notFound } from "../errors.js";
import { budgetService } from "./budgets.js";
[codex] Harden dashboard run activity charts (#4126) ## Thinking Path > - Paperclip gives operators a live view of agent work across dashboards, transcripts, and run activity charts > - Those views consume live run updates and aggregate run activity from backend dashboard data > - Missing or partial run data could make charts brittle, and live transcript updates were heavier than needed > - Operators need dashboard data to stay stable even when recent run payloads are incomplete > - This pull request hardens dashboard run aggregation, guards chart rendering, and lightens live run update handling > - The benefit is a more reliable dashboard during active agent execution ## What Changed - Added dashboard run activity types and backend aggregation coverage. - Guarded activity chart rendering when run data is missing or partial. - Reduced live transcript update churn in active agent and run chat surfaces. - Fixed issue chat avatar alignment in the thread renderer. - Added focused dashboard, activity chart, and live transcript tests. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm exec vitest run server/src/__tests__/dashboard-service.test.ts ui/src/components/ActivityCharts.test.tsx ui/src/components/transcript/useLiveRunTranscripts.test.tsx` - Result: 8 tests passed, 1 skipped. The embedded Postgres dashboard service test skipped on this host with the existing PGlite/Postgres init warning; UI chart and transcript tests passed. ## Risks - Medium-low risk: aggregation semantics changed, but the UI remains guarded around incomplete data. - The dashboard service test is host-skipped here, so CI should confirm the embedded database path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local shell and GitHub workflow, exact runtime context window not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots, or documented why targeted component tests are sufficient here - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 10:34:21 -05:00
const DASHBOARD_RUN_ACTIVITY_DAYS = 14;
function formatUtcDateKey(date: Date): string {
return date.toISOString().slice(0, 10);
}
function getRecentUtcDateKeys(now: Date, days: number): string[] {
const todayUtc = Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate());
return Array.from({ length: days }, (_, index) => {
const dayOffset = index - (days - 1);
return formatUtcDateKey(new Date(todayUtc + dayOffset * 24 * 60 * 60 * 1000));
});
}
export function dashboardService(db: Db) {
const budgets = budgetService(db);
return {
summary: async (companyId: string) => {
const company = await db
.select()
.from(companies)
.where(eq(companies.id, companyId))
.then((rows) => rows[0] ?? null);
if (!company) throw notFound("Company not found");
const agentRows = await db
.select({ status: agents.status, count: sql<number>`count(*)` })
.from(agents)
.where(eq(agents.companyId, companyId))
.groupBy(agents.status);
const taskRows = await db
.select({ status: issues.status, count: sql<number>`count(*)` })
.from(issues)
.where(eq(issues.companyId, companyId))
.groupBy(issues.status);
const pendingApprovals = await db
.select({ count: sql<number>`count(*)` })
.from(approvals)
.where(and(eq(approvals.companyId, companyId), eq(approvals.status, "pending")))
.then((rows) => Number(rows[0]?.count ?? 0));
const agentCounts: Record<string, number> = {
active: 0,
running: 0,
paused: 0,
error: 0,
};
for (const row of agentRows) {
const count = Number(row.count);
// "idle" agents are operational — count them as active
const bucket = row.status === "idle" ? "active" : row.status;
agentCounts[bucket] = (agentCounts[bucket] ?? 0) + count;
}
const taskCounts: Record<string, number> = {
open: 0,
inProgress: 0,
blocked: 0,
done: 0,
};
for (const row of taskRows) {
const count = Number(row.count);
if (row.status === "in_progress") taskCounts.inProgress += count;
if (row.status === "blocked") taskCounts.blocked += count;
if (row.status === "done") taskCounts.done += count;
if (row.status !== "done" && row.status !== "cancelled") taskCounts.open += count;
}
const now = new Date();
[codex] Harden dashboard run activity charts (#4126) ## Thinking Path > - Paperclip gives operators a live view of agent work across dashboards, transcripts, and run activity charts > - Those views consume live run updates and aggregate run activity from backend dashboard data > - Missing or partial run data could make charts brittle, and live transcript updates were heavier than needed > - Operators need dashboard data to stay stable even when recent run payloads are incomplete > - This pull request hardens dashboard run aggregation, guards chart rendering, and lightens live run update handling > - The benefit is a more reliable dashboard during active agent execution ## What Changed - Added dashboard run activity types and backend aggregation coverage. - Guarded activity chart rendering when run data is missing or partial. - Reduced live transcript update churn in active agent and run chat surfaces. - Fixed issue chat avatar alignment in the thread renderer. - Added focused dashboard, activity chart, and live transcript tests. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm exec vitest run server/src/__tests__/dashboard-service.test.ts ui/src/components/ActivityCharts.test.tsx ui/src/components/transcript/useLiveRunTranscripts.test.tsx` - Result: 8 tests passed, 1 skipped. The embedded Postgres dashboard service test skipped on this host with the existing PGlite/Postgres init warning; UI chart and transcript tests passed. ## Risks - Medium-low risk: aggregation semantics changed, but the UI remains guarded around incomplete data. - The dashboard service test is host-skipped here, so CI should confirm the embedded database path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local shell and GitHub workflow, exact runtime context window not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots, or documented why targeted component tests are sufficient here - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 10:34:21 -05:00
const monthStart = new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 1));
const runActivityDays = getRecentUtcDateKeys(now, DASHBOARD_RUN_ACTIVITY_DAYS);
const runActivityStart = new Date(`${runActivityDays[0]}T00:00:00.000Z`);
const [{ monthSpend }] = await db
.select({
Sync/master post pap1497 followups 2026 04 15 (#3779) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The board depends on issue, inbox, cost, and company-skill surfaces to stay accurate and fast while agents are actively working > - The PAP-1497 follow-up branch exposed a few rough edges in those surfaces: stale active-run state on completed issues, missing creator filters, oversized issue payload scans, and placeholder issue-route parsing > - Those gaps make the control plane harder to trust because operators can see misleading run state, miss the right subset of work, or pay extra query/render cost on large issue records > - This pull request tightens those follow-ups across server and UI code, and adds regression coverage for the affected paths > - The benefit is a more reliable issue workflow, safer high-volume cost aggregation, and clearer board/operator navigation ## What Changed - Added the `v2026.415.0` release changelog entry. - Fixed stale issue-run presentation after completion and reused the shared issue-path parser so literal route placeholders no longer become issue links. - Added creator filters to the Issues page and Inbox, including persisted filter-state normalization and regression coverage. - Bounded issue detail/list project-mention scans and trimmed large issue-list payload fields to keep issue reads lighter. - Hardened company-skill list projection and cost/finance aggregation so large markdown blobs and large summed values do not leak into list responses or overflow 32-bit casts. - Added targeted server/UI regression tests for company skills, costs/finance, issue mention scanning, creator filters, inbox normalization, and issue reference parsing. ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/costs-service.test.ts server/src/__tests__/issues-goal-context-routes.test.ts server/src/__tests__/issues-service.test.ts ui/src/lib/inbox.test.ts ui/src/lib/issue-filters.test.ts ui/src/lib/issue-reference.test.ts` - `gh pr checks 3779` Current pass set on the PR head: `policy`, `verify`, `e2e`, `security/snyk (cryppadotta)`, `Greptile Review` ## Risks - Creator filter options are derived from the currently loaded issue/agent data, so very sparse result sets may not surface every historical creator until they appear in the active dataset. - Cost/finance aggregate casts now use `double precision`; that removes the current overflow risk, but future schema changes should keep large-value aggregation behavior under review. - Issue detail mention scanning now skips comment-body scans on the detail route, so any consumer that relied on comment-only project mentions there would need to fetch them separately. ## Model Used - OpenAI Codex, GPT-5-based coding agent with terminal tool use and local code execution in the Paperclip workspace. Exact internal model ID/context-window exposure is not surfaced in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-15 21:13:56 -05:00
monthSpend: sql<number>`coalesce(sum(${costEvents.costCents}), 0)::double precision`,
})
.from(costEvents)
.where(
and(
eq(costEvents.companyId, companyId),
gte(costEvents.occurredAt, monthStart),
),
);
const monthSpendCents = Number(monthSpend);
[codex] Harden dashboard run activity charts (#4126) ## Thinking Path > - Paperclip gives operators a live view of agent work across dashboards, transcripts, and run activity charts > - Those views consume live run updates and aggregate run activity from backend dashboard data > - Missing or partial run data could make charts brittle, and live transcript updates were heavier than needed > - Operators need dashboard data to stay stable even when recent run payloads are incomplete > - This pull request hardens dashboard run aggregation, guards chart rendering, and lightens live run update handling > - The benefit is a more reliable dashboard during active agent execution ## What Changed - Added dashboard run activity types and backend aggregation coverage. - Guarded activity chart rendering when run data is missing or partial. - Reduced live transcript update churn in active agent and run chat surfaces. - Fixed issue chat avatar alignment in the thread renderer. - Added focused dashboard, activity chart, and live transcript tests. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm exec vitest run server/src/__tests__/dashboard-service.test.ts ui/src/components/ActivityCharts.test.tsx ui/src/components/transcript/useLiveRunTranscripts.test.tsx` - Result: 8 tests passed, 1 skipped. The embedded Postgres dashboard service test skipped on this host with the existing PGlite/Postgres init warning; UI chart and transcript tests passed. ## Risks - Medium-low risk: aggregation semantics changed, but the UI remains guarded around incomplete data. - The dashboard service test is host-skipped here, so CI should confirm the embedded database path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local shell and GitHub workflow, exact runtime context window not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots, or documented why targeted component tests are sufficient here - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 10:34:21 -05:00
const runActivityDayExpr = sql<string>`to_char(${heartbeatRuns.createdAt} at time zone 'UTC', 'YYYY-MM-DD')`;
const runActivityRows = await db
.select({
date: runActivityDayExpr,
status: heartbeatRuns.status,
count: sql<number>`count(*)::double precision`,
})
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.companyId, companyId),
gte(heartbeatRuns.createdAt, runActivityStart),
),
)
.groupBy(runActivityDayExpr, heartbeatRuns.status);
const runActivity = new Map(
runActivityDays.map((date) => [
date,
{ date, succeeded: 0, failed: 0, other: 0, total: 0 },
]),
);
for (const row of runActivityRows) {
const bucket = runActivity.get(row.date);
if (!bucket) continue;
const count = Number(row.count);
if (row.status === "succeeded") bucket.succeeded += count;
else if (row.status === "failed" || row.status === "timed_out") bucket.failed += count;
else bucket.other += count;
bucket.total += count;
}
const utilization =
company.budgetMonthlyCents > 0
? (monthSpendCents / company.budgetMonthlyCents) * 100
: 0;
const budgetOverview = await budgets.overview(companyId);
return {
companyId,
agents: {
active: agentCounts.active,
running: agentCounts.running,
paused: agentCounts.paused,
error: agentCounts.error,
},
tasks: taskCounts,
costs: {
monthSpendCents,
monthBudgetCents: company.budgetMonthlyCents,
monthUtilizationPercent: Number(utilization.toFixed(2)),
},
pendingApprovals,
budgets: {
activeIncidents: budgetOverview.activeIncidents.length,
pendingApprovals: budgetOverview.pendingApprovalCount,
pausedAgents: budgetOverview.pausedAgentCount,
pausedProjects: budgetOverview.pausedProjectCount,
},
[codex] Harden dashboard run activity charts (#4126) ## Thinking Path > - Paperclip gives operators a live view of agent work across dashboards, transcripts, and run activity charts > - Those views consume live run updates and aggregate run activity from backend dashboard data > - Missing or partial run data could make charts brittle, and live transcript updates were heavier than needed > - Operators need dashboard data to stay stable even when recent run payloads are incomplete > - This pull request hardens dashboard run aggregation, guards chart rendering, and lightens live run update handling > - The benefit is a more reliable dashboard during active agent execution ## What Changed - Added dashboard run activity types and backend aggregation coverage. - Guarded activity chart rendering when run data is missing or partial. - Reduced live transcript update churn in active agent and run chat surfaces. - Fixed issue chat avatar alignment in the thread renderer. - Added focused dashboard, activity chart, and live transcript tests. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm exec vitest run server/src/__tests__/dashboard-service.test.ts ui/src/components/ActivityCharts.test.tsx ui/src/components/transcript/useLiveRunTranscripts.test.tsx` - Result: 8 tests passed, 1 skipped. The embedded Postgres dashboard service test skipped on this host with the existing PGlite/Postgres init warning; UI chart and transcript tests passed. ## Risks - Medium-low risk: aggregation semantics changed, but the UI remains guarded around incomplete data. - The dashboard service test is host-skipped here, so CI should confirm the embedded database path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local shell and GitHub workflow, exact runtime context window not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots, or documented why targeted component tests are sufficient here - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 10:34:21 -05:00
runActivity: Array.from(runActivity.values()),
};
},
};
}