paperclip/scripts/run-typecheck-build-gaps.mjs

94 lines
2.5 KiB
JavaScript
Raw Normal View History

Speed up PR CI critical path (#5147) ## Thinking Path > - Paperclip orchestrates AI agents for autonomous companies, so developer throughput on the control plane repo directly affects how fast the product can evolve. > - The PR workflow is part of that throughput surface because every change waits on it before review and merge. > - This branch started from measured evidence that the PR critical path was dominated by work that was either serialized unnecessarily or placed on the wrong part of the graph. > - The biggest concrete problems were: the canary dry run living inside `verify`, the server isolated suites running one-by-one in a single lane, and duplicate CI work that the PR path was paying for without increasing coverage proportionally. > - This pull request restructures the PR workflow so those costs are reduced without removing the important coverage that was already protecting release and test quality. > - Follow-up fixes on the branch hardened the new entrypoints so they work on clean GitHub runners and so the reduced PR typecheck path stays self-maintaining as workspace packages evolve. > - The benefit is materially faster PR wall-clock time while keeping canary packaging checks, serialized-suite isolation, plugin SDK consumers, and explicit TypeScript coverage where builds do not already provide it. ## What Changed - Moved the PR canary dry run into its own `Canary Dry Run` job so it still runs on PRs but no longer extends the `verify` critical path. - Split the custom Vitest runner into `general`, `serialized`, and `all` modes, and added shard support for the isolated server suites. - Added `test:run:general` and `test:run:serialized` scripts, then rewired PR CI to fan the serialized server suites out across a 4-way matrix. - Added the required `@paperclipai/plugin-sdk` build preflight before the new reduced-scope typecheck and test entrypoints so they succeed on clean CI runners. - Replaced the hardcoded PR build-gap list with `scripts/run-typecheck-build-gaps.mjs`, which discovers workspace packages whose `build` scripts skip TypeScript and runs only their explicit `typecheck` scripts. - Removed the redundant `pnpm build` from the PR `e2e` job because the Playwright onboarding path boots Paperclip from source. ## Verification - `ruby -e "require 'yaml'; YAML.load_file('.github/workflows/pr.yml'); puts 'workflow ok'"` - `node scripts/run-vitest-stable.mjs --mode general --dry-run` - `node scripts/run-vitest-stable.mjs --mode serialized --shard-index 0 --shard-count 4 --dry-run` - `pnpm run typecheck:build-gaps` - `pnpm test:run:general` - `pnpm test:run:serialized -- --shard-index 0 --shard-count 4` - `pnpm build` - `pnpm paperclipai onboard --yes --run` - `curl http://127.0.0.1:3299/api/health` ## Risks - Branch protection or required-check configuration may need to be updated for the new standalone `Canary Dry Run` job and the serialized-suite matrix job names. - `scripts/run-typecheck-build-gaps.mjs` assumes packages that need explicit PR-time typechecking are the ones whose `build` scripts omit `tsc`; if build conventions change, that heuristic needs to stay aligned. - Serialized test sharding preserves per-suite isolation, but the first few CI runs should still be watched for shard-balance or naming assumptions in downstream tooling. ## Model Used - OpenAI GPT-5.4 via the Codex local adapter, using high reasoning effort with shell, git, and file-edit tool use in a local worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-03 20:20:14 -07:00
#!/usr/bin/env node
import { readFileSync } from "node:fs";
import path from "node:path";
import { spawnSync } from "node:child_process";
const repoRoot = process.cwd();
function fail(message) {
console.error(`[typecheck:build-gaps] ${message}`);
process.exit(1);
}
function run(command, args) {
const result = spawnSync(command, args, {
cwd: repoRoot,
stdio: "inherit",
});
if (result.error) {
console.error(`[typecheck:build-gaps] Failed to spawn ${command}: ${result.error.message}`);
process.exit(1);
}
if (result.status !== 0) {
process.exit(result.status ?? 1);
}
}
function readJson(filePath) {
return JSON.parse(readFileSync(filePath, "utf8"));
}
function listWorkspacePackages() {
const result = spawnSync("pnpm", ["ls", "-r", "--depth", "-1", "--json"], {
cwd: repoRoot,
encoding: "utf8",
});
if (result.error) {
fail(`Unable to spawn pnpm to list workspace packages: ${result.error.message}`);
}
if (result.status !== 0) {
fail("Unable to list pnpm workspace packages.");
}
return JSON.parse(result.stdout);
}
function buildSkipsTypeScript(pkg) {
const buildScript = pkg.scripts?.build;
if (typeof buildScript !== "string") {
return false;
}
return !/\btsc\b/.test(buildScript);
}
const workspacePackages = listWorkspacePackages();
const buildGapCandidates = workspacePackages
.filter((workspacePkg) => workspacePkg.path !== repoRoot)
.map((workspacePkg) => ({
name: workspacePkg.name,
path: workspacePkg.path,
pkg: readJson(path.join(workspacePkg.path, "package.json")),
}))
.filter(({ pkg }) => buildSkipsTypeScript(pkg));
const packagesMissingTypecheck = buildGapCandidates.filter(
({ pkg }) => typeof pkg.scripts?.typecheck !== "string",
);
if (packagesMissingTypecheck.length > 0) {
const missingNames = packagesMissingTypecheck.map((workspacePkg) => workspacePkg.name).join(", ");
fail(
`Workspace packages with build scripts that skip tsc must define a typecheck script. Missing: ${missingNames}`,
);
}
const buildGapPackages = buildGapCandidates.filter(
({ pkg }) => typeof pkg.scripts?.typecheck === "string",
);
console.log(
`[typecheck:build-gaps] typechecking ${buildGapPackages.length} workspace(s): ${buildGapPackages.map(({ name }) => name).join(", ") || "(none)"}`,
);
if (buildGapPackages.length === 0) {
process.exit(0);
}
ci: speed up PR verify workflow (#6137) ## Thinking Path > - Paperclip orchestrates AI agents through a control-plane repo that relies on GitHub Actions as part of its release and verification safety net. > - The PR workflow in `.github/workflows/pr.yml` is the core CI path protecting pull requests before merge. > - Baseline measurement work in [PAPA-335](/PAPA/issues/PAPA-335) showed the old single `verify` job was the critical-path bottleneck, with general tests and build serialized together. > - Follow-up implementation in [PAPA-338](/PAPA/issues/PAPA-338) and [PAPA-339](/PAPA/issues/PAPA-339) split that work into parallel lanes and removed redundant clean-runner prebuild work. > - [PAPA-340](/PAPA/issues/PAPA-340) now needs real post-change PR workflow evidence, not local inference, to compare against the May 15, 2026 baseline and decide whether phase-2 work is still justified. > - This pull request publishes the already-implemented CI speedup branch so GitHub can run the actual `PR` workflow against it. > - The benefit is that CI timing decisions are based on measured runs from the exact workflow shape we intend to ship. ## What Changed - Split the PR workflow so `policy` fans out into separate `Typecheck + Release Registry`, grouped `General tests`, and `Build` jobs. - Kept the serialized server matrix, canary dry run, and e2e jobs intact while removing the old monolithic `verify` bottleneck. - Reworked grouped general-test execution in `scripts/run-vitest-stable.mjs` so the workflow can run balanced non-serialized lanes. - Replaced redundant clean-runner prebuild gates with the idempotent `ensure-build-deps` path used by the relevant CI entrypoints. ## Verification - `ruby -e "require 'yaml'; YAML.load_file('.github/workflows/pr.yml'); puts 'yaml-ok'"` - `node scripts/run-vitest-stable.mjs --mode general --dry-run` - `node scripts/run-vitest-stable.mjs --mode general --group general-server --dry-run` - `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-a --dry-run` - `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-b --dry-run` - `pnpm test:run:general -- --group general-workspaces-b` - `pnpm test:run:general -- --group general-workspaces-a` - `pnpm test:run:general -- --group general-server` - `pnpm run typecheck:build-gaps` - `pnpm --filter @paperclipai/plugin-hello-world-example typecheck` ## Risks - Required-check and branch-protection settings may still reference the old single `verify` job name. - Parallel CI lanes can expose hidden ordering assumptions or clean-runner bootstrap gaps that local grouped dry-runs did not surface. - Because the branch is behind current `master`, merge conflicts or unrelated upstream drift could affect the measured runtime until the branch is rebased. > Checked `ROADMAP.md`; this work is CI throughput maintenance for the existing PR verification path, not duplicate feature work. ## Model Used - OpenAI Codex via Paperclip `codex_local`, GPT-5-class coding agent with repository read/write, shell execution, and GitHub CLI/tool use. The runtime does not expose a more specific backend model ID in-session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-16 11:28:25 -07:00
run("pnpm", ["--filter", "@paperclipai/plugin-sdk", "ensure-build-deps"]);
Speed up PR CI critical path (#5147) ## Thinking Path > - Paperclip orchestrates AI agents for autonomous companies, so developer throughput on the control plane repo directly affects how fast the product can evolve. > - The PR workflow is part of that throughput surface because every change waits on it before review and merge. > - This branch started from measured evidence that the PR critical path was dominated by work that was either serialized unnecessarily or placed on the wrong part of the graph. > - The biggest concrete problems were: the canary dry run living inside `verify`, the server isolated suites running one-by-one in a single lane, and duplicate CI work that the PR path was paying for without increasing coverage proportionally. > - This pull request restructures the PR workflow so those costs are reduced without removing the important coverage that was already protecting release and test quality. > - Follow-up fixes on the branch hardened the new entrypoints so they work on clean GitHub runners and so the reduced PR typecheck path stays self-maintaining as workspace packages evolve. > - The benefit is materially faster PR wall-clock time while keeping canary packaging checks, serialized-suite isolation, plugin SDK consumers, and explicit TypeScript coverage where builds do not already provide it. ## What Changed - Moved the PR canary dry run into its own `Canary Dry Run` job so it still runs on PRs but no longer extends the `verify` critical path. - Split the custom Vitest runner into `general`, `serialized`, and `all` modes, and added shard support for the isolated server suites. - Added `test:run:general` and `test:run:serialized` scripts, then rewired PR CI to fan the serialized server suites out across a 4-way matrix. - Added the required `@paperclipai/plugin-sdk` build preflight before the new reduced-scope typecheck and test entrypoints so they succeed on clean CI runners. - Replaced the hardcoded PR build-gap list with `scripts/run-typecheck-build-gaps.mjs`, which discovers workspace packages whose `build` scripts skip TypeScript and runs only their explicit `typecheck` scripts. - Removed the redundant `pnpm build` from the PR `e2e` job because the Playwright onboarding path boots Paperclip from source. ## Verification - `ruby -e "require 'yaml'; YAML.load_file('.github/workflows/pr.yml'); puts 'workflow ok'"` - `node scripts/run-vitest-stable.mjs --mode general --dry-run` - `node scripts/run-vitest-stable.mjs --mode serialized --shard-index 0 --shard-count 4 --dry-run` - `pnpm run typecheck:build-gaps` - `pnpm test:run:general` - `pnpm test:run:serialized -- --shard-index 0 --shard-count 4` - `pnpm build` - `pnpm paperclipai onboard --yes --run` - `curl http://127.0.0.1:3299/api/health` ## Risks - Branch protection or required-check configuration may need to be updated for the new standalone `Canary Dry Run` job and the serialized-suite matrix job names. - `scripts/run-typecheck-build-gaps.mjs` assumes packages that need explicit PR-time typechecking are the ones whose `build` scripts omit `tsc`; if build conventions change, that heuristic needs to stay aligned. - Serialized test sharding preserves per-suite isolation, but the first few CI runs should still be watched for shard-balance or naming assumptions in downstream tooling. ## Model Used - OpenAI GPT-5.4 via the Codex local adapter, using high reasoning effort with shell, git, and file-edit tool use in a local worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-03 20:20:14 -07:00
for (const workspacePkg of buildGapPackages) {
run("pnpm", ["--filter", workspacePkg.name, "typecheck"]);
}