feat(commitperclip): add automated PR quality and security gates (#6469)

Fixes #6470

## Thinking Path

> - Paperclip is an open-source AI agent platform receiving a high
volume of community PRs — currently 2,398 open
> - The contributor experience is broken: PRs sit for months with no
feedback, contributors don't know why they're stuck, and maintainers
spend review time on PRs that are missing basics
> - Common problems: no linked issue, no test coverage, incomplete PR
template, manually-edited lockfile — all catchable before human review
> - At the same time, accepting untrusted PRs from unknown contributors
is a real attack surface: malicious packages, secret injection,
tampering with CI scripts, and code touching the sensitive paths from
the April security advisories
> - This PR adds automated gates that run on every PR: quality failures
get a clear comment telling contributors exactly what to fix, security
concerns are silently flagged as draft advisories and block merge via a
pending check run
> - The benefit is a dramatically faster feedback loop for good-faith
contributors and a meaningful security layer for the maintainers
reviewing them

## What Changed

- **`.github/workflows/commitperclip-review.yml`** — new workflow using
`pull_request_target` (runs in base branch context, has secrets, never
executes PR code). Runs quality gates + security gates on every PR
open/update.
- **`.github/dependabot.yml`** — weekly automated dependency
vulnerability PRs for npm and GitHub Actions.
- **`.github/scripts/get-bot-token.mjs`** — generates a short-lived
commitperclip installation token from `COMMITPERCLIP_KEY` secret.
- **`.github/scripts/run-quality-gates.mjs`** — orchestrates 5 quality
gates, posts/updates a single consolidated comment on the PR.
- **`.github/scripts/check-pr-template.mjs`** — validates all 5 required
template sections, Thinking Path depth (≥3 sentences), Model Used not
placeholder.
- **`.github/scripts/check-pr-linked-issue.mjs`** — requires `Fixes
#NNN` or issue URL in PR body.
- **`.github/scripts/check-pr-test-coverage.mjs`** — requires at least
one test file in the diff.
- **`.github/scripts/check-pr-lockfile.mjs`** — blocks manual
`pnpm-lock.yaml` edits (only the refresh bot may change it).
- **`.github/scripts/check-pr-dependencies.mjs`** — informational
comment when new npm packages are added.
- **`.github/scripts/check-pr-security.mjs`** — 6 silent security
checks: secret patterns, CI workflow tampering, build script changes,
supply chain (new packages in lockfile), suspicious test patterns
(outbound network/shell exec/env var reads), and changes to the 9
sensitive path prefixes from the April advisories. When any fire:
creates a draft security advisory + sets `security-review` check to
`in_progress` (blocks merge). When clean: sets `security-review` to
`success`.
- **`actions/dependency-review-action@v4`** — per-PR dependency
vulnerability check (fails if new dep has known CVE).
- **44 unit tests** across all gate modules (`node:test`, no external
deps).

## Verification

Run all unit tests locally:
```bash
node --test .github/scripts/tests/*.test.mjs
```
Expected: 44 pass, 0 fail.

End-to-end: open a PR missing the template, linked issue, and test files
→ commitperclip posts a consolidated comment listing all failures. Open
a PR with all gates satisfied → `✅ All checks passing` comment posted,
all check runs green.

## Risks

**`pull_request_target` security model:** This workflow runs in base
branch context and has access to secrets. It explicitly checks out `ref:
master` (never PR code) and reads the PR diff via GitHub API only — no
PR code is ever executed. This is the correct pattern for running
secret-bearing checks on fork PRs; deviating from it (e.g. checking out
the PR branch) would be a security vulnerability.

**False positives on security gates:** The sensitive-path gate flags any
PR touching the 9 path prefixes from the April advisories. Legitimate
fixes to those paths will trigger draft advisories. This is intentional
— those paths warrant a human look regardless. The `security-review`
check can be manually resolved by a maintainer once reviewed.

**commitperclip not yet installed:** Until the app is installed on this
repo and the `COMMITPERCLIP_KEY` secret is added, the workflow will fail
on the token generation step. The quality gate comment won't post, but
Dependency Review will still run independently.

## Model Used

Claude Sonnet 4.5, 200k context window, extended thinking enabled, tool
use: read/edit files, bash execution, GitHub API calls

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass (44/44)
- [x] I have added or updated tests where applicable (44 unit tests
across all gate modules)
- [ ] If this change affects the UI, I have included before/after
screenshots (N/A — CI only)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---

## One-time setup needed from you, Dotta

1. **Install commitperclip app** on this repo:
https://github.com/apps/commitperclip/installations/new
2. **Add `COMMITPERCLIP_KEY`** as a repository secret (Actions →
Secrets) — ask @brandonburr for the key
3. **Add `security_advisories: write` and `checks: write`** to the
commitperclip app permissions (commit-capital org → Settings → Apps →
commitperclip → Permissions)
4. **Install Socket.dev** from GitHub Marketplace for supply chain
scanning
5. **Branch protection** (optional but recommended): require
`commitperclip-review` and `security-review` checks to pass before merge

## Dashboard integration note

The `commitperclip-review` check run result maps cleanly to your PR
triage dashboard. A single filter on your Worker:

```javascript
const gatesCheck = checkRuns.find(r => r.name === 'commitperclip-review');
if (gatesCheck?.conclusion === 'failure') return null; // filter from queue
```

For security flags: `GET
/repos/paperclipai/paperclip/security-advisories?state=draft` — advisory
titles include `PR #NNN` for cross-referencing. PRs with a matching
draft advisory have `security-review` in `in_progress` state (grey
spinner, can't merge via branch protection).

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Devin Foley <devin@devinfoley.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>

.github/scripts/run-quality-gates.mjs

@@ -0,0 +1,145 @@
+#!/usr/bin/env node
+/**
+ * run-quality-gates.mjs
+ * Orchestrates all quality gates. Fetches PR data once, runs all gates,
+ * posts or updates a single consolidated comment via commitperclip.
+ *
+ * Env: GH_TOKEN, GH_REPO, PR_NUMBER, PR_AUTHOR, PR_BRANCH
+ * Exit: 0 if all quality gates pass, 1 if any fail.
+ */
+import { fileURLToPath } from 'node:url';
+import { ghFetch } from './get-bot-token.mjs';
+import { fetchAllPullRequestFiles } from './fetch-pr-files.mjs';
+import { checkTemplate } from './check-pr-template.mjs';
+import { checkLinkedIssue } from './check-pr-linked-issue.mjs';
+import { checkTestCoverage } from './check-pr-test-coverage.mjs';
+import { checkLockfile } from './check-pr-lockfile.mjs';
+import { checkDependencies } from './check-pr-dependencies.mjs';
+
+const COMMENT_SIGNATURE = '— commitperclip';
+
+function buildComment(author, failures, informational) {
+  if (failures.length === 0 && informational.length === 0) {
+    return `✅ All checks passing — ready for Greptile review and maintainer approval.\n\n${COMMENT_SIGNATURE}`;
+  }
+
+  const lines = [
+    `Hey @${author}! Before this PR can be reviewed, a few things need attention:\n`,
+  ];
+
+  if (failures.length > 0) {
+    lines.push('**Missing or incomplete:**');
+    for (const f of failures) lines.push(`- [ ] ${f}`);
+  }
+
+  if (informational.length > 0) {
+    if (failures.length > 0) lines.push('');
+    lines.push('**Informational:**');
+    for (const i of informational) lines.push(`- ${i}`);
+  }
+
+  lines.push(
+    '\nOnce updated, push a new commit and these checks will re-run automatically.\n',
+    COMMENT_SIGNATURE
+  );
+
+  return lines.join('\n');
+}
+
+export async function findExistingComment(fetchFromGitHub, token, repo, prNumber) {
+  for (let page = 1; ; page += 1) {
+    const comments = await fetchFromGitHub(
+      `/repos/${repo}/issues/${prNumber}/comments?per_page=100&page=${page}`,
+      token
+    );
+
+    const existing = comments.find(
+      c => (c.user.login === 'commitperclip[bot]' || c.user.login === 'commitperclip') &&
+           c.body.includes(COMMENT_SIGNATURE)
+    );
+    if (existing) return existing;
+
+    if (comments.length < 100) return null;
+  }
+}
+
+async function upsertComment(token, repo, prNumber, body, existing) {
+  if (existing) {
+    await ghFetch(`/repos/${repo}/issues/comments/${existing.id}`, token, {
+      method: 'PATCH',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ body }),
+    });
+  } else {
+    await ghFetch(`/repos/${repo}/issues/${prNumber}/comments`, token, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ body }),
+    });
+  }
+}
+
+async function main() {
+  const { GH_TOKEN, GH_REPO, PR_NUMBER, PR_AUTHOR, PR_BRANCH } = process.env;
+
+  if (!GH_TOKEN || !GH_REPO || !PR_NUMBER) {
+    console.error('ERROR: GH_TOKEN, GH_REPO, PR_NUMBER env vars required');
+    process.exit(1);
+  }
+
+  // Sanitize inputs before use in URL construction (prevents SSRF)
+  const prNumber = parseInt(PR_NUMBER, 10);
+  if (!Number.isInteger(prNumber) || prNumber <= 0) {
+    console.error('ERROR: PR_NUMBER must be a positive integer');
+    process.exit(1);
+  }
+  if (!/^[a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+$/.test(GH_REPO)) {
+    console.error('ERROR: GH_REPO must be in owner/repo format');
+    process.exit(1);
+  }
+
+  // Fetch PR data once — gates use this, no redundant API calls
+  const [pr, files] = await Promise.all([
+    ghFetch(`/repos/${GH_REPO}/pulls/${prNumber}`, GH_TOKEN),
+    fetchAllPullRequestFiles(ghFetch, GH_REPO, prNumber, GH_TOKEN),
+  ]);
+
+  const prBody = pr.body ?? '';
+  const author = PR_AUTHOR ?? pr.user.login;
+  const branch = PR_BRANCH ?? pr.head.ref;
+
+  // Run all quality gates (pure functions run sync, deps check is async)
+  const prTitle = pr.title ?? '';
+  const [templateResult, issueResult, testResult, lockfileResult, depsResult] =
+    await Promise.all([
+      Promise.resolve(checkTemplate(prBody)),
+      Promise.resolve(checkLinkedIssue(prBody, prTitle)),
+      Promise.resolve(checkTestCoverage(files, prTitle)),
+      Promise.resolve(checkLockfile(files, author, branch)),
+      checkDependencies(files, GH_TOKEN, GH_REPO, prNumber, pr.base?.ref),
+    ]);
+
+  const allFailures = [
+    ...templateResult.failures,
+    ...issueResult.failures,
+    ...testResult.failures,
+    ...lockfileResult.failures,
+  ];
+  const informational = depsResult.informational ?? [];
+  const allPassed = allFailures.length === 0;
+
+  const commentBody = buildComment(author, allFailures, informational);
+
+  // Post comment if there are failures/informational, or update existing comment
+  const existing = await findExistingComment(ghFetch, GH_TOKEN, GH_REPO, prNumber);
+  if (allFailures.length > 0 || informational.length > 0 || existing) {
+    await upsertComment(GH_TOKEN, GH_REPO, prNumber, commentBody, existing);
+  }
+
+  console.log(JSON.stringify({ passed: allPassed, failures: allFailures, informational }));
+  process.exit(allPassed ? 0 : 1);
+}
+
+if (process.argv[1] === fileURLToPath(import.meta.url)) {
+  main().catch(e => { console.error(e.message); process.exit(1); });
+}