CASE № 0001 — AI CODE REVIEW FILED 2026 · OPEN SOURCE

Every pull request deserves a verdict.

Comment @verdict on any PR. A reasoning model reads the diff and the files around it, thinks for as long as it needs, and returns a ruling — inline findings ranked P0–P3, a walkthrough summary, and a verdict.

polypaper / pull #283
CM
Co-Messi commented just now Author

@verdict

👀 1
verdict Bot reviewed in 2m 14s

What this PR does: shares a cached active-market universe across search queries to kill the per-query API fan-out.
Risk areas:

  • P1 singleflight future set with an exception — concurrent callers crash instead of falling back
  • P2 CancelledError leaves the in-flight future unresolved

Verdict: 🚨 Needs fixes before merge

FIG. 01 — TRIGGER & RULING, ACTUAL OUTPUT FORMAT
$0.008 PER REVIEW, API COST
1M TOKEN CONTEXT WINDOW
P0–P3 SEVERITY ON EVERY FINDING
15 min ONE-TIME SETUP
SECTION 01

The procedure

01

Summon

$ gh pr comment 283
┌──────────────────┐
│ @verdict         │
│ @verdict focus   │
│   on security    │
└──────────────────┘

Comment @verdict on any PR — drafts included. Add freeform instructions and the reviewer obeys them: focus on security, be strict about edge cases, anything.

02

Deliberate

diff ──▶ full files
  │
  ▼
reasoning model
  thinks 1–5 min
  (not 5 seconds)

A reasoning model — DeepSeek V4 Pro, 1M-token context — reads the patch plus the full files around it, then deliberates. Real chain-of-thought, minutes when the diff demands it.

03

Rule

walkthrough summary
inline P0–P3 pins
      │
      ▼
✅ / ⚠️ / 🚨

Findings land as inline comments pinned to exact lines — added, context, or deleted. The summary states what the PR does, the risk areas, and the verdict.

SECTION 02

Four degrees of guilt

Every finding carries a sentence. No vague "consider maybe possibly" — a ranked, ordered docket, P0 first.

P0 Critical Data loss, security holes, crashes, broken core features. Blocks the merge.
P1 High Wrong logic, significant bugs, missing error handling for real scenarios.
P2 Medium Edge cases that bite in prod, performance problems, clearly bad practice.
P3 Low Style and naming — flagged only when it genuinely helps readability.

PROPORTIONALITY RULE — small diffs only get P0/P1. Verdict does not manufacture nitpicks to look busy. Clean code gets an empty docket and an honest ✅.

SECTION 03

No premise left unverified

Most AI reviewers are confident surface linters. Verdict's review doctrine is built from real false-positive autopsies.

§ 1

Verify the premise

Before filing a "missing transformation" finding, it must locate the code that produces the value. If the normalizer already lower-cases the input, the finding is suppressed — not filed with confidence.

§ 2

No fabricated evidence

Example values must derive from the actual input as it would really appear — or be labelled hypothetical. Invented "proof" is how reviewers lose your trust.

§ 3

Purpose over suspicion

Findings rank by impact on what the PR is for. One pass always asks: does this hold under concurrency, scale, and adversarial input? — the bugs that actually corrupt production.

§ 4

Honest acquittals

If the code is clean, the verdict says so plainly. An empty findings list is a valid ruling — padding reviews with noise is a worse failure than silence.

✓ LOOKS GOOD TO ME ⚠ MINOR CONCERNS ✗ NEEDS FIXES
SECTION 04

Specification sheet

4.1

Self-hosted, yours

A small Node server behind a Cloudflare Tunnel. Your keys, your machine, your code never touches a third-party review vendor. Or run it on a Cloudflare Worker — both ship in the repo.

4.2

Reads beyond the diff

Full file content accompanies every patch, so the reviewer sees the function you changed and the code that calls it. Fewer hallucinated "missing imports".

4.3

Custom instructions

@verdict focus on auth logic — whatever follows the mention becomes a directive to the reviewer for that run. No config files, no YAML.

4.4

Every line commentable

Findings pin to added, context, and deleted lines using GitHub's line-and-side API. Deleting load-bearing code gets caught too.

4.5

Big PRs welcome

200k characters of prompt budget with per-file caps, so one giant lockfile can't starve the review. 21-file, 3000-line PRs are routine.

4.6

Fails loudly

If anything breaks — network, timeout, model — Verdict posts the error with phase timings to the PR. A reviewer that fails silently is a reviewer you stop trusting.

SECTION 05

The fee schedule

AVAILABLE NOW

Self-hosted

$0 + ~$0.008/review API cost
  • Full source, MIT-spirited — clone and run
  • Your DeepSeek key, your GitHub App
  • Unlimited repos, unlimited reviews
  • 15-minute setup, documented end to end
Get the source →
IN DRAFT COMING SOON

Hosted

~ cheap, metered, no seats
  • Zero setup — install the GitHub App, done
  • Stablecoin payments (USDC) accepted
  • Pay per review, not per developer
  • Same engine, same doctrine
Not yet on the docket

Court is in session.

Install the app, comment @verdict, get a ruling.

Install on GitHub →