Protocol v0 · Jul. 27, 2026
Method
Ten sections, versioned. Every verdict on this site cites the section that produced it.
- 0 · claim unitA claim is a tuple — system digest × reader × judge+prompt hash × settings × dataset × time-and-endpoint — never a bare number.
- 1 · registrationManifest hashed and anchored before the run. Published diff against the registration ships with every result.
- 2 · procurementNo vendor endpoints, no benchmark keys. Standard plans, public artifacts, digest-verified.
- 3 · executionPinned containers, network sealed except reader/judge. Post-hoc exclusions require quantified impact analysis.
- 4 · judgingNo score without a judge audit: false-accept and false-reject rates published first. Panels for close calls. Blinded.
- 5 · statisticsDual uncertainty always — run σ and sampling CI. Paired designs. UNRESOLVABLE is a formal verdict.
- 6 · signingin-toto attestation, keyless signing, Rekor anchor. Verify any page in your terminal.
- 7 · disputesVendor sign-off window, both-config dual runs, materiality thresholds declared in advance.
- 8 · sealingTranscripts sealed 10 years. Re-judged when better judges exist — revisions chain, strikethrough preserved.
- 9 · conflictsNamed operator. Own products unranked. Buyer-pays only. Silence is not for sale.
Honesty clause. Verifiability measures the same thing as optimizability. We sell procedural integrity. Whether a benchmark predicts the real world, no field has solved — we measure the gap instead of hiding it.