Protocol v0 · Jul. 27, 2026

Method

Ten sections, versioned. Every verdict on this site cites the section that produced it.

  • 0 · claim unitA claim is a tuple — system digest × reader × judge+prompt hash × settings × dataset × time-and-endpoint — never a bare number.
  • 1 · registrationManifest hashed and anchored before the run. Published diff against the registration ships with every result.
  • 2 · procurementNo vendor endpoints, no benchmark keys. Standard plans, public artifacts, digest-verified.
  • 3 · executionPinned containers, network sealed except reader/judge. Post-hoc exclusions require quantified impact analysis.
  • 4 · judgingNo score without a judge audit: false-accept and false-reject rates published first. Panels for close calls. Blinded.
  • 5 · statisticsDual uncertainty always — run σ and sampling CI. Paired designs. UNRESOLVABLE is a formal verdict.
  • 6 · signingin-toto attestation, keyless signing, Rekor anchor. Verify any page in your terminal.
  • 7 · disputesVendor sign-off window, both-config dual runs, materiality thresholds declared in advance.
  • 8 · sealingTranscripts sealed 10 years. Re-judged when better judges exist — revisions chain, strikethrough preserved.
  • 9 · conflictsNamed operator. Own products unranked. Buyer-pays only. Silence is not for sale.

Honesty clause. Verifiability measures the same thing as optimizability. We sell procedural integrity. Whether a benchmark predicts the real world, no field has solved — we measure the gap instead of hiding it.