DoubleCharge

An eval where frontier models (Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon) write and review the code behind a Pay button, a webhook handler and a wallet. The code runs in a simulated payment system that fails on purpose: replies lost after the card is charged, webhooks delivered twice, double clicks, a server killed at every call wherever the caller retries. No language model grades anything. The verdict is the money: was the customer charged once? The models mostly get it right: 84 of 90 trials survived every failure. All six failures were the same bug, a request acting on what it read before another request changed the order, and Claude Opus 5.5 approved that bug as safe to merge in both of its reviews. A model that calls code unsafe must attach an attack, which the grader runs: 97 of 99 reviews of buggy code proved the bug that way. I built the sandboxed grader, a virtual clock, crash injection at every call of a retried request and a seeded chaos scheduler. The grader proves itself on every CI run against 19 deliberately broken versions and 16 recorded attacks, and every published number reproduces offline for $0. Built for the micro1 Eval Hackathon, November 2026.