Research
A Cheap Read-Only Verifier Captures Nearly All the False-Pass Benefit of a Full Planning Stack
arXiv 2609.20474 (17 Sep 2026) separates what an agent harness actually contributes by pairing prewritten task-specific plans against shuffled policy text matched word for word, isolating guidance content from mere prompt bulk. Across 265 matched cells on tau-squared-bench Retail, real plans improved oracle-verified success by 7.17 percentage points (90% task-clustered bootstrap interval 1.15 to 13.36), concentrated in harder tasks. A read-only terminal verifier rejected 61% of oracle-invalid episodes while withholding 17% of correct ones at under one cent per episode, and the standalone verifier captured nearly all the false-pass benefit of the full planning-plus-verification stack at a fraction of the cost.
↳ Follow the thread