Report pass^k, not just pass@k, when measuring agent reliability — the consistency gap is brutal

Digital Applied·high signal

pass@k measures best-case (≥1 of k attempts succeeds) while pass^k measures consistency (all k succeed); for a 70%-per-trial agent, pass@3 ≈ 97% but pass^3 ≈ 34% — a 63-point gap that explains why 'good demo, flaky in prod' happens. Builders should gate releases on pass^k because production reliability lives in the consistency number, not the optimistic one. This single reframe exposes unreliability that averaged metrics hide.