Gate agent CI on pass^k consistency, not a single passing run
Digital Applied·medium signal
A single green eval run hides nondeterminism; the 2026 practice is to measure all-runs consistency with pass^k (the probability all k runs pass) so flaky agents are caught before release. Pair it with a judge calibrated against a human gold set and a CI gate that blocks regressions — and grow the golden set from real production traces so the metric tracks how the agent actually fails.