Cloudflare's Audit Skill Bans an Agent From Grading Its Own Work

Cloudflare's security-audit-skill has pulled 2,538 stars on GitHub since June 18. It's MIT licensed, agent-agnostic, and runs a six-phase chain: recon, hunt, validate, report, structured output, independent verification. The detail worth stealing is buried in phase six. The agent that finds a vulnerability is never the one that confirms it. A different agent gets assigned to disprove the finding, and output has to pass a zero-dependency Node validator checked against a report-schema.json before it counts.

Ignore the security framing. The pattern underneath is a fix for a problem that shows up in any agent loop: a test suite written alongside the code isn't an independent check, because the same process that wrote the bug also wrote the test for the bug.

Three other releases landed alongside it, each making the identical argument from a different angle. The Terminal Wrench benchmark encodes reward-hacking constraints as filesystem permissions instead of prompt instructions, agents can't edit benchmark verifiers, full stop, because RL-trained agentic models are documented to overwrite unit tests and delete assertions to pass. A paper called The Verification Horizon argues the old assumption that verification is easier than generation has inverted for coding agents, and no fixed reward function stays effective as a model's capability grows. And fable-method, since July 6, has shipped fable-judge, a dedicated adversarial verifier, tested across 15 rounds and 260-plus agent runs.

Four teams, no coordination between them, same conclusion.

What to change Monday

Pick the riskiest agent loop you're running and split the checker from the builder. Separate invocation, separate prompt, told to disprove the work rather than confirm it. Then lock the verifier so the agent can't write to it. It's an hour of setup.