No AI model fully resolves a single simulated breach

Researchers tested 23 frontier AI models against a simulated network breach, per the new SecRespond benchmark, arXiv 2607.26791. Zero of the 23 closed out a single range end to end, a real gap for teams pitching AI agents at incident response.

Ten cyber ranges made up the test, covering four entry-point types, 21 ATT&CK techniques, and five operating systems, evaluated on the OpenCode harness.

Each agent gets a forensic disk snapshot of a breached host, plus alerts, vulnerability scans, and baseline security checks. It has to produce three reports, on intrusion, baseline risk, and vulnerability risk, plus a remediation plan.

Same failure mode across every model. Agents write solid summaries of what the alerts already caught, but skip the disk snapshot as a lead worth chasing on its own. Intrusions that never tripped an alert mostly go unfound.

I'd bet these models keep writing clean reports on what the alerts already flagged, and keep missing what they didn't. Explaining a known incident and hunting for an unknown one are different skills, and right now only the first one works.