Guard model cuts AI agent attack success 77% pre-execution

A new guard model called StepGuard audits individual tool calls before an AI agent executes them, instead of scoring the completed run afterward, which is where most current guardrails sit. On the AgentDojo and AgentDyn benchmarks it cuts mean attack success by 77.3% while mean utility drops just 2.8 points, according to the paper.

That timing distinction matters more than it sounds. A trajectory-level guard reads the whole transcript after an agent has already called a tool, deleted a file, or sent a request. By the time it flags something wrong, the damage is done. StepGuard sits before execution, so it can block the risky step itself rather than write an incident report about it.

The training method is specific: paired safe and unsafe trajectories that share identical context and diverge only at one risky step. That forces the model to learn what makes a single action dangerous, not just what a bad outcome looks like in hindsight. The paper also describes a rebalancing scheme meant to fight two failure modes at once, guards that miss real attacks and guards that block legitimate actions too often.

The headline number worth sitting with is the utility cost. A 77% drop in attack success for 2.8 points of utility is a good trade if it holds outside the benchmark. Most safety filters buy security by making the agent noticeably worse at its job, and builders route around them for that reason. StepGuard reports the highest average accuracy among open-weight guards in the paper's comparison, putting it near GPT-5.4 on the same tests.

The paper doesn't say how the guard performs against attacks designed specifically to evade step-level screening rather than trajectory-level review, which is the next place I'd expect an adversary to push.