A step-labeled corpus of 71,024 agent steps shows surface features catch only 8.2% of partial hijacks

arXiv 2609.06972·medium signalXBlueskyLinkedInCopy link

AgentDrift labels every one of 71,024 steps across 12,536 synthetic tool-call trajectories in five domains as benign, injection point, hijacked, or failed injection, filling a gap where existing benchmarks only score whether an attack succeeded and guard models judge a whole trace. The corpus deliberately includes 1,500 failed attacks the agent resisted and 1,500 hard negatives carrying legitimate content that resembles an attack, so a detector must separate attempt from success, and the authors report their own LLM judge was fooled by the hard negatives. A surface-feature logistic regression recovers 55.4% of attacks (F1 0.647) but only 8.2% of partial hijacks and 23.1% of delayed executions, so roughly half the attack space requires modeling the behavioral sequence rather than scanning individual steps.