Vera Framework Auto-Generates Agent Safety Tests; 93.9% Attack Success on Production Frameworks

arXiv·medium signal

The Vera framework (arXiv 2607.01793, submitted July 2, revised July 4) automates LLM-agent safety testing via literature-driven risk discovery, combinatorial case generation, and sandboxed adaptive execution with evidence-grounded verifiers that judge tool-call evidence rather than model self-report. Testing four production agent frameworks yielded a 93.9% average attack-success rate under multi-channel attacks. The authors release Vera-Bench with 1,600 executable cases across 124 risk categories — a directly usable red-team asset for agent builders.