ExploitBench: Capability Ladder Decomposes Exploitation into 16 Measurable Flags for LLM Agents

arXiv·high signal

ExploitBench introduces a graded benchmark that breaks exploitation into 16 progressive capabilities — from crash through sandbox primitives, arbitrary read/write, and code execution — with deterministic oracles and randomized challenge-response verification. Results show a 'sharp capability split' between public models (which crash but can't escalate) and private frontier models (arbitrary code execution on ~50% of targets).