Agents' Last Exam (ALE): 1,000+ Economically Valuable Tasks, Hardest Tier at 2.6% Pass Rate

Agents' Last Exam·medium signal

ALE introduces a benchmark of 1,000+ economically meaningful tasks mapped to the U.S. occupational taxonomy and spanning finance, healthcare, legal, engineering, and manufacturing — domains where most prior benchmarks run short-horizon tasks scored by LLM judges. The hardest tier averages just 2.6% full pass rate. The explicit thesis: SWE-bench made coding 'quantifiable and improvable,' and ALE aims to do the same for the rest of GDP-weighted knowledge work.