Microsoft Releases STATE-Bench: Open-Source Memory-Agnostic Benchmark Measuring Whether Agents Actually Improve with Experience

Microsoft Open Source Blog·medium signal

Microsoft published STATE-Bench (Stateful Task Agent Evaluation Benchmark) on May 19, an open-source benchmark that goes beyond retrieval tests to measure whether agent memory translates to improved task completion in realistic enterprise scenarios. Each task includes a pre-populated database of artifacts (bookings, orders, carts), a customer with a specific problem, and deterministic state assertions defining success. The benchmark evaluates four dimensions: task completion, cross-run consistency, operational efficiency, and user communication quality — addressing the gap where most memory benchmarks only test retrieval plumbing, not downstream performance.