StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns

arXiv·high signal

Stress-tests coding agents over 100 consecutive change requests on a REST API server (codebases up to 6,000 lines), across six harnesses and seven open-source LLMs. All tested models fail within 5-6 turns, but adding test feedback with retries improves performance up to 12x, and harness quality alone drives up to 6x variation. Strong evidence that harness design and feedback loops matter as much as the model for long-horizon agentic coding.