Benchmark scores AI loop controllers apart from coder

LoopArena's best-performing loop controller clears just 24.69% strict success across tested coding tasks, according to the LoopArena paper.

That matters for anyone building agent harnesses. It suggests the surrounding loop logic, not the code-writing model, caps performance in these tests.

A single fixed Worker agent writes the code in every test. Separate Controller models see only a structured summary after each round, then decide what to try next, whether to verify, or when to stop.

The paper's cheaper Type II evaluation setting reproduces that same Controller ranking at a Spearman correlation of 0.9747. The correlation is close enough to rank candidate controllers without running full end-to-end tasks.

Paired against that shortcut, tested Controllers cut estimated inference cost by an average of 64.4%. Code for the benchmark is posted on GitHub under AMAP-ML.

The Worker never changes across tests. If a better Controller still can't push strict success past 24.69%, the bottleneck is deciding when to stop, not how to code. Worth watching whether other benchmarks start separating Controller from Worker, since bundling the two hides which part is broken.