Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-18 · high
Five coding harnesses that pass identical tests burn up to ten times more tokens than each other, and extra spend can't recover a fact that's missing.
Why now: The paper posted to arXiv as 2608.16630 on August 18, 2026.
A new study tested seven models across five coding harnesses, supplying facts, withholding them, and injecting faults to see what broke. Five harness setups in the study passed every test, yet differed by more than tenfold in tokens spent. Each one rebuilds the same facts at a different rate, and that gap is real cost for anyone running these tools daily.
They also swapped in each model's own parametric memory instead of supplying the fact directly. What decided the outcome wasn't how far a fact sat from the edit. It was whether the fact was available at all. A supplied fact worked as well far from the change as right next to it, per the paper.
When a fact isn't supplied, the agent doesn't stop and say it doesn't know. It fabricates a file or guesses a value and keeps going. Anyone watching agent runs by checking whether a read happened will see the hole already filled with something wrong.
Spending more tokens doesn't fix a fact that's genuinely absent, the study found. And where a CLAUDE.md-style convention file disagrees with what the code actually does, agents follow the stale file over the code.
Each link below shares sources, entities, or timing with this story.
The rule text can survive context compaction while the behavior it enforces quietly stops, and grepping the summary for that text won't catch the difference.
Four model tiers spanning a 15x price gap failed at the same rate: no model buys its way out of a stale-data problem.
The technique filters terminal output an agent reads one line of, and resolution rates held steady on 50 SWE-bench Lite tasks.
A new checkpoint system saves an agent's memory and its environment together, so a failed attempt feeds the next try instead of getting discarded.
A preregistered test of 18,000 multi-agent missions shows failures cluster instead of scattering, which breaks the math teams use to size redundancy.
An 8,135-trial study finds skill files mostly lock in a procedure, and a 100-item skill pool nearly kills retrieval accuracy.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.