Early cache compaction costs AI agents accuracy

Cache compaction hurts AI agent accuracy when it happens too early, per the arXiv paper testing it on two agent benchmarks.

That matters for anyone running multi-turn agents: coding assistants, search agents, anything that keeps a growing context window across turns. Trimming that cache to save memory can quietly cut into how well the agent answers its next question. Most compaction research doesn't test for that failure mode.

Nearly all existing compaction work scores against a future query that's already known ahead of time, a setup borrowed from offline benchmarks. Agents don't get that luxury. They generate their next query as they go, so there's no fixed target to compact against until that query actually exists.

The study tested two compaction methods, token eviction and attention matching, across several proxy-query sources on the BrowseComp-Plus and WideSearch benchmarks. Compacting a turn right after it happened often hurt accuracy under both methods. Waiting until the agent's own next query existed, then using that real query as the proxy, recovered most of the lost accuracy.

Token eviction, the plainer of the two methods, held up better than attention matching when the proxy query was a poor match. It preserved most of the accuracy even after cutting the cache by 80%.

For anyone building agent loops with long context windows, that points to a specific fix. Skip the fixed compaction schedule. Trim once the next tool call or query exists.