ETH Zurich: LLM-Generated AGENTS.md / CLAUDE.md Context Files Reduce Agent Task Success by 3%, Raise Costs 20%+

InfoQ·high signal

An ETH Zurich study tested Claude 3.5 Sonnet, GPT-5.2, GPT-5.1 mini, and Qwen Code across 138 real-world Python tasks and found that LLM-generated context files (AGENTS.md, CLAUDE.md) consistently degraded task success rates by 3% while increasing inference costs by over 20%, because agents mechanically followed instructions and over-explored. Human-written files provided only marginal gains (+4% success, +19% cost), and architectural overviews gave essentially no benefit—agents couldn't translate general guidance into targeted problem-solving. The practical verdict: avoid auto-generating context files and reserve human-written ones for non-inferable details like custom build commands or domain-specific tooling.