ETH Zurich: LLM-Generated AGENTS.md / CLAUDE.md Context Files Reduce Agent Task Success by 3%, Raise Costs 20%+

Stop auto-generating your context files. The data says they're actively hurting you.

ETH Zurich tested Claude 3.5 Sonnet, GPT-5.2, GPT-5.1 mini, and Qwen Code across 138 real-world Python tasks and found that LLM-generated context files (AGENTS.md, CLAUDE.md) consistently degraded task success rates by 3% while increasing inference costs by over 20%. The mechanism: agents mechanically followed auto-generated instructions and over-explored — visiting more files, running more commands, and burning more tokens without improving outcomes. InfoQ

Human-written files performed marginally better — +4% success at +19% cost — but architectural overviews provided essentially zero benefit. Agents couldn't translate general guidance like "this project uses a hexagonal architecture" into targeted problem-solving on specific bug fixes.

The practical verdict is precise: avoid auto-generating context files entirely. Reserve human-written CLAUDE.md for information the agent cannot infer from the codebase — custom build commands, non-standard tooling, domain-specific terminology. Everything else is noise that costs you tokens and makes the agent worse. This directly challenges the "vibe-code your setup" trend where teams use Claude to generate their own CLAUDE.md. That practice now has empirical evidence against it. Keep it under 30 lines. Make every line something the agent literally cannot figure out by reading package.json and the directory structure.