Human-Curated CLAUDE.md Files Beat LLM-Generated Ones by 4.5-6.5 Points. LLM-Generated Files Actually Made Things Worse.

An ArXiv study analyzing Claude Code's design space found something that should make every "auto-generate your context files" workflow uncomfortable. Human-curated CLAUDE.md files improved task success rates by roughly 4 percentage points. LLM-generated CLAUDE.md files reduced success rates by 0.5-2% AND increased inference costs by over 20%.

Read that again. Having the AI write its own instructions made it perform worse and cost more.

I've been maintaining my own CLAUDE.md for months now, and this confirms what I noticed empirically. The entries that work are the ones I wrote after watching Claude make the same mistake three times. "Don't mock the database in integration tests." "Always check if the branch exists before creating it." "Use WAL mode for all SQLite connections." Each one traces back to a real failure. They're specific, opinionated, and born from pain.

The entries that don't work are the speculative ones. "Consider edge cases carefully." "Follow best practices for error handling." That kind of generic guidance is what LLMs generate when you ask them to write context files. It sounds reasonable but doesn't change behavior. Worse, it bloats the context window, which means the actually useful rules get less attention.

The study also confirmed something I'd suspected about Claude Code's extension mechanisms: MCP servers, plugins, skills, and hooks all operate at different points of the agent loop with different context costs. Knowing which lever to pull matters. A hook that preprocesses output is cheap. An MCP server that's always loaded is expensive. Choose accordingly.

What builders should do: Stop auto-generating CLAUDE.md files. Write entries only when you encounter a repeated agent mistake. Keep them short, specific, and grounded in observed failures. If you haven't seen the failure three times, you don't need the rule. Delete any speculative guidance. Your context budget is finite. Spend it on rules that actually change behavior.