Nonstandard Errors in AI Agents: 150 Claude Code Agents Produce Divergent Empirical Results on Identical Tasks

arXiv 2603.16744·high signal

Researchers deployed 150 autonomous Claude Code agents to independently test six financial market hypotheses using the same NYSE TAQ data, finding substantial agent-to-agent variation ('nonstandard errors') analogous to human researcher disagreement. Different model families exhibit stable 'empirical styles' — Sonnet 4.6 and Opus 4.6 make systematically different methodological choices. Critically, AI peer review had minimal effect on dispersion, but exposure to exemplar papers reduced estimate interquartile range by 80–99%, though convergence happened via imitation rather than understanding.