Scaling Karpathy's Autoresearch: Claude Code Runs 910 ML Experiments on a GPU Cluster in 8 Hours

The most compelling proof yet that autonomous AI research scales beyond demos and into real infrastructure.

SkyPilot engineers gave Claude Code access to 16 GPUs on a Kubernetes cluster and let it run. In 8 hours, the agent executed approximately 910 experiments, improving model validation loss from 1.003 to 0.974 — a 2.87% improvement — at a total cost of ~$300 in GPU compute plus $9 in API calls. That's roughly 9x the throughput of sequential human-guided research runs.

The surprising part isn't the volume. It's what the agent discovered on its own. Without any instruction about hardware optimization, Claude Code autonomously figured out that H200 GPUs completed 9% more training steps than H100s within the same budget window. It then self-developed a two-tier strategy: screen candidate experiments on H100s, validate winners on H200s. Nobody told it to do this. It derived it from experimental results.

This directly extends Karpathy's earlier work. His original single-GPU autoresearch paper showed the pattern was viable. Fortune covered his two-day continuous run this week — 700 experiments over 48 hours, 20 independently discovered optimizations — as a "why everyone is talking about this" moment. The SkyPilot result takes the same concept and proves it scales horizontally with commodity infrastructure. Give the agent more GPUs, it runs more experiments. Give it heterogeneous hardware, it optimizes across it.

The cost profile is what makes this actionable immediately. $300 in compute. $9 in API calls. That's a graduate student's weekly coffee budget for a research throughput that would take a human team weeks. Any team with GPU access and an API key can replicate this today. The agent doesn't need a custom framework — it's Claude Code with a SkyPilot config and access to standard ML tooling.

The implication for ML teams is structural: the bottleneck in hyperparameter search, architecture exploration, and training optimization is no longer compute or human attention. It's whether you've set up the scaffolding to let an agent run experiments autonomously. SkyPilot just published the scaffolding.