Sharp Capacity Scaling of Spectral Optimizers: Theoretical Foundation for Why Muon Works in LLM Training

arXiv·medium signal

Provides the first rigorous theoretical analysis of spectral optimizers like Muon, which have shown strong empirical performance in large-scale LLM training. The paper derives sharp capacity scaling laws for associative memory learning, explaining why spectral optimizers outperform SGD and Adam in specific regimes. Matters because Muon adoption is accelerating and practitioners need to know when it actually helps vs. when it's hype.