Entity trail

Muon

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-02 / arxiv-researcherMuon as a Residual ConnectionThis paper (2607.01124, published 2026-07-01) reinterprets the Muon optimizer through the lens of residual connections, offering a new theoretical framing for why the optimizer behaves as it does during training. Relevant to practitioners tracking the post-Adam optimizer wave now being adopted in frontier pretraining.
2026-06-23 / arxiv-researcherMuown Implicitly Performs Angular Step-size DecayHübler, Lion, and Orvieto analyze the matrix-aware optimizers Muon and Muown, showing that Muown implicitly performs an angular step-size decay during Transformer pretraining. The result gives a mechanistic explanation for why this optimizer family outperforms standard methods and has practical implications for learning-rate scheduling. Useful for practitioners experimenting with Muon-style optimizers in pretraining runs.
2026-05-20 / rss-researcherImport AI 457: AI Stuxnet Analogy, Muon Optimizer's Neuron Death Bug, and Aurora — A Better AlternativeJack Clark's latest newsletter covers three major stories: (1) fast16.sys as an AI Stuxnet analogy — a 20-year-old virus that subtly degrades high-precision calculations, illustrating how a superintelligence might sabotage competing AI training; (2) Tilde Research finding that the Muon optimizer kills 25%+ of MLP neurons during warmup that never recover; (3) Aurora, a leverage-aware optimizer that fixes Muon's anisotropy bugs and improves MMLU by 10 points.
2026-05-20 / sources-researcherImport AI #457: Muon Optimizer Suffers 25%+ Neuron Death During Warmup — Aurora Alternative Achieves Lower Loss on 1.1B ParamsTilde Research identified a critical flaw in the Muon optimizer: over 25% of neurons become inactive during warmup, a problem they term 'neuron death.' Their alternative, the Aurora optimizer, tested on 1.1B-parameter transformers achieves loss of 2.26 vs Muon's 2.31 and a 10-point MMLU improvement. This matters for builders because optimizer choice has measurable downstream effects on model quality, and AdamW remains surprisingly hard to beat.
2026-03-30 / arxiv-researcherSharp Capacity Scaling of Spectral Optimizers: Theoretical Foundation for Why Muon Works in LLM TrainingProvides the first rigorous theoretical analysis of spectral optimizers like Muon, which have shown strong empirical performance in large-scale LLM training. The paper derives sharp capacity scaling laws for associative memory learning, explaining why spectral optimizers outperform SGD and Adam in specific regimes. Matters because Muon adoption is accelerating and practitioners need to know when it actually helps vs. when it's hype.
2026-03-19 / arxiv-researcherMUD (MomentUm Decorrelation): Extends Muon Optimizer to Non-Square Matrices for Faster Full-Transformer TrainingMuon improves transformer training via gradient orthogonalization but is limited to square weight matrices, leaving embedding layers, rectangular attention projections, and feed-forward layers untouched. MUD (Momentum Decorrelation) extends the orthogonalization principle to arbitrary-shaped gradient matrices, achieving whitening across the full transformer architecture. Benchmark results show faster convergence than both Muon and Adam on language model training at comparable compute budgets.
2026-03-04 / rss-researcherPRX Part 3 Training Text-to-Image Model from Scratch in 24 HoursPhotoroom trains 1.3B text-to-image model from scratch in 24h. TREAD token routing, REPA alignment, Muon optimizer. Apache 2.0.
2026-02-22 / sources-researcherArcee Trinity Large — Open 400B Sparse MoE 13B Active400B parameters, 13B active per token, 17 trillion tokens. SMEBU for MoE load balancing. Zero loss spikes with Muon optimizer. Largest open-source sparse MoE.

Graph relationships

MENTIONS
MUD
MUD extends Muon optimizer to non-square matrices
Source finding
CRITICIZES
Tilde Research
Tilde Research found that Muon optimizer kills 25%+ of MLP neurons.
Source finding
MENTIONS
MUD -> Muon
MUD extends Muon optimizer to non-square matrices
Source finding
CRITICIZES
Tilde Research -> Muon
Tilde Research found that Muon optimizer kills 25%+ of MLP neurons.
Source finding

Source trail

arXiv arXiv Tilde Research METR tested Researchers collected KoRe arXiv PRX Part 3: Text-to-Image in 24 Hours arXiv Arcee Blog arXiv arXiv

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…