Kimi Publishes "Attention Residuals" — Rethinking Core Transformer Architecture

Moonshot AI (Kimi) published a paper proposing attention residuals to replace the residual connections used in every transformer since ResNet — routing skip-connection information through learned attention weights instead of direct addition. Results "look legit" at tested scales per r/LocalLLaMA (117 upvotes). If this validates at frontier scale, it's a meaningful departure from core transformer topology that's been stable since 2017. From the same team powering Cursor's Composer 2.