Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
MoE inference turns memory-bound during the low-batch decoding that coding assistants and voice apps use, per a new arXiv paper.
Why now: The paper posted to arXiv on August 17.
DeaMoE cuts per-step expert weight loading up to 50.9% by grouping experts into parameter-sharing departments, per a paper posted to arXiv on August 17.
Coding assistants and real-time voice apps run in exactly the low-batch regime where that weight loading turns memory-bound. The paper says post-training compression and fine-grained expert designs haven't fixed it, trading accuracy or adding communication overhead instead.
DeaMoE groups experts into departments that share most parameters, with a small private set kept per expert. A two-stage router then picks experts while avoiding redundant weight loads.
On a 7B model on an A40 GPU, the paper reports a 1.33x speedup in time-per-output-token end to end. Microbenchmarks on DeepSeek-V3 peaked higher: 2.00x on the A40, 1.97x on an H100.
A technique that cuts loaded weight without retraining or added cross-device communication changes the cost math for anyone serving MoE models at low concurrency.
The paper doesn't report accuracy deltas from the department grouping, only speed and weight-loading numbers.
Each link below shares sources, entities, or timing with this story.
Left alone with a Quran recitation dataset and an eval script, one agent memorized test rows while the other generalized, and only one held up on new data.
Small-batch decoding, which is the regime coding assistants and voice apps actually run in, makes MoE inference memory-bound on expert weight loading (arXiv 2608.14385). DeaMoE groups experts into departments sharing most parameters plus a small private set per expert, with a...
Weights and the full training stack are public, and the 35B agent beats GPT-5.5 plus Codex while capped at 12GB of VRAM.
The two judges scoring these 14,560 attacks disagreed by more than 3x on how often DeepSeek's agent partially complied.
A blinded judge checks root cause and impact against 95 real CVEs, and no frontier model made the ten-model lineup.
Planted skills captured the model's coordinator in 80% of test cases while runtime nearly doubled and task completion stayed unchanged.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.