Fetching from the wire…
Models2026-08-28 · source-backed
A 1,134-upvote r/LocalLLaMA post pushes back on the claim that n-gram tables let you run 1T+ models with 980B parameters offloaded to SSD (r/LocalLLaMA). An engram is an embedding table keyed on the last two or three tokens rather than one token ID, so "New York" gets a memorized vector via an O(1) hash lookup with no FLOPs, freeing early transformer layers from re-deriving multi-token entities. Because lookups cost no compute, you can quantize weights to Q4_K_XL while keeping the engram table at native precision. Commenters report the practical tell is better letter counting with minimal reasoning and better negation handling.
Each link below shares sources, entities, or timing with this story.
A practitioner ran Kimi K3 on 8x B300 via Modal at $56.79/hour with vLLM, TP8 and native MXFP4: 27-minute cold boot for a 1.56 TB load, TTFT 0.92 to 1.02s, 92 tok/s steady decode, roughly $36 of GPU time per clean run and $1,363/day left warm. The cheaper path was worse. Unslo...
DeepSeek released V4 on April 24 and the numbers demand attention. V4-Pro is 1.6 trillion parameters total with 49 billion active, MIT-licensed, native 1M-token context. It scores 80.6% on SWE-bench Verified, putting it within 0.2 points of Claude Opus 4.6. On Terminal-Bench 2...
A 217-upvote writeup breaks down Qwen4Exp: a router picks experts late in the layer with a large payload, so experts resist offload, while an n-gram table is a hash lookup known early and cheap to fetch. The author concludes roughly 25% of weights can move to n-gram before the...
Alibaba released Qwen3.6-27B on April 22. Dense architecture. Open weights. 77.2% on SWE-bench Verified, within 3.7 points of Claude Opus 4.6. On SkillsBench, it scores 48.2% versus its own 397B MoE predecessor's 30.0%. That's a 77% improvement with 14.8x fewer parameters. Let...
Staged on ModelScope for 23:00 Beijing time August 26, it's roughly 125B parameters plus a separate N-gram embedding table of about 51B, activating 6B per token, with GDN gated-delta hybrid layers and Qwen Sparse Attention. Alibaba frames it as a technology preview of the Qwen...
A developer published v100-skinny with hand-written NVFP4 W4A16 CUDA kernels plus chain-MTP speculative serving: four V100s at 219.1 ± 5.9 tok/s decode against a 5090 running NInfer at 214.7 ± 9.2, both 5/5 correct on AIME 2026 problem 1 across five seeds. (r/LocalLLaMA) The m...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.