Fetching from the wire…
Models2026-08-27 · source-backed
A 217-upvote writeup breaks down Qwen4Exp: a router picks experts late in the layer with a large payload, so experts resist offload, while an n-gram table is a hash lookup known early and cheap to fetch. The author concludes roughly 25% of weights can move to n-gram before the tradeoff inverts, turning a 176B model into 125B in RAM plus 51B elsewhere (r/LocalLLaMA). The thread's correction to the original SSD claim is the operational detail: the n-gram index is hashed, so access is random and RAM is the only tier that works.
Each link below shares sources, entities, or timing with this story.
A 1,134-upvote r/LocalLLaMA post pushes back on the claim that n-gram tables let you run 1T+ models with 980B parameters offloaded to SSD (r/LocalLLaMA). An engram is an embedding table keyed on the last two or three tokens rather than one token ID, so "New York" gets a memori...
Staged on ModelScope for 23:00 Beijing time August 26, it's roughly 125B parameters plus a separate N-gram embedding table of about 51B, activating 6B per token, with GDN gated-delta hybrid layers and Qwen Sparse Attention. Alibaba frames it as a technology preview of the Qwen...
PR 27742 adds gated delta net layers, 512-expert MoE with top-10 selection, hyper-connections, query-key sparse attention and the per-layer n-gram embeddings as a mmap table that can sit in RAM or on disk (GitHub). Reported 55 tok/s on 4x3090 with the Q4 GGUF, and one commente...
215 points on Show HN, 4.5-5 tokens/sec on an M5 Mac, plus a 35B model on an iPhone. Critics in the thread landed the real objection: prefill is the bottleneck, roughly half an hour to process 10k tokens on an M5. The architecture bet is what's interesting, though. MoE's ~3B a...
An r/LocalLLaMA post hit 809 upvotes and 143 comments on Daniel Han's claim that the unreleased 27B will run on 17GB RAM/VRAM setups. Unsloth's X account confirms the number verbatim. Alibaba has published no benchmark table, license, or activated-parameter count for the 27B....
DeepSeek-V4-Flash-0731 landed July 31 under MIT with a DSpark speculative-decoding module attached. Terminal Bench 2.1: 82.7. Toolathlon-Verified: 70.3. DSBench-FullStack: 68.7. DeepSWE: 54.4. NL2Repo: 54.2. The model card claims it beats DeepSeek-V4-Pro (Preview) "despite its...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.