Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-31 · high
A LocalLLaMA user ran Muse Glimmer 30B at 3.00bpw and reports no noticeable quality drop for agent work against the 17GB official quant.
Why now: Posted to r/LocalLLaMA on August 31.
A LocalLLaMA user got Muse Glimmer 30B running fully in 12GB of VRAM on an EXL3-SC quant at 3.00bpw H4, with a Q8 KV cache. The model's official K-quant needs 17GB, more than a 12GB card holds, with no noticeable quality drop against it for agent work.
A LocalLLaMA thread on EXL3 quants has the numbers: 100K tokens of context at around 30 tokens a second, all inside that 12GB budget. A 17GB model that doesn't fit on a 12GB card normally means spilling to system RAM or dropping to a smaller model instead. EXL3's 3.00bpw format skips both trade-offs.
The same tester tried Qwen 3.8 27B at a more aggressive SC2.20bpw H3 quant and called it usable. For coding, they still went back to Unsloth's UD_Q4_K_XL, a GGUF quant.
The post doesn't name the hardware beyond the 12GB VRAM ceiling. It has no formal eval score, just one person's read against their daily-driver quant.
Each link below shares sources, entities, or timing with this story.
Alibaba released Qwen3.6-27B on April 22. Dense architecture. Open weights. 77.2% on SWE-bench Verified, within 3.7 points of Claude Opus 4.6. On SkillsBench, it scores 48.2% versus its own 397B MoE predecessor's 30.0%. That's a 77% improvement with 14.8x fewer parameters. Let...
Somebody diffed the configs. Zero architectural changes. Same 64 layers, same 5,120 hidden dimension, same hybrid Gated DeltaNet → FFN / Gated Attention → FFN block structure as Qwen3.6-27B. The r/LocalLLaMA post showing this hit 945 upvotes and 157 comments, and Hugging Face...
Alibaba published a fine-grained MoE with 2.4T total / 95B active, 512 experts, and a 92-layer hybrid full/linear attention backbone. vLLM shipped day-0 support verified on NVIDIA and AMD with ready 4-bit checkpoints (NVFP4 at 1.32 TiB for an 8xB300 node, MXFP4 at 1.45 TiB for...
Google DeepMind released Gemma 4 on April 2 with four model sizes (E2B, E4B, 26B MoE, 31B Dense) under Apache 2.0. Multimodal (text, vision, audio). 256K context. Native thinking and tool-calling optimized for agentic workflows. Day-zero ecosystem support across vLLM, llama.cp...
PR #19378 landed in llama.cpp this week, and I think most people are underselling what it means. Backend-agnostic tensor parallelism via --split-mode tensor makes multi-GPU inference work across AMD, Intel, and Apple Silicon. Not just CUDA. Everything. For context: llama.cpp h...
Moonshot AI dropped Kimi K2.6 today and the numbers are hard to ignore. One trillion parameters total, 32 billion active per token across 384 experts, 256K context window, and native multimodal input. It scores 58.6 on SWE-Bench Pro versus GPT-5.4's 57.7 and Claude Opus 4.6's...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.