Fetching from the wire…
01
02
03
04
05
06
07
08
Top 5 · 2026-05-06 · source-backed
Google released open-source Multi-Token Prediction (MTP) drafters for the Gemma 4 model family. The concept: pair a heavy target model (Gemma 4 31B) with a lightweight drafter that predicts several future tokens in parallel. The target model verifies the predictions in a single forward pass. Result: up to 3x faster inference with identical output quality.
The drafters are Apache 2.0 on Hugging Face and Kaggle. They work with transformers, MLX, vLLM, SGLang, and Ollama. That compatibility list is why this matters. You don't need a new serving stack. You don't need to change your code. Drop in a drafter model alongside your existing Gemma deployment and inference gets faster.
At 645 points and 315 comments on Hacker News, this was the highest-engagement AI story of the day. The enthusiasm makes sense. Local inference speed has been the practical bottleneck for anyone running models on their own hardware. A 3x speedup changes what's viable. Tasks that felt too slow for interactive use become responsive. Batch jobs that took hours finish in one.
I've been running Gemma models locally on an M4 Max for personal project work, and speed is always the tradeoff you accept for privacy and cost savings. A 3x improvement is the difference between "tolerable" and "actually good." Especially for iterative coding workflows where you're making lots of small requests.
The technical approach, speculative decoding, isn't new. What's new is Google packaging it as a turnkey open-source solution that works across the major serving frameworks. That's the kind of practical engineering that moves adoption. Not a paper showing 3x speedup under lab conditions, but actual model weights you can download and run today.
If you're running Gemma 4 locally or on your own infrastructure, add the MTP drafters. Free performance.
Each link below shares sources, entities, or timing with this story.
Ollama uses MLX / Shared entities / Same source domain / Shared topic / What happened next
Linked by a graph relationship (Ollama uses MLX); both cover Apache, Gemma, Google, MLX; reported by the same outlet (blog.google).
Ollama supports Gemma / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Ollama supports Gemma); both cover Apache, Gemma, Google, Ollama; reported by the same outlet (blog.google).
Ollama supports Gemma / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Ollama supports Gemma); both cover Apache, Gemma, Google, Kaggle; overlapping topics (gemma, google, model).
Ollama supports Gemma / Shared entities / Same source domain / Shared topic / What happened next
Linked by a graph relationship (Ollama supports Gemma); both cover Apache, Gemma, Google, Hugging Face; reported by the same outlet (blog.google).
Linked by a graph relationship (Ollama supports Gemma); both cover Gemma, Google, Ollama; reported by the same outlet (blog.google).
Ollama supports Gemma / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Ollama supports Gemma); both cover Apache, Gemma, MLX; reported by the same outlet (blog.google).
Ollama supports Gemma / Shared entities / Shared topic / What happened next / Tension
Linked by a graph relationship (Ollama supports Gemma); both cover Gemma, Google, Hugging Face; overlapping topics (gemma, google).
Ollama uses MLX / Shared entities / Shared topic / What happened next
Linked by a graph relationship (Ollama uses MLX); both cover Gemma, MLX, Ollama; overlapping topics (faster, gemma, model).