Google Ships Multi-Token Prediction for Gemma 4. Local Inference Just Got 3x Faster.

Google released open-source Multi-Token Prediction (MTP) drafters for the Gemma 4 model family. The concept: pair a heavy target model (Gemma 4 31B) with a lightweight drafter that predicts several future tokens in parallel. The target model verifies the predictions in a single forward pass. Result: up to 3x faster inference with identical output quality.

The drafters are Apache 2.0 on Hugging Face and Kaggle. They work with transformers, MLX, vLLM, SGLang, and Ollama. That compatibility list is why this matters. You don't need a new serving stack. You don't need to change your code. Drop in a drafter model alongside your existing Gemma deployment and inference gets faster.

At 645 points and 315 comments on Hacker News, this was the highest-engagement AI story of the day. The enthusiasm makes sense. Local inference speed has been the practical bottleneck for anyone running models on their own hardware. A 3x speedup changes what's viable. Tasks that felt too slow for interactive use become responsive. Batch jobs that took hours finish in one.

I've been running Gemma models locally on an M4 Max for personal project work, and speed is always the tradeoff you accept for privacy and cost savings. A 3x improvement is the difference between "tolerable" and "actually good." Especially for iterative coding workflows where you're making lots of small requests.

The technical approach, speculative decoding, isn't new. What's new is Google packaging it as a turnkey open-source solution that works across the major serving frameworks. That's the kind of practical engineering that moves adoption. Not a paper showing 3x speedup under lab conditions, but actual model weights you can download and run today.

If you're running Gemma 4 locally or on your own infrastructure, add the MTP drafters. Free performance.