Gemma 4 QAT checkpoints drop the smallest model from 11.4GB to 1.1GB

Google DeepMind shipped Quantization-Aware Training checkpoints for every Gemma 4 size, and the headline number is genuinely useful: the smallest model goes from 11.4GB to 1.1GB. That's 0.84GB if you go text-only. Up to ~72% lower VRAM and 2x faster inference on mobile NPUs, with quality preserved because QAT bakes the quantization into training rather than bolting it on after. Source: Google / MarkTechPost

The reason this lands today, next to a story about AI data center buildout hitting 0.8% of US GDP, is that local inference is the pressure valve. Not everything needs to hit a frontier API. A 1.1GB model that runs on a phone NPU or a cheap edge box is the difference between an agent that costs per-token and one that costs nothing after you've downloaded it.

The release came with same-day support across the stack. Q4_0 GGUF for llama.cpp, a new mobile-specialized quant format, compressed tensors for vLLM, plus Ollama and vLLM running it out of the gate. That same-day part matters more than people give it credit for. A model you can't deploy until the tooling catches up is a press release. A model that runs in Ollama the day it drops is a tool.

Now the gotcha, because there's always one. Unsloth's Daniel Han flagged that naive QAT-to-Q4_0 conversion actually loses accuracy. The dynamic GGUF recovers it. So if you grab these and do the obvious conversion, you can end up worse off than you expected and blame the model. Use the dynamic GGUF builds. This is exactly the kind of detail that costs you an afternoon of confused benchmarking if you don't know it going in.

What to do: if you've got any workload running a small model locally, classification, routing, cheap summarization, the grunt work an Adaptive RAG router hands off, re-benchmark it against quantized Gemma 4. The VRAM math alone might let you consolidate two GPUs down to one, or move a task from cloud to a box under your desk. And keep the dynamic-GGUF note pinned somewhere, because you will forget it and you will waste the afternoon.