Gemma 4 Local Inference: Community Benchmarks Show 95 tok/s on MacBook (E2B) but Reveal KV Cache Memory Issues at Long Context

r/LocalLLaMA / Hacker News·medium signal

Day-1 community benchmarks from r/LocalLLaMA and Hacker News reveal practical Gemma 4 deployment realities: E2B hits 95 tok/s on MacBook, E4B gets 57 tok/s, but the 26B MoE model drops to ~2 tok/s with swap on 24GB Macs. Critical finding: the 31B model at 256K context requires ~22GB just for KV cache on top of model weights — Google did not adopt KV-reducing techniques from Qwen 3.5. The 26B MoE is the sweet spot, achieving near-13B quality at 4B speed with only 3.8B active parameters.