r/LocalLLaMA Benchmark Breaks the 'q8_0 KV Cache Is Lossless' Folk Wisdom — Gemma 4 Degrades ~9x More Than Qwen 3.6

r/LocalLLaMA (LocalBench)·medium signal

A widely-shared r/LocalLLaMA benchmark measured token-by-token KL divergence across ~250K tokens while varying only KV-cache precision (f16/q8_0/q4_0) on the same BF16 GGUF. Gemma 4 26B-A4B jumped to KL 0.377 at q8_0 (1.088 at q4_0), while both Qwen 3.6 models stayed under 0.04 at q8_0 and remained usable even at q4_0. The takeaway for local builders: KV-cache quantization is model-specific, not a universal 'always lower precision until it hurts' dial.