Per-Head Adaptive KV Cache Quantization Proposed: Lossless q4_0 on Hybrid Models at 4x Compression

GitHub / llama.cpp·medium signal

A llama.cpp GitHub issue proposes per-head adaptive KV cache quantization, achieving completely lossless q4_0 compression on hybrid attention models like Qwen3.5 (BLEU 1.000 across 10 test configs at 4x compression). Key insight: the bottom 2% of attention heads by entropy ('sink heads') contribute disproportionately to quantization error — skipping just 3 of 144 heads outperforms optimal bit redistribution. A 33K-token conversation ran at 36.5 tok/s where f16 would OOM on 12GB VRAM.