TurboQuant: Google Research Ships Near-Optimal KV Cache Compression — 3 Bits Per Coordinate, Zero Accuracy Loss

Google Research Blog·high signal

Google Research and NYU published TurboQuant (presented at ICLR 2026), a two-stage vector quantization algorithm compressing LLM KV caches to 3 bits per coordinate with zero accuracy loss. It combines PolarQuant (rotation-based coordinate transform) with a 1-bit QJL residual correction, achieving 6x memory reduction and up to 8x faster attention on NVIDIA H100 GPUs while operating within ~2.7x of the information-theoretic optimum. Community open-source implementations began appearing in early June 2026 — directly actionable for anyone running inference at scale.