Grouped Value Attention stops storing keys entirely, cutting persistent KV cache scalars 45-47% against matched GQA

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2609.13285, submitted 2026-09-08, observes that GQA still writes both a key and a value at every decode step and proposes storing only grouped values, reconstructing content keys through a learned linear map that can be absorbed into the query so content keys never materialize. A small shared decoupled RoPE channel carries position through a separately cached positional key. At 350M parameters on 30B FineWeb-Edu tokens, the 16-dimensional positional variant hits 44.18 average accuracy across five tasks against 44.36 for GQA and 43.88 for MLA, so roughly half the cache for roughly the same score, with custom decoding kernels still in progress.