OasisKV moves the KV cache off HBM entirely and gets 6.5–9.7x less cache per request for 0.1 points of reasoning accuracy.

arXiv 2608.08097 exploits decode-time attention sparsity: keep a 2,048-token budget resident and speculatively prefetch the rest via lookahead prediction. Reported results are 1.69x speedup on reasoning workloads, up to 2.1x on multi-GPU long-context serving, roughly 2x throughput under prefill-decode disaggregation, and 2.2–2.6x less decode-node host memory. Accuracy lands within 0.7 points of full attention overall. If this replicates, long-context serving cost is a memory-hierarchy problem, not a hardware-budget one.