DEX-Comp Trains RAG Context Compression Only on Queries the Uncompressed System Fails, Hitting 16x Compression and 4-24x Speedup

arXiv 2609.05152·medium signalXBlueskyLinkedInCopy link

arXiv 2609.05152 argues that soft context compression trained by distilling an uncompressed RAG system's outputs is capped by that system's performance. DEX-Comp uses a two-stage recipe: Pure Distillation warm-starts on the uncompressed RAG's correct responses only, then Hard Exploration runs reinforcement learning solely on queries the uncompressed RAG fails, forcing computation patterns suited to compressed representations. Across five open-domain QA benchmarks at retrieval depths from top-5 to top-30 it compresses context 16x and accelerates inference 4x to 24x while matching or exceeding the uncompressed baseline.