A Retrieval Index That Diagnoses Its Own Failures and Rewrites Its Keys

arXiv 2609.19656·medium signalXBlueskyLinkedInCopy link

arXiv 2609.19656 (submitted 17 Sep 2026, 22 upvotes on HuggingFace Daily Papers) targets the part of retrieval quality that depends on how index keys expose each document's knowledge, and the fact that the right representation varies by retrieval environment so no fixed strategy generalizes. SELF-INDEX gives the index an Optimizer that autonomously diagnoses retrieval shortfalls, selectively refines the optimization strategy, and reprocesses the index, replacing the human loop of diagnose, refine, reprocess. For anyone maintaining a RAG corpus that quietly degrades as queries drift, this is the loop that currently runs on a person's attention.