Late chunking: embed the whole document first, then split — up to 67% fewer top-20 retrieval misses

KX Systems / Michael Ryaboy·medium signal

Late chunking reorders the RAG pipeline: run the entire document through a long-context embedding model (8,192+ tokens) to get contextually-enriched token embeddings, then apply chunk boundaries afterward, so each chunk carries surrounding context. Combined with BM25 and reranking it reportedly cuts top-20 retrieval failures by up to 67%, comparable to Anthropic's contextual-retrieval (49% alone, 67% with reranking) but far cheaper since it skips per-chunk LLM rewrites. Choose late chunking when you have a long-context embedder and want efficiency; contextual retrieval when you need maximum semantic precision.