Two-stage retrieve→rerank with Matryoshka dimensions: retrieve at 256 dims, rerank at full 3072

FreeAcademy·medium signal

The 2026 production RAG default is now a two-stage pipeline: pull top 50–100 candidates with cheap bi-encoder embeddings, then jointly score each (query, chunk) pair with a cross-encoder reranker (Cohere Rerank 3.5, Voyage rerank-2.5, Jina v2). The efficiency trick is Matryoshka embeddings — use truncated 256-dim vectors for the fast first pass and full 3072 dims only where precision matters, avoiding paying for high dimensionality on every candidate.