Cross-Encoder Re-ranking — 18–42% Precision Gain with Net Cost Savings.

Add a cross-encoder after vector retrieval: each (query, chunk) pair scored independently rather than by embedding similarity. 50–200ms latency overhead offset by passing fewer, better chunks to the LLM. At scale, generation savings exceed re-ranker cost. Best options: Cohere Rerank v3.5, Jina Reranker v2, BGE-Reranker-v2-m3. Abhishek Gautam