Reranking 50 documents from the old index matched native retrieval on a Qwen 4B to 8B embedding upgrade

r/LocalLLaMA (cross-posted to r/MachineLearning)·medium signalXBlueskyLinkedInCopy link

A builder posted embedflow, which sidesteps the backfill cost of switching embedding models: instead of re-embedding the corpus, pull the top K documents from the old index and rerank them with the new model. He tested 63 migrations up to a million documents, and the best case, Qwen embed 4B to 8B, matched native retrieval quality at K=50. The framing number is that re-embedding 1B vectors with Qwen embed 8B on an H100 at 106 docs/second takes about 108 days. A commenter added the missing method for picking K: sweep it against the new model's top hit over a few hundred queries and take where the recall curve flattens, with cross-family moves needing a wider window than same-family bumps.