Reranking

Reranking is the second stage in a two-stage retrieval pipeline: the first stage (BM25, vector search, or hybrid) cheaply surfaces a candidate set of perhaps 50-200 documents, and a second-stage reranker re-scores that candidate set with a more expensive model to produce the final top-k. The typical reranker is a cross-encoder — a transformer that takes the query and candidate document together and outputs a single relevance score — which captures query-document interactions that bi-encoder embedding retrieval cannot (the two texts are jointly encoded rather than compared by vector distance over independently-computed embeddings). The cost is latency; a cross-encoder is orders of magnitude slower per pair than a vector lookup, which is why reranking operates only on the narrowed candidate set.

The 2026 landscape has matured around a handful of products and open models. **Cohere Rerank 3** and **Voyage Rerank 2** are the leading managed APIs with multilingual strength important for APAC corpora. Open-weight rerankers — **BGE-reranker-v2**, **Mixedbread mxbai-rerank-large**, **Jina Reranker v2**, and **ColBERTv2 / late-interaction variants** — cover self-hosted use. Multi-stage rerankers (cross-encoder for top-100, ColBERT for top-1000) are appearing in high-stakes retrieval. Latency budgets for reranking typically run 100-400ms on top-100 candidates with a modern large reranker, which is affordable for most interactive RAG but prohibitive for high-QPS search.

For APAC mid-market teams, reranking is worth the latency cost **when first-stage recall is good but precision is middling** — i.e. the right passage is in the top-50 but not the top-5. Instrument retrieval to measure recall@50 versus precision@5 before deciding. If recall@50 is low, fix the first stage (better embeddings, better chunking, better hybrid weights); reranking cannot recover passages that were never surfaced. Multi-lingual corpora especially benefit — the strong multilingual rerankers (Cohere, Voyage, Jina) outperform most first-stage retrievers on Japanese, Korean, and Traditional Chinese content.

The non-obvious failure mode is **reranking garbage-in**. A reranker's job is to reorder a candidate set; if the relevant passage is not in the set, reranking cannot put it there. Teams see a weak first stage, add a reranker, measure recall@5, and conclude the reranker is ineffective — when the actual failure was upstream. The right debugging protocol is: measure recall@N at the first-stage candidate size (usually N=50 or 100); if it is low, fix retrieval before adding a reranker. Reranking turns recall into precision — it cannot manufacture recall the first stage did not produce.