Hybrid search is sold as a free upgrade: run BM25, run a vector search, merge the two lists, get the best of both. The merge step is where most of the behaviour actually lives, and the default merge — Reciprocal Rank Fusion — throws away every score either retriever produced. It only looks at positions.
That is not a bug. RRF was designed that way because lexical scores and embedding similarities are not on comparable scales. But it means a single integer constant in your config decides whether your system favours documents that one retriever loves or documents both retrievers merely tolerate. If you have never looked at that constant, you have never actually chosen your ranking policy.
What RRF actually computes
Reciprocal Rank Fusion was introduced by Cormack, Clarke and Buettcher at SIGIR 2009. The formula is small enough to write in one line:
score(d) = Σ_i 1 / (k + rank_i(d))
For each retriever i that returned document d, you take its 1-based rank in that retriever's list, add a constant k, and invert. Sum across retrievers. Documents missing from a list contribute nothing from that list.
The original paper used k = 60, and that value propagated into production systems — Elasticsearch and OpenSearch both default their RRF rank constant to 60. Nothing about 60 is derived from your corpus, your query mix, or your retrievers. It is an inherited default.
Notice what is absent: the BM25 score, the cosine similarity, and any notion of how far ahead the top result was. A lexical hit that scores twice as high as everything below it and a lexical hit that barely edges out rank 2 both contribute exactly 1/(k+1).
Why the rank constant changes the winner
k controls how steeply the contribution curve decays. Small k makes the top of each list enormously valuable. Large k flattens the curve so that appearing in both lists at all matters more than where you appear.
Three documents, three values of k:
| Document | Ranks | k = 0 | k = 10 | k = 60 |
|---|---|---|---|---|
| A | Lexical #1 only | 1.000 | 0.0909 | 0.0164 |
| B | Lexical #20, vector #20 | 0.100 | 0.0667 | 0.0250 |
| C | Vector #3, lexical #50 | 0.353 | 0.0936 | 0.0250 |
At k = 0, the document that one retriever ranked first wins by an order of magnitude. At k = 60, a document that both retrievers put twentieth wins. The documents did not change. The corpus did not change. The ranking policy changed, silently, inside a parameter most teams never touch.
This is the practical meaning of "RRF favours consensus": at the standard k = 60, two mediocre agreements beat one strong specialist opinion. If your vector index is the only component that understands paraphrase, a high k systematically suppresses exactly the results you added it for.
The candidate depth cliff nobody logs
Each retriever returns a finite list — often 50 or 100 candidates before fusion. A document at rank 101 in a depth-100 list does not get a small contribution. It gets zero. It is as if that retriever had never seen it.
That creates a discontinuity you can trip over with a one-character config change:
{
"retriever": {
"rrf": {
"retrievers": [ /* bm25, knn */ ],
"rank_window_size": 100,
"rank_constant": 60
}
}
}
Raising rank_window_size from 50 to 100 does not just add tail results. It promotes documents that were previously single-list contributors into dual-list contributors, and at k = 60 that roughly doubles their score. Results in your top 5 can be reordered by a change that looks like it only affects position 60 onward.
Another quiet failure: chunking. If you retrieve at passage level and present at document level, a long document with eight near-duplicate chunks occupies eight slots in the vector list, pushing other documents past the depth cutoff before fusion even runs. Deduplicate to one chunk per document before the merge, or your depth parameter means something different for long pages than for short ones.
Why you cannot threshold an RRF score
RRF scores are bounded by the number of retrievers and the constant — with two retrievers and k = 60, nothing can exceed about 0.0328. They are also entirely relative to the query. A query where everything retrieved is garbage produces the same score distribution as a query with five perfect answers, because ranks always start at 1.
This breaks the most common RAG safety valve: "only pass chunks above score X into the prompt." There is no calibrated X. Any cutoff you pick is really "take the top N", dressed up as a quality filter.
If you need an abstain path, it has to come from somewhere other than the fusion score:
- A cross-encoder reranker over the fused candidates, whose output is calibrated per query-document pair.
- A threshold on the pre-fusion retriever scores, applied before merging.
- A coverage check — does any retrieved chunk contain the query's rare terms at all?
How to instrument fusion in your own stack
The diagnostic that pays for itself is per-retriever attribution on the final list. For every query, record each result's rank in each input list and whether it appeared in one or both.
def rrf(lists, k=60):
scores, origin = {}, {}
for name, docs in lists.items():
for rank, d in enumerate(docs, start=1):
scores[d] = scores.get(d, 0) + 1 / (k + rank)
origin.setdefault(d, {})[name] = rank
top = sorted(scores, key=scores.get, reverse=True)
return [(d, scores[d], origin[d]) for d in top]
Aggregate origin across a few thousand queries and you get the number that actually tells you whether hybrid search is earning its cost: the share of final top-10 slots filled by documents only one retriever found. If the vector retriever contributes almost no unique results, you are paying for embeddings to re-rank BM25. If it contributes nearly all of them, your lexical index may be misconfigured — check analyzers and stopwords before celebrating.
What to do with this
- Find your rank constant and write down why it has that value. If the answer is "it came with the library", treat 60 as a hypothesis, not a setting.
- Sweep k against a labelled query set — 10, 20, 60 — and measure recall at the depth you actually show. Low k favours specialists; high k favours agreement. Choose based on which retriever carries your hard queries.
- Log pre-fusion ranks, not just the merged list. Without them, no fusion problem is debuggable after the fact.
- Deduplicate chunks per document before fusion so candidate depth means the same thing for every page length.
- Never threshold on an RRF score. Put a reranker or a pre-fusion score check in the abstain path instead.
One caveat on scope: this describes open hybrid retrieval stacks — Elasticsearch, OpenSearch, Vespa, LangChain-style pipelines, and most self-built RAG systems. Google has not documented using RRF in AI Overviews or Gemini grounding, and you should not assume it does. The transferable idea is narrower and safer: when a system merges lexical and semantic candidates, the merge rule, not the retrievers, usually decides what survives.