Migrating a RAG Reranker's Input Format Between Providers

Rerank endpoints look interchangeable: a query, a list of documents, a model name. The request body converts in ten minutes. What does not convert is the threshold somebody tuned six months ago, and the pipeline will keep running with it.

The shape that is the same everywhere

Every current rerank API takes a query string, a list of documents, and a model identifier, and returns results that identify each document by its index into the list you sent rather than by echoing the text back. That last part is the portable bit and it is worth building on deliberately: if your pipeline joins rerank results back to your chunk store by index rather than by returned text, the join survives a provider change. If it joins by matching the returned string, it does not, because whether documents come back at all is a per-provider default.

So the first refactor is not a provider swap. It is making your reranking step take a list of (chunk_id, text) pairs, send only the texts, and map indices back to ids on the way out. Do that first and the rest of this page is configuration.

The fields that differ

Three current APIs, three vocabularies for the same four ideas. Cohere’s Rerank v2 takes model, query, documents as a list of strings, top_n, and max_tokens_per_doc with a documented default of 4096; its results carry index and a relevance_score documented as normalised into the range 0 to 1. See Cohere’s Rerank reference.

Voyage’s rerank endpoint takes query, documents as a list of strings with a documented maximum of 1,000 documents, model, top_k rather than top_n, return_documents defaulting to false, and a boolean truncation defaulting to true rather than a token count. Its response is an object whose data array holds index, relevance_score and optionally document, plus a usage object carrying total_tokens. See Voyage AI’s reranker reference. Jina’s reranker at /v1/rerank takes query, documents, top_n and return_documents; see Jina’s reranker page.

Note what that list implies for a wrapper. top_n against top_k is a rename. max_tokens_per_doc against a boolean truncation is not: one lets you choose the budget and the other only lets you choose whether truncation happens. A wrapper that exposes a numeric per-document budget cannot honestly implement it on a provider that does not accept one, so it should truncate client-side and say so, rather than accept the parameter and ignore it.

A relevance score is not a portable number

This is the failure that ships. Most RAG pipelines do not use rerank scores as an ordering only; somewhere there is a line that drops documents below a constant, and that constant was chosen by looking at one provider’s score distribution.

A score documented as normalised to 0 to 1 and a score documented only as relative to the query are different objects. Even between two providers that both bound the score to 0 to 1, the distributions are shaped by different training objectives, so the same numeric cut keeps a different fraction of documents. The visible symptom is not an error. It is a generation step that starts producing thinner answers because the context it was handed shrank, or a cost increase because it grew.

Two ways out, and they are not equivalent. The robust one is to stop thresholding on the score and threshold on rank instead — take the top k, always, and spend the tuning effort on k. Rank is the one thing every reranker agrees on. The other is to recalibrate: run your held-out query set through the new provider, look at the score distribution for documents you have labelled relevant and irrelevant, and pick the cut that reproduces your old precision. Recalibration keeps the adaptive behaviour of a threshold, at the cost of having to redo it every time the model version changes.

Truncation changes which document wins

Each provider caps how much of a document it will actually read. Under a 4096-token per-document cap, a 6,000-token chunk is scored on its first two thirds. Under a provider whose cap is smaller, it is scored on less; under one where truncation is a boolean you did not set, it may be rejected instead.

The consequence is specific and easy to miss: a document whose relevant passage sits near the end changes rank between providers even though neither provider is wrong. If your chunks are long — a whole page, a whole section — you are relying on truncation behaviour you did not choose. The fix is upstream of the reranker, in how the corpus is chunked: if no chunk exceeds the smallest cap among the providers you might use, truncation stops being a variable.

One related trap. Cohere’s v2 documentation states that documents are strings and that structured data should be formatted as YAML strings for best performance. If your pipeline was passing structured records and relying on the provider to interpret named fields, that serialisation is now yours to write, and the choices in it — which fields to include, in what order, with what labels — are retrieval-quality decisions rather than plumbing. Write them once, in one function, and version that function alongside the index.

Deciding whether the new reranker is better

The decision rule has to be set before the numbers arrive, because rerankers are close enough that any post-hoc rule will find a way to prefer whichever one you already chose.

1. Sample queries from production logs — a few hundred, stratified so that rare query types are represented rather than drowned.
2. For each query, retrieve the candidate set exactly as production does and rerank it under both providers. Take the union of the top ten from each as the pool to label.
3. Label each pooled document as relevant or not, once, without knowing which reranker surfaced it. Pooling with blind labelling is what stops the incumbent winning by construction.
4. Score recall at your production k and mean reciprocal rank, per query, keeping the per-query pairs rather than only the means.
5. Apply the rule: accept the new reranker only if it wins on more queries than it loses under a paired sign test at your chosen level, and no query that previously had a relevant document in the top k now has none. The second clause is the one that catches a reranker with a better average and a worse tail.

Keep that labelled pool. It is the same artifact you need for evaluating the pipeline as a whole, and building it is most of the cost of this migration.