Why Vector Search Ranks a Mistranslated Chunk Above the Correct One

A corpus has two translations of the same source paragraph. One is fluent and says something the original does not. The other is stilted and correct. The retriever returns the fluent one, every time, and the generator repeats its error with a citation attached.

The shape of the failure

This is not a rare pathology. It shows up in any corpus where content exists in more than one language and at least one version came from machine translation, post-editing, or a translator working at speed — which is most product documentation, most support knowledge bases, and almost all localised marketing copy. The two candidate chunks are near-duplicates by topic, so both are retrieved; the ranking between them is decided by a similarity score of a few decimal places, and nothing in that score is measuring accuracy.

The reason it is hard to notice is that the output looks right. The answer is fluent, on-topic, and sourced to a document that genuinely exists in your corpus. Evaluation that checks whether the answer is grounded in a retrieved chunk passes. Only a reader who knows the source language catches it.

What the embedding was trained to score

A dense retriever is trained with a contrastive objective: pull the embedding of a query and the embedding of a passage that answers it together, push apart the query and passages that do not. The training pairs come from things like question-and-answer sites, click logs, citation links and title-body pairs. What that teaches is a notion of topical relevance. There is no signal anywhere in it that distinguishes a true statement about a topic from a false one about the same topic, because no training pair ever asked the model to.

So the model is doing its job correctly. Cosine similarity between a query vector and a passage vector answers “is this passage about what the query is about, phrased in a way that matches how questions about it get answered”. A mistranslation that inverts a condition — writing “you must” where the source says “you need not” — changes almost none of the content words. Negation and modality are exactly the features that survive embedding least well, which is a known and much-discussed weakness of sentence embeddings and not a quirk of any one vendor’s model.

There is a second effect on top. Retrieval models are also, implicitly, fluency models: passages that read like well-formed prose in the training distribution sit in denser, better-mapped regions of the embedding space. A passage with unusual collocations lands further out. When two candidates are equally on-topic, the more idiomatic one wins the tie-break, and that is a preference for style acting as a proxy for nothing at all.

Why the accurate translation is the odd one out

Now add the specific character of a careful translation. Translators preserving meaning across a structural mismatch produce text with measurable statistical differences from natively-written text in the same language — longer and more explicit connectives, more literal renderings of source idioms, source-language word order showing through in subordinate clauses. This is the phenomenon translation studies calls translationese, and it is well enough established that it is used as a classification target: models can be trained to tell translated text from original text with high accuracy.

The consequence for retrieval is uncomfortable. The properties that make a translation faithful — resisting the pull of the target language’s idiom in order to keep a distinction the source makes — are the same properties that push it away from the fluent centre of the embedding space. A loose translation that reads like it was written natively has thrown away the source-language fingerprint along with, sometimes, part of the meaning. The retriever rewards it for the first and cannot see the second.

The problem is worse when the query is in the target language, which it usually is. A user asking in Spanish produces a query vector shaped by Spanish norms, and the chunk that best matches those norms is the one that reads most like native Spanish. Cross-lingual retrieval, where you query in one language against documents in another, is a different problem with different failure modes — see cross-lingual embedding alignment — but it does not have this one in the same form.

How it compounds through the rest of the pipeline

• Deduplication picks the wrong survivor. Pipelines that drop near-duplicate chunks usually keep the one that is more central, or the first one seen. Neither rule prefers the accurate version, and after dedup the correct chunk is not merely outranked — it is gone.
• Reranking does not automatically save you. A cross-encoder reranker scores a query and passage jointly and is much better at negation and condition, so it helps. But it is trained on the same kind of relevance data, so it is still scoring relevance rather than fidelity to a source it has never seen.
• The generator resolves the conflict silently. If both chunks reach the context window and contradict each other, the model produces one answer. It rarely surfaces the contradiction, and when it does it typically picks the more confidently-worded chunk — which is again the fluent one.
• Evaluation confirms the wrong answer. Groundedness or faithfulness checks ask whether the answer follows from the retrieved context. It does. The error is upstream of everything those metrics look at.

What actually helps

The structural fix is to stop treating a translation as an independent document. Index the source-language text as the retrievable unit and carry translations as attached renderings, so that retrieval happens once against one canonical version and translation happens at presentation time. This removes the competition between two versions of the same passage entirely, which is better than trying to win it. The cost is that your queries and your index are now in different languages, so you need a genuinely multilingual embedding model and you inherit the problems described in retrieval over a mixed-language corpus instead.

Where that is not possible, the practical measures are: store a provenance field on every chunk recording whether it is source or derived and by what process, and use it as a ranking feature rather than a filter; run a cross-encoder reranker, accepting that it is a partial fix; and hybridise with lexical retrieval, because BM25 over the source-language terms is at least matching on tokens that were actually in the original. Finally, when two retrieved chunks are near-duplicates but disagree, pass both to the generator with their provenance and instruct it to say so — a surfaced contradiction is a bug report, and a silently-resolved one is not.