Language Drift in Multilingual Conversations

The first two paragraphs are in the language the user wrote in. The third contains an English clause. By the sixth turn the model has switched entirely and has not remarked on it. This is a specific, explicable failure of the same self-conditioning loop that governs every other drift in a long conversation.

What drift looks like

It has several distinguishable forms, and they do not have the same cause:

• Wholesale switching. A response, or the tail of one, in the wrong language entirely.
• Lexical intrusion. The syntax stays but technical nouns arrive in English — often correct usage in the domain, and often not what was asked for.
• Structural intrusion. Headings, list markers, labels, and JSON keys revert to English while the prose does not.
• Reasoning-trace drift. A reasoning model thinks in one language and answers in another, or mixes them within the trace. Its final answer may be fine, which makes this the easiest form to miss and the most alarming to a user who can see the trace.
• Script slippage. Within a language, output moving between scripts — simplified and traditional characters, Latin transliteration of a Cyrillic-script language.

Four causes

The vocabulary is shared. One subword tokeniser covers every language, and many tokens are ambiguous between them — short pieces, digits, punctuation, proper nouns, anything Latin-script. There is no language variable in the model’s state; the language of the output is an emergent property of the conditional distribution, held in place only by the tokens already generated. Nothing enforces it, so it can be nudged.

The training data is unbalanced. English dominates web pretraining corpora, and it dominates instruction-tuning and preference data by an even larger margin. That means the “helpful assistant answering a technical question” behaviour is most strongly associated with English text, so anything that makes the context look more like that scenario — a code block, a stack trace, a technical register — pulls towards English.

Foreign material in the context does the nudging. An English error message, an English retrieved document, an English library name, a quoted log line. Each of those tokens raises the probability of English continuations, and once one English clause is generated it becomes evidence in the transcript that this conversation is in English. That is the same self-conditioning loop described on long-conversation degradation, and it is why drift accelerates rather than staying constant.

Nothing in training penalised it. For a reinforcement-learned reasoning model optimised against outcome correctness, the language of the reasoning trace is simply not in the objective. Mixing languages costs nothing if the final answer scores.

A documented case, and its fix

The clearest public account comes from DeepSeek’s R1 report (2025), and it is unusually candid. Training their reasoning model with pure outcome-based reinforcement learning produced strong reasoning along with two readability problems, one of which was language mixing: traces that switched between languages, particularly when the query was in neither English nor Chinese.

Their intervention was to add a language-consistency reward during reinforcement learning — a term proportional to the fraction of the reasoning trace written in the target language. The part worth noting is what they reported about the cost: the reward slightly degraded the model’s benchmark performance while improving readability, and they kept it anyway because the traces were for humans to read. That is a rare public statement of an alignment trade-off with both sides named, and it confirms the mechanism directly — language consistency is a thing you have to pay for, not a thing you get.

The term itself is older. Lee, Cho and Kiela’s Countering Language Drift via Visual Grounding (2019) described the same phenomenon in agents fine-tuned with reinforcement learning: without a term anchoring them to natural language, they drifted away from it because nothing in the reward cared.

Pinning the output language

• State the target language explicitly, by name, and say what happens to quoted material. The common failure is an instruction that says “answer in Dutch” and then a context full of English documentation, leaving the model to guess whether the quotes are exempt. Say it: technical terms and quoted error strings remain in the original; everything else is translated.
• Do not rely on the user’s language as the only signal. A user writing in their second language, or mixing, provides a weak and inconsistent anchor. Detect the language once, store it as an explicit setting, and put that setting in the system prompt.
• Re-anchor at the end. The language instruction is subject to the same position effects as any other constraint, and a short reminder immediately before generation is the highest-leverage change available.
• Give one example in the target language. A single short exemplar in the right language shifts the distribution more reliably than any amount of instruction text, because it is evidence about this conversation rather than a rule about it.
• Tag foreign-language context. Wrap retrieved English documents in a delimiter and say what they are. Unlabelled English in the context reads as part of the conversation.
• Separate the reasoning language from the answer language. If a model reasons better in English, let it, and require the final answer in the target language. Fighting the trace costs quality for no user-visible benefit — unless you show the trace, in which case decide deliberately.

Detecting it in production

Language identification is a solved, cheap problem, so this is one of the few failure modes you can detect exactly rather than probabilistically.

from lingua import LanguageDetectorBuilder

detector = LanguageDetectorBuilder.from_all_languages().build()

def language_purity(text, target, min_chars=25):
    """Per-sentence detection, not whole-document: a document-level call
    returns the majority language and hides exactly the mixed case you
    are looking for."""
    sentences = [s for s in split_sentences(text) if len(s) >= min_chars]
    if not sentences:
        return 1.0, []
    off = [s for s in sentences
           if (d := detector.detect_language_of(s)) and d.iso_code_639_1.name.lower() != target]
    return 1 - len(off) / len(sentences), off

purity, offending = language_purity(response, target="nl")
if purity < 0.9:
    # Regenerate with the language instruction restated, or fall back.
    log_drift(purity, offending)

Strip code blocks, inline code and quoted strings before measuring, or every technical answer will look like drift. Track purity as a percentile over time rather than as an average — drift is bimodal, responses are either clean or badly mixed, and a mean of 0.94 can be hiding a bad 5% that is your entire complaint volume. And plot it against turn index, because in a long conversation drift is a function of depth.