What to Do When Automatic Language Detection Is Wrong

The detector returned none, or returned a label with a score under your threshold, and a reply still has to be produced in some language. The default that ships in most systems is English, and it is the worst available choice: it is a guess with no evidence behind it, made in the one place where evidence is abundant.

The signals you already have

Before falling back to anything, inventory what is already in the request. Most applications have four or five independent signals and use exactly one of them.

• An explicit stored preference. The user picked a language in settings, or in the installer, or when they signed up. This outranks everything including the detector, because it is a stated intention rather than an inference.
• The language of the previous confident turn. In a conversation, the last message that cleared the threshold is a very strong prior for the next one. A two-word reply of “ok” in the middle of a Portuguese thread is Portuguese.
• Accept-Language. Sent by every browser, derived from the operating system’s configured languages. Weak on its own, decent as a tiebreak, and almost free.
• The UI locale the request came from. If the page was served at /de/support, the user chose German at some point — or at least did not leave.
• Country signals. Billing country, phone prefix, IP geolocation. Weakest of the set and the most likely to be insulting when wrong, because country is not language. Use to break a tie, never as a primary.

The decision tree

Evaluate in this order and stop at the first hit. The ordering is by strength of evidence, and every branch produces both a language and a confidence label you carry forward.

def resolve_language(msg, session, account, request):
    # 1. A stated preference beats every inference.
    if account.language_explicitly_set:
        return account.language, "explicit"

    # 2. A confident detection on this message.
    lang, score, reason = detect(msg.text)
    if reason == "ok":
        return lang, "detected"

    # 3. The language of the last confident turn in this conversation.
    if session.last_confident_language:
        return session.last_confident_language, "session"

    # 4. The locale of the page or app surface the request came from.
    if request.ui_locale:
        return request.ui_locale, "ui"

    # 5. The best supported match from Accept-Language.
    header = negotiate(request.accept_language, SUPPORTED)
    if header:
        return header, "header"

    # 6. Nothing. Ask, or use the product default -- and mark it as a guess.
    return DEFAULT_LANGUAGE, "guess"

Two details in that function matter more than the ordering. The first is that it returns the reason alongside the language. A downstream step that knows the answer came from “guess” can behave differently from one that knows it came from “explicit” — it can offer a switch, log for review, or decline to auto-translate. Throwing the provenance away at the boundary is what makes bad guesses invisible.

The second is that step 3 requires you to store the last confident language on the session. It is one column and it removes the single most common visible failure: a conversation that answers in the right language for four turns and then switches to English because the fifth message was “thanks”.

Reading Accept-Language properly

The header is a ranked list, not a value. A typical one looks like this:

Accept-Language: fr-CA,fr;q=0.9,en-US;q=0.8,en;q=0.7

Every entry has an optional quality value from 0 to 1, defaulting to 1, and the correct read is: French as spoken in Canada first, then French generally, then American English, then English. Three implementation mistakes are near-universal.

• Taking only the first entry. If you do not support fr-CA you must fall through to fr, not to your default. Truncating a tag at the hyphen and retrying is the required behaviour, not an optimisation.
• Ignoring the q values. They are the user’s ranking. Matching your first supported language against the header’s order rather than the header’s against yours inverts the preference.
• Treating q=0 as absent. A quality of exactly zero means “not acceptable” and is a stronger statement than omission.

Do not implement this by hand. Every platform ships a negotiator, and browsers expose the resolved list directly — MDN documents navigator.languages as the same ranked list on the client side.

When to ask, and how not to

Asking is the highest-quality signal available and the most expensive to collect, so the question is when it earns its cost. Three rules hold up.

Ask when the action is irreversible or costly in the wrong language — sending an email, generating a document, filing a ticket into a language-specific queue. Do not ask before a cheap, reversible action like rendering a page; guess with the ladder above and put a one-click switch next to the result. And never ask twice: an answer to this question is an explicit preference, so it goes into step 1 of the tree and the question never recurs.

When you do ask, ask in a way that does not require the user to already understand your interface. Render the language names in their own language (Deutsch, Français, 日本語, العربية), not in English, and put the two or three candidates the ladder produced at the top rather than making the user find their language in an alphabetical list of sixty.

Making the wrong answer cheap

The strategic move is not to get the guess right more often. It is to make being wrong survivable, because on short and mixed input you will be wrong at some rate no matter what.

• Let the model mirror the user instead of obeying a label. For conversational output, an instruction of the form “reply in the same language the user wrote in” frequently beats passing your resolved locale, because the model reads the actual message. It has its own failure modes — see what to do when the output language ignores the instruction — and where you need a hard guarantee, the explicit approach in forcing a single output language is the one to use.
• Put the switch where the mistake is visible. A language selector in a settings page three clicks away is not a recovery path. One next to the reply that came out wrong is.
• Log the reason code with the outcome. If you record which branch produced each answer and which answers users overrode, you learn within a week whether your threshold is too high or too low — from your own traffic, which is the only place that number can honestly come from.