Why Code-Switched Customer Support Messages Get Misrouted

The ticket usually reads: “Spanish-speaking customers are sometimes getting English agents, and sometimes not, and I cannot reproduce it.” It is reproducible. The trigger is the ratio of English to Spanish words in the message, and the boundary moves every time the customer types.

The ticket

The observable facts, in the order they normally get reported:

• A subset of customers get replies in a language they did not write in.
• The same customer is routed correctly on one message and incorrectly on the next, in the same thread.
• The language detector, tested in isolation on a monolingual sample, reports high accuracy — commonly quoted as 98% or better on the vendor’s benchmark.
• Complaints concentrate geographically: US Hispanic customers, Indian customers, North African customers. Not a random slice of traffic.

That last one is the diagnostic. A random distribution across users points at infrastructure; a distribution that follows bilingual populations points at the language layer.

The line of code

Somewhere in the intake path there is something equivalent to this:

lang = detect(message.body)          # returns a single ISO code
queue = QUEUE_BY_LANG.get(lang, DEFAULT_QUEUE)
ticket.assign(queue)

Every part of that is reasonable and the composition is broken. The detector was built for documents and is being handed a two-sentence chat message. It returns a single label because that is its return type. And it computes that label from aggregate character n-gram statistics, which on a mixed message is effectively a weighted vote over the words present.

So the queue is decided by which language contributed more text. Consider two consecutive messages from one bilingual customer:

Msg 1: "Hola, no puedo hacer login, me sale un error"
        Spanish words: 8   English words: 1 (login)
        detect() → es      → Spanish queue    ✓

Msg 2: "El error dice: Payment method declined, please update your
        billing information. Que hago?"
        Spanish words: 5   English words: 9 (a pasted error message)
        detect() → en      → English queue    ✗

The second message is more English than Spanish because the customer pasted your product’s own English error string into it. The customer’s language did not change. The word count did, and the word count is what the rule is reading.

Why it is intermittent

Three effects compound to make the symptom look random.

Technical nouns are English. In most products the untranslated vocabulary — login, upload, refund, password, checkout — is English regardless of the customer’s language. So the English share of a message rises with how technical it is. Your most detailed, most valuable reports are the most likely to misroute.

Pasted content dominates short messages. A screenshot transcription, an error string or a copied confirmation email can be longer than everything the customer wrote. The detector weights it equally.

The matrix language does not track word count. As code-switching describes, the language supplying a clause’s grammatical frame is frequently the minority contributor by word count, because it supplies short function words while the other language supplies long nouns. The quantity being measured and the quantity you care about are not the same quantity, and they diverge precisely on the messages that matter.

The fix

1. Route on the customer, not on the message. A language preference on the account is stable across messages and comes from the customer rather than from a guess. If you have one, per-message detection should not be able to override it. This one change resolves most of the ticket.
2. Where you must infer, infer from function words only. Count only closed-class tokens — determiners, pronouns, auxiliaries, prepositions — and ignore nouns entirely. Pasted error strings and technical vocabulary stop voting, which removes the largest source of instability.

   ES_FUNC = {"el","la","los","las","un","una","de","que","no","me",
              "mi","yo","pero","como","hago","cuando","porque","muy"}
   EN_FUNC = {"the","a","an","of","that","is","are","i","my","but",
              "how","when","because","very","do","did","with"}
   
   def matrix_language(tokens):
       es = sum(t.lower() in ES_FUNC for t in tokens)
       en = sum(t.lower() in EN_FUNC for t in tokens)
       if es + en < 3:
           return None            # not enough evidence — do not guess
       if abs(es - en) <= 1:
           return None            # too close — do not guess
       return "es" if es > en else "en"

3. Make “unknown” a real outcome. The two None returns above are the substantive change. A detector that must answer will answer wrongly on exactly the ambiguous cases; a router that can say “I do not know” sends those to a bilingual queue or asks the customer. Setting a confidence floor on the detector achieves the same thing and is covered in language detection confidence thresholds.
4. Strip your own strings before detecting. You know your error messages, your email templates and your UI labels. Remove known product strings and quoted blocks from the message before running detection. This is a lookup against text you already own and it removes the second-largest source of skew.
5. Log the decision, not just the outcome. Record the detector output, the confidence, the function-word counts and the queue chosen. Without this the bug is unreproducible from the ticket, which is why it sat open.

The second bug, in the reply

Fixing the routing usually reveals a second problem in the same path. Where replies are drafted by a model, the model is typically shown the customer message and told to reply in the same language. Given a mixed message it makes its own judgement, and its judgement is not the one your router made — so a ticket in the Spanish queue receives an English-drafted reply, or worse, a reply that switches language mid-paragraph in a register no support team would use.

Pass the resolved language to the model as an explicit instruction rather than letting it infer: state the output language in the system prompt as a parameter, not as “reply in the customer’s language”. That construction and its failure modes are covered in forcing a model to respond in one language.

There is a third failure in the same path if any of your traffic arrives in a non-Latin script. A message written partly in Devanagari and partly in Latin, or partly in Arabic script and partly in Arabizi, can defeat the function-word approach entirely, because the romanised half has no entry in either list. Profile the script first — it is a deterministic character-property lookup — and route mixed-script records to a bilingual queue rather than trying to resolve them, as described in detecting language in a mixed-script document.

Finally, verify the fix with a metric rather than with the absence of complaints. The number to watch is the reassignment rate — tickets moved between language queues by an agent after intake — segmented by whether the intake path found a confident language. Before the change that rate is high and concentrated in bilingual regions; after it, the uncertain bucket should absorb the ambiguity and the reassignment rate on confident routes should drop. If reassignments merely move from one queue to another without falling, the detector was not the problem and you are looking at a queue definition that does not match how your customers actually write.