Handling Code-Switched Prompts Where the User Mixes Languages

A user writes “Mera order abhi tak deliver nahi hua, can you check the status?” and your pipeline has to answer one question before it can do anything else: what language is this, and what language should the reply be in? Both plausible answers are wrong some of the time, and picking by counting characters is wrong most of it.

Two different inputs that look alike

A detector that returns two languages for one message is reporting a fact about the text and nothing about the user’s intent. The same signal is produced by two very different situations.

The first is code-switching proper: a bilingual speaker using two languages as one system. Hinglish, Spanglish, Arabizi, Taglish and Singlish are all stable, conventional and enormous in volume, and a speaker writing that way is not making an error and does not want to be answered as if they were. Switching within a sentence is normal here, and the switch points are not random — they follow grammatical constraints of the kind Shana Poplack described in her 1980 paper on Spanish-English switching, Sometimes I’ll start a sentence in Spanish y termino en español, whose title is itself the phenomenon.

The second is accidental mixing: a quoted email in another language, a pasted error message, a boilerplate signature, a product name, a browser autocomplete in the wrong language, or the user starting to type in one language and giving up. Here the second language is data, not voice, and answering in it is a visible mistake.

Deciding between them is the whole problem, and the useful thing is that the two have different structural signatures.

The matrix language test

When the mix is genuine code-switching, one language is doing the grammatical work. In the model Carol Myers-Scotton set out in Duelling Languages (1993), that language is the matrix language: it supplies the sentence frame, the function words and the inflection, while the other language contributes content words slotted into that frame.

That gives a test far better than counting. In the Hinglish example above, the verbs and the postpositions are Hindi and the nouns are English — order, deliver, check, status are all borrowed content, but the sentence is built on a Hindi frame. By character count the message looks majority English. By matrix language it is Hindi with English insertions, and a reply in the same register is what the user is asking for.

The practical version of the test, in order of reliability:

• Function words and inflection win over content words. Pronouns, auxiliaries, postpositions, case markers, verb endings and copulas identify the frame. Nouns and technical terms identify very little, because borrowing nouns is what every language does.
• Sentence-final morphology is decisive in languages that carry it there — Hindi, Japanese, Korean, Turkish. Whatever language the verb ending is in, that is the frame.
• The first clause is a weak signal and is often what a document-level detector latches onto. Prefer a segment-level detector that labels each span, then reason over the labels.

The detection mechanics — why a document-level detector returns one confident and wrong answer on mixed text — are covered in language detection on code-switched text.

Signals that the mix is accidental

Accidental mixing has a shape that intentional switching does not. Look for these before deciding it is code-switching:

• The switch lands on a hard boundary. A paragraph break, a blank line, a quote marker, a code fence, a signature delimiter. Real code-switching happens inside sentences and at clause boundaries; a paste happens at a block boundary.
• The second-language span is self-contained and grammatical on its own. A complete, well-formed sentence or paragraph in language B, which would stand alone if you deleted everything around it, is almost always quoted material.
• It looks like a template. Order confirmations, system notifications and email footers repeat. If the same span appears across many users’ messages, it is boilerplate and should be treated as an attachment rather than as speech.
• The pair is implausible. Hindi and English, Spanish and English, Arabic and French are common bilingual pairs with millions of speakers. A span of Finnish in a Portuguese message is far more likely to be pasted than switched.
• The script changed without the language changing. Romanised Hindi, Arabizi and Greeklish are the same language in Latin letters, and a naive detector calls them English or nothing at all. This is a transliteration case, not a code-switching one — see Arabizi and its numeral substitutions.

A reply policy

Write the policy down, because the alternative is that it varies per feature. A defensible default:

1. Reply in the matrix language, in its dominant script. If the user wrote romanised Hindi, replying in Devanagari is technically the same language and reads as a correction. Match the script the user chose.
2. Do not mirror the switching. Generating code-switched output deliberately is a hard problem and the failure mode is a reply that reads as a parody. Keeping the borrowed nouns the user used — the English product terms in a Hindi frame — is enough, and is closer to what a bilingual agent would write.
3. Never comment on the language. No “I notice you are writing in a mixture”, and above all no correction. The user is not confused.
4. Quote foreign material in its original language. If an accidental span was a pasted error message or a quoted email, echo it as it was; translating it into the reply language loses the string the user needs to search for.
5. When the matrix is genuinely undecidable, ask, or fall back to the account language. A one-line message of three words in two languages has no matrix. The account’s stored language preference is a better prior than a coin flip, and it is the right answer more often than any detector on a string that short.

Note the bidirectional-text case as its own problem: an Arabic message with English product names, or Hebrew with Latin URLs, will render incorrectly in a naive UI regardless of how you pick the reply language, because ordering is a display question rather than a detection one. That is mixed Arabic and English bidirectional text.

Implementing it

// Segment-level detection, then a matrix decision, then a prompt fragment.
type Segment = { text: string; tag: string; confidence: number; start: number };

function chooseReplyLanguage(segments: Segment[], accountTag: string) {
  // 1. Drop spans that look like quoted or pasted material rather than speech.
  const speech = segments.filter((s) => !isQuotedBlock(s) && !isBoilerplate(s));

  // 2. Score by grammatical weight, not by length. Function words and
  //    inflected verbs identify the frame; bare nouns identify almost nothing.
  const weights = new Map<string, number>();
  for (const s of speech) {
    const w = functionWordCount(s) * 3 + inflectedVerbCount(s) * 3 + contentWordCount(s);
    weights.set(s.tag, (weights.get(s.tag) ?? 0) + w);
  }

  const ranked = [...weights.entries()].sort((a, b) => b[1] - a[1]);
  if (ranked.length === 0) return { tag: accountTag, reason: "no speech segments" };

  const [top, second] = ranked;
  // 3. A near-tie is not a decision. Fall back rather than guess.
  if (second && top[1] < second[1] * 1.3) {
    return { tag: accountTag, reason: "matrix undecidable" };
  }
  return { tag: top[0], reason: "matrix language" };
}

function promptFragment(reply: { tag: string }, userScript: string) {
  return [
    "The user's message mixes languages. This is normal and deliberate.",
    "Write your reply in " + reply.tag + ", using the " + userScript + " script.",
    "Keep any product or technical terms in the language the user used for them.",
    "Do not comment on the user's language or correct it.",
    "Quote any pasted error text exactly as it appears, without translating it.",
  ].join("\n");
}

The weights are a starting point rather than a tuned model, and the near-tie fallback matters more than the exact multipliers: a wrong confident answer is worse here than a default, because replying to a Hindi speaker in English is a smaller error than replying to an English speaker in Hindi. Log the decision and its reason on every request, and revisit the ratio when you have real distribution data.

One thing to check early: whether your embedding and retrieval stages handle these inputs at all. A code-switched query embedded by a model that clusters by language rather than by meaning will retrieve poorly no matter how well you picked the reply language, which is a separate failure covered in embedding code-switched text.