Why Jailbreak Prompts Work Better When Translated

Translating a refused request into a language with little training data often gets it answered. That is not a quirk of any one model. It follows from a structural asymmetry between where a model’s abilities come from and where its refusals come from, and the same asymmetry predicts which defences will work.

What the research established

The finding is public and specific rather than folklore. In Low-Resource Languages Jailbreak GPT-4 (Yong, Menghini and Bach, 2023), harmful English instructions were machine-translated into languages such as Zulu, Scots Gaelic, Hmong and Guarani before being sent. The paper reports that the combined translation attack succeeded on the great majority of a standard harmful-behaviour set, against a near-zero success rate for the same instructions submitted directly in English. Whatever one thinks of the exact rate, the size of the difference is the result.

Deng and colleagues, in Multilingual Jailbreak Challenges in Large Language Models (2023), gave the effect a gradient: their MultiJail set translates 315 harmful English prompts into nine languages grouped as high, medium and low resource, and unsafe-response rates rise as the resource level falls. The monotonic relationship with resource level is the part that matters, because it points at the cause.

A third line of work generalises it beyond translation. Yuan and colleagues’ GPT-4 Is Too Smart To Be Safe (2023) showed the same effect using ciphers and encodings rather than natural languages. Once you see that, translation stops looking like a language problem and starts looking like an instance of a general one: any encoding the model can decode but the alignment data never covered.

Capability and refusal do not generalise alike

Here is the argument. A model acquires two different things from two very differently sized processes.

Its capabilities come from pretraining on a web-scale corpus measured in trillions of tokens. That corpus is multilingual because the web is, and the training objective builds a shared representation in which a concept expressed in Swahili and the same concept in English end up in similar places. This is why a model can usefully answer at all in a language you might assume it barely knows.

Its refusals come from alignment: supervised examples and preference data, produced by people, numbering in the tens or hundreds of thousands rather than the trillions. That data is written and reviewed against a policy document, by annotators recruited and calibrated for the task, and it is overwhelmingly English — not because anyone decided low-resource languages should be unsafe but because that is where the annotators, the red teams and the guidelines were. Where labs publish anything about composition it points the same way; Meta’s Llama 2 paper (2023) reports a pretraining mix that is roughly 90% English and states plainly that its safety tuning and testing were English-focused.

So the two things are learned at wildly different scales, from differently distributed data. Capability follows the big multilingual corpus and transfers. Refusal follows the small English set and transfers weakly, because a decision boundary defined by examples in one language is only as well placed in another as the shared representation happens to make it. The distance between the two is the attack surface, and it is widest exactly where pretraining data is thin, which is what MultiJail’s resource gradient shows.

It is not that the model misunderstands

A common reading is that the model “does not understand” the low-resource prompt and so fails to notice it is harmful. That explanation does not survive contact with the results, and the reason is worth stating because it changes the defence.

If the model did not understand the request, it could not produce a useful answer to it either. The attacks are interesting precisely because the responses are on-topic and usable after translating back. Comprehension is demonstrated by the compliance. What is missing is not understanding of the content but the learned association between this kind of content in this language and refusing.

That distinction rules out a whole family of proposed fixes. Improving the model’s general fluency in a language does not close the gap; it can widen it, because it raises the usefulness of a successful attack without touching the alignment data. Safety in a language is a separate thing to train, and it needs separate data — English-centric safety training and its blind spots works through what that does to the boundary itself.

The filter is a separate system with its own gaps

Most production stacks put a moderation classifier in front of the model and often another behind it. These are small, cheap models trained on their own labelled data, and their language coverage is typically narrower than the generative model’s — for the labelling reasons set out in moderation gaps in low-resource languages.

A translated attack therefore defeats two systems independently and for the same underlying reason. Reasoning about it as one boundary leads to the wrong instrumentation: when an attack succeeds you want to know whether the filter scored it low or never had a chance, and those are different fixes. Log the classifier scores per category alongside the generation, in every language, and you can tell them apart.

Which defences actually follow

• Do not gate on detected language. Refusing or downgrading non-English input is the reflexive response and it is a product decision disguised as a security control. It penalises real users, and code-switched or transliterated input evades it anyway.
• Translating to English before moderating is not sound. It sounds like it should work and it introduces a new failure mode: the translator is itself a model, it can normalise away the thing you were trying to detect, and a sufficiently obfuscated input translates to something innocuous. It also inserts a component that can be attacked directly. Use it as one signal, never as the gate.
• Moderate the output as well as the input. The output is in the attacker’s chosen language too, so this only helps if your output classifier covers that language — but the model has by then done the work of making the harmful content explicit, which is often easier to detect than the request was.
• Enumerate your covered languages and write them down. A stack has a language coverage list whether or not anyone has ever written it out. Writing it out turns an unknown into a risk assessment, and makes “we are not covered here” a reportable state rather than a silent pass.
• Red-team in the languages you serve, with native speakers. Machine-translated red-team sets measure the translation as much as the model. This is the expensive recommendation and it is the only one that produces data the model can be trained on.