Prompting Effectively in a Low-Resource Language

A prompt that works in English is doing a lot of work you cannot see: it relies on the model inferring structure, register and terminology from a dense prior. In a language with a thousandth of the coverage, that prior is not there, and the prompt has to contain what the prior would have supplied.

What low-resource means operationally

The label is relative and it is worth replacing with something you can check before you plan around it. Four proxies, all of which you can look up in an afternoon:

• Wikipedia edition size. The list of Wikipedias published by the Wikimedia Foundation gives article counts per language. This spans four to five orders of magnitude and is the single fastest signal, because for small languages that edition is frequently the largest clean corpus in existence.
• Presence in web-crawl language statistics. Corpus projects derived from Common Crawl publish per-language byte counts. A language at a hundredth of a percent of the crawl was seen a hundredth of a percent as often.
• Tokeniser efficiency. Send a paragraph and count the tokens per character. Heavy fragmentation means the script was under-represented when the vocabulary was fitted, which is both a cost problem and a capability one — see the tokeniser vocabulary bottleneck.
• Benchmark presence. If the language is not in FLORES, not in any multilingual evaluation suite, and has no published results, nobody has measured it — which means nobody optimised for it either. That is a language with no standard benchmark.

One consequence to internalise before writing any prompt: a model that can produce a language may not be able to follow instructions in it. Those are separate capabilities learned from separate data, and pretraining coverage is always broader than instruction-tuning coverage.

The four failure modes

Thin coverage does not produce generically worse text. It produces four specific, recognisable failures, and the prompt is built to block each one.

• Silent language substitution. The model answers in a related, better-resourced language and does not mention it. Tigrinya requests answered in Amharic; Bhojpuri in Hindi; Kikuyu in Swahili; Nynorsk in Bokmål. Where the two share a script this is invisible to anyone who does not read the language, which is why it survives review. The general behaviour is in what a model does with a language it does not know.
• Instruction collapse. Constraints that hold reliably in English — word limits, required sections, output format, refusal rules — stop being honoured. Not because the model misunderstood, but because the behaviour of honouring them was trained in other languages.
• Invented vocabulary. Asked for a technical term the language may not have settled, the model produces a plausible calque with total confidence. This is the sharpest edge of the general tendency described in why low-resource languages hallucinate more, because a fabricated term is fluent and undetectable without a speaker.
• Script and orthography drift. Languages written in more than one script, or with unsettled spelling conventions, produce output that mixes them: Serbian sliding between Cyrillic and Latin, Kurdish between Arabic and Latin scripts, or a language that recently reformed its orthography mixing both versions within a paragraph.

The technique: supply the structure

The technique is one idea applied five ways: whatever the model would have inferred from a dense prior, put in the prompt as an explicit artefact. An English prompt can say “write a support reply” and rely on the model knowing the genre. A Wolof prompt cannot, so you give it the skeleton.

• Instructions in a high-resource language, content in the target. Usually English; sometimes the regional lingua franca is better if it is closer to the target and well covered — French for several West African languages, Hindi for several South Asian ones, Swahili for parts of East Africa.
• Name the language unambiguously. The English name, the endonym, and the ISO 639-3 code together. “Tigrinya (ትግርኛ, ISO 639-3 tir), written in the Ge’ez script” is far harder to substitute away from than “Tigrinya”.
• Give the output shape literally. Not “write three paragraphs” but the actual headings, the actual field names, the actual ordering, with a length budget in characters rather than words — the word-boundary assumption fails in several of these languages, as argued in writing prompts natively.
• Supply a glossary. Every domain term you care about, mapped from the source language to the exact target-language term you want. This is the only reliable defence against invented vocabulary, and it converts an unverifiable output into a checkable one.
• Give more examples than you would in English. Two to four native demonstrations, chosen to differ in shape. The prior they are competing against is weak, so they carry proportionally more of the load — the mechanism in few-shot examples in the target language.

Two decoding settings follow from the same reasoning. Keep temperature low: high-temperature sampling in a thin distribution is more likely to wander into a neighbouring language than to produce interesting prose. And raise max_tokens generously, because heavy tokenisation means the same content consumes several times the token budget it would in English, and a truncated answer looks like a quality failure.

Building the prompt

1. Write the instruction block in English, and state the target language with its endonym and ISO code plus the script.
2. Add an explicit anti-substitution clause naming the languages you expect it to drift toward: “Do not answer in Amharic or English. If you cannot produce Tigrinya, reply with exactly UNSUPPORTED and nothing else.” Giving the model a legal way to decline is what turns a silent failure into a visible one.
3. Paste the glossary as a two-column block, and say the terms are mandatory.
4. Add two to four native examples in the exact output shape, before any per-request content so they sit in a cacheable prefix.
5. Specify the output structure literally, with character budgets.
6. Set a low temperature and a generous token limit.
7. Run the checks below before any output reaches a user.

{
  "model": "<your-model>",
  "temperature": 0.2,
  "max_tokens": 1600,
  "messages": [
    {
      "role": "system",
      "content": [
        "Write in Tigrinya (ትግርኛ, ISO 639-3 tir), Ge'ez script.",
        "Do NOT answer in Amharic, Tigre or English.",
        "If you cannot produce Tigrinya, output exactly: UNSUPPORTED",
        "",
        "MANDATORY TERMS (source -> Tigrinya). Use these exact forms;",
        "do not coin alternatives:",
        "  invoice  -> <term>",
        "  due date -> <term>",
        "  refund   -> <term>",
        "",
        "OUTPUT SHAPE, exactly these three sections in this order:",
        "  1. Greeting, one sentence, max 120 characters",
        "  2. Explanation, max 600 characters",
        "  3. Next step, one sentence, max 120 characters",
        "Return nothing else. No English. No transliteration."
      ].join("\n")
    },
    { "role": "assistant", "content": "<native example 1>" },
    { "role": "assistant", "content": "<native example 2>" },
    { "role": "user", "content": "<the case details>" }
  ]
}

Verifying without a native speaker on call

You will need a speaker eventually; the point of these checks is that they catch the mechanical failures first, so the speaker’s time goes on the judgements only a speaker can make.

• Script assertion. Check that the output’s characters fall in the expected Unicode block and that the proportion of Latin characters is near zero. This catches script drift and English leakage outright.
• Substitution probe. Run a language identifier over the output. It will not reliably distinguish close relatives — that is the problem in telling similar languages apart — but a confident prediction of the neighbouring language is a strong signal, and worth alerting on even with a weak detector.
• Glossary assertion. Every mandatory term must appear verbatim where the source text contained its counterpart. A missing term means the model coined something.
• Structure and budget. Three sections, in order, within the character limits. Instruction collapse shows up here first and it is free to detect.
• Back-translation, with a warning. Translating the output back is worth doing and is not sufficient. Back-translation through the same model launders its own errors: a fabricated term round-trips into the term you asked for, because the model believes its own coinage. Use it to catch gross meaning failures, never to certify terminology.
• Review one output per template, once, with a speaker. Templates are stable; individual generations are not. Reviewing the template is affordable and catches register and tone, which none of the above can.