Skip to content
Multigrid

Language Detector

Identify the language of a snippet from its script and its function words, with the evidence for the guess printed next to it.

Best guess
Dutch

Not enough evidence. Under about twenty words a function-word score is close to a coin flip; add more text.

Letters
66
Words
16
Share of letters in Latin script
100.0%
Estimated tokens
≈ 24
Estimated tokens per word
1.50

Scripts present

  • Latinsee the scores below66 (100.0%)

Latin-script scores, and the words they came from

  • Dutch43.8%
    matched: de, zijn, niet, het, is, dat
  • Italian12.5%
    matched: per
  • English6.3%
    matched: is
  • Spanish6.3%
    matched: de
What this assumes: two different mechanisms, with very different reliability. Script detection is a Unicode range lookup and is close to certain: Hangul is Korean, kana is Japanese, and Han without kana is Chinese. The Latin-script ranking is a heuristic — the share of words that are in a fixed list of about twenty high-frequency function words per language, plus a small bonus for script-distinctive characters. It is not a trained n-gram model and the percentage beside each language is a share of the scores computed here, not a probability. Twelve languages are scored; anything else in Latin script will be assigned to the closest of those twelve. It fails on short strings, on code, on lists of proper nouns, and on mixed-language text, where it reports the majority and says nothing about the rest. Token counts are estimates. Estimated, not tokenized. A real count needs the model's vocabulary; the authoritative number is usage.prompt_tokens on the response.

Detect for routing, not for a record

Language detection in an LLM pipeline is nearly always a routing question: which prompt template, which retrieval index, which moderation list, which model. Routing tolerates a wrong answer badly at the margins and well in the middle, which is why the evidence matters more than the label. A result carried by fifteen matched function words across sixty words of text is a decision you can act on. The same label on a four-word product name is not, and this page will say so rather than printing a confident percentage over nothing.

The tokens-per-word figure is the reason a detector belongs next to a token counter at all. The same sentence costs a different number of tokens in different languages, because the vocabularies were fitted mostly to English text: Latin-script European languages sit a little above English, and scripts with few merges in the vocabulary can cost several times more per word. If you route by language, that ratio is the input to the budget for each route.

Where this is genuinely unsafe: distinguishing closely related languages that share function words — Danish and Norwegian, Indonesian and Malay, Czech and Slovak, Serbian and Croatian in Latin script. This tool merges some of those and simply gets others wrong. If the distinction carries legal or contractual weight, do not use a heuristic; ask the user and store the answer.

The reasoning behind this

Related tools

Language Detector · Multigrid