Transliterating Hebrew Text Into English Characters

When a model transliterates unvocalised Hebrew it is not converting letters. It is guessing which word you meant, and then transliterating that. Everything unreliable about the output follows from the fact that the vowels were never written down.

The information that is not on the page

Hebrew is an abjad: the letters are consonants, and the vowels are carried by a separate system of dots and dashes called niqqud which is omitted from essentially all adult writing. Newspapers, contracts, websites and forms are unvocalised. Children’s books, poetry, liturgy and dictionaries are vocalised. So the ordinary case — the text you actually have — is missing the vowels.

It is not entirely missing them. Two letters, ו and י, double as vowel markers in a convention called matres lectionis, and modern Israeli spelling (ktiv male, full spelling) inserts them more liberally than biblical spelling (ktiv haser) does. That narrows the possibilities without closing them, and it introduces its own ambiguity, because now a ו is either the consonant v or the vowel o/u and only the word tells you which.

This is a different problem from the one Arabic romanisation has, even though Arabic is also an abjad. Arabic has multiple competing standards for a mapping that is otherwise well defined; Hebrew has that too, but on top of it the input is under-determined. No standard helps, because the standard specifies what to do with a vowel and there is no vowel there.

One word, five readings

The three letters ס־פ־ר are the classic demonstration. Unvocalised, ספר is any of these:

ספר   unvocalised — five common readings

סֵפֶר    sefer     a book
סוֹפֵר   sofer     a scribe, an author
סִפֵּר    sipper    he told (a story)
סָפַר    safar     he counted
סְפָר    sfar      a frontier, a border region

A model asked to transliterate the bare form will return one of them — most likely sefer, since it is the most frequent — with no indication that it chose. Put the word in a sentence and the model will usually choose correctly, because now it has the syntax and the collocations. That is the actual mechanism: transliterating unvocalised Hebrew is a disambiguation task, and disambiguation needs context, so a list of isolated words is the worst possible input.

Names are the worst case within the worst case, because a name has no syntax to help and no fixed pronunciation to fall back on. The same letters can be a family name pronounced one way in one community and another way elsewhere, and no amount of context recovers a pronunciation the text never encoded.

What niqqud fixes

Supplying the vowel points removes the ambiguity in the vowels completely. סֵפֶר can only be sefer. If you control the input — a liturgical corpus, a dictionary, a dataset that has been vocalised — send the vocalised form and this class of error disappears.

Niqqud also disambiguates two things that are not obviously vowels. The dot inside a letter, the dagesh, distinguishes ב v from בּ b, כ kh from כּ k, and פ f from פּ p — a consonant distinction that is invisible in unvocalised text and that changes the word. And a second use of the same dot marks gemination, so סִפֵּר has a doubled p that unvocalised text does not show.

The letter ש carries its own diacritic that is technically part of niqqud: a dot on the right makes שׁ sh, a dot on the left makes שׂ s. Unvocalised, שר is sar (a minister) or shar (he sang), and the letters are identical.

Asking a model to add niqqud and then transliterate does not fix anything, and it is worth being clear about why: the vocalisation step is the guess. You have moved the guess earlier and made it look like data. If the niqqud did not come from the source, it carries no more information than the transliteration would have.

The half niqqud does not fix

Even with a fully vocalised source, the Latin output is not determined, because the consonant conventions are unsettled and several published systems disagree.

• ח appears as ch, kh, h and ḥ. This is why חיים is Chaim, Haim, Hayyim and Chayim, all for the same person.
• צ appears as tz, ts, z and ṣ — יצחק as Yitzhak, Itzhak, Yitzchak.
• כ without dagesh is kh or ch, which collides with ח in both directions.
• ק is k in most practical systems and q in academic ones, which is the difference between Yaakov and forms that preserve the distinction from כ.
• א and ע, the glottal stop and the pharyngeal, are usually written as nothing at all in practical romanisation, and as ʾ and ʿ in scholarly ones.

The Academy of the Hebrew Language, the body that regulates the language, publishes transliteration rules with a simplified variant and a precise variant precisely because these two audiences want different things; ISO 259 and the ALA-LC tables make different choices again. The Academy’s own material is the place to check the current form of the rules rather than a secondary summary — the Academy of the Hebrew Language publishes them directly.

What to do in a pipeline

The design follows from the two ambiguities being independent. Fix the second by decree and manage the first with context.

1. Keep the Hebrew as the canonical field, always. Normalise it first: the five final-form letters ך ם ן ף ץ are positional variants of כ מ נ פ צ and must be folded before any comparison, and niqqud code points should be stripped for the match key even if you keep them for display.
2. State the consonant convention explicitly in the system prompt — one letter-to-letter table for ח, כ, צ, ק, ש, א and ע — rather than naming a standard and hoping.
3. Never transliterate isolated words. Send the sentence, or send the word with a label saying what kind of thing it is (a surname, a place, a verb).
4. For anything user-facing, treat the transliteration as a display hint and let the user override it. A person’s own spelling of their own name is authoritative and no system can derive it.
5. Index a vowel-free match key alongside the full form, for the same reason Arabic needs one: readers search with the spelling they know, not the one you generated.

If the Hebrew is arriving from scanned documents rather than typed input, the niqqud question turns into a recognition question first — OCR of Hebrew with niqqud covers why the marks are the part that gets lost. And the underlying feature here is not specific to Hebrew; vowel ambiguity in abjad scripts is the general case.