OCR for Mongolian Traditional Vertical Script

Feed a page of traditional Mongolian to a general OCR engine and it usually returns nothing at all, or one enormous garbage line. That is not the recogniser failing. It is layout analysis reporting, correctly by its own rules, that the page contains no text lines.

The assumption that breaks first

Traditional Mongolian script — Mongol bichig, Unicode block U+1800–U+18AF — runs from the top of the column to the bottom, with columns advancing from left to right across the page. The left-to-right column order is worth noting on its own: vertical Chinese, Japanese and Korean advance right to left, so a pipeline that has been taught “vertical means right-to-left columns” will reverse the reading order of a Mongolian page while producing entirely valid-looking text.

The deeper break is earlier. Every mainstream OCR pipeline begins by finding text lines as horizontal bands, and the standard techniques all encode that assumption in their arithmetic. A horizontal projection profile sums ink per image row and looks for peaks and troughs; on a vertical page every row contains ink from every column, so the profile is flat and there are no troughs to cut on. Run-length smearing joins components along the x-axis. A CTC or attention decoder consumes a sequence of image slices taken along the width. None of these are configurable to a different axis; the axis is baked into the implementation.

So the failure is total rather than degraded, and it happens before the part of the system anyone thinks of as the OCR. This is the same class of problem described in vertical reading order, with one extra difficulty: Mongolian is also cursive.

Rotation solves layout, not recognition

The obvious fix is to rotate the page ninety degrees so the columns become rows. It works, for the layout stage, and it introduces two new problems.

The first is that rotation is a page-level operation and real documents are not page-level uniform. Modern Inner Mongolian publications mix traditional vertical Mongolian with horizontal Chinese, and archival material mixes it with horizontal Cyrillic Mongolian or Russian. Rotate the page to fix the Mongolian and the Chinese is now vertical. Any pipeline that handles real documents has to detect and rotate regions, not pages, which means running layout analysis before you know which orientation to use — a chicken-and-egg the usual fix for which is to run a cheap orientation classifier on each detected block first.

The second is that a rotated Mongolian line still needs a Mongolian model. The letters are connected along a continuous vertical stem — the backbone that runs down the middle of the column — and after rotation that becomes a horizontal baseline with letters hanging off it. It is structurally like Arabic: a cursive script with a connecting stroke, where the glyph for a letter depends on its position in the word. That shared structure is why the segmentation-free approach that works for Arabic works here too, and it is covered from the other side in OCR for Arabic cursive script.

Letters the image does not contain

This is the part that makes Mongolian genuinely different, and it is the part most tooling handles badly.

Mongolian letters take initial, medial, final and isolated forms, as Arabic letters do. Unlike Arabic, several distinct letters collapse into the same medial shape. The medial forms of a and e are identical; o and u share a medial form; ö and ü share one; t and d overlap in several positions. Which letter is meant is determined by the word, by Mongolian vowel harmony, and by convention — not by the ink.

The consequence is precise and unusual: for those positions, the information required to produce the correct character is not present in the image. A recogniser reporting 0.94 confidence on a medial vowel is reporting the output of a softmax over classes that are visually indistinguishable; the number describes the model’s prior, not evidence from the page. If you are filtering low-confidence characters for human review, those positions will never be flagged, and they are precisely the positions most likely to be wrong.

What follows practically is that lexicon-constrained decoding is not a post-processing nicety here. A greedy decode over per-character argmax cannot be correct on ambiguous positions except by luck. A beam search constrained by a Mongolian lexicon or a character-level language model is the mechanism that resolves them, and it is doing the job the image cannot. Report confidence at the word level after decoding, where it means something, rather than per character, where it does not.

Selectors, separators and a moving standard

The Unicode representation adds its own edges. Free Variation Selectors U+180B, U+180C and U+180D select among shape variants that the default shaping rules would not produce, so two strings that render identically in one font may differ in whether a selector is present. Ground truth transcribed by different people will disagree about them.

The Mongolian Vowel Separator, U+180E, is worse. Its general category changed in Unicode 6.3.0 from a space separator to a format character, which means older software treats it as whitespace and newer software does not. Tokenisers, regular expressions using \s, and trimming functions therefore disagree about word boundaries depending on which Unicode version their runtime was built against. If a Mongolian corpus tokenises differently in two services that are supposedly running the same code, this is usually why.

A pipeline that works

1. Detect text blocks and classify each one’s orientation before doing any line finding. A small classifier on block aspect ratio and ink-density profile is enough to separate vertical Mongolian from horizontal Chinese or Cyrillic.
2. Segment vertical blocks into columns with a vertical projection profile, cutting on the gutters between columns.
3. Rotate each column ninety degrees counter-clockwise so the backbone becomes a baseline, and hand the result to a line recogniser.
4. Recognise with a segmentation-free sequence model trained on Mongolian line images. Do not attempt per-letter segmentation: the connecting stem means letter boundaries are not visually marked.
5. Decode with a lexicon-constrained or language-model-guided beam search. This step is load-bearing, not an accuracy tweak.
6. Reassemble columns in left-to-right order. Assert this explicitly in code, because every vertical-script default you inherit from a CJK pipeline will be the other way round.
7. Evaluate word error rate, not character error rate. Character accuracy on a script with visually merged letters rewards a model for guessing the more frequent member of each ambiguous pair.