OCR for Korean Text Mixing Hangul and Hanja

Korean OCR is close to solved on contemporary material. On a 1975 newspaper, a legal judgment or an academic monograph it falls apart, and it falls apart in a specific way: the Hanja do not come back as errors, they come back as plausible Hangul with high confidence.

What mixed-script Korean looks like

Until roughly the 1990s, serious Korean prose was written in gukhanmun honyong — Chinese-derived Hanja for the Sino-Korean vocabulary, phonetic Hangul for grammar, particles and native words, interleaved within the same sentence and often within the same phrase. Newspapers, statutes, court decisions and scholarly writing all used it. The convention faded in general publishing but survives in legal and academic contexts, and it is universal in anything archival.

Two further conventions complicate the layout. Hanja are often glossed in parentheses with their Hangul reading immediately after, or the reverse, so a single noun occupies two scripts and a bracket pair. And older Korean print is frequently set vertically, top to bottom in columns running right to left — the same layout assumption break covered in vertical reading order, which has to be resolved before recognition can start.

The label set is the whole problem

Hangul and Hanja occupy different Unicode ranges and, more importantly, radically different label-set sizes. Precomposed Hangul syllables run from U+AC00 to U+D7A3 — 11,172 of them, of which the national standard KS X 1001 designates 2,350 as the commonly used set. The same standard includes 4,888 Hanja. CJK Unified Ideographs begin at U+4E00 and run to U+9FFF, with extension blocks beyond that.

A Korean recogniser is typically built with a classifier over the common Hangul set, because that keeps the output layer small and the training data tractable. That decision is invisible in the API and decisive in the output. A softmax over 2,350 classes cannot emit a Hanja. Presented with 學, it distributes probability over the Hangul syllables it does have and returns the nearest one — and because the probabilities are renormalised over the classes that exist, the confidence attached to that answer can be perfectly high.

This is the difference between an accuracy problem and a coverage problem, and it changes what you do about it. Better images, better binarisation and a bigger model do nothing, because the information is not being lost in the image; it is being lost in the output layer. The only fix is a recogniser whose character set contains the characters in your documents.

Korean Hanja are not simplified Chinese

The second trap is reaching for a Chinese model to fill the gap. Korea uses traditional character forms, as Taiwan and Hong Kong do, not the simplified forms standard in mainland China. A recogniser trained predominantly on simplified Chinese has seen 学 far more than 學, 国 far more than 國, and will map traditional forms onto simplified ones — which is a substitution, not a recognition error, and produces a document that is no longer the document.

There are also character forms used in Korea that differ from both Chinese conventions, and a small set of Korean-invented characters. If you are borrowing a Chinese model, borrow a traditional one, and expect a residual error class you will need Korean-specific training data to close. The distinction and its downstream effects are the subject of simplified and traditional Chinese handling.

The post-corrector deletes the distinction

Suppose the recogniser gets it right. The next stage will often undo it. Language-model post-correction on Korean is trained on modern all-Hangul text, where Hanja are rare, so a correctly recognised Hanja looks like an anomaly and gets rewritten to its Hangul reading.

That rewrite destroys the exact information the mixed script existed to carry. Korean has a great many Sino-Korean homophones, and the Hanja is what disambiguates them. The Hangul 사기 corresponds to several unrelated words — 士氣 (morale), 詐欺 (fraud), 沙器 (porcelain), 史記 (historical records) — and in a legal or historical text the difference is the meaning of the sentence. Converting to Hangul is not normalisation; it is a lossy transform, and it is irreversible.

If you need the reading as well, keep both: emit the original code point and attach the reading as metadata or as a parallel field. Never replace in place. The same discipline applies to romanisation, where the choice of system is itself contested — Korean romanisation systems covers why the round trip does not close.

Routing by script

1. Resolve layout first. Detect whether the page is set vertically, and if so rotate the columns before line segmentation rather than trying to make the line finder cope.
2. Run script identification per connected component or per character-sized region, not per page and not per line. A mixed-script line is mixed at the character level, so a page-level or line-level script label is wrong by construction.
3. Route Hangul regions to a Hangul recogniser and ideograph regions to a traditional-form CJK recogniser. Two specialised models routed by script beat one model with a 15,000-class output layer trained on the data you are likely to have.
4. Reassemble in reading order, preserving the parenthetical gloss structure. Do not collapse the Hanja and its bracketed Hangul reading into one form at this stage.
5. Normalise to NFC. Hangul has both precomposed syllables and decomposed jamo representations, and a string in one form will not compare equal to the same text in the other — Korean NFC comparison is the failure this prevents.
6. Report character error rate separately per script. An aggregate figure on a page that is 85% Hangul will look fine while every Hanja on it is wrong.

The last step is the one that turns this from a mystery into a measurable defect. A single accuracy number over a mixed-script corpus is dominated by whichever script is more frequent, which is precisely the script that was not the problem.