Extracting Fields From a National ID Card

A bilingual identity card gives you the holder’s name twice, in two scripts, and they are not transliterations of each other in any reversible sense. Storing one and discarding the other loses information; storing both without saying which is authoritative moves the problem downstream to whoever has to match against it.

The same field, printed twice

Dual-script identity cards are the norm across a large part of the world. A card may carry Arabic and Latin, Chinese and Latin, Cyrillic and Latin, Devanagari and Latin, or a regional language alongside a national one. The two renderings sit in different places on the card, often in different type sizes, and frequently only one of them is repeated in the machine-readable zone.

They differ in ways that are not mechanical. A Latin rendering of an Arabic name is a transliteration whose spelling was chosen by the issuing authority, and several defensible spellings exist for the same name. A Chinese name and its Latin form may reflect Mandarin or Cantonese romanisation. Some cards render the name in the second script and others render a translated form of a title or place. Treating one as derivable from the other by a rule you write is wrong in both directions.

They also fail differently in extraction. A model reading a card will often return only the Latin field, because Latin is what it produces most fluently, and will silently omit the other. Or it will return the non-Latin field already transliterated, so what you store looks like an extraction and is actually a translation the model performed. The defence is to ask for each field with its script named, and to require the raw glyphs for the non-Latin one.

Which script is authoritative

There is a defensible answer rather than a preference, and it comes from the document itself. Most national identity cards used for travel within a region are ICAO-conformant TD1 documents, and Doc 9303 requires the machine-readable zone to be written in the 37-character set — Latin capitals, digits and the filler. Doc 9303 also specifies how national characters are transliterated into that set. So the issuing authority has already made the decision about which Latin form is normative, and it is printed on the card in the MRZ.

That gives a rule with a source behind it:

• Machine matching uses the MRZ form. It is fixed-width, check-digit protected, and it is what any other ICAO-conformant system will hold for the same person.
• Display uses the script the holder’s own jurisdiction prints. Showing a person a transliteration of their own name when the card carries the original is a poor experience and a source of support tickets.
• The printed Latin field is a third value, not a duplicate of the MRZ. It can carry diacritics and lower case that the MRZ cannot, and it can differ from the MRZ because the MRZ truncates long names. Store it separately.

A matching implementation follows: compare on the MRZ form after normalising fillers to spaces, and treat a match on the printed form as supporting rather than decisive. The check-digit machinery that makes the MRZ trustworthy is worked through in extracting a passport machine-readable zone, and it applies unchanged to a TD1 card.

Name structure is not universal

The second-order problem is that first_name and last_name are not universal concepts, and an ID card is exactly where that assumption breaks.

• Order differs from print order. Doc 9303 puts the primary identifier first in the MRZ regardless of how the card prints the name, so an MRZ reading ZHANG<<WEI<MING corresponds to a card that may print the family name first or last depending on the script.
• Patronymics are a third component. Several systems carry a patronymic or matronymic that is neither a middle name nor a surname, and squeezing it into a middle-name field makes it unrecoverable.
• Compound surnames. Two family names, or a surname with a particle, may be separated by a single < in the MRZ — which is the same separator used between given names. Only the position relative to the << boundary distinguishes them.
• Mononyms. A person with one name is not a data error. An MRZ can carry an empty secondary identifier, and a schema requiring a given name will make the model split the single name in half.

Storing primary_identifier and secondary_identifiers as the MRZ does, rather than first and last name, removes most of this at the cost of a slightly unfamiliar field name. It is the trade worth making.

TD1 cards carry their own check digits

A TD1 document is three lines of 30 characters rather than two of 44, and the field positions differ from TD3 — but the check digit arithmetic is identical, and there is a composite digit at the end of the second line covering defined substrings across the first two lines. Consult Doc 9303 Part 5 for the TD1 field positions rather than adapting the TD3 offsets by eye; the temptation to guess is strong and the failure mode is silent, because a wrong slice can still produce a digit that matches by chance one time in ten. The general treatment of a checksum-validated identifier field applies unchanged.

A schema that stores both

{
  "document": { "type": "TD1", "issuing_state": "UTO", "number": "SYN0001234" },
  "name": {
    "mrz": { "primary": "ERIKSSON", "secondary": ["ANNA", "MARIA"] },
    "printed": [
      { "script": "Latn", "value": "Anna Maria Eriksson" },
      { "script": "Arab", "value": "<original glyphs preserved>" }
    ],
    "authoritative_for_matching": "mrz"
  },
  "checks": {
    "document_number": true, "birth_date": true,
    "expiry_date": true, "composite": true
  },
  "mrz_printed_name_differs": true,
  "mrz_difference_reason": "truncation"
}

Tag each printed value with an ISO 15924 script code — Latn, Arab, Cyrl, Hans, Deva — rather than with a language, because what varies here is the writing system and a language tag does not tell a renderer which direction to set the text in. The rest of the pipeline can then pick a rendering by script without a lookup table of its own.

mrz_difference_reason is worth the field because there are only a few legitimate reasons the two forms differ — truncation, transliteration, diacritic loss — and anything outside that set is a read error or a document worth a second look. Recording the reason turns a mismatch from a mystery into a category.