Handling a Field That Is Present but Illegible

The field is right there on the page. A stamp is across it, or the fax compressed it into a smear, or a staple went through the third digit. You have two ways to record that and they lead to different failures: a best guess that is silently wrong, or an explicit gap that stops a downstream process. The choice is decidable, and the arithmetic is not complicated.

Three kinds of unreadable

“Illegible” covers three situations that behave differently, and lumping them together is why the decision feels harder than it is.

• Degraded. The glyphs are damaged but there is only one thing they can plausibly be. A 300 dpi scan of a carbon copy, a dot-matrix invoice, a third-generation fax. Recognition is uncertain per character but the uncertainty is often resolvable from context — a date field with a damaged month can only be one of twelve values.
• Ambiguous. The marks are clear and admit more than one reading. A handwritten 1 that is a European 7 without the bar, a 0 against an O in an alphanumeric identifier, a 5 against an S. This is the dangerous class, because a model reads it confidently and picks one, and nothing about the image says it was a coin flip.
• Occluded. Part of the field is physically not there: a stamp over it, a cut-off right margin, a redaction bar, a hole punch. There is no information to recover, only inference from the rest of the document.

The distinction matters because occluded fields must never be guessed and ambiguous fields must never be trusted, while degraded fields are usually fine and are the main reason people build a “low-confidence but keep it” path in the first place.

The cost of a guess

Write down the two options for one field. Let p be the probability the guess is right, C_wrong the cost of a wrong value flowing downstream undetected, and C_gap the cost of an explicit illegible marker — which is not zero, because something has to handle it: a queued exception, a paused workflow, a phone call to the vendor.

Storing the guess has expected cost (1 − p) × C_wrong. Storing the gap has cost C_gap. Guessing is the better choice exactly when (1 − p) × C_wrong < C_gap, that is when p > 1 − C_gap / C_wrong.

Put plausible numbers through it. Suppose an explicit gap on a bank account number costs about 4 minutes of an operator’s time to resolve — call it £3 fully loaded, as a labelled assumption — and a wrong account number that reaches a payment run costs the recall process, the failed payment fee and the reconciliation: call it £400, again as an assumption you should replace with your own figure. Then guessing is only worthwhile above p = 1 − 3/400 = 0.9925. In other words: for that field, at those costs, essentially any doubt at all should become a gap.

Now the same arithmetic for a free-text description field on the same document, where a wrong value costs perhaps £2 in downstream confusion and the gap still costs £3. The threshold is 1 − 3/2, which is negative — there is no confidence low enough to justify flagging it. Guess, always, and move on.

Two things fall straight out of this and are worth stating plainly. The decision is per field, not per document, because C_wrong varies by three orders of magnitude across fields on the same page. And it depends on p being an actual probability, which a raw model score is not until somebody has checked it against measured accuracy. Feeding an uncalibrated score into this inequality produces a confident answer to the wrong question.

When the checksum reads it for you

Here is the case that gets missed, and it is a genuinely large win on identifier fields. If the field carries a check digit, an unreadable character is often not a loss at all — it is a solvable equation.

A mod-10 scheme over a fixed-length identifier admits exactly ten candidate values for one unknown digit, and the check equation is satisfied by exactly one of them. So a 13-digit ISBN with the fourth digit obliterated has one arithmetically valid completion, and you can recover it exactly, with no guessing and no model involvement. The same holds for IBAN under its mod-97 check and for any other scheme where the check function is a bijection in the unknown position.

The limits are sharp and you should encode them rather than hoping:

• One unknown, recoverable. Two, usually not. Two unknown digits in a mod-10 field leave ten valid completions out of a hundred candidates, so you have narrowed the field by 90% and resolved nothing. Enumerate them and hand the reviewer a shortlist — that is still a much faster review than reading the page.
• The check digit itself must be legible. If the unknown character is the check digit, there is nothing to check against and no recovery.
• You must know which position is unknown. This is the real engineering constraint, and it is why character-level output matters: a vision model that returns a string has thrown away which character it was unsure about. An OCR engine that emits per-character confidence has not.

Mark a checksum-recovered value as recovered rather than as read. It is correct, but it was not observed, and a later investigation into a bad record should be able to tell the two apart.

Why you must not let the model repair

The tempting instruction is “if a character is unclear, infer the most likely value from context”. Do not. A language model is extremely good at producing a plausible identifier, and plausible is exactly the failure mode you cannot detect downstream: the value has the right length, the right prefix, the right shape, and it belongs to a different account.

The general phenomenon has its own page in vision hallucination, but the extraction-specific form is worth naming: a model asked to fill a gap will fill it, and its reported confidence for the filled value reflects how natural the completed string looks, not how much of it was on the page. That is why a per-character or per-region confidence beats a per-value one, and why an explicit “illegible” option in the schema is not politeness — it is the only way the model can decline.

The instruction that works is the inverse: tell the model that reporting a character as unreadable is an acceptable and expected outcome, give it a syntax for doing so, and make the syntax machine-readable. A placeholder character in the value string, plus a list of unreadable positions, gives you exactly what the checksum recovery above needs.

What to store

Illegible is a status alongside present and absent, not a low confidence on a present value — because a reviewer who sees a value will read it, and a reviewer who sees a gap will go and find it. The field-level record wants five things:

{
  "field": "bank_account_number",
  "status": "illegible",
  "value": null,
  "partial": "4021??8317",
  "unreadable_positions": [4, 5],
  "reason": "occluded",
  "source": { "page": 2, "bbox": [0.61, 0.44, 0.83, 0.47] },
  "candidates": []
}

partial is what makes the review fast: a reviewer looking at a crop with eight of ten characters already filled in is doing a two-character job. candidates is populated when a checksum or a lookup against a known-values table narrows the answer, and an interface that offers a reviewer two buttons instead of a text box removes the typo they would otherwise introduce. And source is what lets the queue show the crop at all, which is the difference between a four-minute review and a forty-second one.

One last rule: never let an illegible field become an empty string on the way to a consumer. An empty string passes most type checks, satisfies most required-field validators, and means “the value is nothing” to every reader downstream. It is the same information loss as collapsing absence into null, and it happens at exactly the boundary where somebody wrote a convenience function to flatten the record.