Extracting Structured Fields From a Handwritten Order Form

A handwritten order form has two fields that matter and they have opposite properties. A misread product code is almost always detectable by arithmetic. A misread quantity is not detectable at all, which is where the entire financial exposure sits — and the confidence policy should be shaped by that asymmetry rather than by one threshold across both.

The confusion pairs, and where they cost money

Handwritten digits fail in predictable pairs, and the pairs are worth enumerating because they drive everything else. One and seven, which collide in both directions depending on whether the writer crosses their sevens and flags their ones. Zero and the letter O, and zero and six when the loop is left open. Five and S. Two and Z. Four and nine when the four is closed. Three and eight. One and a European seven is the single most consequential pair on a quantity field, because it is a factor of seven.

A quantity field also has failure modes that are not digit recognition at all:

• Decimal comma against decimal point. “1,5” is one and a half in much of Europe and might be read as fifteen with a stray mark, or as one thousand five hundred if the comma is taken as a grouping separator. The convention is a property of the customer and the form, not of the pen stroke.
• Ditto marks. The same repetition convention that appears in handwritten notes: a quantity of “same” means the value on the line above, and transcribing it literally leaves the line with no quantity at all.
• Units written into the field. “2 boxes”, “1 doz”, “3 pr”. The numeral is not the order quantity until the unit is applied, and a dozen read as one is an order for a twelfth of what was wanted.
• Corrections. A quantity crossed out and rewritten, or overwritten in place. The later value wins, but only if you can tell which is later — and an overwrite is genuinely ambiguous. Capture both and flag it rather than choosing.
• Drift between rows. A quantity written slightly high, landing in the row above. The same row-band problem repeating groups have, with money attached.

The product code can check itself

If the product code on your form is a GTIN — the identifier behind a retail barcode, published by GS1 — then it carries a check digit and you can validate an extraction arithmetically without any reference to your catalogue.

For a 13-digit GTIN, take the first twelve digits and multiply them alternately by 1 and 3 starting from the left, sum the products, and the check digit is whatever brings that total up to the next multiple of ten. Taking 5012345678900 as a worked example:

digits   5  0  1  2  3  4  5  6  7  8  9  0
weights  1  3  1  3  1  3  1  3  1  3  1  3
products 5  0  1  6  3 12  5 18  7 24  9  0

sum   = 90
check = (10 - (90 mod 10)) mod 10 = 0      -> 5012345678900 is valid

The property that makes this useful is not that it catches errors generally but which errors it catches. Change any single digit by an amount between one and nine and the weighted sum changes by that amount times either one or three, and neither product is a multiple of ten — so every single-digit substitution is caught. That is exactly the class of error handwriting produces: a seven read as a one, a zero read as a six.

It is equally worth knowing what slips through. Transposing two adjacent digits changes the sum by twice their difference, which is a multiple of ten precisely when the digits differ by five. So a transposition of 2 and 7, or 3 and 8, or 4 and 9, passes the check undetected. That is a small and specific hole, and it is one your catalogue lookup closes, since the transposed code almost certainly is not a product you stock. GS1 publishes the calculation in its General Specifications.

Combine the two and you get a strong recovery path. If a code fails its check digit, generate variants over the known confusion pairs, keep only those that pass the check, and then look those up. Accept only when exactly one survives both filters. Two survivors is not a choice between candidates — it is a review case.

Quantity cannot

A quantity is an arbitrary small integer. There is no check digit, no catalogue to look it up in, and no internal redundancy. Every plausible misread produces another perfectly valid quantity, and nothing downstream will reject it.

What you have instead are weak external constraints, and they are worth using precisely because they are all there is. A quantity that is not a whole number where the product is only sold in whole units is wrong. A quantity that exceeds anything this customer has ever ordered of this line, or that exceeds your stock by an order of magnitude, is suspicious. A line total handwritten on the form is a redundancy you should always capture, because quantity times unit price against a written line total is an arithmetic check the form is offering you for free — and where a handwritten order total exists, the sum of the line totals should foot to it.

Those checks catch the large errors. They do not catch a two read as a three on a routine line, and nothing will. That residual is the risk you are actually managing, and the sensible response is to size it.

Deriving a review threshold from line value

Confidence alone is the wrong gate, because it treats a line worth four pounds and a line worth four hundred identically. What matters is expected cost, which is the probability of an error multiplied by what the error costs. Both inputs below are assumptions, chosen for the arithmetic and not measured here — substitute your own.

Assume that above your accepted confidence level the quantity field is wrong on one line in two hundred, so an error rate of 0.005. Assume a human check costs twenty seconds at a fully loaded eighteen pounds an hour, which is ten pence per line. And assume, conservatively, that an undetected quantity error costs the full value of the line, once in reshipping and once in goodwill.

expected cost of NOT reviewing a line = error_rate x line_value
                                      = 0.005 x V

review is worth it when   0.005 x V  >  0.10
                                   V  >  0.10 / 0.005
                                   V  >  20

Under those assumptions the break-even line value is twenty pounds: review every line above it, accept every line below it, and the total cost is lower than either reviewing everything or reviewing nothing. The number itself is only as good as the three assumptions, but the shape of the answer is robust — the threshold is a value in money, it moves inversely with your error rate, and it has nothing to do with where you would have put a confidence cut-off.

The refinement is to use both. Expected cost is error rate times value, and a per-line error probability is roughly one minus its confidence, so route to review when the product of that and the line value exceeds the cost of checking. A low-confidence cheap line and a high-confidence expensive line can then both be handled correctly, which a single threshold on either axis cannot do.

The policy that follows

• Validate the code, gate the quantity. The product code gets arithmetic and a catalogue lookup and needs almost no human attention. The quantity gets the value gate. Applying one policy to both wastes review capacity on the field that can check itself.
• Never let the model normalise a quantity. Store the characters as written, the parsed number, the unit text and the resolved unit separately. “1 doz” must survive as written or the twelve is unexplainable later.
• Make the arithmetic checks blocking, not advisory. A line whose quantity times price disagrees with the written line total does not go through, whatever the confidence says. The document contradicting itself outranks any score.
• Keep the crop. Store the image region for every quantity alongside the value. A reviewer settling a disputed order needs to see the pen stroke, and a crop is a few kilobytes against a line worth a few hundred pounds.
• Report per field, not per document. A document accuracy figure hides that the codes are effectively perfect and the quantities are not, and it is the per-field number that tells you where to spend.

The same asymmetry is worth looking for on any handwritten form. Wherever a field carries a checksum, a controlled vocabulary or an arithmetic relationship to another field, it is nearly free to verify; wherever it does not, that is where the review budget belongs, and a confidence score is a much weaker instrument than people expect.