Extracting Purchase Order Numbers Buried in Invoice Headers

The symptom is an accounts-payable queue full of no matching purchase order for this invoice on documents where a human can see the PO number printed at the top. The extraction returned po_number: null, or worse, returned the delivery-note number with high confidence.

The error you are actually chasing

Three-way matching compares an invoice to a purchase order and a goods receipt. If the invoice cannot be tied to a PO, none of that runs, and the invoice drops out to manual handling — which is where the cost of a document pipeline actually lives. So the failure to fix is not “the model did not read the number”. It is “the number we returned is not one of ours”.

That distinction changes the design. A PO number is not free text. It comes from a set you own: the open purchase orders in your own ERP, typically tens of thousands of them, each with a known format and a known supplier. You are not extracting an arbitrary string; you are choosing a member of a known set, or correctly deciding that no member applies.

What the reference field really contains

On a structured e-invoice this is easy: EN 16931 defines BT-13 as the purchase order reference — an identifier of a referenced purchase order issued by the buyer — distinct from BT-10, the buyer reference used for internal routing. Peppol BIS Billing 3.0 requires an invoice to carry one or the other. If the document arrives as UBL, read the element and stop.

On a PDF it is a header block that looks like this:

Your ref:  4500123456 / CC 8820 / Contract C-2291
Del. note: 1200456        Cust. no: 77341
Order:     PO-2026-0448   Project: NW-RETROFIT-2

Five to eight identifiers, at least two of which are ten digits long, none of them labelled in a way that generalises across suppliers. Some suppliers put the PO under “Your reference”, some under “Order”, some under “Customer order number”, and some print it only in the line-item table because different lines belong to different POs — which is common and which a header-only extractor cannot represent at all.

The label is unreliable, but the format is not. Most ERP systems mint PO numbers to a fixed shape: ten digits beginning with a particular series in one system, a prefixed and dated pattern in another, a plain sequence in a third. You know which shapes yours uses because you generated them.

Generate candidates, do not pick one

The first change is to stop asking the model for the PO number. Ask for every identifier-shaped token in the header and line area, with its label and location, and let a later stage decide:

"reference_candidates": [
  { "value": "4500123456", "label": "Your ref",  "region": "header", "line_no": null },
  { "value": "8820",       "label": "CC",        "region": "header", "line_no": null },
  { "value": "C-2291",     "label": "Contract",  "region": "header", "line_no": null },
  { "value": "1200456",    "label": "Del. note", "region": "header", "line_no": null },
  { "value": "PO-2026-0448","label": "Order",    "region": "header", "line_no": null }
]

This is a much easier task for a model than disambiguation, because it is transcription plus locality. It is also measurable in a way the original task is not: recall on candidates is checkable by a human in seconds, and a missed candidate is a different bug from a wrong choice.

A regex pass over the raw text layer should run alongside, not instead. The two failure modes are complementary: the regex misses anything the text layer garbled or that sits inside an image, and the model misses tokens that look unimportant. Union the two candidate sets, keep the provenance, and deduplicate on the normalised value.

Constrain against the open-PO set

Now the closed-set step, which is where the accuracy actually comes from. For each candidate, normalise and look it up:

1. Normalise: strip a leading PO or P/O prefix, separators, and leading zeros into a canonical form, keeping the original.
2. Look the canonical form up in the open-PO index. Discard anything that is not a live PO.
3. Filter surviving matches by supplier: the PO’s vendor must be the invoice’s supplier, resolved by tax identifier as in extracting the vendor name. This single filter removes almost every wrong-but-valid match.
4. If exactly one candidate survives, that is the PO number, regardless of what it was labelled. Record the label anyway.
5. If more than one survives, prefer the candidate whose PO has an open balance covering the invoice amount, then route the rest to review with all surviving candidates attached.

The point of this ordering is that a delivery-note number that happens to look like a PO number will not be in the PO index, and a valid PO belonging to a different supplier will not survive the supplier filter. You have replaced a judgement with two lookups.

What is left after the set is exhausted

• Per-line POs. A consolidated invoice covering four orders carries a PO per line. If your schema has one header field, this invoice is unmatchable by construction — it is a multi-entity document and needs a schema shaped for one. Put po_number on the line as well as the header and let the header be null when the lines disagree.
• Transposed digits. A candidate that misses the index by one character is worth a near-match pass against open POs for the same supplier, surfaced as a suggestion for a human rather than applied silently.
• Closed or fully received POs. The number is right and the index is the wrong index. Search closed POs before declaring no match, and report the state.
• Genuinely no PO. Non-PO spend is real — utilities, subscriptions, professional fees. The pipeline needs a legitimate “no PO, route by cost centre” path, or it will manufacture PO numbers to satisfy a required field.

Two amendments to the same PO can also change the number’s revision without changing the number, which matters when you match quantities and prices rather than just the identifier; that is the subject of matching a purchase order to its amendments. And when several candidate identifiers survive every filter, the invoice belongs in the review queue with the candidates ranked, not in the pipeline with the first one chosen — the trade-off between those two is what routing on a confidence threshold is about.