Extracting Structured Fields From a Proof of Delivery Document

A proof of delivery is not a data document that happens to have a signature on it. It is an evidentiary record whose entire point is a determination made at the door, and the fields that carry that determination are the ones a naive schema throws away.

What the document actually asserts

The freight paperwork you already extracted — the bill of lading, the manifest — describes what was supposed to move. The proof of delivery describes what arrived, in whose hands, in what condition, and whether anybody objected at the time. That last clause is the whole document. A delivery receipt taken without exception is treated very differently from one annotated at the kerb, because in US motor carriage the receipt is the contemporaneous record on which a later loss-or-damage claim rests. The Carmack Amendment, 49 U.S.C. § 14706, sets the carrier liability framework, and the FMCSA claim-handling rules at 49 CFR Part 370 describe what a claim must contain. You are not adjudicating any of that. You are extracting the fields somebody else will adjudicate, and the cost of flattening them is that the record stops being able to answer the question it exists for.

So the design question is not “which fields does this document have”. It is: which of these fields, if lost or silently defaulted, would change what the record proves? Four survive that test — who received it, whether they signed, what exceptions were noted, and how many pieces were actually accepted.

The signature is a presence question

Almost every schema written for this document has a signature_name string, and almost every one of them is wrong in the same way. The signature block usually holds two different things: a printed or typed name, which is text and can be read, and a handwritten mark, which is a graphic and generally cannot. Asking a vision model to transcribe a scrawl produces a plausible name that nobody wrote. That is not an OCR problem you can prompt your way out of; the information is not present in a legible form, and a model asked for text where there is none will supply some.

Split the two. signature_present is a boolean determination about whether a mark exists in the signature region — a question a model can answer well. received_by_printed is a text read of the printed-name line, which is often blank even when the signature line is not. Keep them independent, because the four combinations are all real and they mean different things: a signature with no printed name is a normal consumer delivery, a printed name with no signature is usually a driver filling in a release, and neither is the same as a blank receipt.

The related field that gets conflated with both is the relationship of the receiver to the consignee. “Signed by agent”, “front desk”, “neighbour at 14” and “security” are all deliveries, and a schema with only “consignee signed: yes/no” forces all of them into one bucket.

A status enum, not a nullable field

Here is the failure this page exists for. A proof of delivery with no signature is extracted as signature: null, and downstream every null looks identical: a value that was not on the page, a value the model could not read, and a value that was legitimately never collected. Those are three different facts and one of them is a normal successful delivery. Contactless release, photo-on-delivery and signature-waived shipments all produce a valid, complete document with nothing on the signature line.

Give the document a status field with a closed vocabulary and let the signature fields be evidence for it rather than a proxy:

{
  "delivery_status": "released_no_signature",
  // signed_by_consignee | signed_by_agent | released_no_signature
  // | refused | attempted_not_delivered | partial
  "signature_present": false,
  "received_by_printed": null,
  "receiver_relationship": null,
  "release_authority": "standing instruction on file",
  "delivered_at": "2026-03-14T16:22:00-05:00",
  "delivered_at_source": "driver device timestamp",
  "pieces_expected": 4,
  "pieces_accepted": 4,
  "exceptions": []
}

attempted_not_delivered earns its place because those documents exist and look almost identical: the same form, the same stops, a reason code and no receiver. Extracting one as a delivery with a null signature is a data error that will not show up until somebody counts deliveries.

Note the two timestamp fields. A delivery can carry a handwritten time at the door, a device timestamp from the driver’s scanner and a later upload time, and they disagree by minutes or hours. Record which one you took, and apply the ordinary date-field validation rules to each separately rather than to a merged value. Local time at the delivery point is also the norm on the paper, so an offset that you inferred from the stop address should be marked as inferred rather than presented as though it was printed.

Exception notations and piece counts

Exceptions are the annotations written on the receipt at the moment of delivery: shortage, damage, refusal of part of the load. In US LTL practice these are handled under the general heading of OS&D — over, short and damaged — and they arrive as free text in a margin, as a tick in an exception box, or as both saying slightly different things. Two structural traps follow.

• An exception is a list, not a flag. One delivery can be two cartons short and have a crushed corner on a third. A boolean damaged loses the second fact and a single string loses the ability to count either.
• Pieces expected and pieces accepted are separate numbers. A short delivery is precisely the case where they differ, so a schema with one piece_count cannot represent the only situation anybody cares about. If the form prints a total and the itemised lines do not sum to it, keep both and flag the mismatch rather than choosing.
• “Subject to inspection” is not an exception. Pre-printed reservation language appears on every copy of some carriers’ forms. If your extraction treats pre-printed text as a written annotation, every delivery becomes claused. Anchor exception capture to the handwriting and the tick boxes, not to the presence of a phrase.

Photo references point outside the page

Modern proofs of delivery carry a photo, and what appears on the document is a reference to it: a thumbnail, a filename, a URL, a shipment-scoped image id. The reference is extractable and the image is not on the page. Store the identifier as an identifier — a string with its scheme recorded — and resolve it separately against wherever the carrier actually keeps the file. A schema that declares photo: true throws away the only part you had.

Two practical notes. Thumbnails on a scanned page are usually far below the resolution at which anything in them is readable, so treat the thumbnail as confirmation that a photo exists rather than as an image to analyse; the general trade-off is covered in image detail levels. And a delivery photo often contains a house number, a car number plate or a person, which makes it personal data that arrived attached to a logistics record. If those images will be sent anywhere for analysis, they belong in the same review as any other PII in your pipeline, not in a different one because they came from a driver’s phone.

The check that catches most real errors here is not a confidence threshold. It is arithmetic and consistency, which is an ordinary cross-field validation rule applied to counts rather than money: pieces accepted plus shortages equals pieces expected, an exception list that is non-empty implies a status other than a clean signed delivery, and a delivered_at that precedes the pickup timestamp on the linked manifest is a document you have matched to the wrong shipment.