Extracting Structured Fields From a Moving Company Bill of Lading

A household goods bill of lading shares a name with the freight document and almost nothing else. Its payload is an inventory of a few hundred numbered items, each carrying condition symbols from a legend the carrier printed on its own form — which is why a hardcoded list of condition codes is wrong before you start.

Not the freight document of the same name

An ocean or trucking bill of lading is about a consignment: container numbers, seal numbers, gross weight, ports, Incoterms. Its identifiers are standardised and checkable. A household goods bill of lading, governed in the US by the Federal Motor Carrier Safety Administration under 49 CFR Part 375, is a contract of carriage between a mover and a household, and its payload is a descriptive inventory of a family’s possessions.

The consequence is that nothing on it has a check digit. There is no ISO 6346 container number to validate, no standardised commodity code, no weight that has to match a manifest. What it has instead is an internal consistency: the inventory is written at origin and signed, and the same inventory is annotated again at destination and signed again. The document validates against itself across time rather than against an external standard, and building the extraction around that is the whole job.

Practically the packet is several documents stapled together: the bill of lading proper, which is the contract and carries the parties, the dates, the estimate and the valuation election; the descriptive inventory, which is the numbered item list; and often an estimate or order for service. They are frequently scanned as one PDF. Split them before extraction — the inventory is a repeating-row structure and the bill of lading is a form, and asking one prompt to handle both produces the worst of each.

The legend is on the form, so read it first

Every mover’s inventory sheet prints a legend, usually down the left margin or across the header, mapping single letters to conditions and digits to locations on an item. Movers write SC for scratched, D for dented, G for gouged, R for rubbed, W for worn, PBO for packed by owner, CP for carrier packed, and pair them with numbers identifying which surface of the item they apply to.

Here is the point that separates this page from a generic extraction page: those legends are not standardised across carriers. Two movers use different letters for the same condition, and worse, the same letter for different conditions. A code that means “marred” on one form means “mildewed” on another. If you hardcode the legend from the first carrier whose forms you processed, you will silently mis-decode every subsequent carrier, and the output will look entirely reasonable. It is the sharpest case of the problem covered in schema design for unseen variants, because the variant is not a new field but a new meaning for an old one.

So the extraction has two stages. First read the legend block off the form itself and produce a mapping. Then read the inventory rows and decode the symbols using that mapping, not a global one. The decoded value and the raw symbol both go into the record:

{
  "carrier": "Example Van Lines",
  "legend_source": "extracted_from_form",
  "legend": { "SC": "scratched", "D": "dented", "M": "marred",
              "1": "top", "2": "bottom", "3": "left", "4": "right" },
  "items": [
    { "number": 118, "description": "Dining table, oak",
      "symbols_raw": "SC1, D4",
      "conditions": [
        { "code": "SC", "meaning": "scratched", "location": "top" },
        { "code": "D",  "meaning": "dented",    "location": "right" }
      ],
      "packed_by": "CP" }
  ]
}

When the legend block is missing, illegible or cropped out of the scan, do not fall back to a default mapping; treat it the way an illegible field is treated anywhere else. Emit the raw symbols with the meaning left null and a flag saying the legend was unavailable. An undecoded symbol is honest; a wrongly decoded one is a claim about somebody’s furniture.

Two readings of one item

The same item number is annotated twice: once at origin, when the mover records its condition before loading, and once at destination, when the customer checks it off. The difference between those two annotations is the damage claim. That is the entire commercial purpose of the document.

An extractor that treats the two pages as independent documents and produces two rows for item 118 has destroyed the relationship. The record must be a pair, keyed on the item number, with an explicit diff:

item 118  origin:      SC1
          destination: SC1, D4, missing leg glide
          new_at_delivery: ["D4", "missing leg glide"]

Three failure modes attach to the pairing and each is worth a specific check. Item numbers repeat across sheets when a move uses more than one inventory booklet, so the key is the sheet identifier plus the item number, not the number alone. Items are added at origin out of sequence and squeezed into the margin, so the printed order is not the numeric order. And items are legitimately absent from the destination sheet, which usually means undelivered rather than undamaged — so a missing destination annotation must be its own state, not an empty condition list.

Valuation is the field that decides the claim

One field on the bill of lading proper determines what any of the damage annotations are worth. Under the FMCSA rules the customer elects a level of liability coverage, and the two standard options are Full Value Protection, which is the default, and a waiver down to a released value stated per pound per article, which the customer must affirmatively sign for. The released value option is dramatically cheaper and dramatically less useful — it is a weight-based cap, so a light and expensive item is barely covered.

Extract the election as an enum with an explicit not_marked value, capture the released-value rate as printed rather than from memory, and capture whether the waiver carries a separate signature, because the waiver is only effective if signed. This is the same absent-versus-negative distinction that runs through the vehicle title lien field, and it fails the same way: a blank box read as a declined option.

The estimate fields alongside it are worth the same care. A binding estimate and a non-binding estimate are different products with different rules about what the customer must pay at delivery, and the form distinguishes them by a checkbox rather than by wording. Extract the type, not just the amount.

Everything here is handwritten

The printed parts of this document are the legend and the column headings. Everything that matters — the item descriptions, the symbols, the exception notes at delivery — is written by hand, on carbonless duplicate paper, frequently while standing up. The third copy in the pack is a purple-grey smear.

Two things follow. First, ask for the best copy: if the customer photographs the top sheet rather than their carbon copy, legibility improves more than any prompt change will achieve. Second, expect the numeric fields to be where the errors concentrate, because a handwritten 1 and 7, 4 and 9, or 3 and 5 are genuinely ambiguous in isolation and the item number has no redundancy to recover from. Use the sequence itself as the redundancy: item numbers on an inventory run consecutively, so a jump from 117 to 119 to 118 is almost always a misread rather than a mover who numbered out of order. Reconciling against the sequence catches more digit errors than any confidence threshold. The general ground on handwriting recognition is covered elsewhere in the library and is worth reading before you tune anything here.