Extracting a Table Whose Columns Don't Align Between Pages

Pages one and two of the report extract cleanly. From page three, amounts start appearing under Quantity and the descriptions run into the column beside them. Nothing about the table changed visually. What changed is that you defined the columns once, on page one, and then used those coordinates on every page after it.

The symptom, and where it starts

Positional column extraction has a characteristic failure signature. The error appears abruptly at a page boundary rather than gradually within a page; it affects a run of adjacent columns rather than one; and the misplaced values are systematically shifted by exactly one column, not scattered. If you see values shifted by one from a specific page onward, you have a column-boundary problem, not a model problem, and no amount of instruction in the prompt will fix it because the damage is done before the model is called.

The distinct case where a single page’s header covers several columns is a merged header cell, handled separately. This page is about the join between pages.

Why the coordinates move

There are four independent causes and they compound.

• The renderer re-lays out each page. A generator that sizes columns to content will size them to the content of each page. One long supplier name on page three widens a column and everything to its right shifts.
• A column is empty on this page. Some generators collapse a column with no values on a given page. The visual table still has the right number of headings, but a positional splitter counting gaps counts one fewer.
• The scan is not square. Paper fed through a document scanner arrives with a small rotation and a small scale error, and both differ per sheet. This is the dominant cause in scanned reports and the reason the problem is described as common there.
• The page was photographed, not scanned. Then you have perspective as well as rotation, and column boundaries are not even parallel vertical lines any more.

How much drift it takes

It is worth knowing the order of magnitude, because it explains why a rotation too small to notice by eye breaks extraction. Take a table seven inches wide, which is a normal printable width on A4 or Letter with ordinary margins, and assume a page skew of one and a half degrees — a plausible feeder misalignment, and small enough that most people would call the scan straight. Both of those are assumptions stated here so the arithmetic can be checked, not measurements.

horizontal displacement = width x tan(skew)
                       = 7.0 in x tan(1.5 deg)
                       = 7.0 x 0.0262
                       = 0.183 in  ~= 13 points at 72 pt/in

A thirteen-point horizontal displacement across the width of the page is larger than the gutter between two tightly set numeric columns, which in a dense financial table is often six to ten points. So a one-and-a-half-degree skew is sufficient, on its own, to push the right-hand columns of a page across a fixed boundary derived from a different page. Halve the skew and you halve the displacement; the relationship is linear in both width and angle for small angles.

The practical consequence is that deskewing each page before you measure anything is not an optimisation. It is a precondition, and it belongs in the same stage of your OCR pipeline as binarisation.

Anchor on the header, not the position

The fix is to stop treating column boundaries as a property of the document and start treating them as a property of the page. For each page independently: find the header row, read the header labels, derive the x-interval of each label, and build the page’s own boundary list. Then map the page’s columns onto the canonical schema by matching header text, not by index.

1. Deskew the page and recompute word boxes, so that x coordinates on this page mean something.
2. Locate the header row: the topmost band of text whose normalised strings have high overlap with the canonical header set you learned from page one.
3. For each canonical field, find the header run on this page whose normalised text matches. Normalise aggressively here — case, whitespace, soft hyphens, and the footnote markers that appear on some pages and not others.
4. Derive boundaries from the gaps between the body text of this page, not from the header extents, and associate each boundary region with the canonical field whose header sits above it.
5. Emit rows keyed by canonical field name. From here on, nothing downstream ever sees an x coordinate.

Most table libraries let you supply boundaries per page rather than globally — Camelot’s stream mode takes explicit column x positions, and pdfplumber’s table settings accept explicit vertical lines — so this is a matter of calling them once per page with page-specific arguments rather than writing a new extractor.

Pages that repeat no header

Long tables frequently print the header only on the first page. Now there is nothing to match on, and the two available strategies both have a failure mode you should choose deliberately.

The first is to carry forward the previous page’s canonical field order and re-derive only the boundaries from this page’s own body text. That is right whenever no column is missing, and it is the strategy to default to. The second is to type the columns: determine for each detected region whether its values are dates, amounts, quantities or free text, and match those types against the canonical schema. Type matching survives a collapsed column and fails when two adjacent columns have the same type — two amount columns are exactly the case it cannot resolve.

Use both: carry the order forward, then verify with types, and route the page to a human when they disagree rather than picking a winner. That is a straightforward application of confidence-based review routing where the confidence signal is a structural disagreement rather than a model score.

Checking the join arithmetically

A multi-page table usually gives you a free correctness check that a single page does not: a carried-forward subtotal, a running balance, or a grand total at the end. Foot every page and compare. A shifted column almost always breaks the arithmetic, because the values that landed in the amount column came from somewhere else. This is more reliable than any per-field confidence score, and the general form of it is on cross-field amount validation.

Two details make the check usable. Compare in integer minor units, not floats, for the reasons on currency validation. And when the total does not foot, report the difference rather than a boolean: a discrepancy equal to exactly one line’s value points at a dropped row, while a discrepancy that is a rounding-sized residue points at a units or sign problem.