Designing an Extraction Schema That Survives a Multi-Entity Document

A residential lease names two tenants, a landlord, a guarantor and a managing agent. It states five dates and four amounts, and three of the amounts are the same number. A schema with tenant_1 and tenant_2 will work on the first document and stop being trustworthy on the second.

Where numbered fields go wrong

Numbering parties encodes the order they were printed in, which is not a property of anything. Three failures follow directly.

• The number is not stable across documents. The same person is tenant_1 on the lease and tenant_2 on the renewal, because someone typed the names in a different order. Any join between the two documents on the numbered field is wrong.
• The number is not stable across extractions. Re-run the extraction after a model change and the order can differ, silently invalidating every stored correction keyed on it — the path-stability problem described on nested and flat schemas.
• The number has no bound. You wrote fields for two tenants. The document with four does not fit, and the schema cannot say that it did not fit.

Role is not ordinal

The distinction that resolves this: a role comes from the document type and is a closed vocabulary, while an ordinal comes from the printing and is an accident. A lease has a lessor, a lessee, possibly a guarantor and possibly an agent; those four words are decided by the instrument, not by the page. That two people occupy the lessee role is a fact about cardinality within a role, not a fact about two different roles.

So the model is: an array of parties, each carrying a role drawn from an enum, plus an ordinal that records the printed order for reference only and is never used as an identifier. The enum is the part that repays effort, because it is where the domain knowledge lives, and because a constrained decoding mode can enforce it — the general testing of that is on enum constraint compliance.

One caution about the vocabulary: use the words the instrument uses, not a normalised business synonym. A document that says lessee and a document that says tenant may mean the same role in ordinary speech and different things in the jurisdiction the document belongs to. Extract the term as printed, map to your canonical role separately, and keep both. Mapping is a decision you may need to revise; extraction is not.

The shape that works

{
  "parties": [
    {
      "role": "lessee",
      "role_verbatim": "Tenant",
      "ordinal": 1,
      "name": "A. Okonkwo",
      "is_organisation": false,
      "source": { "page": 1, "bbox": [72, 402, 268, 416] }
    },
    {
      "role": "lessee",
      "role_verbatim": "Tenant",
      "ordinal": 2,
      "name": "R. Okonkwo",
      "is_organisation": false,
      "source": { "page": 1, "bbox": [72, 418, 271, 432] }
    },
    {
      "role": "guarantor",
      "role_verbatim": "Guarantor",
      "ordinal": 1,
      "name": "Meridian Housing Trust",
      "is_organisation": true,
      "source": { "page": 4, "bbox": [72, 128, 302, 142] }
    }
  ]
}

Three details in that structure are doing real work. The ordinal is scoped within a role, so adding a guarantor does not renumber the tenants. The verbatim role preserves what the document said. And every party carries its source region, without which a reviewer cannot check the assignment and source highlighting has nothing to point at. Two synthetic surnames that match, as above, are also the case worth testing deliberately: joint tenants are frequently related, so name similarity is not evidence of a duplicate extraction.

When three amounts are the same number

This is the failure that gets missed, because it produces output that looks correct. On a lease, the monthly rent, the first month’s payment and the security deposit are commonly all the same figure. An extraction that finds the amounts on the page and assigns them by proximity or by order will happily put the same value in all three fields and be right by luck, and will be wrong in the same way on the document where the deposit is five weeks rather than one month.

The rule is to key on the label, never on the value. A monetary field is identified by the text that introduces it, and the extraction should be required to return the label it matched alongside the amount. If two fields report the same source region, that is a definite error regardless of whether the values look plausible, and it is a check you can run without any ground truth: no two distinct fields may derive from the same span.

Duplicate-span detection is one of the highest-yield validations in a multi-entity extraction, and it costs nothing once every field carries its source. It also catches the opposite failure, where a document states an amount twice with different values — a figure in words and a figure in digits that disagree, which is a genuine drafting error and should reach a human rather than being resolved by preferring one.

Dates of the same kind

The same discipline applies to dates, and documents carry more of them than people expect. A lease has an execution date, a commencement date, an expiry date, a rent review date and a notice deadline; a shipment has a booking date, a departure date, an estimated arrival and a delivery date. All are dates, all look identical to a pattern matcher, and the difference between them is entirely in the label.

Give each its own named field with a role-like name that says what the date does — commences_on, expires_on, executed_on — rather than a generic array of dates, since unlike parties these are distinct roles with cardinality one. Then add the relational checks that only exist because there are several: commencement is not after expiry, execution is not after commencement by an implausible margin, a notice deadline falls inside the term. Each individual date still needs the calendar and range validation on validating a date field; the ordering checks are additional, and they catch the swap that per-field validation cannot see because both values are valid dates.

When the role cannot be determined

Some documents genuinely do not say. A signature block with two names and one title, a party introduced only in a recital, an address that could belong to either side. The wrong response is to assign the most likely role, because that produces a confident field indistinguishable from a certain one.

Keep the party, set the role to null, keep the source region, and let the record be incomplete. A schema in which role is nullable can represent “there is a fourth party here and I do not know what they are”, which is true and actionable; a schema in which it is not forces a guess. Route the document on the missing role rather than on a confidence number — a structural gap is a better routing signal than a score, and the routing mechanics are on confidence threshold review routing.

Finally, record which model version produced the assignment. Role assignment is the part of a multi-entity extraction most sensitive to a model change, because it depends on reading the document’s structure rather than copying a string, so it is the field set most likely to shift under you — which is what a model-version audit trail exists to make visible.