Handling a Required Field That Is Missing From the Source Document

Your schema says purchase_order_number is required. The invoice in front of you does not have one, because the customer ordered by email. The extraction returns null, the validator raises “required field missing”, and a human is now looking at a document to confirm the absence of something that was never there. Repeat a few thousand times and the review queue is mostly non-events.

Three states, not two

A field on an extracted record is in one of at least three states, and only the first is a value:

• Present. The document contains the field and the pipeline read it. There is a value, a confidence and a source location.
• Absent. The document does not contain the field. This is a positive finding about the document — it is the answer to the question, not a failure to answer it. A retail receipt has no purchase order number; a domestic shipment has no export licence; a sole proprietor’s invoice has no company registration number because the jurisdiction does not issue one.
• Not extracted. The pipeline did not determine whether the field is present. The page it would have been on failed to render, the response was truncated at max_tokens, the request errored after two retries, the document was 40 pages and you only sent 20, or the model simply omitted the key. Nothing at all has been learned about the document.

A fourth state is worth carrying separately if your documents warrant it: illegible, meaning the field is visibly present and its value could not be read. That one has its own decisions and its own page — the choice between a low-confidence guess and an explicit illegible flag — but note that it is distinguishable from “absent” only if something in your pipeline is looking at layout rather than at text.

What null costs you

Store absent and not-extracted as the same null and four separate downstream decisions stop being computable. This is the argument, and it is worth being concrete about each one.

Retry

Not-extracted is retryable and absent is not. Re-running a document because the field was genuinely missing burns a second vision call for a guaranteed identical answer, and on a large batch that is a real line on the bill. Worse, a retry loop keyed on null will retry the same document forever if the pipeline is well behaved and the document simply lacks the field.

Review routing

A reviewer shown an absent field can only confirm the absence, which takes them the same thirty seconds as a real correction and produces no information. A reviewer shown a not-extracted field is doing something useful. If both arrive as “missing”, your queue silently fills with the useless kind, your average handling time goes up, and the metric that would have told you — corrections per review — drops without anybody being able to say why.

Coverage measurement

“We extract the tax ID on 82% of documents” is a number about your pipeline only if the other 18% contain a tax ID. If half of them do not, your extraction rate on documents that have the field is 91% and you have been chasing a problem that is mostly a property of the corpus. You cannot recover this split retrospectively from a column of nulls; the information was discarded at write time.

Downstream validation

Conditional requirements are extremely common in document work: an EU cross-border invoice needs a VAT identification number, a domestic one may not; a shipment needs a customs value only if it crosses a border. The rule “VAT ID must be present when both parties are in different EU member states” is enforceable against a typed absence and unenforceable against a null that might mean your OCR died.

Making the model say which

The model can only report absence if you ask for it. An extraction schema that types every field as a nullable string invites the model to emit null for both cases, and the model has no way to signal the difference even when it clearly knows it — it saw the whole page and there was no PO number anywhere on it.

So make the absence explicit in the value shape. A field becomes an object with a status discriminant rather than a bare scalar:

{
  "purchase_order_number": {
    "status": "absent",
    "value": null,
    "evidence": "no purchase order line in the header block or the remit-to panel"
  },
  "invoice_number": {
    "status": "present",
    "value": "INV-2026-00814",
    "evidence": "header, right of the vendor logo"
  }
}

The evidence string is doing more work than it looks like. It is not for the reviewer, primarily — it is a cheap forcing function that makes the model commit to having looked somewhere, and it gives you something to read when a field starts being reported absent on documents where you know it exists. Keep it short and keep it optional-to-consume; nothing should parse it.

Note what this schema does not ask the model to do: it never asks the model to report “not extracted”. That state is not observable from inside a successful response. Not-extracted is assigned by your pipeline — when the call failed, when the response did not parse, when a required key is simply absent from the returned object, when the finish reason indicates truncation. Treat a key the model omitted entirely as not-extracted, never as absent, because a truncated response and a thoughtful omission look identical in the JSON and only one of them is a claim about the document.

Strict schemas make optionality awkward

Providers that enforce a JSON Schema during decoding generally constrain what optionality you may express, and several require every property to be listed as required — you model an optional field as a union with null rather than by omitting the key. That restriction is helpful here rather than annoying: it means the returned object always has every key, so a genuinely missing key is unambiguous evidence that something went wrong in transport or decoding rather than in the document. The general behaviour is covered in JSON mode versus structured outputs, and which providers enforce what is in structured output support.

Storing absence

In the database, resist the urge to model this as a nullable column plus a nullable reason column, because nothing stops the two from disagreeing. A field-level row with a non-null status column and a nullable value column keeps the invariant checkable: status present requires a value, every other status forbids one. That is a single check constraint, and a constraint that the database enforces is worth more than a convention six services agree to respect.

The field-level row is also what makes correcting one field without re-running the document possible, and what a reviewer’s correction attaches to when they discover that the field was present, in a place nobody looked. That correction — absent, corrected to a value — is one of the most informative events your pipeline can produce, because it is the only one that tells you a whole region of the document is being missed rather than misread.