Pattern: Extract, Then Reason

“Read this invoice and tell me whether to approve it” is two tasks pretending to be one. The model reads badly and reasons badly at the same time, in a single opaque step, and when the answer is wrong there is nothing to inspect. Splitting it produces an artefact in the middle, and the artefact is most of the benefit.

One call doing two jobs

The combined prompt is the natural first draft because it matches how you would ask a person. It also has three properties you would not accept anywhere else in a system.

It is untestable at the level that matters. If the approval decision is wrong, you cannot tell whether the model misread the total or read it correctly and applied the policy wrongly. Those need completely different fixes — one is an extraction problem solvable with a schema and better formatting, the other is a rules problem that may not belong in a model at all — and the single call gives you no way to tell them apart.

It is uncacheable and unreusable. The extraction is deterministic-ish work about a document that does not change; the decision depends on policy that changes monthly. Fused together, a policy change forces you to re-read every document, and a second feature that needs the same fields has to re-extract them.

And it has no audit trail. “Approved” with a paragraph of justification is not evidence, because the justification is generated text that may or may not describe the actual basis of the answer. A structured intermediate is evidence, because you can check it against the source.

Why splitting helps

The mechanism is worth stating precisely, because it also tells you when the split will not help.

A fused call has one error rate that hides two. Reading errors and reasoning errors both surface as a wrong final answer, and they compound: a misread field produces a correctly-reasoned wrong answer, which is indistinguishable from a correctly-read badly-reasoned one. If the probability of a clean read is r and of correct reasoning given a clean read is d, the fused call is right at about r × d — and you can observe only the product, so you cannot tell which factor is costing you.

Splitting does two things to that. It makes both factors separately observable, which converts an unimprovable number into two improvable ones. And it frequently raises r outright, because extraction with a constrained schema is a task with a closed answer space and a verifiable result, where the techniques that work are well understood — the whole of structured extraction applies, including field-level validation the fused version had no place to put.

The second stage often improves too, for a less obvious reason: it now operates on a short structured object rather than on ten pages of text. The distraction and position effects that degrade reasoning over long inputs largely disappear when the input is twelve fields.

The shape, with the intermediate

The middle artefact is the design decision. It should contain everything the second stage needs and nothing else, and each field should be traceable to the source.

// Stage 1 — read. No judgement, no policy, no conclusions.
type Extracted = {
  vendor:        { value: string;  span: [number, number] };
  total_cents:   { value: number;  span: [number, number] };
  currency:      { value: string;  span: [number, number] };
  line_items:    { description: string; cents: number }[];
  po_number:     { value: string | null; span: [number, number] | null };
  // Every field carries where it came from, so a disputed value is checkable
  // against the document rather than against the model's recollection.
};

// Between the stages — deterministic, free, and where most bugs are caught.
function check(e: Extracted) {
  const sum = e.line_items.reduce((a, l) => a + l.cents, 0);
  if (sum !== e.total_cents.value) throw new ExtractionMismatch("total");
  if (!KNOWN_CURRENCIES.has(e.currency.value)) throw new ExtractionMismatch("currency");
  return e;
}

// Stage 2 — decide. Sees the object, never the document.
//   Often not a model call at all: if the policy is expressible as code,
//   this is the point at which you discover that, and the discovery is
//   the largest win the pattern offers.
function decide(e: Extracted, policy: Policy): Decision { ... }

Three details do the work. The spans make every extracted value checkable against the source, which turns a truth question into a string comparison. The arithmetic check between the stages is free and catches a real share of misreads before anything downstream sees them. And decide is written as an ordinary function first — if the policy turns out to be expressible in code, you have removed a model call, a source of non-determinism and an entire class of failure, and the second stage becomes something you can unit test. That outcome is common enough that it is worth attempting deliberately.

Measuring it on your task

Whether the split improves accuracy for you is a question with a cheap experiment attached, and it is worth running rather than assuming, because the answer varies by task and by document quality.

• Label a set of documents at both levels. The correct field values and the correct final decision. This is the only expensive part, and thirty to fifty documents is usually enough to see a difference that matters.
• Run the fused version. Record the final-answer accuracy. This is your baseline and it is one number.
• Run the split version. Record field-level extraction accuracy and decision accuracy given correct fields, separately. You now have two numbers where you had one.
• Compare, then look at the disagreements. The comparison of end-to-end accuracy is the headline, but the useful output is the breakdown: if extraction is the weak factor, the fix is schema and formatting work; if reasoning-given-correct-fields is weak, the fix is policy logic and the model may not be the right tool for it at all.

Report both factors, not just the product. A team that knows its extraction is accurate and its decision step is the weak link has a tractable problem; a team that knows only that end-to-end accuracy is a certain figure has a mood. The sample sizes needed to call a difference real are the subject of the statistics page, and this is a case where the difference is often small enough that the question matters.

When not to split

The pattern has real costs: two calls instead of one, an intermediate schema to maintain, and a stage boundary that discards information. Three cases where it is the wrong choice.

• The reasoning genuinely needs the raw text. Tone, intent, sarcasm, rhetorical structure, anything where the wording is the evidence. A schema is a lossy projection, and if the thing you need to reason about does not survive the projection, do not project. The tell is that you cannot write down the intermediate type without a field called notes.
• Latency dominates and the task is easy. Two sequential calls roughly double time to answer. For a short input where the fused version is already accurate, the split buys observability you may not need at a latency price the user does feel.
• The intermediate would be as large as the input. If extraction cannot compress — a long document where everything matters — you have paid for two passes over the same material and gained only the audit trail. Sometimes the audit trail alone justifies it; often it does not.

The general form of this trade — decomposition versus one large prompt — is worked through in prompt chaining, and the extract-then-reason split is its most reliable instance because the two halves have genuinely different characters rather than being two parts of one reasoning chain.