Instructor and Structured Extraction

Instructor is a small idea applied consistently: declare a Pydantic model, ask for it, and if the response does not validate, send the validation error back to the model and ask again. The library is mostly that loop. Understanding what it puts in the repair message is the difference between using it well and being confused by it.

The model as a typed function

The usage shape has been stable since the project started: wrap a provider client, pass a response model alongside the messages, get a validated instance back instead of a string.

class Invoice(BaseModel):
    supplier: str
    invoice_number: str
    total_pence: int = Field(description="Total including VAT, in pence")
    due_date: date | None = None

client = instructor.from_openai(OpenAI())

invoice = client.chat.completions.create(
    model=MODEL,
    response_model=Invoice,
    max_retries=2,
    messages=[{"role": "user", "content": raw_text}],
)

invoice.total_pence   # an int, or an exception was raised — never a string

What you gain over calling the provider’s own structured-output feature directly is not the JSON. It is that Pydantic validation runs on the result, that a failure is turned into another attempt rather than an exception at your call site, and that the same code works across providers whose structured-output support differs.

The repair loop, in full

This is the part worth being precise about, because it explains both the successes and the surprises.

1. The schema becomes a tool or a response format. The Pydantic model is converted to JSON Schema and handed to the provider using whatever mechanism that provider supports — a function definition, a response-format constraint, or a prompt instruction as a last resort. Which mechanism is used affects how often step three happens at all.
2. The response is parsed and validated. Not just “is it JSON” — every Pydantic rule runs, including your own field and model validators.
3. On failure, the error is appended and the call is repeated. The invalid output goes back as an assistant message and the validation error goes back as a user message. The model sees exactly what it produced and exactly what was wrong with it. This is why the second attempt so often succeeds: the error text is unusually good feedback.
4. After the retry budget is exhausted, it raises. Your code sees an exception, not a half-valid object.

The cost implication is direct and often missed. A retry is a fresh call with a longer conversation — the original prompt, plus the bad output, plus the error. With a retry budget of three, a page of input text and a stubborn schema, a single extraction can cost four times what you budgeted. Set the budget deliberately; two is usually right, and if you routinely need three, the schema is the problem.

Designing a schema the model can satisfy

Most extraction failures are schema failures. Five rules cover the majority of them.

• Every field gets a description. The descriptions go into the schema the model reads, so they are prompt text with a guaranteed delivery mechanism. “Total including VAT, in pence” prevents an entire class of error that no amount of system-prompt wording reliably prevents.
• Optional means optional. A required field the source document does not contain forces the model to invent one. This is the most common cause of fabricated values in extraction, and it is entirely self-inflicted. Make absence representable.
• Enums beat free strings. A literal union of five statuses is checkable and stable; a string field described as “the status” will return six spellings of the same three concepts.
• Keep nesting shallow. Failure rates climb with depth on every model. Two levels is comfortable, four is asking for trouble, and a flat list of objects is nearly always achievable instead.
• Put a reasoning field first if the task is hard. A short field ordered before the answer, asking for the evidence, gives the model somewhere to do the work — and gives you something to read when the answer is wrong. Fields are generated in order, so the position is what makes this work.

Validators are prompt engineering

The feature that distinguishes this library from a JSON parser: an ordinary Pydantic validator becomes an instruction the model receives on the next attempt. You do not have to describe the rule in the prompt at all — you enforce it, and the enforcement message is the description.

class Quote(BaseModel):
    text: str
    source_span: str

    @model_validator(mode="after")
    def span_must_be_verbatim(self):
        if self.source_span not in CURRENT_DOCUMENT.get():
            raise ValueError(
                "source_span must be copied verbatim from the document; "
                f"the text {self.source_span!r} does not appear in it"
            )
        return self

Note the error message. It is written to be read by a model: it says the rule, it quotes the offending value, and it does not say “invalid input”. Vague validator messages produce a second attempt that fails the same way, and vague messages are the single most common reason a retry budget is wasted.

This gives you a genuine grounding check for extraction — a quote that must appear in the source cannot be hallucinated — which is a stronger guarantee than any prompt instruction, and directly useful for citation work.

What a retry cannot fix

The honest limits, because the loop is convincing enough that people expect too much of it.

What the pattern costs

Two costs, both predictable, and both worth computing before setting a retry budget rather than after reading an invoice.

Cost 1 — the schema is in every request.
  The JSON Schema derived from your model is sent with each call.
  A flat model with 8 described fields is roughly 150–300 tokens;
  a nested model with 30 fields can exceed 1,000. At 100k extractions:

    300 tokens × 100,000 = 30M input tokens, spent on the schema alone.

Cost 2 — a retry is a longer call, not a repeat of the first.
  attempt 1 input:  P                      (prompt + schema)
  attempt 2 input:  P + O + E              (+ bad output, + error text)
  attempt 3 input:  P + O + E + O' + E'

  With P = 2,000, O = 400, E = 120 tokens and a failure rate f:

    expected input ≈ P + f·(P + O + E) + f²·(P + 2O + 2E)
    at f = 0.15:  2,000 + 0.15(2,520) + 0.0225(3,040) ≈ 2,447 tokens
    at f = 0.50:  2,000 + 0.5(2,520)  + 0.25(3,040)   ≈ 4,020 tokens

The shape of that is the useful part. At a low failure rate retries are nearly free and there is no reason not to allow two. At a high failure rate they double your cost and latency without fixing the underlying problem, which is the schema or the task. The number to instrument is therefore the retry rate itself: if it drifts above roughly one in five, that is a signal about your schema, not a cost line to absorb.

Latency follows the same curve and matters more in a request handler, because a retry is a second full round trip. Where extraction sits in front of a user, one retry and a graceful failure beats three retries and a timeout — the general reasoning is in graceful degradation.

Instructor versus native structured output

Providers now offer schema-constrained decoding that guarantees syntactically valid JSON matching a schema. That removes one of the two reasons for this library and leaves the other intact. Constrained decoding cannot enforce that a quote appears in the document, that a date is in the future, or that a total is consistent — those are semantic rules, and semantic rules are what the repair loop is for.

The sensible configuration is both: native constrained output for shape, Pydantic validators plus a bounded retry for meaning. The trade-offs of the underlying mechanisms are covered in JSON mode versus structured outputs, and the testing discipline in testing structured output.