Checking Model Output Against a Database

The reliable way to stop a model asserting things that are not true is not a better prompt. It is to make every factual claim a structured object with enough information to look it up, check each one against a system of record, and refuse to render any claim that did not verify.

The idea: claims, not prose

A paragraph of generated prose containing five facts cannot be checked, because there is no reliable way to extract the five facts back out of it. Asking a second model to check the first is not verification; it is two opinions, and the second model has the same failure mode as the first.

So invert the output. The model produces a structured answer: a template of sentences plus a list of typed claims, each with the identifiers needed to resolve it. The application verifies the claims, then renders. Nothing reaches the user that did not pass.

{
  "answer_template": "Your order {order_id} shipped on {ship_date} and is "
                     "expected to arrive on {eta}. It contains {item_count} items.",
  "claims": [
    {"key": "order_id",   "type": "identifier",
     "entity": "order", "entity_id": "ord_88431", "field": "id",
     "value": "ord_88431"},
    {"key": "ship_date",  "type": "date",
     "entity": "order", "entity_id": "ord_88431", "field": "shipped_at",
     "value": "2026-07-29"},
    {"key": "eta",        "type": "date",
     "entity": "order", "entity_id": "ord_88431", "field": "estimated_delivery",
     "value": "2026-08-05"},
    {"key": "item_count", "type": "integer",
     "entity": "order", "entity_id": "ord_88431", "field": "line_count",
     "value": 3}
  ]
}

The template is the only place free text appears, and every slot in it is filled from a verified claim. A model that wants to state a fact must state it as a claim; there is no other channel, which is what makes the check exhaustive rather than best-effort.

The claim schema

The whitelist on field is a security control as much as a correctness one. Without it, a prompt-injected instruction can produce a claim addressing a field the user should not see, and the checker will dutifully verify it and render it. The list of readable fields per entity type belongs in code, not in the prompt — the reasoning is the same as in indirect prompt injection.

The checker

from dataclasses import dataclass

READABLE = {
    "order":    {"id", "shipped_at", "estimated_delivery", "line_count",
                 "total_minor", "status"},
    "customer": {"id", "name", "tier"},
}

@dataclass
class Result:
    key: str
    status: str          # confirmed | contradicted | not_found | not_checkable
    claimed: object
    actual: object = None
    source: str | None = None


def check(claim: dict, db) -> Result:
    entity, field = claim["entity"], claim["field"]

    if field not in READABLE.get(entity, ()):
        return Result(claim["key"], "not_checkable", claim["value"])

    row = db.fetch(entity, claim["entity_id"])       # parameterised, by id
    if row is None:
        return Result(claim["key"], "not_found", claim["value"])

    actual = row[field]
    if actual is None:
        return Result(claim["key"], "not_found", claim["value"])

    if equal_for_type(claim["type"], claim["value"], actual):
        return Result(claim["key"], "confirmed", claim["value"], actual,
                      source=f"{entity}:{claim['entity_id']}#{field}")

    return Result(claim["key"], "contradicted", claim["value"], actual,
                  source=f"{entity}:{claim['entity_id']}#{field}")


def render(payload: dict, db) -> dict:
    results = {c["key"]: check(c, db) for c in payload["claims"]}

    if any(r.status == "contradicted" for r in results.values()):
        return {"action": "regenerate_from_facts",
                "facts": {k: r.actual for k, r in results.items()
                          if r.actual is not None}}

    if any(r.status in ("not_found", "not_checkable") for r in results.values()):
        return {"action": "refuse",
                "unverified": [k for k, r in results.items()
                               if r.status != "confirmed"]}

    text = payload["answer_template"].format(
        **{k: r.actual for k, r in results.items()}
    )
    return {"action": "answer", "text": text,
            "citations": {k: r.source for k, r in results.items()}}

One detail in the last block is doing quiet work: the rendered text is formatted from r.actual, the database value, not from r.claimed. Even a confirmed claim renders the authoritative value, so a formatting difference between the two never reaches the user, and the model’s string is never the thing displayed.

Three outcomes, three behaviours

The refusal wording matters more than the mechanism. “I could not verify the delivery estimate for this order, so I have not given one” is a usable answer that tells the reader exactly what is missing. “I do not know” is not, and neither is a confident answer with one silently dropped sentence. This is the same discipline as abstention, made mechanical.

Numbers, dates and tolerance

Exact equality is right for identifiers and wrong for nearly everything else. Compare by type:

• Money. Compare integer minor units. Never compare formatted strings, and never compare floats — the classic failure is a claim of “1299.00” against a stored 129900 minor units being marked contradicted by a naive comparison.
• Dates. Normalise to a date, and decide explicitly whether the claim is about a timestamp or a calendar day. A shipment at 23:40 UTC on the 29th is the 30th in some timezones and both answers are defensible; only one is yours.
• Decimals. Define a tolerance per field, in the field’s own units, and store it beside the whitelist. A weight claimed as 4.2 kg against a stored 4.23 kg is confirmed at a 0.05 tolerance and contradicted at exact equality; the right answer is a product decision, so make it once and write it down.
• Enums. Case-fold and map through the alias table from your shared glossary, so that “Shipped”, “dispatched” and SHIPPED do not read as three different claims.
• Staleness. Return the record’s updated_at alongside the value, and treat a value older than the field’s freshness budget as not verified. A confirmed claim against a record that stopped updating three weeks ago is confirmed against stale data, which is a different kind of wrong.

The claim the checker cannot express

The honest limit of this design: it verifies claims that fit the claim schema, and the model can still put an unverifiable assertion into the template text itself — “this is unusual for orders of this size” — which passes through untouched because it contains no slot.

• Constrain the template. Validate it against a small library of approved sentence shapes rather than accepting arbitrary prose. Restrictive, and the right trade wherever the answer is transactional.
• Reject templates whose free text makes factual statements. A cheap classifier over the non-slot text, flagging comparatives, quantities and causal claims, catches most of it. Imperfect, and much better than nothing.
• Count the slots. A template with a great deal of prose and one slot is a model routing round the mechanism. Alert on the ratio; it moves before anybody notices the content.
• Log every result. The rates of confirmed, contradicted and not-found per field are the best hallucination monitor you will ever get, because they are measured against a system of record rather than against a judge. A field whose contradiction rate jumps is either a model regression or a schema change, and both are worth a page.