Citations and Sources in an AI Interface

A citation next to a generated sentence looks like evidence. Whether it is evidence depends entirely on something no interface checks by default: that the cited text actually says the thing the sentence claims.

Three ways a citation fails

They are worth separating because only one of them is caught by the checks most teams have.

• The source does not exist. A fabricated URL, a fabricated paper, a plausible DOI that resolves to nothing. Highly visible once anyone clicks, and mechanically preventable: never let the model write a citation, only let it select from identifiers you supplied. See why models invent references.
• The source exists and does not support the claim. This is the common one and it is invisible to every automated check you are likely to have. The document is real, retrieved, and in context; the generated sentence overstates it, merges it with something else, or attaches it to a nearby claim it does not cover. A link checker passes. A reviewer skimming passes. Only reading both catches it.
• The granularity is useless. The citation is correct and points at a 200-page PDF. Formally accurate, practically a refusal to help, because verifying now costs more than the answer saved.

Notice that the second and third failures share a consequence: a citation the user cannot cheaply check is a trust signal that carries no information. It makes the answer feel more reliable without making it more reliable, which is the definition of the thing you are trying not to build.

A citation is a verification device

The framing that makes every subsequent decision fall out: a citation exists to reduce verify_time. It is not a credibility badge and it is not academic convention. So it is specified by an inequality — the time to check a cited claim must be small enough that a user actually does it, for the claims that matter.

Everything follows. If checking requires opening a new tab, finding the document, and searching it, verification costs a minute and nobody does it at scale. If the cited span is visible in a side panel with the relevant sentence highlighted, it costs a few seconds and it happens. The design target is seconds, and that target rules out most of the citation UI people ship.

Anchor to a span, not a document

The single highest-value change available in citation design, and it is mostly an indexing decision rather than a visual one.

Attribution after generation

Asking the model to emit citations inline is the cheap approach and it is the one that produces failure mode two, because the model is generating the citation marker as text — a plausible continuation, under the same mechanics as everything else it writes. It is not performing a lookup.

The more reliable structure is a separate pass: generate the answer, then attribute it against the retrieved chunks, and render only the citations that survive.

// Post-hoc attribution. Cheap filter first, model check only on
// the candidates that pass it.
async function attribute(answer, chunks) {
  const out = [];

  for (const sentence of splitSentences(answer)) {
    // 1. Cheap: lexical + embedding overlap to shortlist.
    const candidates = chunks
      .map(c => ({ c, score: overlap(sentence, c.text) }))
      .filter(x => x.score > OVERLAP_FLOOR)
      .sort((a, b) => b.score - a.score)
      .slice(0, 3);

    // 2. Expensive: does the candidate actually entail the sentence?
    //    One small-model call per candidate, not per chunk.
    let support = null;
    for (const { c } of candidates) {
      const verdict = await entails(c.text, sentence);  // yes | no
      if (verdict === "yes") { support = c; break; }
    }

    out.push(support
      ? { sentence, chunkId: support.id,
          span: locate(support, sentence), state: "grounded" }
      : { sentence, state: "unverified" });
  }
  return out;
}

The cost is real: an extra pass over the answer, and one small-model call per candidate. The mitigation is that the entailment check is a short, constrained, yes/no task, which is exactly the shape a much cheaper model handles well. Whether it is worth it is a domain question, and the answer is obviously yes wherever a wrong citation would be quoted onwards.

The pattern also produces the unverified state used in confidence display for free, which is a better use for the pass than the citations alone.

When no citation is the right answer

There is a real temptation to cite everything, because citations look rigorous. Two cases where the correct output is none:

• When the claim is not from a source. Synthesis, summarisation across many documents, and the model’s own general knowledge are all legitimate outputs that have no span to point at. Marking them as unsourced is more useful than attaching the nearest plausible document.
• When attribution failed. If the entailment check comes back negative, showing no citation is correct. The alternative — showing the best candidate anyway — is precisely how failure mode two gets shipped, and it is worse than nothing, because a wrong citation converts a sceptical reader into a trusting one.

Which is the general principle underneath the page: the value of a citation system is set by its precision, not its coverage. A system that cites 40% of sentences and is always right is more useful than one that cites 100% and is sometimes wrong, because the first can be trusted without checking and the second cannot.

That principle is unusual enough to be worth stating in the negative, because coverage is what gets reported and precision is what gets assumed. A dashboard showing “98% of claims cited” is measuring the wrong thing, and improving it is actively harmful if the improvement came from lowering the entailment threshold. If you instrument one number here, instrument the rate at which attribution fails and is honestly reported as unverified. A rise in that number is retrieval getting worse or generation drifting off-context, and it is visible weeks before anyone notices the answers are less reliable.

What the citation does to the reader

There is one more reason to hold citations to a high bar, and it is not about correctness at all. A citation changes how the sentence next to it is read. It signals that somebody checked, and it borrows the authority of the source for text the source may not support — which means a citation system with poor precision does not merely fail to help, it actively suppresses the scepticism that was the reader’s only remaining defence.

This is the same effect described under confidence display: any affordance that signals reliability reallocates attention away from verification. Citations are the strongest such signal available, because they carry the visual grammar of scholarship. That is exactly why they are worth building properly, and exactly why the failure mode in the middle of this page — a real source that does not support the claim — is the most damaging thing on the list rather than the most forgivable.