Showing Confidence Without Faking Precision

“I am 87% confident” is a sentence a model generated. It is not a measurement, it was not computed, and nothing checked it. The interesting question is not whether to display it — it is what, if anything, in the whole system does constitute evidence about correctness.

Four places a number could come from

There are exactly four candidate sources in a normal stack, and it is worth being precise about what each is a probability over, because that is where the confusion lives.

1. The model states it

You ask for a confidence score and it produces one. Mechanically this is the same operation as producing any other text: a plausible continuation. The number is drawn from what confidence expressions look like in the training data, not from any internal quantity. It is not a probability over anything.

2. Token logprobs

Genuine probabilities, and usable for real work — but probabilities over which token comes next, not over whether a claim is true. A model can be highly certain about the surface form of a sentence whose content is invented, because the fluent continuation and the true continuation are different questions. Logprobs are strong evidence for a constrained choice — a single-token classification label — and weak evidence about a paragraph of facts.

3. Agreement across samples

Run the same prompt n times and see whether the answers agree. This one is real: it measures the model’s stability on this input, and instability is genuine evidence that the answer is unreliable. Note the direction of the implication — disagreement is informative, agreement is much weaker, because a model can be consistently wrong.

4. External verification

Checking the claim against something that is not the model: a retrieved document, a database, a compiler, a unit test, an API. This is the only source that measures correctness rather than the model’s behaviour, and it is the only one from which a confident display is defensible.

Why the percentage is the wrong shape

Even where a number is genuinely derived, rendering it as a percentage makes two claims the underlying quantity does not support.

The first is calibration. A displayed 70% implies that across many such claims, roughly seven in ten are true. That is a strong property and it has to be established empirically for your task and your model. Guo, Pleiss, Sun and Weinberger’s 2017 paper On Calibration of Modern Neural Networks is the standard reference for the general result that modern networks are typically miscalibrated and tend toward overconfidence, and that post-hoc correction is needed to fix it. None of that correction is happening in a model that simply says a number in a sentence. See calibration in language models.

The second is resolution. Two significant figures implies you can distinguish 87% from 84%. Even a well-founded confidence estimate over free text rarely supports more than three or four distinguishable levels, and a display with more precision than the signal has is exactly the kind of false precision that gets believed.

What to show instead

Replace the scalar with a statement about provenance, which is both honest and more actionable — it tells the user where to look rather than how worried to be.

Three or four states, visually distinct, never colour alone. The decision a user makes with this is binary in practice — check it or do not — so a display with four levels already exceeds the resolution of the action it informs.

Computing a stability signal

Where the value is small and structured, agreement across samples is cheap to compute and honest to display. The cost is n times the generation, which is why this belongs on extracted fields rather than on prose.

// Extract one field n times at non-zero temperature and report
// agreement. Only defensible for values with a canonical form.
async function extractWithStability(prompt, n = 5) {
  const runs = await Promise.all(
    Array.from({ length: n }, () => extractOnce(prompt, { temperature: 0.7 }))
  );

  const counts = new Map();
  for (const value of runs.map(normalise)) {
    counts.set(value, (counts.get(value) ?? 0) + 1);
  }

  const [value, agreed] = [...counts].sort((a, b) => b[1] - a[1])[0];

  return {
    value,
    agreed,                     // e.g. 4
    of: n,                      // e.g. 5
    unanimous: agreed === n,
    // Display this, not a percentage. "4 of 5 runs agreed" is a
    // statement about what happened. "80% confident" is not.
    label: agreed + " of " + n + " runs agreed",
  };
}

Two honesty constraints on this. It measures the sampler, not the world: five agreeing runs of a model that misread the document agree on the wrong answer. And it must not be relabelled as a probability on the way to the screen — the moment “4 of 5” becomes “80%”, you have reintroduced the calibration claim you avoided.

The cost of getting this wrong

Confidence display is not a neutral addition, and this is the argument for restraint. Skitka, Mosier and Burdick’s 1999 work on automation bias established the pattern in automated decision aids: people under-verify when a system provides a confident recommendation, producing both errors of omission — missing a problem the automation did not flag — and errors of commission — following an incorrect prompt from the automation against other available evidence.

Applied here: a confidence indicator does not merely inform, it reallocates the user’s attention. A high score attached to a wrong answer is worse than no score, because it withdraws the scrutiny that would have caught it. That asymmetry is the argument for the provenance display above — “grounded in this span” invites a check, while a high percentage substitutes for one.

The design rule that follows: if a signal cannot cause the user to check something, it should not be on screen. Confidence that only reassures is decoration with a cost.

The asymmetry has a second consequence for where confidence display is worth building at all. Its value comes entirely from the low-confidence cases, since those are the ones that redirect attention. So a system that is confident nearly all the time gains almost nothing from the indicator and pays the automation-bias cost on every output. Before building one, look at the distribution of whatever signal you intend to display: if it is flat, you are adding a control surface with no variance, and the honest conclusion is that verification belongs somewhere else in the flow entirely.

One case does justify a numeric display, and it is worth naming so the rule above does not overreach. Where you have a real evaluation set, for a narrow constrained task — a classification into six labels, an extraction of one field — and you have measured accuracy at each logprob band on your own data, then a bucketed indicator is a calibrated claim you can defend. Note everything that took: a fixed task, a measured relationship, and buckets rather than a continuous score. That is the standard a percentage has to meet, and it is the reason almost nothing in a chat interface meets it.