Designing for Probabilistic Output

Conventional software has two states: it worked, or it told you it did not. A feature built on a language model has a third, and the third one is the entire discipline — it produced something, the something is wrong, and nothing anywhere in the system knows that.

The shift

The contract of a deterministic component is that the same input produces the same output, and that failure is signalled. Both halves are load-bearing for interface design. Because output is repeatable, you can show it as a fact. Because failure is signalled, you can put an error state next to the success state and be confident that between them they cover everything.

A language model honours neither half. It returns a probability distribution over next tokens and something samples from it, so two identical requests diverge by construction — see the runtime definition of the thing. And a wrong answer is not an exception path: it is a high-likelihood continuation that happens not to be true, delivered with exactly the same fluency, structure and HTTP status as a right one.

Everything else in this cluster is downstream of that. If you take only one thing from it: the model has no error state for being wrong, so the interface has to supply one, and the only thing that can supply it is a person looking at the output.

Five properties, five consequences

It is worth being specific about which property forces which decision, because a recommendation you cannot trace back to one of these is usually just taste.

The verification gap

There is one inequality that decides whether an AI feature helps at all, and it has nothing to do with model quality in the abstract. A feature is worth using when the cost of getting the output and checking it is less than the cost of doing the work yourself:

generate_time + verify_time  <  do_it_yourself_time

and, separately, the risk term:

  p_wrong * cost_of_an_undetected_error  <  value_of_the_time_saved

Both lines have to hold. The first one is why a feature that produces a plausible 800-word document can be a net loss: reading it carefully enough to trust it takes longer than writing 300 words yourself would have. The second is why the same model that is a triumph in a code editor — where the compiler and the test suite verify for free — is a liability generating a customer refund amount, where the verification is a human reading a number that looks right.

Design work is mostly an attack on verify_time. Citations that anchor to a span, diffs instead of rewritten documents, structured output that renders as a form rather than a paragraph, and confidence disclosure that says which parts are grounded — all of these exist to make checking cheaper, not to make the model better.

The second line deserves its own attention, because the term that dominates it is not p_wrong. Teams argue about model quality, which moves that probability by a few percentage points at considerable expense, while cost_of_an_undetected_error varies across features by orders of magnitude and is set entirely by where you put the output. The same model, at the same error rate, is a good idea suggesting a tag and a bad idea setting a price. That is a placement decision, it is free, and it moves the larger term.

It also explains a pattern worth recognising in your own roadmap: an AI feature that has been “nearly ready” for two quarters, waiting for the model to improve, is usually a feature on the wrong side of the second inequality rather than the first. No model release fixes it. Moving the output into a reviewable position does.

Reversibility is the design variable

When you cannot prevent a wrong output, the next best thing is to make a wrong output cheap. This is why reversibility does more work in AI interfaces than in any other kind, and why it is worth classifying every AI-driven action by it before designing anything:

• Reversible and private — a draft in a text box, a suggested rewrite, a proposed filter. The correct amount of confirmation friction here is zero. Ship it optimistically and put an undo next to it.
• Reversible but visible — a status change other people can see, a re-ordered board. Undo still works, but the interface has to make the undo discoverable within seconds, because after that the damage is social rather than technical.
• Irreversible — an email sent, a payment made, a row deleted, a message posted. No amount of model quality justifies running these without an explicit human confirmation that shows the actual effect. The whole of approval design lives in this row.

Note what this classification is not: it is not a judgement about how good the model is at the task. A very good model doing an irreversible thing still needs the confirmation, because the tail is what you are designing for, and the tail exists at every quality level.

Reclassifying a feature you already shipped

Most teams arrive at this cluster with a feature already live. Four questions, in order, and each one has a concrete fix attached:

• Where does output land? If it lands anywhere that is not editable before it takes effect, that is the bug. Move it into a draft state.
• What does the user do to check it? If the answer is “read it and hope”, your verification cost is unbounded and the feature is on the wrong side of the inequality above for at least some of your users.
• What happens at p95 latency? Not the median. Open the feature, imagine forty seconds of nothing, and look at what is on screen. See the forty-second problem.
• What does a second attempt cost? In money, and in user effort. If regenerating is one click and re-entering the context is twenty, users will do the expensive one because it is the one they can see.