Multi-Turn Conversation Design

A conversation is not a sequence of independent calls. It is one call that gets longer, more expensive and more contaminated with every turn, and almost every multi-turn design problem is a consequence of that single implementation fact.

What multi-turn actually changes

The model is stateless. A conversation exists because your product resends the entire history on every turn, so three things move together as the conversation grows: the cost per turn, the latency per turn ( prefill scales with prompt length), and the probability that something in the context is now working against you.

Note the shape of that. In an ordinary product, the tenth interaction is no more expensive than the first. Here it is several times the first, for output the user often values less, and the growth is invisible in the interface unless you put it there. It is also why caching the stable prefix matters so much more in chat than anywhere else.

Mistakes persist

This is the consequence that surprises people. In turn three the model asserts something wrong. In turn nine, that assertion is still sitting in the context, and everything since has been generated conditioned on it. The model is not lying repeatedly; it is being consistent with a document that contains an error — context poisoning.

Now consider what the interface offers the user when they notice. In nearly every chat product, the only affordance is to type “no, that’s wrong, it’s actually X”. That appends a correction, which means the context now contains both the error and the correction, and the model must weigh them. It often does, and it often reverts a few turns later — which users experience as the model “forgetting” the correction, and which is really the original error still being in the room. See why long conversations degrade.

How repair works in human conversation

Conversation analysis has studied this precise problem since long before it was a software concern. Schegloff, Jefferson and Sacks’ 1977 paper The Preference for Self-Correction in the Organization of Repair in Conversation is the standard reference, and two of its findings are directly applicable.

• Repair is organised and preferred to be self-initiated and self-completed — the speaker of the trouble source is overwhelmingly the one who fixes it, and other-correction is comparatively rare and marked.
• Repair happens close to the trouble source — within the same turn or in the immediately following turns, rather than being deferred. Conversation has structural positions where repair belongs.

Both are the opposite of what chat interfaces support. The user is forced into other-correction (correcting the model), and it is forced to happen far from the trouble source (at the end of the transcript, many turns later). The interface has removed the two properties that make repair work in the medium it is imitating.

Editable history is the repair mechanism

Editing an earlier turn and re-running from there is the interface equivalent of repair at the trouble source. It is a small feature and it changes the character of the product, because it is the only affordance that removes the bad text rather than arguing with it.

Conversation as a tree, not a list:

  turn 1  user     "summarise this contract"
  turn 2  model    ...
  turn 3  user     "what's the notice period"
  turn 4  model    "90 days"            <- wrong
  turn 5  user     "no, it's 30"
  turn 6  model    ...conditioned on BOTH 4 and 5

vs. editing turn 3:

  turn 1  user     "summarise this contract"
  turn 2  model    ...
  turn 3' user     "what's the notice period in clause 14"
  turn 4' model    "30 days"
                   ^ the wrong answer is not in context at all

Branch semantics:
  - editing turn n discards turns > n from the active path
  - discarded branches are kept and reachable, never deleted
  - the UI shows a branch switcher at the edited turn

The rule that everything below an edit is discarded is not a UI convention; it is forced. Those turns were generated conditioned on text that no longer exists on this path, so keeping them would construct a history that never happened. Keeping the branch reachable is a separate decision and the right one, for the same reason regenerate should not destroy the previous sample — see recovery affordances.

Making the context budget visible

Eventually the history exceeds what you will send, and something drops it, truncates it or summarises it. From the user’s side this is inexplicable: the model that knew their name in turn four does not in turn forty, and nothing said why.

• Say when history is dropped. A quiet marker in the transcript — “earlier messages are no longer being sent” — placed where the boundary is, converts an inexplicable failure into an understood limit and suggests the fix.
• Say when history is summarised. Compaction is lossy in a way the user cannot see, and it is worse than truncation in one respect: the summary is fluent, so the model behaves as if it has the detail. Let the user open the summary.
• Make starting fresh obvious. The best fix for a degraded conversation is a new one, and users resist it because they think they will lose the context. Offer to carry forward an explicit, visible, editable brief instead of the whole transcript.
• Do not show a token counter. It is precise, meaningless to almost everyone, and it invites optimisation of the wrong thing. A three-state indicator — fine, getting long, dropping history — carries the whole of the actionable information.

Underneath all four is a single principle worth stating on its own: in a multi-turn interface, the context is part of the document, and the user should be able to see and change it. It is the input to every subsequent generation, it accumulates errors, it grows until something silently discards part of it, and in most products it is completely invisible. No other interface would hide the document from the person editing it.

Products that take that seriously end up looking less like a messaging app and more like an editor with a transcript attached — pinned facts that always travel, a visible working set, individual messages that can be excluded from the context without being deleted from the record. That is a larger design commitment than adding an edit button, and it is where multi-turn interfaces are likely to end up, because every problem on this page is a consequence of the transcript being both the history and the input while only being presented as the history.