Error Messages When the Model Fails

6 min read · updated August 3, 2026

“Something went wrong. Please try again.” is correct for about a third of AI failures and actively harmful for the rest, because for the rest, trying again cannot possibly help and you have just told the user to spend money finding that out.

Everything that can go wrong

Errors arrive from at least four layers, and the user-facing consequences differ enough that collapsing them into one message destroys the only information you had.

Failure	Description
Transport	Connection dropped, DNS, TLS, the stream died mid-token. Retryable, usually transient, and the user did nothing wrong. This is the only class where 'try again' is straightforwardly true.
Rate limited (429)	Yours or the provider's capacity, not the request. Retryable but only after a wait, and the wait is often stated in a header. Telling the user to retry immediately guarantees a second 429.
Provider 5xx / overloaded	Retryable with backoff, and the single best case for automatic failover to another provider rather than for any message at all.
Timeout	Ambiguous by construction: the request may have completed on the provider's side and been billed. Retrying may duplicate a side effect, which is why idempotency matters more here than anywhere.
Context length exceeded	Deterministic. Retrying the identical request fails identically. The only fix is fewer tokens, and the interface knows that — so the message should offer the fix, not the retry.
Content filter	The provider blocked the input or the output. Not retryable unchanged. Distinct from a model refusal, and users experience the two very differently.
Truncated output	The generation hit max_tokens. Not an error at the transport layer at all — status 200, a finish reason of 'length', and an answer that stops mid-sentence. Silently the most common broken experience.
Malformed structured output	Valid HTTP, invalid JSON or a schema violation. Retryable and often succeeds on a second sample, because it is a sampling accident rather than a capability failure.
Empty output	Zero tokens, no error. Rare, jarring, and easy to render as a blank box that looks like a UI bug rather than a model outcome.
Confidently wrong	The failure with no error at all. Nothing in this page can detect it; it is why the rest of this cluster exists.

The ones you cannot tell apart

Three of those genuinely cannot be separated from the client side, and pretending otherwise produces confident, wrong error copy.

Timeout versus slow success. Your deadline expiring tells you nothing about the provider’s state. The request may be finishing right now. An idempotency key is what converts this from a guess into a safe retry.
Content filter versus model refusal. Some providers return a distinct code; some return a normal completion in which the model declines. The second one is a 200 with prose in it, and no error handling will ever see it. See refusal handling.
Truncation versus a short answer. Only the finish reason tells you, and it is the field most commonly ignored. If you read one field out of the response other than the content, read that one.

The honest design response to genuine ambiguity is not a vaguer message. It is a message that describes what you observed — “the response stopped early” — and offers both plausible next actions, rather than asserting a cause you do not know.

Four rules for the copy

Never say “try again” unless trying again can work. For context-length, content-filter and schema errors after several attempts, a retry is a bill with no upside. Offer the change that would help instead: shorten, rephrase, split.
Say who failed. “We couldn’t reach the model”, “the model declined” and “your request was too long” assign responsibility to three different parties, and the user’s next move is different in each case.
Never render the provider’s raw error. It names models and internal fields, its wording changes without notice, and it occasionally leaks fragments of the request. Map it; do not forward it.
Carry a correlation id. One short id, copyable, in the error UI. It is the difference between a support ticket that can be diagnosed and one that cannot, and it costs a line of code.

Normalising the error

One function, at the boundary, converting whatever the provider said into the small set of things the interface actually needs to decide. Everything downstream reads this shape and never the original.

type AiFailure = {
  kind: "transport" | "rate_limited" | "provider" | "timeout"
      | "too_long" | "filtered" | "truncated" | "malformed" | "empty";
  retryable: boolean;      // can an identical request succeed?
  retryAfterMs?: number;   // if the provider told us
  userMessage: string;     // what we show
  action?: "shorten" | "rephrase" | "wait" | "regenerate";
  correlationId: string;
};

const COPY: Record<AiFailure["kind"], Pick<AiFailure,
  "retryable" | "userMessage" | "action">> = {
  transport:    { retryable: true,  action: "regenerate",
    userMessage: "The connection dropped before the answer finished." },
  rate_limited: { retryable: true,  action: "wait",
    userMessage: "We're at capacity right now." },
  provider:     { retryable: true,  action: "regenerate",
    userMessage: "The model provider is having trouble." },
  timeout:      { retryable: true,  action: "regenerate",
    userMessage: "That took longer than we allow. It may still finish." },
  too_long:     { retryable: false, action: "shorten",
    userMessage: "This conversation is too long to send. Start a new one, or remove an attachment." },
  filtered:     { retryable: false, action: "rephrase",
    userMessage: "This request was blocked by a safety filter." },
  truncated:    { retryable: true,  action: "regenerate",
    userMessage: "The answer was cut off at the length limit." },
  malformed:    { retryable: true,  action: "regenerate",
    userMessage: "The model returned something we couldn't read." },
  empty:        { retryable: true,  action: "regenerate",
    userMessage: "The model returned nothing at all." },
};

Two things fall out of writing it this way. The action field drives which button is rendered, so the copy and the affordance can never disagree. And retryable is a single place to audit — every automatic retry in the system reads one boolean, rather than re-deriving retryability from a status code at four different call sites. See normalising errors across providers and which retries are safe to make automatically.

Never lose the input

The most damaging thing an AI interface does on failure is discard what the user typed. It is a small bug and it is the one that ends the session, because the cost of the failure is no longer a wasted wait, it is re-doing work.

Concretely: keep the composer populated after a failed send; keep attachments attached; retain partial streamed output on a mid-stream error and mark it as incomplete rather than removing it; and put the retry control next to the failed message rather than making the user re-submit from the composer. A partial answer plus “this stopped early” is worth more than a clean error and an empty screen, because the partial answer may already contain what they needed.

The mid-stream case is the one that gets implemented badly, because it falls between two owners. The request succeeded — headers returned, status 200, tokens arrived — and then the connection died at token 180 of 400. Code that models a request as either a success or an error has no state for this, so it usually picks error and discards the 180 tokens. The correct model has three outcomes rather than two: completed, failed before any output, and failed after partial output. Only the second is an error in the ordinary sense; the third is a result with a caveat, and it needs its own rendering.

There is a related decision about automatic retries that belongs to this page rather than to the infrastructure. Retrying transparently is right for transport blips, where the user should never learn that anything happened. It is wrong once the retry is slow enough to be felt: a silent retry that adds twenty seconds looks like a hang, and the user cancels a request that was about to succeed. The threshold is roughly the point at which the user would otherwise have been shown a result — past that, tell them what is going on. “The first attempt failed, trying again” costs one line and converts an inexplicable delay into a system that is visibly working.

One last thing that is easy to get backwards: an error message is not the place to apologise at length. Users reading an error want to know what happened, whether their work survived, and what to do next, in that order. A paragraph of regret delays all three, and a product’s error copy is one of the few places where brevity is unambiguously kinder.