Production LLM Error Classes, and How to Categorise Your Own

9 min read · updated August 4, 2026

Somebody else’s error rates will not transfer to your traffic: the mix depends on your model, your routes, your prompt length and your users. What does transfer is the taxonomy, because every LLM application’s failures fall into the same eight classes, and three of those classes return HTTP 200 and appear nowhere in an ordinary error dashboard.

Why a taxonomy first and counts second

The instinct is to count first: pull a week of logs, group by status code, sort descending. That produces a table dominated by 429 and 500 and hides the failures that cost the most, because the expensive failures in an LLM system are usually successful HTTP requests that returned the wrong thing.

A useful classification has to separate three things that status codes conflate: whose fault it is, whether retrying can possibly help, and whether the user experienced a failure at all. The eight classes below are cut along exactly those lines.

The eight classes

Class	Description
1 · Transport	Connection resets, DNS failures, TLS errors, socket hang up, read timeouts. Nothing about the model. Retry is almost always correct and almost always cheap, because no tokens were generated.
2 · Capacity	429 and provider-side 503. The request is well-formed and the provider is refusing right now. Retry with backoff, or fail over. The distinguishing question is whether a retry-after header is present.
3 · Request rejected	400-class errors caused by what was sent: an unsupported parameter, a bad tool schema, an image the model cannot accept. Deterministic — retrying the identical body returns the identical error. These are code bugs wearing a network error's clothes.
4 · Context limit	context_length_exceeded and its variants. Technically class 3, but broken out because the cause is data-dependent rather than code-dependent: the same code path succeeds on most inputs and fails on the long tail.
5 · Commercial	The provider account has no credit, the key is revoked, the organisation is over a hard limit. Never a statement about the caller's request, and the single most commonly misclassified class — see the next section.
6 · Content policy	The provider refused on safety grounds, or returned an empty completion with a filter reason. Retrying the same prompt is futile; retrying a rephrased prompt is a product decision, not a client library decision.
7 · Malformed output	HTTP 200. The model answered and the answer does not parse, does not validate against the schema, or was truncated mid-structure. Invisible to transport-level monitoring.
8 · Wrong output	HTTP 200 and well-formed. The answer is confidently incorrect, cites a source that does not exist, or ignores an instruction. Only detectable by evaluation or by user feedback, never by a status code.

Why HTTP status alone misclassifies

Class 5 is the one that breaks naive classifiers, and it breaks them in a way that costs money in both directions. Providers do not agree on which status code means “your account has no credit”. Some return 400, which a status-based classifier treats as permanent and caller-caused; some return 429, which the same classifier treats as transient and retries with backoff.

Both readings are wrong, and each is wrong differently. Treating it as 400 means the request is returned to the caller as though their own input were malformed — and any healthy alternative routes are never tried. Treating it as 429 means an account in a failed billing state is called three more times with backoff, tripling the request rate against a provider that will refuse every one of them.

The correct classification requires reading the response body and matching on the provider’s own wording about its billing. That matching must be narrow: a phrase a provider writes about its own balance, never a phrase that could describe a bad request. A false positive here converts a genuine class-3 error into a failover and hides the one error the caller could have acted on. Test both directions, and treat the negative direction as the more important one.

The general form of the problem — that provider error shapes do not agree and have to be normalised before anything can reason about them — is covered in normalising provider API errors.

The classes that produce a 200

Classes 7 and 8 are the ones that make LLM error accounting different from ordinary API error accounting, and there is a third silent case that catches people out.

Malformed output (7) is measurable without a human. Parse rate and schema-validation rate are two numbers you can emit on every request. If you are not emitting them, your dashboard is reporting a health it cannot see.
Wrong output (8) is not measurable without either an evaluation set or a user signal. Anyone quoting a hallucination rate without naming which of the two they used is quoting nothing.
Failure hidden by failover. If your stack fails over between providers, a class-2 or class-5 failure produces a 200 for the user and a log row attributed to whichever provider eventually answered. The failure is invisible because the failover worked, and the first symptom is a larger bill, because the fallback route is usually the dearer one. Record the refusal at the moment it is classified, not at the end of the request.

Instrumenting so the counts mean something

To produce a table that is worth acting on, every call needs five fields recorded regardless of outcome. Four of them are free; the fifth is the one usually missing.

Per attempt (not per user-facing request):
  provider        which upstream was actually called
  route           model id as sent, after any aliasing
  class           1..8, decided by the classifier below
  attempt_index   0 for the first try, 1+ for retries and failovers
  outcome         served | retried | failed_over | returned_to_caller

Per user-facing request, additionally:
  attempts        how many upstream calls it took
  parsed          did the response parse / validate  (classes 7)
  cost_micros     what the request cost, including the attempts that failed

CLASSIFIER — order matters, first match wins
  no HTTP response at all                      -> 1 transport
  429 or 503 with retry-after                  -> 2 capacity
  body matches provider billing phrases        -> 5 commercial
  400-class and context length in the message  -> 4 context limit
  400-class otherwise                          -> 3 request rejected
  200 with a refusal or filter reason          -> 6 content policy
  200 and parse/schema failed                  -> 7 malformed
  200 and valid                                -> 8 only if an eval or a user
                                                  signal says so; otherwise OK

Recording attempt_index is what separates a useful table from a misleading one. Without it, a request that failed over twice and succeeded is one successful row, and the two upstream failures that made it expensive are gone.

Reading the resulting table

Once you have a week of counts by class, three ratios say more than the absolute numbers.

Attempts per served request. Anything meaningfully above 1.0 means you are paying for failures. Break it down by provider before touching anything else.
Class 3 as a share of all errors. Class 3 is a code bug. If it is not near zero, you are shipping requests that could never have succeeded, and the fix is a schema check before the call, not a retry after it.
Class 7 rate against your parse rate target. This is the number that responds fastest to work — constrained decoding, a repair pass, or a stricter schema — and it is usually the cheapest large improvement available.

Deliberately absent from this page: any figure for what these rates typically are. Published rates vary by an order of magnitude across models, prompts and traffic mixes, and a number quoted without that context would be used as a benchmark and would mislead. Measure your own; the classifier above is the part that transfers.

Turning classes into a budget you can act on

Counts are only useful once each class has an owner and a target, and the eight classes do not belong to one team. Assigning them is what stops an error review becoming a monthly recitation of numbers.

Classes	Description
1, 2 — transport and capacity	Owned by whoever owns routing and retries. Target: the user never sees these, because failover and backoff absorb them. The metric that matters is not the rate but attempts per served request, since absorbing them costs money.
3, 4 — rejected and context limit	Owned by the application team. Target: near zero for class 3, because it is a bug, and a known, bounded rate for class 4, because it is a data property. Class 4's fix is truncation or chunking policy, not a retry.
5 — commercial	Owned by whoever owns provider accounts. Target: zero, and any non-zero count is an alert rather than a statistic, because it means an account is in a failed state right now.
6 — content policy	Owned by product. Not a defect and not a target: a rate to be understood, and a signal about what users are asking for that the feature refuses.
7 — malformed output	Owned by the application team, and the class with the best return on effort. Set a parse-rate target and treat it as a service objective.
8 — wrong output	Owned by whoever owns quality. Cannot be measured from logs at all; it needs an evaluation set or a user signal, and pretending otherwise is how it goes unowned.

Two failure modes in how these tables get used. Reviewing the total error rate hides everything, because classes 1 and 2 usually dominate the count and are the least interesting. And a target on the wrong class produces the wrong behaviour — a team measured on total errors will suppress class 3 by retrying it, which converts a visible bug into an invisible bill.

The related service-level thinking, and which of these belong in an objective at all, is in service levels for AI features.