Skip to content
Multigrid

Hallucination & failure modes

Why a language model states false things fluently, which distinct failures the word hides, and how to detect each one in your own system rather than arguing about it.

“Hallucination” is doing too much work. It gets applied to a fabricated legal citation, a summary that contradicts the document it was given, a model that agrees with your wrong premise, a truncated JSON object and an answer that was correct in 2023. Those have different causes and different fixes, and calling them one thing is why so much advice about them is useless.

These pages start from the mechanism. A model returns a probability distribution over the next token; nothing in that distribution is a truth value, the training objective never asked for one, and the way models are scored afterwards rewards a confident guess over an admission of ignorance. Everything downstream — calibration, grounding, abstention, detection — is an attempt to add the channel that the architecture does not have.

Where a page needs a number, it names the paper the number came from. Where a page needs a rate for your system, it gives you the code to produce it, because nobody else’s rate transfers to your prompts.

Why LLMs Hallucinate: The Mechanical Explanation

Next-token prediction has no truth term, pretraining rewards fluent continuation, and post-training grading rewards guessing over abstention.

5 min read

Types of Hallucination: A Taxonomy That Routes to a Fix

Six distinct failures get called hallucination, they have different causes, and only one of them is fixed by better retrieval.

4 min read

Measuring Hallucination Rate in Your Own App

A claim-level grading rubric, a labelling protocol with an agreement check, and the code to turn both into a rate with a confidence interval.

5 min read

Grounding: The Only Reliable Hallucination Fix

Grounding is four architectural commitments — provided evidence, span attribution, a verifier and a refusal path — not an instruction in the system prompt.

5 min read

Confidence and Calibration: Does the Model Know It's Wrong?

What calibration means precisely, what the published reliability diagrams show about pretraining versus RLHF, and the code to produce one from your own logprobs.

6 min read

Making a Model Abstain Instead of Guessing

Why abstention has to be paid for in the scoring rule before any prompt can produce it, and the risk–coverage harness that tells you whether yours works.

6 min read

Fabricated Citations and Fake References

Why references are the format models fabricate most fluently, what the published evaluations of cited generation found, and a resolver that makes the failure impossible to ship.

5 min read

Sycophancy: Why the Model Agrees With Your Mistake

Where agreement-seeking comes from in the preference data, four probes that expose it in any model, and the prompt structures that reduce it.

5 min read

Prompt Sensitivity: Same Question, Different Answer

Semantically identical prompts produce materially different accuracy, the published spreads are large, and a single-prompt number is a sample of one.

5 min read

Non-Determinism: Why Temperature 0 Isn't Deterministic

Greedy decoding is deterministic given identical logits, and on a shared GPU server the logits are not identical — mostly because kernels are not batch-invariant.

5 min read

Refusals: When Safety Training Blocks Legitimate Work

Exaggerated safety is a documented, benchmarked failure mode with a known trigger taxonomy — and a three-way grading scheme you can run on your own prompts.

5 min read

Context Poisoning and Bad Retrieved Data

Provided context outranks parametric knowledge by design, which makes the retrieval corpus an attack surface and a quality ceiling at the same time.

5 min read

Numerical Reasoning Failures

Why arithmetic breaks in a way that scales badly with digit count, what the benchmarks show about robustness, and why the fix is a tool call rather than a better prompt.

5 min read

Date and Time Reasoning Bugs

A model has no clock, inherits every numeric weakness, and knows nothing about your timezone — which makes scheduling features a reliable source of silent errors.

5 min read

Long-Conversation Degradation

Instruction adherence falls as a conversation grows, for reasons involving position effects, self-conditioning and early commitment — and there is a plot that shows it.

5 min read

Language Drift in Multilingual Conversations

Why a model answering in Dutch slips into English mid-paragraph, what one model vendor documented about fixing it, and how to pin the output language.

5 min read

Repetition Loops and Degenerate Output

Repetition is self-reinforcing — the probability of a repeated phrase rises with each repetition — which is why decoding settings both cause it and cure it.

5 min read

Truncated Output: Diagnosing an Unfinished Answer

Every stop reason decoded, the specific cause behind each, and the streaming case where the response ends with no stop reason at all.

5 min read

Model Degradation Over Time: Real or Perceived?

The study that started this argument, the critique that complicated it, the six things that genuinely change under you, and the regression harness that answers it for your workload.

4 min read

A Field Guide to LLM API Error Messages

What each status code and error type actually means underneath, which ones are safe to retry, and the failures that arrive with a 200.

5 min read

Other topics