Vision Model Hallucinations: Seeing Things That Aren't There

A vision model does not decline to answer when the image is uninformative. It completes. The result is a description that is fluent, specific, appropriately hedged, and about an image that does not exist — and the failure has enough distinct mechanisms that treating it as one phenomenon guarantees you fix the wrong thing.

Why the pipeline produces them

Two forces, both structural. The image arrives as a bounded number of patch vectors from an encoder trained on a caption-matching objective — a gist, not a record. And the language half is a next-token predictor with strong priors about what tends to appear where. When the visual evidence is weak, the prior wins, and the prior is very confident that a kitchen contains a refrigerator.

This is why the classic hallucinated object is never bizarre. Nobody reports a giraffe in an office photograph. They report a chair, a mug, a person in the background — the statistically obvious co-occurrence. That signature is a useful diagnostic: if the wrong answers are all plausible, you are looking at prior dominance rather than a bug.

Six kinds, by mechanism

The last row deserves emphasis because it is the one you cause. A leading question is an instruction to find something, and a model that is trained to be helpful will find it. “Is there a car in this image? If so, what colour?” is a materially different prompt from “what colour is the car?”, and the difference costs nothing.

How the literature measures it

Two names are worth knowing, because they tell you what a paper means when it claims a reduction.

• CHAIR (Rohrbach et al., 2018, “Object Hallucination in Image Captioning”) — compares the objects mentioned in a generated caption against the ground-truth object annotations for that image, and reports the fraction that are not there. It measures generation, and it depends on a fixed object vocabulary.
• POPE (Li et al., 2023) — reframes the same question as balanced yes/no polling: “Is there a {object} in the image?”, with negatives sampled in three ways — at random, from objects that are popular in the dataset, and from objects that frequently co-occur with what is actually present. That third sampling strategy is the clever part, and models generally do worst on it, which is direct evidence for the co-occurrence-prior mechanism described above.

No scores are quoted here, because the useful ones are per-model and per-version and would be stale before you read this. Go to the current model card. What generalises is the method: ask about absent objects chosen adversarially from co-occurrence statistics, not at random, or your evaluation will be far too easy.

Detecting them in your own system

Ranked by how much they buy relative to what they cost:

• Give the model somewhere to put uncertainty. A required not_visible enum in the schema, or an explicit instruction that “not determinable from this image” is a correct and acceptable answer. Free, and it moves a surprising number of confident errors into honest abstentions.
• Ask for evidence with the claim. Require a short description of where in the image each claim comes from. Not because the location will be accurate, but because a claim with no describable location tends to be one that was invented.
• Self-consistency. Sample the same question twice at a non-zero temperature and compare. Genuine observations are stable; hallucinated ones tend to vary between draws.
• Negative controls in your eval set. Include images that lack the thing you always ask about. If your set only contains positives, a model that answers “yes” unconditionally scores perfectly.
• Cross-check against a specialist. For OCR invention, an OCR engine is ground truth enough. For object presence, an open-vocabulary detector is.

What reduces them

The largest single lever is resolution, and it is unglamorous. A large share of what gets reported as hallucination is the model being asked to describe detail that was destroyed by downscaling before it ever reached the encoder. Crop to the region of interest and send it at native size; a number of “the model made that up” reports resolve into “the model was never shown it”.

After that: ask non-leading questions, keep the requested output narrow — a long free-form description has far more surface area to invent on than a five-field schema — and separate perception from reasoning into two turns, so that the list of what is present can be checked before anything is concluded from it. None of these eliminate the failure. They move it from silent to visible, which is the achievable goal.

A last note on where in your system the mitigation belongs. It is tempting to attack this in the prompt, because the prompt is the thing you can change fastest, and prompt changes do help — a non-leading question and an abstention option are genuinely worth a lot. But the durable fixes are architectural: a resolution policy that crops rather than shrinks, a schema narrow enough that there is little to invent, a verification pass against a specialist tool for the fields that matter, and a human in the loop wherever a wrong answer is expensive. Prompt fixes decay when a model version changes. The architecture does not.