Spotting Overclaiming in AI Papers

Overclaiming is rarely dishonesty. It is usually a true result described in language that outruns it, produced by ordinary incentives. Which is good news for a reader: the signals are structural, they are visible in the abstract and the tables, and none of them require expertise in the subject. Every example sentence below is invented to illustrate the pattern.

Flags in the claim

1. A capability word doing undefined work

“Our method enables genuine reasoning in language models.” Words like reasoning, understanding, planning and knowledge have no agreed operational definition, so a claim built on one cannot be checked. Ask: what was measured, and would the same sentence be defensible with the benchmark name substituted for the capability word? If “improves accuracy on a multi-step arithmetic set” sounds much weaker, the gap is the overclaim.

2. Scope silently widened between results and abstract

The experiments run on one language, one model family and two datasets; the abstract says “for large language models”. This is the single most common form of overclaiming and the easiest to catch: read the abstract, then read the experimental setup, then reread the abstract with the setup in mind.

3. A mechanism asserted rather than shown

“This works because the model learns a hierarchical representation of the task.” Perhaps. Was that measured, or is it the authors’ account of a result they observed? Both are legitimate; only one is evidence, and the sentence usually does not say which it is.

Flags in the evidence

4. No ablation, or an ablation of only the safe parts

A method with five components and a table varying two of them leaves you unable to attribute the gain. If the component the paper is named after is not the one ablated, ask why. This is the subject of reading ablations first.

5. A baseline that is old, untuned, or asymmetrically tuned

Look for the sentence describing the hyperparameter search. If the proposed method got a sweep and the baseline got defaults, the comparison measures effort. If the baseline is a system from three years earlier while current alternatives exist and go unmentioned, the same applies.

6. Single runs presented as differences

“Our method achieves 71.2 versus the baseline’s 69.8.” With no seeds, no variance and no interval, a 1.4-point gap is a number, not a finding. Absence of variance reporting does not mean the result is wrong — it means nobody, including the authors, knows whether it is inside the noise.

7. Evaluation on data the model may have seen

Any benchmark that has been on the public web for years is a candidate for contamination, and a paper claiming a large gain on such a set without addressing it has skipped the first question a reader should ask. See how contamination is detected.

8. Cherry-picked qualitative examples

Hand-selected outputs demonstrating that a method can do something are evidence of possibility and nothing else. The useful version is a random sample with the failures included, and papers that do this are visibly more trustworthy for it.

Flags in the presentation

9. Axes and highlights doing persuasion

Truncated y-axes that turn a two-point difference into a cliff. Bolded entries in a table that are not the best in their column. Log axes chosen so that curves converge visually. None of these are falsehoods and all of them are the figure arguing rather than reporting — which is why reading the tables before the prose pays.

10. A press release and a paper that disagree

When an accompanying blog post makes a stronger claim than the paper, it is usually the paper that is accurate. The gap between the two is the clearest available signal about how the result should be read, and it costs thirty seconds to check.

Things that are not red flags

A checklist without this section produces readers who dismiss everything, which is not scepticism but a different way of not engaging with evidence.

• A small improvement. If it is properly measured with variance reported, a one-point gain that holds is a real result. Field progress is mostly small gains that survive.
• A simple method. Simplicity is a virtue in a method and a disadvantage in a paper, which is a problem with reviewing rather than with the work.
• Industry authorship. Compute access is where the experiments can be run. Judge the method and the reporting, not the affiliation.
• A negative or null result. Rarer than it should be and more informative than most positive ones.
• A long limitations section. This is the opposite of a red flag. A paper enumerating what it did not establish is doing the reader’s work for them.
• Not being peer reviewed. A weak signal, in a field where the preprint is the norm. Judge the artefact and the evidence instead — what review does and does not certify.

A worked misreading

Constructed, but assembled from patterns that recur. Suppose a paper reports a prompting technique with the abstract line: “our approach improves reasoning accuracy by 15%, demonstrating that language models can plan when properly prompted.” A reader passes this on as “new prompting method makes models 15% better at planning”.

What the tables would need to say for that summary to hold, and what they typically say instead:

• 15% of what. A rise from 40 to 46 is fifteen percent relative and six points absolute. Relative improvements over a low base are the most common source of an impressive-sounding number, and both framings are honest — only one is usually chosen.
• On which set. If it is one benchmark of synthetic multi-step puzzles, “reasoning accuracy” is a generous label for a specific score.
• On which models. If the gain appears on one model and is neutral on two others, the abstract’s plural is doing unearned work — and the reader has no way to know without the table.
• Against what. If the baseline is naive prompting rather than the strongest current prompting technique, the improvement is over a straw comparison.
• “Demonstrating that models can plan”. Higher scores on a planning-flavoured benchmark are consistent with planning and consistent with several duller explanations. The word demonstrating is carrying an inference the results cannot bear.

Underneath all five, the paper may well contain a genuine, useful, modest result. That is the usual situation, and it is why the goal of this checklist is recalibration rather than rejection: figure out what was actually shown, then decide whether it changes anything you do.