Fabricated Citations and Fake References

A fabricated citation is the purest form of the failure: an output that is perfectly formatted, entirely checkable, and false. It is also the only hallucination with a fully mechanical detector, which makes shipping one an engineering choice rather than an accident.

Why references specifically

Look at what a reference is from the model’s side. It is a highly regular template — surname, initials, year, title-case phrase, journal name, volume, page range — filled with components whose joint distribution the model has learned extremely well and whose particular combination it may never have seen. Each piece is individually high-probability. Authors who publish in an area co-occur; titles in a field share vocabulary; the journal that publishes that vocabulary is predictable; the year follows from the topic’s vintage.

So the model assembles a reference the way it assembles any other fluent string, and the result is a plausible paper by plausible authors in a plausible venue. There is nothing in the generation process that distinguishes recalling a reference from constructing one. Both are the same operation.

Identifiers make it worse rather than better. A DOI or an arXiv id is a short, structured string with no redundancy a model could use to check itself — 10.1016/j. followed by a journal abbreviation and digits is a pattern, and completing a pattern is exactly the operation being performed. A fabricated identifier looks precisely as legitimate as a real one until something resolves it. The same argument covers fabricated API methods, config keys, CLI flags and package names, all of which are formats with strong internal regularity and long tails. The monofact argument on why models hallucinate predicts this concentration exactly: most references appear once or not at all in any corpus.

The category of incident

Since 2023 there has been a steady stream of court filings, in multiple jurisdictions, containing case citations that did not exist, produced by lawyers who used a chatbot for research and did not check. Judges have responded with sanctions, orders to show cause, and written opinions on the professional duty to verify. Legal-technology researchers now maintain running trackers of these filings, and the count has not stopped growing.

The details of individual cases are not the useful part and are easy to get wrong second-hand, so this page does not recite any. The structural lesson is what transfers: the failure survived multiple layers of professional review because a fabricated citation is indistinguishable from a real one by inspection. A reviewer who knows the field sees a case name that sounds like a case, reported in a reporter that exists, at a plausible page. Human review does not catch this class. Only resolution does.

The same shape appears in medicine, in academic drafting and in technical documentation, and it is why the mitigation below is worth building even where the stakes look lower.

What the evaluations found

The relevant published work is on attributed generation — systems that emit citations alongside claims — because that is the setting where you can measure whether the citation supports the sentence rather than merely whether it exists.

Liu, Zhang and Liang’s Evaluating Verifiability in Generative Search Engines (2023) ran human annotation across four commercial generative search systems and reported that only about half of generated sentences were fully supported by their citations, and that a substantial minority of citations did not support the sentence they were attached to — this in systems that had actually retrieved the sources they were citing. Gao et al.’s ALCE (2023) found the same shape on an open benchmark. The lesson generalises past search: a citation being present, and even being real, does not mean it supports the claim.

That splits verification into two independent checks, and a system that does only the first is a system that has learned to launder.

Resolve, then verify

Both checks are cheap, and the first is embarrassingly cheap:

import re, difflib, requests

DOI = re.compile(r"10\.\d{4,9}/[-._;()/:A-Za-z0-9]+")
ARXIV = re.compile(r"arXiv:\s*(\d{4}\.\d{4,5})", re.I)

def resolve_doi(doi):
    r = requests.get(f"https://api.crossref.org/works/{doi}", timeout=10)
    if r.status_code != 200:
        return None                      # STEP 1 FAILED: no such record
    m = r.json()["message"]
    return {"title": (m.get("title") or [""])[0],
            "year": m["issued"]["date-parts"][0][0],
            "authors": [a.get("family", "") for a in m.get("author", [])]}

def check_citation(cited_title, cited_year, cited_first_author, doi):
    rec = resolve_doi(doi)
    if rec is None:
        return "FABRICATED_IDENTIFIER"
    # STEP 2: the identifier resolves -- but to the same work?
    sim = difflib.SequenceMatcher(None, cited_title.lower(),
                                  rec["title"].lower()).ratio()
    if sim < 0.75:
        return "MISMATCHED_TITLE"        # real DOI, wrong paper
    if abs(int(cited_year) - int(rec["year"])) > 1:
        return "MISMATCHED_YEAR"
    if cited_first_author.lower() not in [a.lower() for a in rec["authors"]]:
        return "MISMATCHED_AUTHOR"
    return "RESOLVED"

# STEP 3, the one people skip: does the resolved abstract or the retrieved
# span actually entail the sentence the citation was attached to?
#   nli_entailment(premise=abstract_or_span, hypothesis=claim_sentence)

Crossref, arXiv, PubMed, OpenAlex and Semantic Scholar all expose free lookup APIs; for internal domains the authority is your own schema, your package registry or your documentation index. Rate limits are the only real constraint, and citations per response are few.

The mismatch categories are worth keeping separate in your telemetry. A fabricated identifier and a real identifier attached to the wrong claim are different bugs — the first is generation, the second is usually a retrieval-to-citation mapping error in your own code, and if you collapse them you will spend your effort on the wrong one.

The policy that removes the class

Verification is the fallback. The design that eliminates the failure is simpler and worth stating as a rule:

• A model may never author an identifier. Citations come from a retrieval step that returned real records; the model’s job is to select among the records it was handed and reference them by an opaque id you assigned. This makes fabrication structurally impossible rather than merely detectable — a chunk id it invents matches nothing.
• Render from the record, not from the model. The displayed reference string is formatted from the resolved metadata you hold. The model never produces user-visible bibliographic text, so it cannot produce a wrong one.
• Fail loudly. A claim whose citation does not resolve or does not entail is held back with the gap named. Silently stripping the citation leaves the claim standing with its evidence removed, which is worse than either alternative.
• Count them. Fabrication attempts per thousand responses is one of the few hallucination metrics that is cheap, objective and needs no human grader. Put it on the dashboard.