Research With AI Without Inheriting Its Mistakes

The workflow below has one rule and everything else follows from it: you find the sources, and the model is only ever allowed to point at text that is already in front of it. Every claim it returns comes with a quoted span, and a twenty-line script checks that each span really appears in the source before you read a word of it.

One rule: the model never supplies a fact

Stated precisely: the model may summarise, extract, classify, restructure and compare text you have supplied in the context. It may not answer a question from its weights. Anything it produces that cannot be traced to a span in a document you provided is treated as absent, not as a lead.

This is stricter than “check its work”, and the difference is the entire point. Checking output is a task you will skip on the eleventh claim of a busy afternoon. A workflow where an unsupported claim is structurally impossible to produce does not depend on your attention.

Why asking it directly cannot be made safe

A model returns the most probable continuation of your prompt. When the fact you asked for appeared thousands of times in training, the most probable continuation is the fact. When it appeared twice, or never, the most probable continuation is a well-formed sentence of the right shape containing a plausible value. Nothing in the output distinguishes the two cases, because fluency is produced by the same machinery in both.

The categories that break are exactly the ones research depends on. Proper nouns, dates, numbers, statute sections and citations are low-probability strings sitting inside high-probability sentence frames, so the frame is reproduced accurately and the slot is filled approximately. That is why a fabricated citation looks so convincing: the author is real, the journal is real, the title is the sort of title that author would write.

Prompting does not fix this. Instructing a model to say “I do not know” helps at the margin and cannot be relied on, because the model has no separate representation of what it knows to consult — abstention is a behaviour it produces, not a state it reports. Grounding is not a prompting technique. It is an architecture: put the text in the context and require the output to point at it.

The workflow

1. Frame the question as a list of specific things you need to establish. Not “research the EU AI Act” but “which obligations apply to a deployer rather than a provider, and from what date”. This is the one step the model is allowed to help with, because a question is not a fact.
2. Find sources yourself. Search engines, the primary document, the library. If you ask a model where to look, treat its suggestions as search terms to try rather than as references — open every one and confirm it exists before it enters the pile.
3. Get the full text into files. One file per source, plain text, with a stable identifier in the filename. PDFs need extraction first; check the extraction did not mangle tables or drop footnotes, because that is where the numbers live.
4. Extract with the span requirement. Run the prompt below over one source at a time. One source per call: batching several into one context is how attributions get swapped between documents.
5. Verify the spans mechanically. The script below fails any claim whose quoted evidence is not a literal substring of the source. Discard failures rather than repairing them.
6. Read the surviving evidence yourself, in context. A true quotation can still be misused. The script proves the words are there; only you can tell whether the sentence before it said “critics have wrongly claimed that”.
7. Write from your notes, not from the model output.

The extraction prompt

The load-bearing instruction is that evidence must be copied character for character. Everything else in the prompt exists to make that instruction enforceable.

You are extracting evidence from a single source document. You may use
only the document below. You have no other knowledge.

For each question, return zero or more findings. A finding is only valid
if the document contains a passage that supports it.

Return JSON matching this shape and nothing else:

{
  "findings": [
    {
      "question_id": "q1",
      "claim": "one sentence, in your own words",
      "evidence": "a verbatim passage copied character-for-character from
                   the document, between 10 and 60 words",
      "confidence": "explicit" | "implied"
    }
  ],
  "unanswered": ["q2", "q3"]
}

Rules:
- evidence must be an exact substring of the document. Do not correct
  spelling, expand abbreviations, normalise quotation marks, or join
  passages that are not adjacent.
- If the document does not address a question, put its id in
  "unanswered". Do not answer it from memory.
- "implied" means the document supports the claim without stating it.
  Use it sparingly and never for numbers, dates or names.

QUESTIONS
q1: ...
q2: ...

DOCUMENT
<<<
...full text...
>>>

Two details earn their place. Forbidding normalisation matters because a model that tidies a curly quotation mark into a straight one breaks the substring check for a reason that has nothing to do with truthfulness, and you want your failures to be meaningful. Theunanswered array matters because a model with somewhere to put “not in this document” uses it; one without has only two options, and inventing is the more probable of them.

Verifying every span mechanically

This is the step that turns a rule into a guarantee. It is short enough to read in full, which is deliberate — a verifier you do not understand is not a verifier.

# verify_spans.py — Python 3.9+, standard library only.
# Usage: python verify_spans.py source.txt findings.json

import json, re, sys, unicodedata

def normalise(s: str) -> str:
    """Fold the differences that are not about truthfulness."""
    s = unicodedata.normalize("NFKC", s)
    s = s.replace("‘", "'").replace("’", "'")
    s = s.replace("“", '"').replace("”", '"')
    s = s.replace("–", "-").replace("—", "-")
    return re.sub(r"\s+", " ", s).strip().lower()

source = normalise(open(sys.argv[1], encoding="utf-8").read())
data = json.load(open(sys.argv[2], encoding="utf-8"))

passed, failed = [], []
for f in data.get("findings", []):
    ev = normalise(f.get("evidence", ""))
    words = len(ev.split())
    if not ev:
        failed.append((f, "no evidence"))
    elif ev not in source:
        failed.append((f, "evidence not found in source"))
    elif words < 10:
        failed.append((f, f"evidence too short ({words} words)"))
    else:
        passed.append(f)

print(f"{len(passed)} verified, {len(failed)} rejected")
for f, why in failed:
    print(f"\nREJECTED ({why})")
    print("  claim:    " + f.get("claim", "")[:140])
    print("  evidence: " + f.get("evidence", "")[:140])

sys.exit(1 if failed else 0)

The normalisation is the part to think about. Folding case, whitespace, quotation marks and dashes removes the failures that are about typography; folding anything more — stemming, punctuation removal, fuzzy matching — starts letting through evidence that does not say what the claim says. Fuzzy matching in a verifier is the same mistake as a hedge in a sentence: it makes the failure go away without making the problem go away.

A minimum length matters more than it looks. A four-word span is a substring of almost any document by accident, so the check passes and proves nothing. Ten words is short enough for a real quotation and long enough that a coincidental match is unlikely.

What it is genuinely good at in research

• Turning a vague question into specific ones. Asking what you would need to establish in order to answer something is a structuring task, and structuring is where these models are strong.
• Search-term generation. Alternate phrasings, the technical term for the thing you described in plain words, the name a different discipline uses. All cheap to verify: either the search returns something or it does not.
• Reading long documents for relevance. “Which of these forty pages mention retention periods” is a filtering task over supplied text, and false positives cost you a minute each.
• Finding disagreement between sources. Given two documents in one context, asking where they conflict works well, and the conflicts are checkable by construction.
• Extraction into a table. Once every source is in the pile, pulling the same six fields out of each is exactly a structured extraction job.

What this does not protect you from

Four holes, and it is better to know where they are than to believe the workflow is airtight.

• A bad source, faithfully quoted. Verification is about fidelity, not truth. The bibliography is still your responsibility.
• Quotation out of context. The span check cannot see the paragraph around it. This is why step six of the workflow exists and why deleting it is the most common way this process fails.
• Silent omission. A model that misses the one paragraph that contradicts your thesis produces a clean, fully verified, wrong summary. Read the source yourself when the stakes justify it; the extraction is a sieve, not a substitute.
• Instructions hidden in the source. If you are pasting in web pages, the text you supply is untrusted input to the model — the same problem as indirect prompt injection. The span requirement limits the damage, because injected instructions that produce claims without supporting spans fail the verifier, but it does not eliminate it.