Getting an AI to Work With Your Own Documents

“Upload a document and ask questions about it” describes two completely different mechanisms that fail in opposite ways. Which one you are using determines whether “it did not find something that is definitely in the file” is a bug, a limit, or your fault.

What uploading actually does

Nothing is learned. The file is not added to the model, it does not remember it tomorrow, and no other user gains access to it through the model. What happens is more mundane: the file is converted to plain text, and some of that text is placed in front of the model alongside your question. The model reads it the way it reads anything else you type.

Three consequences follow immediately. Formatting is mostly lost, so a table that looked clear in a PDF may arrive as a run of numbers with no columns. Anything that was an image — a scanned page, a chart, a photographed receipt — only works if the product runs character recognition on it first, and that step has its own error rate. And there is a hard ceiling on how much text can be in front of the model at once, called the context window.

Two mechanisms, two failure modes

Every product does one of two things with your file, and the difference is the whole subject.

Retrieval failures are the ones people find maddening, because the text is visibly in the file. The system never looked at it. This happens most often when your question and the document use different words for the same thing: you ask about “staff turnover”, the document says “attrition”, and the passage matching never connects them. It is the same mechanism used inside company search products, and how a document is cut into passages determines a surprising amount of the quality.

Telling which one you have

Products rarely say. Two tests take a minute and both are worth doing once per tool, because the answer changes how you work with it.

1. The needle test. Put a distinctive nonsense string near the end of a long document — ZEPHYR-COUNT-7734 in its own paragraph. Ask what ZEPHYR-COUNT-7734 is. If it finds it, the end of your document is reachable. Run it again with the marker in the middle.
2. The census test. Ask a question that requires reading everything: “how many times does the word risk appear, and list every heading in order”. Whole-file systems do this passably. Retrieval systems fail it badly, because no small set of passages contains the answer. If it produces a plausible but incomplete list of headings, you are on retrieval.

If you are on retrieval, stop asking questions that require reading the whole thing. Ask locatable questions instead, and ask them one at a time.

Four questions to answer before you upload

These are worth answering once per tool and once per class of document, not per file. The answers are usually in the vendor’s privacy policy and terms rather than in the marketing pages.

• Is the content mine to send? Client files, medical records, employee data, anything under an NDA and anything covered by a professional duty of confidence are not yours to hand to a third party merely because it is convenient. This is the question that causes actual professional consequences and it is answered before the technical ones.
• Is it used for training? Consumer products and business products commonly differ on this even under one brand. See how to read the clause that answers it.
• How long is it kept, and can I delete it? Not training on something and not keeping it are different promises, and most products make only the first.
• Who else in my organisation can see it? Shared workspaces sometimes make an uploaded file searchable by colleagues. That is a feature, until the file is a disciplinary letter.

If any answer is unacceptable, the fix is usually to redact rather than to abandon. Replacing names with PERSON_A and PERSON_B throughout a document costs two minutes in a word processor and removes most of the objection, and the answer maps back onto the real names trivially.

Preparing a document so it can be read

Five minutes here saves more than any prompt technique.

1. Prefer a text-based file over a scan. If you can select the text with a cursor, it is text. If not, it is a picture of text and everything depends on recognition quality — which is worst on exactly the documents people scan: forms, tables, and anything with handwriting.
2. Split very large documents by section. A 300-page manual as one file forces the tool into retrieval. The same manual as twelve chapter files lets you point at the chapter you mean, which beats any retrieval system.
3. Rescue the tables. Copy critical tables into a spreadsheet and attach that separately. A table converted from PDF is the most common source of a wrong number, because column boundaries disappear and a figure ends up under the wrong heading. Why PDF extraction is still hard is a genuinely unsolved engineering problem, not a bug in your tool.
4. Keep the page numbers. If the text has page or section markers, they survive into the extract and you can ask for them in answers, which makes checking possible.

Asking questions whose answers you can check

The single instruction that changes the reliability of document work more than anything else:

Answer using only the attached document.

For every claim, quote the sentence you took it from, in quotation
marks, with its page or section number.

If the document does not answer the question, say "not in the
document" and stop. Do not use general knowledge to fill the gap.

This does two things. It makes the answer checkable in seconds — you search the document for the quoted sentence. And it makes a particular failure visible: if a quoted sentence is not in the file, you have caught a fabrication that you would otherwise have believed. Quotes that cannot be found are the most reliable warning sign in all of document work.

Ask one question at a time. A list of six questions in one message reliably gets three good answers, two thin ones and one that is ignored. There is no benefit to batching — it costs the same and takes about as long.

What this cannot do

It cannot tell you what is not in a document. “Does this contract contain an indemnity clause?” is a question a retrieval system cannot answer negatively with any confidence, because failing to find something and it not existing look identical from the inside. For absence questions, search the text yourself.

It cannot reliably count, total or compare across a long document. Anything of the form “how many”, “what is the sum” or “which is largest” should be treated as a draft answer and recomputed. For numbers in a spreadsheet, there is a better method in making sense of a spreadsheet.

And it does not know which version it has. If you upload the draft rather than the signed contract, every answer will be confidently about the draft. Nothing in the system will mention this.