Spotting AI-Written Text Without a Detector

There is no reliable way to tell from the text alone. Some signals are worth a raised eyebrow, most of the famous ones are worthless, and nothing in this page — or in any detection product — is sound enough to justify accusing a person.

The short answer

Machine-generated text and human text are drawn from overlapping distributions of the same language. There is no watermark in ordinary output, no artefact that survives editing, and no property of a sentence that only one kind of author can produce. Every signal below is a probability shift, not a determination, and the probability shift is smaller than people expect.

The overlap has also grown in one direction that nobody planned: hundreds of millions of people now read machine-written prose daily, and human writing has drifted towards it. A stylistic tell from 2023 is partly a description of how people write now.

Why a detector must not decide anything

Detection products report a percentage. The percentage is a model’s output about another model’s output, and it carries no evidential weight of the kind an accusation requires. Three facts about them matter more than their marketing.

• They are wrong in both directions, and the two errors are not equally costly. Missing a machine-written essay costs a grade. Wrongly flagging a human-written one attaches an academic misconduct finding to a real person’s record. Students have had their records damaged, their degrees delayed and their integrity questioned on the strength of a detector score, and some of those cases have been documented and reported in the press. That is not a hypothetical risk being raised for balance; it has happened, and the documented cases are worth reading before anyone runs a detector on a cohort.
• The false positives are not randomly distributed. A 2023 study published in the journal Patterns reported that several widely used detectors classified essays written by non-native English speakers as machine-generated at a strikingly high rate, while classifying native-speaker essays correctly. Text that is simple, formulaic, or written to a rigid structure scores as machine-like, which means the students most likely to be wrongly flagged are non-native speakers, some disabled writers, and anyone who was taught to write to a template.
• They can be defeated trivially and accidentally. Light paraphrasing moves a score substantially. So does running the text through a grammar checker. A tool that a careful cheat evades and a careless honest student trips is not doing the job it is sold for. There is more on why the underlying problem is hard in why detectors fail and what watermarking can and cannot add.

The arithmetic of a small error rate

The reason a “99% accurate” claim is not reassuring is that accusations are generated by the false positive rate multiplied by the number of honest submissions, and the second number is large. Work it through with labelled assumptions.

ASSUMPTIONS (chosen for illustration, not measured)
  cohort                   300 students
  assignments per student  8
  submissions              2,400
  share actually using AI  20%   ->   480 machine-assisted
                                      1,920 honest
  detector sensitivity     90%   (catches 9 of 10 that used it)
  detector false positive  4%    (flags 4 of every 100 honest ones)

RESULT
  true positives   480 x 0.90  = 432
  false positives  1,920 x 0.04 = 77
  total flagged                 = 509

  Share of accusations that are wrong: 77 / 509 = 15%

  And at a 1% false positive rate, which is better than most
  published figures for these products:
  false positives  1,920 x 0.01 = 19
  Nineteen students accused wrongly, in one module, in one year.

Nineteen is the optimistic case. It is also, unavoidably, nineteen individual people with a misconduct letter. No threshold setting removes this; raising the bar to cut false positives cuts sensitivity with it. The arithmetic is why institutions that thought carefully about it have moved to assessment redesign instead — see what assessment redesign actually looks like and designing assignments in a world with AI.

Signals that are weak evidence

These are worth noticing. None of them individually means anything, and several together mean only that the text is worth reading more closely.

Signals that are worthless

Each of these circulates widely as a tell. None of them supports any inference at all, and repeating them causes measurable harm to people who write in a particular way.

• Em dashes. The best-known example and the emptiest. Plenty of people have punctuated like this for thirty years. Some word processors insert them automatically.
• Particular words. “Delve”, “tapestry”, “underscore”, “multifaceted”. These are ordinary English words with higher frequencies in some varieties of English than in others, which means a word-frequency tell is partly a nationality tell.
• Perfect grammar and spelling. Also produced by a spell-checker, a proofreader, a careful writer, or the accessibility software a dyslexic student depends on.
• Tripartite structure. Lists of three and “firstly, secondly, finally” are taught in every school writing curriculum in the English-speaking world.
• The absence of typos. Penalising clean copy is the most self-defeating version of this, because it teaches students to introduce errors deliberately.

The only strong evidence is about process

Everything reliable comes from outside the text. Not because it is more sophisticated, but because it is the only class of evidence that can actually distinguish two identical-looking documents.

• Version history. A document written over a week has hundreds of revisions, false starts, moved paragraphs and a research trail. A document pasted in whole has one revision. Most cloud word processors keep this automatically and it is checkable.
• Ability to discuss the work. A five-minute conversation about why a particular choice was made distinguishes authorship better than any textual analysis, and it is fair, because it tests the thing the assessment was supposed to test. It also has to be applied to everybody, not to the students somebody suspects.
• Continuity with earlier work. A step change in capability between two pieces by one person is information — though it is also what learning looks like, so it is a reason to ask, never a conclusion.

What to do when you suspect it

1. Check what your rule actually says. Very often nothing was prohibited, because nobody wrote a rule. In that case there is no offence and the work to do is on the rule, not the student.
2. Assess the work on its merits. If it is vague, unsupported and does not answer the question, mark it accordingly. That is defensible, it is what the marking criteria are for, and it requires no theory about who wrote it.
3. Verify the checkable things. Do the citations exist? Do they say what the text claims? An invented source is a substantive academic fault whoever produced it, and it can be demonstrated rather than alleged.
4. Have a conversation, framed as a conversation. “Talk me through how you approached this” is legitimate. “This scored 87% on a detector” is not, and it puts the burden of disproving a machine on a person who cannot.
5. Never make the accusation on textual grounds alone. Including on the grounds in the table above. They are reasons to look further; they are not findings.

If this feels like it leaves you without a remedy, that is the accurate conclusion, and it is why the durable answer is upstream. Assessment that is hard to shortcut is a design problem with real solutions; detection is a measurement problem with none.