ELIZA and the First Chatbot Illusion

Joseph Weizenbaum published ELIZA in the Communications of the ACM in January 1966. It matched keywords in the user’s sentence, swapped the pronouns and handed the sentence back as a question. People told it things they had not told their families. Weizenbaum spent the following decade writing about why that frightened him.

The paper, the machine and the name

The full title is ELIZA — A Computer Program For the Study of Natural Language Communication Between Man and Machine, Communications of the ACM, volume 9, number 1, January 1966. Weizenbaum was at MIT. The program was written in MAD-SLIP — the SLIP list-processing package layered on the MAD language — and ran on the IBM 7094 under the Compatible Time-Sharing System, which is the reason a conversational program was possible at all: CTSS gave several users interactive terminals on one machine at a time when most computing was still batch.

The name comes from Eliza Doolittle in Shaw’s Pygmalion, and Weizenbaum chose it for a specific reason he states in the paper: like Doolittle, the program could be taught to speak progressively better, and like her, its improved speech would not indicate any change in what lay behind it.

ELIZA is not DOCTOR

This is the distinction the popular accounts collapse, and collapsing it produces the widely repeated claim that ELIZA was a 200-line program.

The separation was the architectural point of the paper. Weizenbaum was demonstrating a scripting system, and the choice of a Rogerian therapist for the demonstration script was a technical convenience: a Rogerian therapist is one of the very few conversational roles in which reflecting the other party’s statements back at them, while knowing nothing whatever about the world, is normal behaviour rather than evasion. Say “I am unhappy” to a person who replies “how long have you been unhappy?” and you cannot tell from that exchange whether they understood you.

So “ELIZA was 200 lines” is a claim about the script. Repeating it as a claim about the program is the same class of error as quoting the New York Times on the perceptron as though it were Rosenblatt.

How the pattern matching works

The mechanism is small enough to state completely, which is unusual for a famous program and is part of why it is worth studying.

1. Scan for keywords. The input is scanned word by word against the script’s keyword list. Each keyword carries a numeric rank; the highest-ranked keyword found wins, so that mother can outrank I in the same sentence.
2. Decompose. The winning keyword owns a set of decomposition rules — patterns with wildcards, of the shape (0 YOU 0 ME), meaning “anything, then the word you, then anything, then the word me”. The first matching pattern splits the sentence into numbered parts.
3. Reassemble. Each decomposition rule owns a list of reassembly templates that reference those parts by number — for instance (WHAT MAKES YOU THINK I 3 YOU). The templates are used in rotation so that repeating yourself does not produce an identical reply.
4. Transform pronouns. The copied fragment is run through a substitution table: I becomes you, my becomes your, am becomes are. This single step supplies most of the apparent comprehension.
5. Fall back. If no keyword matches, the program emits a content-free prompt — “please go on” — or, better, reaches into a memory of earlier transformed inputs and returns to something the user said several turns ago. That memory trick is the single most convincing behaviour in the whole system and it involves no understanding at all.

Input   : It seems that you hate me
Keyword : YOU        (rank beats "hate" in the DOCTOR script)
Rule    : (0 YOU 0 ME)
Parts   : 1="It seems that"  2="YOU"  3="hate"  4="ME"
Template: (WHAT MAKES YOU THINK I 3 YOU)
Output  : WHAT MAKES YOU THINK I HATE YOU

No parse tree. No grammar. No model of the sentence's meaning.
The word "hate" is copied through as an opaque string.

One consequence is worth stating because it explains the whole reception: the program has no representation of anything the user says. It has a representation of the shape of what the user says. Everything the reader experiences as understanding is contributed by the reader.

The reaction, and Weizenbaum’s three shocks

Weizenbaum expected the program to be read as a demonstration of how shallow a convincing conversation can be. He wrote in the 1966 paper that once a program is unmasked and its inner workings explained plainly, “its magic crumbles away” and it stands revealed as a mere collection of procedures. The unmasking did not work.

In the introduction to Computer Power and Human Reason: From Judgment to Calculation (1976) he sets out three things he had not anticipated. They are worth having in his order, because the third is the one people forget:

• People became emotionally involved, quickly and deeply. Users who knew exactly what the program was nonetheless disclosed personal material to it. His most-repeated illustration is his own secretary, who had watched him build the thing, asking him to leave the room so she could talk to it privately.
• Practising psychiatrists took it seriously as therapy. Some proposed that a version of it could deliver psychotherapy automatically at scale, which Weizenbaum regarded as a category error about what a therapist is for rather than an engineering proposal. Kenneth Colby, who had collaborated with him early on and later built PARRY, was among those arguing for computerised therapy, and their relationship did not survive the disagreement.
• People concluded that natural language understanding was solved. A general solution had been demonstrated, the reasoning went, and what remained was engineering. This is the shock with the longest half-life, and it recurs every time a system produces fluent output.

The book that followed is not a technical objection. Weizenbaum’s argument is that there are decisions computers should not be asked to make regardless of whether they could make them competently, because the decisions require the kind of judgement that comes from having lived a human life. That is an argument about what to delegate, and it reads as more current now than it did in 1976.

The ELIZA effect, sixty years on

The name for the phenomenon — the readiness of people to attribute understanding, intention and feeling to a system on the basis of its surface output — came later than the program, and its popularisation is usually credited to Douglas Hofstadter’s writing in the 1990s.

The mechanism is not gullibility. It is that human conversational competence includes a strong prior that anything producing well-formed responses to you has a mind, because until 1966 that prior was perfectly reliable. ELIZA was the first demonstration that the prior could be exploited by three hundred lines of pattern rules. Every system since has exploited it harder, and modern models exploit it without anyone intending to: the same effect appears wherever a system answers confidently, which is why confident wrong answers are so much more damaging than hedged ones.

Where it breaks, in three exchanges

The fastest way to understand what the program is not doing is to look at the inputs that expose it, because each one corresponds to a specific missing piece of machinery.

1. NO KEYWORD, NO CONTENT

   > The mitochondrion is the powerhouse of the cell.
   PLEASE GO ON.

   Nothing in the sentence matches the script, so the fallback fires.
   The program cannot distinguish a profound statement from a
   meaningless one, because it never inspected either.

2. THE TRANSFORMATION APPLIED WHERE IT DOES NOT BELONG

   > I told my brother he should stop lying to me.
   WHY DO YOU SAY YOUR BROTHER SHOULD STOP LYING TO YOU?

   Correct-looking, and produced by a pronoun swap over a copied
   fragment. Now change the syntax slightly and the same rule
   produces something ungrammatical, because there is no parse to
   protect the transformation.

3. NO ACCUMULATION ACROSS TURNS

   > My name is Anna.
   ...
   > What is my name?
   WHY DO YOU ASK?

   The program has no representation of anything you told it. Its
   only memory is a stack of previously transformed inputs used as
   fallback material, not a store of facts it can be asked about.

Those three failures map onto three things a modern system has and ELIZA does not: a representation of meaning rather than of surface form, a model of syntax, and a context that persists across turns. It is worth noticing that the third one is still, in a modern model, a matter of what is in the context window rather than of anything the model remembers — the mechanism changed completely and the constraint did not.

The source code, recovered

For decades no original listing was publicly available, and the hundreds of “ELIZA” implementations in circulation were reconstructions from the 1966 paper’s description rather than copies of Weizenbaum’s program. Many of them differ from it in ways that matter — several omit the memory mechanism entirely, which removes the behaviour that made the original convincing.

In the early 2020s the original MAD-SLIP listing was located in Weizenbaum’s papers in the MIT archives, and a restoration effort subsequently got it running again on an emulation of the CTSS environment. The practical upshot for anyone writing about ELIZA is that claims about what the original did can now, for the first time, be checked against the original — so it is worth checking them, rather than inheriting them from the reconstruction lineage.