Attention Is All You Need, Explained Section by Section

This paper is cited constantly and read rarely, which is a shame, because it is short, unusually clear, and contains an ablation table that teaches more about the architecture than the diagram everyone reproduces.

What the paper was arguing against

Read in 2026 the title sounds like a slogan. In 2017 it was a specific rebuttal. Sequence transduction — translation, summarisation — was done with recurrent networks, which process a sequence one position at a time, and the state of the art added an attention mechanism on top of the recurrence to let the decoder look back at encoder positions. Attention was an accessory. The paper’s claim is in its title: remove the recurrence, keep only the attention, and the model gets better and trains far faster.

The speed argument is the one that mattered most and it is a parallelism argument. A recurrent model cannot compute position t until it has computed t-1, so training time scales with sequence length no matter how much hardware you have. Self-attention computes all positions at once. Everything about the subsequent decade — larger models, larger datasets, the entire scaling literature — depends on that one property, and the paper states it plainly rather than burying it.

Section 3: the model

The architecture section builds the model from the outside in, and it is worth following in that order.

• Encoder and decoder stacks. Six identical layers each. Every layer is a sublayer plus a residual connection plus layer normalisation — the residual and the normalisation are mentioned almost in passing and are load-bearing for training deep stacks at all.
• Scaled dot-product attention. Queries, keys and values; the score is a dot product; the scaling by the square root of the key dimension exists because large dimensions push the dot products into regions where the softmax saturates and gradients vanish. The paper says exactly this, in one sentence, and it is the kind of detail that only appears if someone hit the problem.
• Multi-head attention. Rather than one attention function over the full dimension, several in parallel over lower- dimensional projections, concatenated. The stated motivation is that different heads can attend to information from different representation subspaces.
• Position-wise feed-forward networks. The same small network applied independently at each position. This is where most of the parameters live, which surprises people who assume attention dominates the parameter count.
• Positional encoding. Because self-attention is order-agnostic, position has to be injected. The paper uses fixed sinusoids and notes that a learned alternative performed about the same — a point it makes in the ablation, not the prose.

If you want the mechanism explained without reference to the paper at all, that is a separate job: attention as a lookup that returns a blend.

Section 4: why self-attention

This is the section most summaries drop and it is the intellectually honest core of the paper. Rather than asserting that attention is better, the authors compare layer types on three axes: computational complexity per layer, the amount of computation that can be parallelised, and the path length between any two positions in the network.

The third axis is the argument. In a recurrent network, information from position 1 reaching position 500 passes through 500 sequential steps, and long paths make long-range dependencies hard to learn. In self-attention that path length is constant. The table also concedes the cost: self-attention is quadratic in sequence length where recurrence is linear, so the trade is favourable only while sequences are shorter than the representation dimension. That concession is the entire long-context research programme of the following decade, stated in advance by the paper that created the problem.

The table nobody quotes

The results section reports BLEU on WMT 2014 English-to-German and English-to-French for a base and a big configuration, along with the training cost in FLOPs — reporting the cost alongside the score is what makes the comparison against prior systems meaningful rather than decorative.

Then comes the model variations table, which is the ablation, and which is where the paper earns its reputation. The authors vary one thing at a time on the base model and report what happens:

Read that table before the diagram and you get a different paper: not “here is the architecture that won” but “here is which parts of it we can show are doing something”. That is the shape of a well-constructed method paper, and it is the standard against which to judge the ones that omit it.

What it claimed versus what it became

Being precise about this is the most useful thing a reader can take away, because it is where the paper is most often misquoted.

• It is a machine translation paper. The evaluation is translation plus a constituency parsing experiment. It makes no claim about language modelling, few-shot behaviour, or scaling, because those results did not exist yet.
• It describes an encoder-decoder model. The architecture that dominates generative AI is decoder-only, which comes from the line of work traced in the GPT papers. Someone citing this paper for “the architecture behind ChatGPT” is citing an ancestor, not a description.
• It does not predict its own consequences. The conclusion speculates about extending attention to other modalities and about local attention for long inputs. It does not suggest that scaling this architecture would produce general-purpose language systems. Nobody knew.

That last point is worth sitting with. The single most influential architecture paper of the era did not know what it had. It is a good antidote to the habit of reading a new paper for whether it is revolutionary — a question its authors are usually in no position to answer either.