Inside The Transformer: Attention Explained For Non-Specialists

The transformer is a type of neural network introduced in 2017, and it underpins well-known large language models such as OpenAI’s GPT series. OpenAI’s 2023 technical report, for example, describes GPT-4 as a transformer-based model trained to predict the next token.¹ The name appears throughout product documentation and research papers, yet the idea at its centre, attention, is rarely explained in plain terms.

This article explains what attention does with a worked example, why it replaced earlier designs, and what the main parts of a transformer are for. It builds on the ideas of tokens and embeddings from How An LLM Reads And Writes.

The Problem Attention Solves

The meaning of a word depends on the words around it. Take the sentence “The auditor rejected the invoice because it had no purchase order.” A person knows instantly that “it” means the invoice, not the auditor. Change the ending to “because she found no purchase order” and the focus shifts to the auditor.

Before transformers, the leading language models were mostly recurrent networks, which read text one word at a time and carried a running summary forward. Information from early words had to survive many steps to influence later ones, and because each step waited for the one before it, training was hard to spread across many processors at once.²

In 2017, Vaswani and colleagues, most of them at Google, proposed an architecture “based solely on attention mechanisms”, dropping recurrence entirely.² They demonstrated it on machine translation, where it beat the previous best results on standard English to German and English to French benchmarks while taking less time to train.

What Attention Does

Attention lets a token look at other tokens in the input and decide how much each one matters to its own meaning. In the original design’s encoder, every token could look at every other token. In the text-generating models described later, a token can look only at itself and the tokens before it. For the word “it” in the example, the model works out a set of weights over the earlier words and then builds a new representation of “it” that mixes in information from the words with the highest weights.

The mechanism works with three learned transformations of each token’s vector, usually called the query, key and value. A useful analogy is a library search. The query is what a token is looking for, each key describes what another token offers, and comparing the query against every key produces the attention weights. The values are the information actually passed along, blended according to those weights.

Because each token is compared with all the tokens it is allowed to see in a single step, the model can connect words that are far apart as easily as neighbours. During training, the calculation for a whole passage can run in parallel on modern chips. That parallelism is a large part of why transformers could be trained on far more data than earlier designs.

Several Heads At Once

A single attention calculation captures one kind of relationship. The transformer runs several in parallel, called attention heads, and combines their outputs before moving on. Vaswani and colleagues called this multi-head attention.² The largest GPT-3 model, for comparison, used 96 heads in each layer.⁵

In trained models, different heads often settle on different jobs. A 2019 study of Google’s BERT, a transformer built for understanding text rather than generating it, found heads that mostly looked at the next or previous word, heads that linked verbs to their direct objects, and heads that connected later mentions of something to earlier ones.⁶

Keeping Track Of Word Order

Attention on its own does not know the order of the tokens: “the auditor rejected the invoice” and “the invoice rejected the auditor” would look alike. The original paper fixed this by adding fixed mathematical patterns, called positional encodings, to each token’s embedding before the first layer.² Later work proposed other approaches, such as rotary position embeddings in 2021, which build position into the attention calculation itself.⁷

How A Transformer Is Put Together

The highlighted attention and feed-forward pair forms one transformer block, and large models stack many of these blocks; the largest GPT-3 model had 96 of them.⁵ The original 2017 design had two halves: an encoder that read the source sentence and a decoder that wrote the translation.² The first GPT model, published by OpenAI in 2018, used only a decoder-style stack.⁸ In that kind of stack, each token may attend only to itself and earlier tokens, a restriction called masking. It is what makes next-token prediction work, because the model cannot peek at the answer it is trying to predict.

Why This Matters In Practice

Two practical consequences follow from the design. First, because each token is compared with every token it can see, the work done by self-attention grows with the square of the input length: double the input and that part of the work roughly quadruples.² This is one reason models have a maximum context length and why long prompts cost more to process.

Second, the model has no separate store of facts. What it learned is spread across the parameters of its attention and feed-forward layers, and when Lewis and colleagues proposed retrieval-augmented generation in 2020, they treated updating what a model had absorbed into its parameters as an unsolved research problem.⁹ That is one reason techniques such as retrieval-augmented generation, covered in Using LLMs Well, supply facts in the prompt instead.