Attention

How a real language model, GPT-2, works out which earlier words matter to each word.

This one’s part of How Models Learn Pro — every lesson, every interactive, and new ones as they land. About 14 minutes.

See Pro plans Sign in

Already Pro? Sign in and you’ll go straight in.

Inside “Attention”

What you’ll learn

What does “it” mean?
The whole pattern as a grid
Queries and keys
From scores to shares
Mixing in the values
Many heads at once

Key terms

Attention: A mechanism that lets each token weigh how relevant other tokens are to it, then mix in their information.
Token: A chunk of text, often a word or part of a word, that a language model reads and writes.
GPT-2: A language model released by OpenAI in 2019. The small version has 124 million parameters, 12 layers and 12 attention heads per layer, and was trained on WebText, about 40 GB of web pages.
Query: In attention, a vector describing what a token is looking for.
Key: In attention, a vector describing what a token offers, matched against queries.
Value: In attention, the information a token passes on to the tokens that attend to it.
Dot product: Multiply two vectors number by number and add up the results. Large when they point the same way.
Softmax: Turns a list of scores into positive weights that add up to 1; a score one point higher than another gets about 2.7 times its weight, so large gaps in score become very large gaps in weight.
Causal attention: Attention in which each token can only look at itself and earlier tokens. Used by models that write text one token at a time.
Attention head: One independent set of queries, keys and values. GPT-2 small has 12 per layer, 144 in all.
Induction head: A kind of attention head that finds an earlier copy of the current token and looks at the token that followed it.
Attention sink: A token, often the first one, where heads put attention when nothing else is useful.
Transformer: A neural network design built from stacked layers of attention and other parts. It powers most modern language models.

Sources

Attention Is All You Need (Vaswani et al., 2017) — Introduces the transformer: scaled dot-product attention (query·key divided by √dₖ, softmax, weighted sum of values), multi-head attention with 64-dimensional heads, and masking of later positions.
Language Models are Unsupervised Multitask Learners (Radford et al., 2019) — The GPT-2 paper: transformer language models trained on WebText; the smallest has 12 layers and width 768 (listed as 117M parameters; the released model counts 124M) and a 1,024-token context.
GPT-2 model card, openai-community/gpt2 (Hugging Face) — The exact model whose attention weights, queries, keys and values this lesson shows: the smallest GPT-2, with 124M parameters, trained on WebText (40 GB).
GPT-2 small configuration file, config.json (Hugging Face) — n_layer 12 and n_head 12 (12 layers of 12 heads, 144 in all), n_embd 768 (so 64 numbers per head), n_ctx 1,024.
Transformer: A Novel Neural Network Architecture for Language Understanding (Google Research blog, 2017) — Source of the opening example: in “The animal didn’t cross the street because it was too tired/wide”, “it” refers to the animal or the street, and attention helps resolve it.
In-context Learning and Induction Heads (Olsson et al., 2022) — Defines induction heads, which complete [A][B] … [A] → [B], working with a previous-token head that copies information from the token before.
A Mathematical Framework for Transformer Circuits (Elhage et al., 2021) — Introduces induction heads and previous-token copying, analysed in small attention-only transformers.
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small (Wang et al., 2022) — Finds previous-token, duplicate-token and induction heads in GPT-2 small, the patterns this lesson scores every head on.
Efficient Streaming Language Models with Attention Sinks (Xiao et al., 2023) — Names the “attention sink”: language models put strong attention on the initial tokens even when they are not semantically important.
Speech and Language Processing (3rd ed. draft), Chapter 7: Transformers and Pretraining (Jurafsky & Martin) — Causal (left-to-right) transformer language models: the upper-triangular part of the attention score matrix is set to −∞, so no token can see the words that follow it.