Causal attention

Attention in which each token can only look at itself and earlier tokens. Used by models that write text one token at a time.

See it in action

Try causal attention hands-on in the lesson Attention, part of the AI fundamentals course.

Open the lesson