How Models Learn

Transformer

A neural network design built from stacked layers of attention and other parts. It powers most modern language models.

Stacked layers of attention like this, with a few other parts in between, are the heart of a transformer, the design behind modern chatbots.
From Attention, step “Many heads at once”

See it in action

Try transformer hands-on in the lesson Attention, part of the AI fundamentals course.

Open the lesson