How Models Learn

Training tokens

The amount of text a model learned from, counted in tokens.

The other number that matters is how much text a model learns from, counted in training tokens.
From Pre-training, step “Training tokens: how much it reads”

See it in action

Try training tokens hands-on in the lesson Pre-training, part of the How AI assistants work course.

Open the lesson