How Models Learn

Vocabulary

The full list of tokens a tokeniser can produce; GPT-2’s has 50,257 entries.

GPT-2’s has a vocabulary of 50,257 tokens; Qwen2.5’s, from 2024, has 151,665 (counting its special tokens).
From Tokens and context windows, step “Models count tokens, not words”

See it in action

Try vocabulary hands-on in the lesson Tokens and context windows, part of the How AI assistants work course.

Open the lesson