How Models Learn

Next-token probability

The chance a model gives each possible next token; it writes text by repeatedly choosing among them.

The shading shows the next-token probability the model gave each token it wrote.
From Why models hallucinate, step “A predictor, not a database”

See it in action

Try next-token probability hands-on in the lesson Why models hallucinate, part of the How AI assistants work course.

Open the lesson