Self-supervised learning

Learning in which the training targets come from the data itself, such as the next token of a text, rather than from human labels.

This is called self-supervised learning.
From Pre-training, step “The text marks its own homework”

See it in action

Try self-supervised learning hands-on in the lesson Pre-training, part of the How AI assistants work course.

Open the lesson