Surprise

How unexpected an outcome was, −ln p for an outcome given probability p; also called self-information.

The standard answer is surprise = −ln p, where p is the probability the model gave the outcome and ln is the natural logarithm (log base e).
From Softmax and cross-entropy, step “Logarithms measure surprise”

See it in action

Try surprise hands-on in the lesson Softmax and cross-entropy, part of the The maths behind AI course.

Open the lesson