Attention head
One independent set of queries, keys and values. GPT-2 small has 12 per layer, 144 in all.
It has 124 million learned numbers, arranged in 12 layers with 12 attention heads each.
See it in action
Try attention head hands-on in the lesson Attention, part of the AI fundamentals course.