What you’ll cover

Open a step to read its text without the interactive.

1. Arrows and lists of numbers

   A vector is a list of numbers, such as [3, 1]. You can also draw it as an arrow from the origin: 3 steps right, then 1 step up. Both pictures describe exactly the same thing, and each number is one of the vector’s components.

   In this lesson a and b are names for vectors (teal and rose on the grid). Their components get small numbers: a = [a₁, a₂], so a₁ is the step right and a₂ the step up, and likewise b = [b₁, b₂].

   An arrow has a length, written |a|. The components form the two short sides of a right-angled triangle, so Pythagoras gives it: |a| = √(a₁² + a₂²). For [3, 1] that is √10 ≈ 3.16.

   Drag the tip of a to (3, 4). The length is exactly 5, because 3² + 4² = 25. Now try (−3, −4): the arrow points the other way, but the length is still 5.

2. Adding and stretching

   To add two vectors, add matching components: [2, 1] + [1, 2] = [3, 3]. On the grid, that means placing b’s tail on a’s tip and following both arrows in turn. Going the other way round, a then b or b then a, lands in the same place, which is why the dashed lines make a parallelogram.

   Multiplying a vector by an ordinary number k, called a scalar, multiplies every component by k. The arrow stays on the same line, and its length is multiplied by |k|. A negative k flips it to point the opposite way.

   Set k to −1: k × a has the same length as a but points exactly backwards. Then try k = 0.5 and k = 0, and watch where k × a + b ends up.

3. The dot product

   The dot product turns two vectors into a single number: multiply matching components, then add: a·b = a₁b₁ + a₂b₂. For a = [3, 1] and b = [1, 2], a·b = 3×1 + 1×2 = 5.

   There is a second, geometric way to get the same number. For non-zero vectors, a·b = |a| |b| cos θ, where θ is the angle between the arrows. The readout works it out both ways so you can see they agree.

   Because lengths are never negative, the sign comes from cos θ. The dot product is positive when the angle is under 90°, zero at exactly 90°, and negative when it is wider.

   Drag b to (−1, 3). The dot product becomes 3×(−1) + 1×3 = 0, and the angle shows a right-angle box: the arrows are perpendicular.

4. Shadows: projection

   Imagine light shining straight down onto the line through a. The vector b casts a shadow on that line. That shadow is the projection of b onto a.

   The dot product measures it. The shadow’s signed length is a·b / |a|, so a·b is |a| times how far b reaches along a’s direction. The shadow itself, as a vector, is (a·b / |a|²) a. If a has length 1, the dot product simply is the shadow’s signed length.

   This is why the dot product is so useful: it answers “how much of b points in a’s direction?”

   Drag b slowly around the origin. For a given length of b, the amber shadow is longest when b lies along a, vanishes when b is perpendicular to a, and turns negative (behind the origin) once the angle passes 90°.

5. Cosine similarity: direction only

   The dot product mixes two things: how closely the arrows line up, and how long they are. Often we only care about direction. Dividing by both lengths removes them: cos θ = a·b / (|a| |b|), defined for non-zero vectors.

   This cosine similarity is always between −1 and 1: 1 means the same direction, 0 means perpendicular, −1 means opposite.

   AI systems use it to compare things stored as vectors. In Words as vectors, words with related meanings have embeddings that point in similar directions, and cosine similarity is the usual way to measure that. The raw dot product would also reward vectors for simply being long.

   Drag b from (1, 2) out to (2, 4), along the same line. The dot product doubles from 4 to 8, but the cosine stays at exactly 0.8.

6. More than two numbers

   Nothing in the formulas needs exactly two components. A vector can be a list of 5, 300 or 10,000 numbers, and the rules stay the same: the length is the square root of the sum of the squares, the dot product is the sum of the products, and cosine similarity divides one by the other two.

   We can’t draw 300 dimensions, but any two vectors lie in a flat plane through the origin, so the angle between them is still an ordinary angle.

   Here are two real word vectors from GloVe, the embeddings in Words as vectors: 300 numbers each for king and queen. The table shows the first 8; the last row adds up the products of the other 292. Over all 300, king·queen = 30.775. Dividing by the lengths, 7.008 and 6.931, gives a cosine similarity of 0.634, an angle of 50.7°. (These are computed from the vectors as stored, rounded to 3 decimal places, so the last digit can differ slightly from full-precision GloVe.)

   The first 8 dimensions contribute only 0.482 of that dot product. The similarity is spread across many small agreements, and GloVe gives none of its dimensions a name.

   Change queen to apple. The dot product falls to 7.276 and the cosine to 0.146 (81.6°), and the two words agree in sign in only 137 of the 300 dimensions, against 215 for king and queen. Then press “8 biggest products” to see which dimensions pull hardest.

Key terms

Vector

A list of numbers, which can also be drawn as an arrow from the origin.

Component

One of the numbers in a vector, such as how far its arrow goes along x or along y.

Length (norm)

How long a vector’s arrow is: the square root of the sum of its squared components, written |a|.

Scalar

An ordinary single number, used to stretch, shrink or flip a vector by multiplying each component.

Dot product

Multiply matching components of two vectors and add the results; it equals |a| |b| cos θ.

Projection

The shadow one vector casts on the line through another; its signed length is a·b / |a|.

Cosine similarity

a·b / (|a| |b|): a score from −1 to 1 for how closely two non-zero vectors point the same way, ignoring their lengths.