What you’ll cover

Open a step to read its text without the interactive.

1. Predicting a number

   Last time the model sorted things into two groups. Often we want a number instead: how much will this house sell for? Predicting a number is called regression.

   Each dot is a real house that sold in Ames, Iowa, between 2006 and 2010 — 24 ordinary sales of single-family homes, sampled across the size range from a public record of nearly 3,000 sales. Its living area runs along the bottom (in square metres), its sale price up the side (in thousands of US dollars). Bigger houses tend to cost more, but not exactly — the dots form a rough, noisy band.

   The simplest model for a band like this is a straight line. To price a new house, find its size on the bottom axis, go up to the line, and read off the price.

   Drag the two amber handles until the line runs through the middle of the dots.

2. Two dials: slope and intercept

   Every straight line can be written as price = slope × size + intercept. The slope is how much the price rises for each extra square metre — here, in thousands of dollars. The intercept shifts the whole line up or down.

   These two numbers are the model’s parameters. Learning, for this model, means nothing more than choosing good values for them.

   The rose lines show each residual: the gap between what a house really sold for and what the line predicted. A good line keeps these gaps small.

   Use the sliders. Notice how the slope tilts the line, while the intercept slides it up and down.

3. One number for “how wrong”

   To compare two lines we need a single score. We can’t just add up the gaps: a miss above the line and a miss below would cancel out, and a terrible line could score zero.

   So we square each gap first — literally, the area of the rose squares. Squares are never negative, and a gap twice as big makes a square four times the size. The average area is called the mean squared error. Because the gaps are in thousands of dollars, it comes out in squared thousands of dollars — which is why the readout also shows its square root, the RMS miss (root-mean-square), back in dollars. Like the squares it comes from, the RMS miss counts big misses extra.

   A score like this is called the loss. Lower is better, and zero would mean every prediction was perfect. Training a model means making its loss as small as possible.

   Shrink the squares. How low can you get the loss bar?

4. Let the computer find the best line

   Fiddling with dials works for two parameters, but it is slow and you never quite know whether you have found the bottom. The small black tick on the loss bar marks the lowest loss any straight line can reach.

   For straight lines there is a neat shortcut: a formula that solves least squares directly, jumping straight to the slope and intercept with the smallest possible squared error.

   Even the best line misses a typical house by about $20,000, and its RMS miss is about $47,000. That isn’t a failure of the maths: size is only part of what sets a price — location, age and condition matter too, and this model can’t see them.

   Most bigger models, like neural networks, have no such formula. The next lesson shows the method they use instead, which feels a lot like what you were just doing by hand.

   Get as close as you can by hand, then press “Fit automatically” and see how near you were.

5. The line follows the data

   The best line isn’t chosen by anyone. It is a consequence of the data: move a house and the best line moves with it, because a different line now has the smallest total area.

   Every point gets a say, and houses far from the average size have the most leverage — moving a house at either end tilts the line far more than moving one in the middle.

   Drag the biggest house up and down, then try one in the middle. Click empty space to add houses of your own.

6. One loud outlier

   Squaring has a side effect. A house that is wildly off makes one huge square, and the line bends towards it to shrink that square, even if it makes everything else a bit worse.

   This one is real: the largest house in the whole Ames dataset, 524 m² — nearly four times the size of a typical home — sold in 2008 for just $160,000, about a quarter of what the line through the other houses predicts. It was a partial sale (the house wasn’t finished when it was last assessed), which the dataset’s author, Dean De Cock, says likely doesn’t reflect the house’s market value; he suggests leaving houses like it out. To fit it in, both axes now stretch much further.

   The dashed line shows the fit without it. If we count plain distances instead of squares, one big miss matters less and the line tilts far less — though a house this far out still has a pull. How you measure wrongness changes what the model learns.

   Compare the two slopes, then switch to “Plain misses”. Drag the amber-ringed house up towards a price that fits its size and watch the line swing back.

Key terms

Regression

Predicting a number, such as a price, rather than a category.

Parameter

A number inside a model that is learned from data, like the slope and intercept of a line.

Slope

How much the prediction rises for each one-unit increase in the input.

Intercept

The value the line predicts when the input is zero; it shifts the line up or down.

Residual

The difference between a real value and the model’s prediction for it.

Loss

A single number measuring how wrong a model is on its data. Training tries to make it small.

Mean squared error

A loss: square every residual, then take the average.

Least squares

Choosing the line that gives the smallest possible mean squared error.

Outlier

A data point far from the pattern of the rest, such as a huge house recorded as a partial sale at a fraction of its size-based price.