0% completed
How a Model Learns
On This Page
Part 1: Learning Means Adjusting Numbers
Part 2: Loss, One Number for How Wrong
Part 3: Walking Downhill in Fog
Part 4: Gradient Descent, Step by Step
Part 5: Learning the Pattern, Not the Examples
Part 6: What This Changes for an Engineer
Common Mistakes
Key Takeaways
A delivery app wants to predict how long each delivery will take. Its model has a single learned number: minutes per kilometer. To predict, the model multiplies this number by the trip length.
The model starts with a guess of 1 minute per kilometer. One past trip was 4 km long and took 20 minutes. The model predicts 4 × 1, or 4 minutes, so it is off by 16 minutes. Now raise the learned number by 1, to 2 minutes per kilometer. The prediction becomes 8 minutes, and the model is now only 12 minutes off.
That one small change made the model less wrong. Training is this same change, repeated many times. First, measure how wrong the model is. Next, work out which way each learned number should move. Then move each number a small step that way. A large language model repeats this process for billions of learned numbers and a huge amount of text. This lesson explains the process through four ideas: data, parameters, loss, and gradient descent.
Part 1: Learning Means Adjusting Numbers
A model is a function. It turns an input into an output, and what it does depends on the numbers inside it. A number the model learns during training, rather than one a person writes in, is called a parameter. To learn simply means to change the parameters until the outputs are good.
The delivery model has only one parameter, minutes per kilometer. Its training data includes the three previous deliveries. All numbers in this lesson are illustrative.
- A 2 km trip took 10 minutes.
- A 4 km trip took 20 minutes.
- A 6 km trip took 30 minutes.
You can see that the right value is 5 minutes per kilometer. The model cannot see this. It starts with a guess and must find the right value by trying and correcting.
A large language model learns in the same way, at a far larger scale. It can have billions of parameters instead of one. The next lesson shows how a neural network holds them.
Part 2: Loss, One Number for How Wrong
Before the model can improve, it needs a way to measure how wrong it is. A loss function compares the predictions with the correct answers and returns one number. The number it returns is called the loss. A high loss means the predictions are far off, and a low loss means they are close.
For the delivery model, a simple loss is the average miss in minutes. Here is the loss for the starting guess of 1 minute per kilometer.
| Trip | Correct time | Prediction at 1 min per km | Miss |
|---|---|---|---|
| 2 km | 10 minutes | 2 minutes | 8 minutes |
| 4 km | 20 minutes | 4 minutes | 16 minutes |
| 6 km | 30 minutes | 6 minutes | 24 minutes |
The average miss is 16 minutes, so the loss is 16. Training now has a clear goal: change the parameter so that this number gets smaller.
A language model uses the same idea. Each example is a piece of text, and the model predicts the next piece. The loss is low when the model assigns a high probability to the piece that actually comes next. The text already contains the correct next piece, so no person has to label it.
Part 3: Walking Downhill in Fog
Picture a hiker on a mountain in thick fog, trying to reach the bottom of the valley. She cannot see the valley, or even ten meters ahead. She can only feel which way the ground slopes under her feet. So she takes one step downhill, feels the ground again, and takes another step. She stops when the ground feels flat.
Each part of this picture matches a part of training:
- The hiker's position is the current value of the parameters.
- Her height is the loss at that position.
- The valley floor is where the parameters give the lowest loss.
- The fog means the model cannot see the loss for every possible value at once. It knows only the loss where it is now.
- The slope under her feet is the gradient, which Part 4 explains.
- The length of each step is set by the learning rate, which Part 4 also explains.
This method is called gradient descent. "Descent" means going down, and the model goes down the loss one small step at a time.
Part 4: Gradient Descent, Step by Step
The slope of the loss at the current parameter values is called the gradient. For each parameter, it tells the model two things: whether to increase or decrease the parameter, and how much the loss changes when it does.
In the delivery example, the gradient says that raising the parameter lowers the loss. While the guess is below 5, each extra minute per kilometer cuts the average miss by 4 minutes. So the slope has a size of 4.
The learning rate is a small number that sets the size of each step. The step equals the slope times the learning rate. Here, a learning rate of 0.25 turns a slope of 4 into a step of 1.
The diagram below shows the model moving down its loss curve like a rolling ball.
Look at the loss after each step: 16, then 12, 8, 4, and finally 0. Each step lowers it by the same amount. Real data is noisier than this, so the loss rarely reaches zero.
The learning rate matters a great deal. Suppose it were 0.75, so each step moved the parameter by 3. The model would jump from 1 to 4, and then past the bottom to 7. From there, it would jump back to 4 and keep jumping between 4 and 7 forever. If the learning rate were 0.025 instead, each step would move the parameter by only 0.1. The model would then need 40 steps to arrive.
The playground below runs gradient descent on the delivery model, with the three trips from Part 1. It starts at 1 minute per kilometer, uses a learning rate of 0.25, and measures the loss as the average miss. It uses only the Python standard library.
Press Run and look at each printed step. It shows the parameter, the three predictions, and the loss. The loss falls from 16 to 12, 8, 4, and 0, and the parameter ends at 5 minutes per kilometer after 4 steps.
Now change LEARNING_RATE to 0.75 and run it again. The parameter jumps from 1 to 4 and then to 7, and the loss moves between 4 and 8 without settling. With a learning rate of 0.025, the loss reaches 0 only after 40 steps.
Training repeats one loop many times. The diagram below shows the four steps of that loop.
Step 1: Predict. The model runs on training examples with its current parameters.
Step 2: Measure the loss. The loss function scores how far the predictions are from the correct answers.
Step 3: Compute the gradient. In a neural network, a method called backpropagation computes the slope for every parameter in one pass, working backward through the model.
Step 4: Update the parameters. Each parameter moves a small step in the direction that lowers the loss.
A large model has billions of parameters, so its loss depends on billions of values at once, not on one. The hiker may also stop in a small hollow that is not the lowest point. For large networks, researchers have found this is usually less of a problem than it sounds.
Part 5: Learning the Pattern, Not the Examples
A falling training loss shows that the model fits its training examples better. It does not show that the model will do well on new inputs.
When a model memorizes its training examples instead of learning the general pattern, it is overfitting. An overfit model does very well on data it has seen and poorly on data it has not. For example, an overfit support model might repeat exact replies from its training tickets, even when a new question needs a different answer.
To catch overfitting, teams set aside some examples the model never trains on. These examples are called the validation set, and the loss on them is called the validation loss. At first, the training loss and the validation loss usually fall together. When the validation loss starts to rise while the training loss keeps falling, the model has started to memorize.
Look at the point where the validation loss stops falling and starts to rise. Stopping training at that point is a common way to keep the version that works best on new data.
Part 6: What This Changes for an Engineer
Most engineers will never train a large model from scratch. But the same ideas show up when you fine-tune a model, that is, train an existing model further on your own examples. They also appear when you review a teammate's results or choose a provider's training option.
Read loss charts with two lines, not one. The training loss alone cannot tell you whether a model is ready to ship. Ask for the validation loss, and test the model on real cases it has never seen. The later lesson "Evals: Knowing Your Feature Works" shows how to build those tests.
The model is only as good as its data. A model learns whatever the loss rewards on the data you give it. If half your support tickets have careless answers, the model learns to give careless answers.
Expect to choose a few settings. A training setting that a person picks, instead of a value the model learns, is called a hyperparameter. The learning rate is one hyperparameter. Another is the number of epochs, where an epoch is one full pass over the training data. Fine-tuning services often let you set both.
Using a model does not train it. After training ends, the parameters stay fixed, so chatting with a model does not change them. The lesson Training vs Inference explains this split and what each side costs.
Common Mistakes
Mistake 1: Treating a falling training loss as proof of quality. It shows only a better fit to the training examples. Check the validation loss, and test on new cases.
Mistake 2: Thinking a model learns from each conversation. Its parameters are fixed after training. A model changes only when someone retrains it.
Mistake 3: Training longer to fix bad results. More steps on poor or narrow data often make overfitting worse. Fix the data first, and then train again.
Key Takeaways
- A model is a function with learned numbers. These numbers are its parameters, and training adjusts them.
- Loss turns "how wrong" into one number. Training exists to make that number smaller.
- Gradient descent walks downhill in fog. The gradient gives the direction, and the learning rate sets the step size.
- A low training loss is not enough. The validation loss on held-back data shows whether the model learned the pattern or memorized the examples.
The next lesson, Neural Networks in One Lesson, looks inside the model itself. It shows how neurons, layers, and weights hold the parameters that this lesson taught you to adjust.
Reading Progress
0%
On This Page
Part 1: Learning Means Adjusting Numbers
Part 2: Loss, One Number for How Wrong
Part 3: Walking Downhill in Fog
Part 4: Gradient Descent, Step by Step
Part 5: Learning the Pattern, Not the Examples
Part 6: What This Changes for an Engineer
Common Mistakes
Key Takeaways