Offline Evaluation: Choosing the Metric

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

• Offline evaluation: Scoring a model on past examples whose real outcomes are already known, before any user sees the model.
• Test set: A set of past examples kept apart from training, so the model's score on them is a fair check.
• Metric: One number that summarizes how good a model's predictions are, such as an average error.
• Positive: The class a classifier looks for, such as a fraudulent payment. It is often the rare class