Where Training Data Comes From

Notice that most items never reach a human. People see the random sample and the hard cases, which is where their time does the most good.

Why LLM Errors Do Not Cancel Out

It is tempting to think that errors fade at scale. Suppose the LLM is wrong on 5 percent of items, at random. A million labels would still teach the right pattern. But LLM errors are usually not random. They are systematic, which means they repeat on the same kinds of input.

For example, an LLM may misread a regional dialect. A phrase that is friendly in that dialect can look rude to it. The LLM may also miss adversarial content. This is text written on purpose to get past a filter, such as an insult with altered spelling.

On these groups, the LLM is wrong again and again, in the same direction. More labels only repeat the mistake, and a model trained on them learns it.

So keep audit slices labeled by humans. An audit slice is a set of items, labeled by people, that measures quality on one group. Keep one slice for the random sample. Keep others for the groups where errors cluster, such as each dialect and each known adversarial pattern. To fill these slices, send extra items from each group to the human reviewers.

Keep the test set human-labeled too, or the LLM's labels would grade a model that learned from the LLM's labels. The same mistakes would then appear on both sides and stay hidden.

Part 5: When Positives Are Rare

In the fraud model's data, about 0.2 percent of payments are fraud, or about 2 in every 1,000. This rarity is called class imbalance.

Plain accuracy fails under class imbalance. A model that predicts "legitimate" for every payment is already 99.8 percent accurate, yet it catches no fraud at all. So fraud models are judged with other metrics, which Offline Evaluation: Choosing the Metric covers.

Imbalance also affects training, because the model sees too little fraud to learn from. There are two standard fixes.

Fix 1: Downsampling

Downsampling keeps every fraud case and keeps only a fraction of the legitimate ones. For example, take a month with 10 million payments, of which about 20,000 are fraud.

Keep all 20,000 fraud cases, and keep 1 in 50 legitimate payments, which is about 200,000. Now about 1 in 11 training examples is fraud, instead of 1 in 500. The training set is much smaller, and the model sees fraud often enough to learn from it.

Downsampling has one side effect: the model now believes fraud is far more common than it is, so its scores come out too high. In this example, a score of 0.5 matches a real chance of only about 2 percent.

So recalibrate the scores after training, which means mapping each score back to a real chance. Recalibration uses the known sampling rate, or a held-out set at the real fraud rate. The order of the scores stays the same. This matters whenever a threshold is set from expected costs, because those costs need real chances.

One more rule follows from this: downsample only the training set. Keep the test set at the real fraud rate, so its metrics describe real traffic.

Fix 2: Reweighting

Reweighting keeps all the data and weights the positives more in the loss. The loss is the number that training tries to make smaller. It measures how wrong the model is on the training data. With reweighting, a mistake on a fraud case might count 50 times as much as a mistake on a legitimate payment.

Nothing is thrown away, so training takes longer than with downsampling. Reweighting also pushes the scores upward, so the same recalibration check applies.

Part 6: Choosing Negatives for a Retrieval Model

The ranking model needs a different kind of care. Before ranking, a retrieval model picks a few hundred candidate dishes out of millions. A common choice is a two-tower model. It turns each customer and each dish into a list of numbers, and it scores how well the two lists match. Embeddings and Two-Tower Retrieval explains the model itself.

The positives are easy to find: the dishes each customer ordered. The negatives are the hard part, because the logs do not list every dish a customer did not want. So the team must choose negatives, and each of the four common options below teaches the model something different.

Random negatives. Pick dishes at random from the whole catalog. They are cheap, but they are too easy. A customer who orders sushi is compared with a dessert from a shop in another city. The model learns obvious differences, not fine ones.

In-batch negatives. Training processes examples in groups called batches. Each example pairs one customer with one dish they ordered. For each customer, the other customers' positives in the same batch serve as negatives. They are free, because those dishes are already in the batch. They are also realistic, because real people wanted them. That is why in-batch negatives are the production default for two-tower training.

Popularity-corrected sampling. In-batch negatives have a bias: popular dishes appear in many orders, so they surface as negatives far too often. The model then learns to push popular dishes down, even for customers who would like them. The fix adjusts each dish's score during training by how often that dish is sampled. Serving uses the plain scores. A popular dish is then not punished just for being common.

Hard negatives. A hard negative is a near miss: an item that looks like a good match but was not chosen. One example is a pizza place next to the one the customer ordered from. Another is a dish the customer opened and did not order. Hard negatives teach the exact boundary between "right" and "almost right." Random and in-batch negatives are rarely this close, so they teach it poorly.

Hard negatives carry a risk, because many of them are items the user liked, just not this time. The customer who ordered from one pizza place may order from the one next door next week. If training uses only hard negatives, the model is punished for matching the user's real tastes. It can collapse, which means it learns to score many good matches low. So hard negatives should be a small share of the mix.

The usual order is to start with in-batch negatives, correct them for popularity, and then add a few hard negatives.

Part 7: Splitting Data Without Leaking the Future

Use a Temporal Split

Offline evaluation tests a model on data it did not train on. How you split the data decides whether that test is honest.

A random split shuffles all examples and deals them into a training set and a test set. For data shaped by time, a random split scatters future data into training by accident.

Suppose a fraud ring started in March. With a random split, March examples land in both sets. The model learns the ring's pattern from training, and then it "predicts" the same pattern in the test. Real serving never has that advantage, because in production the model always predicts payments from a period it has not seen.

A temporal split divides the data by time. For example, train on January to April, validate on May, and test on June. Now the test matches real use, where the model predicts a period it has never seen. Scores often drop compared with a random split, and the lower number is the honest one.

The Leakage Checklist

Leakage makes offline results look better than real results. Check for these four patterns before you trust any score.

1. Future data through a random split. For data shaped by time, use a temporal split, as described above.

2. Features computed after the label moment. The label moment is the moment the example describes, when the prediction would have been made. Every feature must use only data from before it. For example, take a merchant fraud-rate feature, where the merchant is the restaurant. Suppose it is computed over a window that includes the transaction being predicted. The rate then contains that payment's own fraud outcome, which was known only weeks later. Even a window that ends before the payment must count only the fraud cases already confirmed at that moment. A point-in-time-correct join prevents this, as Features and Feature Stores explains.

3. Label-adjacent features. These are features that are filled in only after the outcome is known. For example, a refund flag may be set only after fraud is confirmed. In training, it predicts fraud almost perfectly. At serving time it is always empty, because the outcome has not happened yet. Ask one question of every feature: "Would this value exist at the moment of the prediction?"

4. Near-duplicates across splits. The same example, or a nearly identical one, appears in both the training set and the test set. For example, one customer may post the same review on three restaurants. One fraud ring may send hundreds of almost identical payments. The model has already seen the test items, so every metric is inflated. Remove the duplicates, or keep each related group in a single split.

Training While Labels Are Still Arriving

Delayed labels create one more problem. Suppose chargebacks take about six weeks to arrive, and the team needs a new fraud model this week. The last six weeks have incomplete labels. Many fraud cases from those weeks still look legitimate, because their chargebacks have not arrived yet.

Waiting six weeks every time is too slow, because fraud patterns change. The answer has two parts.

Train on a mature window. Use an older window whose labels have settled and are final, such as weeks 7 to 26 before today.

Add provisional labels for the recent weeks. For the last six weeks, use faster signals as stand-ins, which are also called proxies. These are analyst decisions, customer reports of unknown charges, and cards that the bank blocked. Mark each one as provisional, and replace it with a final label when the chargebacks arrive.

The diagram below shows the two windows.