Where Training Data Comes From

2. Human Labeling Is a System, Not a Task

When a design needs annotation, do not just say "we'll have humans label it" and move on. Human labeling is a subsystem with its own quality controls, the same way a database has its own quality controls. Mentioning even two of these reads as real experience:

• Guidelines are versioned artifacts. A label means whatever the guideline document says it means, nothing more. So the guideline gets a version number, and that version is recorded in the dataset's lineage. Change the definition of "spam" and every metric computed across that boundary is quietly comparing different labels.
• Inter-annotator agreement is the quality metric. Send the same item to two or three reviewers and measure how often they agree. Low agreement is almost never a reviewer problem. It usually means the guideline itself is ambiguous. The fix is sharper instructions and worked examples, not more labels.
• Gold questions audit the reviewers. Seed the queue with items whose correct answers are already known, then track each reviewer's accuracy on them, continuously. This catches fatigue and drift without re-reviewing everything.
• Cost per label drives sampling. At real prices, you cannot label the entire firehose of data. So you label where a label is worth the most: predictions the model is least certain about, cases where reviewers or models disagree, and slices where the product is newest.

The cheapest labeling pipeline is one the product already runs for other reasons. The analyst queue in the fraud design exists to make decisions about individual transactions. But every decision it makes is also a fresh, high-quality label. Those labels flow straight into the retraining loop from Monitoring, Drift, and Retraining. Saying "the review queue is also my label factory" connects the product design to the training design in one sentence.

3. The LLM in the Labeling Loop

There is a modern middle path between the two: cheap-but-biased behavioral signals on one side, accurate-but-expensive humans on the other. It is called LLM-assisted labeling. A frontier model labels items against the same written guideline a human reviewer would use, at a small fraction of the cost and at whatever scale you need. The production pattern is a hierarchy, not a full replacement of people. The LLM labels everything. Humans verify a random sample, plus every case where the LLM is uncertain or disagrees with the current production model. The measured agreement between LLM and human then decides, per slice of the data, where the LLM can be trusted alone.

Here is the tricky part: LLM labels inherit LLM blind spots. The errors are systematic, not random, so they do not average out. They cluster in dialects, niche domains, and adversarial content, which is exactly where moderation and fraud problems tend to live. So the audit slices stay human-labeled. Agreement gets re-measured whenever the prompt, the model, or the policy changes.

4. Class Imbalance: Training on a 0.2% World

Fraud runs at roughly 0.2% of transactions. That is 500 legitimate examples for every single fraud case. Train naively on that stream and two things go wrong. You waste most of your compute on negatives. And the model quietly learns it can score well just by ignoring the positive class entirely.

The standard fix is to downsample negatives: keep every fraud case, but keep only a fraction, say 1 in 10, of the legitimate transactions. Train on a set that is 10x smaller and far less lopsided. Here is the catch: the model now believes fraud is 10x more common than it really is. Its output scores are inflated, and they must be recalibrated, meaning adjusted back to true real-world probabilities, before anyone treats them as probabilities. The calibration deep dive in Design Ad Click Prediction covers exactly how. The alternative is reweighting: keep all the data, but weight the positive examples more heavily in the loss function. That trades extra compute for skipping the resampling step.

Imbalance also breaks the most obvious metric. A model that predicts "legitimate" for every single transaction is 99.8% accurate, and completely useless. That is why imbalanced problems get evaluated on precision, recall, and the trade-off curve between them, instead of plain accuracy. Offline Evaluation: Choosing the Metric treats that choice properly.

5. Manufacturing Negatives

Retrieval and recommendation training has the opposite problem: no explicit negatives at all. Logs record what a user clicked, but nobody records the millions of items a user would never have clicked. To train a two-tower retrieval model, you must manufacture the negatives yourself. The choice of how matters:

• Random negatives: sample items uniformly from the catalog. Easy, but usually too easy. The model learns to tell cats apart from pianos, not to tell good candidates from great ones.
• In-batch negatives: inside each training batch, every other user's positive item serves as this user's negative for free. No extra sampling work, and it is the production default for two-tower training.
• Popularity-corrected sampling: popular items show up as in-batch negatives far too often, precisely because they are popular positives elsewhere. So the sampling gets corrected for item frequency, or the model ends up learning to punish popularity itself, which is not what you want.
• Hard negatives: a small dose of near-misses, items that were retrieved but not clicked, or close neighbors of the true positive, sharpens the exact boundary the retriever needs to learn.

Here is the tricky part: going all-hard backfires. Many "hard negatives" are actually items the user would have liked, just not this time. A diet of only hard negatives feeds the model contradictory signal, and training collapses.

💡 "What are your negatives?" is a standard interviewer move the moment you say "train a two-tower model." Have the ladder ready: in-batch by default, popularity-corrected, plus a small fraction of hard negatives, and say why all-hard fails.

6. Split Hygiene

How you split data into train, validation, and test decides whether your offline numbers mean anything at all. For anything time-shaped, and nearly every production problem is, use temporal splits. Train on weeks 1 to 10, validate on week 11, test on week 12. A random split scatters the future into the training set by accident. The model gets quietly graded on predicting a Monday it already saw pieces of from Tuesday's data. Within each example, features must also be point-in-time correct: computed only from what existed at the prediction moment, as Training vs Serving established.

Then run the leakage checklist, because each item on it produces a model that looks brilliant offline and fails online:

1. Features computed after the label moment: an aggregate like "merchant's fraud rate" that was computed over a window including the very transaction being predicted.
2. Label-adjacent features: fields that only get populated because the outcome already happened, like a refund flag or a case-resolution code riding along in the feature set.
3. Near-duplicates across splits: the same user, document, or an almost-identical example landing in both train and test, which quietly inflates every metric.

7. A Short Word on Privacy

Training data is user data. One deliberate paragraph in an interview covers it well. Minimize PII (personally identifiable information) at ingestion: train on hashed ids and derived features, not raw identifiers or raw text, unless the task truly requires them. Make deletion propagate. When a user deletes their account, that obligation reaches derived data too. That means training sets, and even embeddings built from their history. The standard mechanism is versioned datasets with a retention window, plus scheduled rebuilds. And treat labels as sensitive in their own right. A fraud verdict or a moderation decision attached to a person is exactly the kind of field where "who can see the labels" is an access-control question, not an afterthought.

8. In the Interview

The 30-second answer (to "where does your training data come from?"): "Three label sources, each with a cost and a delay. Natural outcomes, where the world grades us: delivery times in an hour, chargebacks in six weeks, free but late. Behavioral signals like clicks: free and instant, but biased toward whatever we chose to show. And human annotation under versioned guidelines with agreement metrics, for judgments no outcome volunteers, with an LLM pre-labeling at scale and humans auditing samples. I'd join labels to point-in-time features, split by time so nothing leaks the future, downsample the 99.8% negative class and recalibrate the scores, and for retrieval training I'd manufacture negatives: in-batch by default, popularity-corrected, plus a few hard ones."

Likely follow-ups:

1. "Chargebacks take six weeks. How do you train this week?" Train on data old enough for its labels to have matured, so the training window ends six weeks back. For recent data, use early proxies as provisional labels: rule hits and analyst-queue decisions arrive in days. And accept that outcome monitoring runs six weeks late. That is exactly why the score-distribution monitors exist.
2. "Your annotators disagree 30% of the time. What do you do?" Treat it as a guideline bug before an annotator bug. Cluster the disagreements, sharpen the guideline with worked examples, re-version it, and re-measure agreement. For the residue that is genuinely ambiguous, collect multiple judgments per item and train on the majority or the distribution. Report agreement alongside the model metric.
3. "Why not label everything with the LLM and drop the humans?" Because LLM errors are correlated: one model's blind spots repeated a million times over, invisible from inside the system. Human-labeled audit slices are the ground truth that catches it. Disagreement and low-confidence cases still route to people. Agreement gets re-checked on every prompt, model, or policy change.

9. TL;DR