Introduction: ML Systems Foundations

The Five Pieces of a Predictive ML System

An interviewer says: "Design the system that predicts delivery time for a food order."

You may not know the data or traffic yet. Begin with a high-level design and identify the assumptions you need to confirm.

The following five parts provide a useful starting point.

Many predictive ML systems can be explained using five parts. A model is a function whose behavior is learned from data. Predictive means it produces estimates, scores, or categories.

Recommendations, fraud detection, and delivery estimates often need these parts, but their data, architecture, and operational requirements differ.

1. A training pipeline. A workflow that prepares data, trains a model, and evaluates it. It may run on a schedule or after a specific trigger.

2. A feature layer. The code and storage that turn raw data into model inputs.

3. An inference path. The steps that produce one prediction. Inference means running the model to get a prediction.

4. A serving platform. The machines and routing that answer live requests.

5. A monitoring loop. The measurement that notices when the system is getting worse.

Use these five parts to organize your design, then adapt them to the requirements.

This module teaches them one at a time. It stays product-agnostic, meaning it is not tied to one product.

There are no full designs here yet. Module 3 assembles these parts into real designs. Module 1 gave you the overall picture. This module gives you the individual parts.

What One Prediction Actually Does

A list of parts is hard to remember. The same system, written out as steps, is easier.

Here is an example delivery estimate, followed by a possible model-update process.

1. The customer opens the checkout screen. The app calls your prediction service.
2. The service collects features. Some are read from storage. Some are computed from the request itself.
3. The features are arranged in the exact order the model expects.
4. The model runs and returns a number, such as 34 minutes.
5. Business rules adjust that number, and the estimate goes back to the customer.
6. The service writes a log line. It records the features, the prediction, and the order id.
7. About 40 minutes later the food arrives. The real delivery time is now known.
8. That night the training pipeline joins the logged features to the real delivery times.
9. A new model is trained, tested, and shipped if it beats the current one.

The request path must meet a defined response-time target. Outcomes and model updates arrive later, often hours or days after the request.

That difference is the most important idea in this module.

One Divide Explains Most Problems

The recurring theme is one divide: offline against online.

Training often runs in background jobs over historical data. It has its own completion deadlines and compute requirements.

Online serving handles live requests within a response-time limit. Batch inference instead produces predictions in background jobs.

Training and serving need compatible feature definitions and model inputs, even though they process data differently.

Show two important connections in the interview: model artifacts move toward serving, and prediction records and outcomes support evaluation and training. Production systems also share schemas, configuration, and other metadata.

The model file moves from training to serving. Training produces it. Serving loads it and runs it.

Logs move from serving back to training. Serving records what it predicted. The real outcome is added later. Training reads all of it next time.