0% completed
Introduction: The AI Interview Landscape
On This Page
Know the Vocabulary
Part 1: Why AI System Design Interviews Exist
How It Differs From a Classic Design Interview
What Interviewers Expect
Part 2: What Makes an AI Design Different
The Definition
What an AI Design Question Is Not
Part 3: One Company, Three Kinds of AI
Part 4: The Map of the Course
Part 5: The Lessons in This Module
The Seven Steps You Will Use Everywhere
Part 6: How to Use This Course
Part 7: Common Mistakes This Course Prevents
Mistake 1: Starting With a Model Name
Mistake 2: Drawing Only the Model
Mistake 3: Using One Family's Tools for Another
Mistake 4: Stopping When the Answer Reaches the User
Mistake 5: Trusting the Model's Output
Mistake 6: Reading Without Practicing
FAQs
Key Takeaways
Check Your Understanding
Know the Vocabulary
This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.
- Model: A function whose behavior is learned from data instead of written by a developer. It takes an input and returns an output, such as a score, a piece of text, or a proposed action.
- AI system: A software system in which a model makes some of the decisions or produces some of the output.
- AI system design interview: An interview round in which you design a product or feature that uses a model. You explain the design out loud, usually in about 45 minutes.
- Large language model (LLM): A model trained on a very large amount of text. It reads text and writes new text, such as an answer to a question.
- Predictive system: A system whose model returns a score, a rank, or a label, and the product acts on that output.
- Generative system: A system whose model produces new content, such as text, code, or an image.
- Agentic system: A system in which a model chooses some of the next steps while a task runs. Tools carry out those steps as real actions.
- Question family: One of three groups of AI design questions, sorted by what the system mainly produces. The three families are predictive, generative, and agentic.
- Label: The correct answer for one past example, such as the delivery time that really happened. A model learns from examples that have labels.
- Latency: The time between a request and its answer.
- Fallback: What the system does, as decided in advance, when a part fails or is too slow.
- Feedback loop: The part of the system that records real outcomes and uses them to measure and improve the model.
- Scope: The part of a problem you agree to design in the interview. Everything else is out of scope.
- Running example: One made-up company that many lessons return to. Each new idea builds on a case you already know.
- Seven-step framework: The course's order for answering a design question. The steps are frame the problem, define success, map the data, pick the model, design the serving path, keep it working, and count the costs.
Part 1: Why AI System Design Interviews Exist
Many products now contain a model somewhere. A delivery app estimates arrival times, and a bank scores each payment for fraud. A help center answers questions in plain language, and a coding tool suggests the next line of code.
The model is rarely the hard part to obtain, because teams often reuse a known method or call a hosted model. The hard part is the system around the model. That system collects the data, delivers each answer in time, and checks that the answers stay good. It also controls the cost and decides what happens when the model is wrong.
Companies need engineers who can build that system. Many of those engineers are software engineers, not researchers. So companies test the skill directly, in an AI system design interview.
How It Differs From a Classic Design Interview
A classic system design interview asks you to design a service such as a chat app or a file store. You discuss storage, scale, speed, and failures.
An AI system design interview asks the same kind of question, but a model sits inside the system. Here are two examples of such questions:
- "Design a system that shows each customer how long the delivery will take."
- "Add an assistant that answers customer questions about their orders."
Both answers still use familiar parts, such as caches, databases, queues, and load balancers. But each answer also needs new parts, because a model behaves differently from ordinary code, as Part 2 explains.
What Interviewers Expect
Many candidates begin with the name of a model and then draw one box labeled "model." Interviewers then ask the questions this course answers. Where does the training data come from, and how fast must the answer arrive? How do you know the answers are good, and what happens when the model is wrong? What does each request cost?
These questions are about the system, not about model math. This lesson gives you a map of the course that answers them. The five lessons after it give you the method you will use in every answer.
Part 2: What Makes an AI Design Different
The Definition
An AI design question is a system design question in which a model produces part of the output. Four facts about models change the design.
-
Quality is a range, not right or wrong. A normal service usually returns the correct record, or it fails with an error. A model can return an answer that is partly right, or wrong, while the request still succeeds. So the design must say how quality is measured.
-
Quality can drop while the code stays the same. Customers, prices, and habits change over time. A model learned from the past, so it can become less accurate, often slowly. This change in the data is called drift. So the design needs monitoring and a way to update the model.
-
Each request can cost real money. A small model can be cheap to run, but an LLM answer can take seconds and cost about a cent, at example prices. At millions of requests a day, that becomes a large bill. So cost becomes part of the design.
-
The model can be wrong, or it can be tricked. A model can make an honest mistake. It can also read text written to change its behavior, such as "ignore your rules." This kind of attack is called prompt injection, and checking the input alone cannot stop it. So rules that matter must live in code outside the model.
What Changes When the System Learns explains these differences in more detail. It also shows what each one adds to the design.
What an AI Design Question Is Not
An AI design question is not a math exam. For most software engineering roles, you treat the model as a part with inputs, outputs, a latency, and a cost. You explain why you chose it, but you rarely derive how it learns. Roles that build models go deeper, as the next lesson explains.
An AI design question is not a test of model names. Model names change every few months, but the reasons for a choice last much longer, such as "a small model is fast enough here."
An AI design question is not a new subject. Everything you know about system design still applies. What is new is how the old parts fit around a model.
Part 3: One Company, Three Kinds of AI
This course returns again and again to one made-up food-delivery company. Customers order meals from restaurants in its app, and drivers bring the meals to them. The company uses AI in several places, and two of them return in many lessons.
- Delivery-time prediction. At checkout, the app shows how long the delivery will take.
- The support assistant. Customers ask questions about their orders, and an assistant built on an LLM answers them.
The fastest way to meet the example is to follow one order through it. The steps below show one customer's evening. The times, prices, and limits are illustrative assumptions.
Step 1: The app shows an estimate at checkout. At 6:02 PM, a customer opens checkout. A model estimates that the delivery will take 32 minutes. The number appears within about 100 milliseconds (ms). The model returns a number, so this part of the system is predictive.
Step 2: The company records the real delivery time. The order arrives 50 minutes later than the app promised. That real time becomes a label, and it helps train the next version of the model. This is the feedback loop at work.
Step 3: The customer asks the support assistant a question. At 7:30 PM, the customer writes: "My order came 50 minutes late. Can I get money back?" The support assistant first looks up the company's late-delivery policy. Then it writes an answer from that policy. Orders more than 30 minutes late receive a credit of 20 percent of the order total. The output is new text, so this part is generative. The answer takes a few seconds, not milliseconds.
Step 4: The assistant applies the credit itself. An assistant that is allowed to act can take this step without a person. The order total was 32, so the credit is 6.40. Ordinary code checks the amount against an example limit of $10. A larger amount would go to a person. The output is an action that moves money, so this part is agentic.
The diagram below shows the same four steps.
Notice that one company and one order needed three different designs. The estimate had to arrive in milliseconds. The answer took seconds, and it had to match the policy. The credit moved money, so code had to check it before it ran.
Notice also that the hardest questions sit outside the model. Where did the policy text come from? Who decided that the limit is $10? What happens to the estimate if the model is down? These are system questions, and this course teaches you to answer them.
This module's capstone looks at the same company. It lists several AI features the company could build. Then it chooses one to scope first, the support assistant. Later lessons build both features in full, one part at a time.
Part 4: The Map of the Course
The course has nine modules, and this first module gives you the method. The next six teach the parts and the designs, one question family at a time. The last two train the interview itself and review the whole course.
The diagram below shows the nine modules. They are grouped by the question family each one teaches.
Notice that the family columns read in course order, from left to right. The predictive modules come first, because the later modules reuse their ideas. Serving, evaluation, and monitoring all return in the generative and agentic modules.
The table below lists every module in order, with the question it answers. Each link opens the module's first lesson.
| Question the module answers | Module |
|---|---|
| What do AI design interviews ask, and how do you structure an answer? | The AI Interview Landscape |
| What system surrounds a model, from training data to serving and monitoring? | ML Systems Foundations |
| How do you design recommendations, feeds, search, ads, fraud detection, and harmful content detection? | Classic ML Design Questions |
| How fast can an LLM answer, and what does each answer cost? | LLM Serving & Inference |
| How does a model answer from your own documents and cite its sources? | RAG & Retrieval |
| How do you turn model calls into a whole product, such as a chat assistant, and keep it safe? | GenAI Product Designs |
| How do you let a model take actions through tools, within limits? | Agentic Systems |
| How do you run a full design question in 45 minutes, out loud? | System Design Interview Case Studies |
| What should you drill in the last two weeks before the interview? | Rapid Review |
The running example moves through these modules. ML Systems Foundations builds delivery-time prediction from end to end. LLM Serving & Inference and RAG & Retrieval build the support assistant's answers. GenAI Product Designs turns it into a full chat product, and Agentic Systems lets it act. Many other lessons use products you already know, such as a video platform, a code assistant, and a music app.
Part 5: The Lessons in This Module
This module teaches the method, not a single design. The first two lessons describe the interview and what changes when a system learns. The next two give you a way to sort any question and a way to answer it. The capstone puts them together on the food-delivery company.
| Question the lesson answers | Lesson |
|---|---|
| What forms do AI design questions take, and what do interviewers look for? | How AI System Design Shows Up in Interviews |
| What does a model add to the system design you already know? | What Changes When the System Learns |
| How do you tell a predictive, a generative, and an agentic question apart? | The Three Question Families |
| Which seven steps give every answer its order, and how long does each one take? | A Framework for AI Design Answers |
| How do you turn a vague AI question into a clear scope? | Capstone: Scoping an AI Design Question |
The Seven Steps You Will Use Everywhere
The framework lesson teaches seven steps in a fixed order. Every later module adds content to them, so learn the names now.
- Frame the problem. Name the family and the closest known system, which is a design you have already studied. Say what is in and out of scope, then confirm the scope with the interviewer.
- Define success. Put numbers on quality, latency, scale, and cost.
- Map the data. Say where the data and the labels come from, how late the labels arrive, and how fresh the inputs must be.
- Pick the model. Start with a simple baseline, and say what evidence would justify something larger.
- Design the serving path. Draw the path of one request, with a time limit and a fallback for each model call.
- Keep it working. Close the feedback loop, watch quality, and ship each change safely.
- Count the costs. List the main ways the system can fail, and estimate what it costs to run.
Step 5 usually gets the largest share of a 45-minute interview. The framework lesson gives each step its share of the time. Step 1 stays with this module, and the later modules supply the content for Steps 2 to 7.
Part 6: How to Use This Course
The course uses a few kinds of lessons. Each kind asks something different of you.
| Lesson kind | What it does | How to use it |
|---|---|---|
| Module introduction | Draws a map of the module and follows one example through it | Read it first, and come back to its map later |
| Concept lesson | Builds one idea with one example, then shows how to say it in an interview | Say its short interview answer out loud |
| Capstone | Asks one design question that uses the whole module | Answer it yourself before you read the walkthrough |
| Case study | Runs one full interview question on a 45-minute clock | Run it out loud, with a timer, before you read it |
| Review lesson | Drills short answers, warning signs, and decisions | Use it in the last two weeks before the interview |
Use the same routine for each lesson.
Step 1: Read the vocabulary before the lesson. Each lesson opens with its terms, so read them once and return when a term is unclear.
Step 2: Follow the example through every step. Each idea is built on one example, often the food-delivery company. Follow the numbers, not only the words.
Step 3: Say the short interview answer out loud. Many lessons include a short spoken answer and likely follow-up questions. Spoken answers take longer than you expect, and they show where your words run out.
Step 4: Answer the check questions before you open them. Most lessons end with a few check questions. Answer each one in your own words first. Most lessons also end with a longer set of practice questions.
Step 5: Try each capstone and case study first. Set a timer and answer out loud before you read. The gaps you find become your study list.
If your interview is soon, read this module and every module introduction first. Then study the modules closest to the role. Finish with the case studies and the review module.
Part 7: Common Mistakes This Course Prevents
Mistake 1: Starting With a Model Name
An answer that opens with "I would use the newest model" skips the problem. Start with what the system must do, for whom, and how fast. The model choice comes after the targets.
Mistake 2: Drawing Only the Model
A single "model" box has no data, no fallback, and no way to improve. Draw the path of a request, the data that trains the model, and the loop that measures it.
Mistake 3: Using One Family's Tools for Another
A delivery-time estimate is a predictive question. Answering it like a chatbot, with an LLM and a long prompt, adds seconds and cost. A small predictive model fits the task better. Name the family first, then use the tools that fit it.
Mistake 4: Stopping When the Answer Reaches the User
Many answers end when the result appears on the screen. But the interviewer also wants to know how quality is measured, what can fail, and what it costs. Reach the feedback loop and the costs, even in a few sentences.
Mistake 5: Trusting the Model's Output
The model will sometimes be wrong, slow, or tricked. Decide in advance what the system does then. Use a fallback, a check in code, or a person for high-risk actions.
Mistake 6: Reading Without Practicing
Reading a good answer feels easier than producing one. Practice out loud, on a timer, before you read the walkthrough.
FAQs
Q: Do I need a machine learning background for this course?
No. The course explains each term the first time it appears. You need to know basic system design, such as databases, caches, and queues. You do not need to know how to derive a training algorithm.
Q: My interview is about LLMs, so can I skip the predictive modules?
No. The LLM and agent modules reuse the predictive modules' ideas about serving, evaluation, monitoring, and safe release. Many products also mix families, such as a feed ranked by a predictive model with LLM summaries.
Q: Will my interview use the food-delivery company?
Probably not. The company is a made-up teaching example. The moves you practice on it transfer to any product, such as a bank, a music app, or a search engine.
Q: Are the numbers in the lessons real figures?
No. They are illustrative assumptions, chosen to make the arithmetic easy to follow. In an interview, state your own numbers as assumptions, and show how they lead to your decisions.
Q: Should I read the modules in order?
Yes, if you can. Each module builds on the ones before it. The case studies and the review module use everything that came earlier.
Key Takeaways
- AI system design interviews test the system around the model. The interviewer wants the data, the serving path, the quality checks, the fallbacks, and the cost, not model math.
- Four facts make AI designs different. Quality is a range, quality can drift, each request can cost money, and the model can be wrong or tricked.
- Questions come in three families. Predictive systems produce scores, generative systems produce content, and agentic systems take actions.
- One running example connects the course. The food-delivery company's delivery-time prediction and its support assistant return in many lessons.
- The seven-step framework orders every answer. Frame the problem, define success, map the data, pick the model, design the serving path, keep it working, and count the costs.
- In a system design interview, name the family first and confirm the scope. Then walk the seven steps, and always reach the feedback loop and the costs.
The next lesson, How AI System Design Shows Up in Interviews, describes the forms these questions take. It also shows what interviewers look for in each one.
Check Your Understanding
1. A support assistant returns an answer with a success code, and no error appears in the logs. Can the answer still be wrong?
<details> <summary>Show answer</summary>Yes. A model's quality is a range, not right or wrong. The request can succeed while the answer is partly wrong or wrong. So the design must measure answer quality directly, not only errors and latency.
</details>2. In the food-delivery example, which family does each part belong to: the delivery estimate, the policy answer, and the applied credit?
<details> <summary>Show answer</summary>Predictive, generative, and agentic. The estimate is a number, so it is predictive. The policy answer is new text, so it is generative. The credit is an action that moves money, so it is agentic.
</details>3. A delivery-time model has not changed in six months, but its estimates keep getting worse. What is a likely cause?
<details> <summary>Show answer</summary>Drift. The world changed while the model stayed the same. New restaurants, new traffic, or new habits can make old patterns wrong. Monitoring finds the drop, and retraining the model on recent labels usually restores its accuracy.
</details>4. The instructions to the assistant say "never refund more than $10." Is that enough to enforce the limit?
<details> <summary>Show answer</summary>No. The model can be wrong, or text written to trick it can change its behavior. A limit that matters must be checked in ordinary code before the refund runs. A larger refund then goes to a person.
</details>Reading Progress
0%
On This Page
Know the Vocabulary
Part 1: Why AI System Design Interviews Exist
How It Differs From a Classic Design Interview
What Interviewers Expect
Part 2: What Makes an AI Design Different
The Definition
What an AI Design Question Is Not
Part 3: One Company, Three Kinds of AI
Part 4: The Map of the Course
Part 5: The Lessons in This Module
The Seven Steps You Will Use Everywhere
Part 6: How to Use This Course
Part 7: Common Mistakes This Course Prevents
Mistake 1: Starting With a Model Name
Mistake 2: Drawing Only the Model
Mistake 3: Using One Family's Tools for Another
Mistake 4: Stopping When the Answer Reaches the User
Mistake 5: Trusting the Model's Output
Mistake 6: Reading Without Practicing
FAQs
Key Takeaways
Check Your Understanding