Introduction: The AI Interview Landscape

1. What This Module Covers

System design interviews have changed. For a decade, the classic question was "Design Twitter" or "Design a URL shortener." You will still get those. But a new family of questions has joined them: "Design YouTube's recommendation system." "Design a ChatGPT-style assistant for our product." "Design an agent that resolves customer-support tickets." AI-first companies now build whole interview rounds around these questions. At other companies, these questions are being added to the standard interview rounds. And interviewers expect this from software engineers now, not just ML specialists.

Here is where most candidates go wrong. They spend their preparation time on model architecture: the internal structure of the models themselves. They learn what a transformer is (the neural network design behind ChatGPT and similar products), down to attention heads, which are small parts of that structure. Then they sit down for the actual interview, and nobody asks about any of it. The interviewer asks what system design interviewers have always asked. What are the requirements? Where does the data come from? What does the request path look like? What breaks, and what does it cost? The models are new. The interview is not.

This module gives you the overall picture before the details. You will learn four things:

1. The forms AI system design questions take.
2. What actually changes once a system has a model inside it.
3. How to spot which of the three question families you are facing.
4. A repeatable framework for structuring any answer.

The rest of the course fills in the depth. This module's job is simpler: make sure you never start an answer in the wrong place.

2. The Lessons

3. Two Terms Used Throughout

1. Model. A function learned from data instead of written by hand. It takes an input, like a user's watch history or a text prompt. It produces an output, like a ranked list or the next word. In this course, treat a model exactly like a database or a cache: just another component sitting in your architecture diagram. It has:
   • a latency (how long a call takes to return)
   • a cost per call
   • a failure mode
   • a way to deploy and update it
2. Inference. Running a trained model to get a prediction for a live request. Training happens offline, on a schedule, over huge datasets. Offline means away from live traffic, inside background jobs. Inference happens online, per request, inside your latency budget. Online means right now, while a user is waiting. Compare it to a restaurant: training is writing the cookbook, and inference is cooking one dish when an order arrives. Most of the architecture in this course exists to make that one dish, inference, fast, cheap, and safe. Keep the two paths separate in your thinking and in your diagrams. That habit alone prevents many of the mistakes candidates make.

4. Before You Start

Answer these three questions for a product you use daily. Rough answers are fine. The module will sharpen them.

1. Pick one AI feature you use. It could be a feed, an autocomplete, or a chatbot. What would "this feature is working well" mean as a number someone could put on a dashboard?
2. When that feature gives a bad output, like a boring feed or a wrong answer, how would the company ever find out?
3. If the feature disappeared for an hour, would the product still work? What would users see instead?