Introduction: The AI Interview Landscape

A new kind of question has joined the system design interview.

"Design YouTube's recommendation system." "Design a ChatGPT-style assistant for our product." "Design an agent that resolves customer-support tickets."

The classic questions are still there. You will still be asked to design Twitter or a URL shortener.

AI-first companies now build whole interview rounds around the new questions. Other companies add one or two of them to their standard rounds. Interviewers expect these answers from software engineers now, not only from ML specialists.

Most preparation goes to the wrong subject. It goes into model architecture, which is the internal structure of the model itself.

People learn what a transformer is, which is the neural network design behind ChatGPT and similar products. They learn about attention heads, which are small parts of that structure. Then nobody asks about any of it.

The interviewer asks what system design interviewers have always asked. What are the requirements? Where does the data come from? What does the request path look like? What breaks, and what does it cost?

The models are new. The interview is not.

What This Module Covers

This module gives you the whole picture before the details. You will learn four things:

1. The forms AI system design questions take.
2. What changes once a system has a model inside it.
3. How to tell which of the three question families you are facing.
4. A repeatable framework for structuring any answer.

The rest of the course fills in the depth. This module has a smaller job. It makes sure you never start an answer in the wrong place.

The Lessons

Two Terms Used Throughout

Model. A model is a function learned from data instead of written by hand.

It takes an input, like a user's watch history or a line of text. It produces an output, like a ranked list or the next word.

In this course, treat a model like a database or a cache. It is one more part in your architecture diagram, and you should be able to state four things about it:

• its latency, which is how long one call takes to return
• its cost per call
• its failure mode, which is what it does when it goes wrong
• how it is deployed and updated

Inference. Inference means running a trained model to get one prediction for one live request.

Training is the other path. Training runs offline, on a schedule, over large datasets. Offline means away from live traffic, inside background jobs.

Inference runs online, once per request, inside your latency budget. Online means right now, while a user is waiting.

Almost every part of the architecture in this course exists to make inference fast, cheap, and safe.

Keep the two paths separate in your thinking and in your diagrams. That one habit prevents many design mistakes.

Before You Start

Answer these three questions for a product you use every day. Rough answers are fine. The module will sharpen them.

1. Pick one AI feature you use. It could be a feed, an autocomplete, or a chatbot. What would "this feature is working well" mean as a number on a dashboard?
2. When that feature gives a bad output, like a boring feed or a wrong answer, how would the company find out?
3. If the feature disappeared for an hour, would the product still work? What would users see instead?

💡 In the interview: open an AI question the same way you open any system design question. Ask what the feature does, who uses it, and which number would show that it works. Then say where the model sits: "there is an offline path that trains the model, and an online path that serves one request." An interviewer who hears that in the first two minutes knows you will not spend the session on model internals.

Key takeaway: an AI system design question is a system design question with a learned part inside it. This module covers four things. It covers the forms these questions take, and what changes once a system learns. It also covers how to tell the three question families apart, and a framework for structuring an answer. Two terms carry through all of it. A model is a function learned from data. Like any other part of your design, it has a latency, a cost, and a failure mode. Inference is one run of a trained model for one live request, and it stays on a separate path from training. Start with the product problem, not the model architecture.