A Framework for AI Design Answers

Here is roughly how forty-five minutes splits across them: 3 minutes on step 1, 5 on step 2, 5 on step 3, 5 on step 4, 12 on step 5, 8 on step 6, and 5 on step 7, with the rest held as slack for follow-ups. That weighting shifts by family: step 5 dominates generative questions, while steps 3 and 4 grow heavier in predictive ones. But no step ever drops all the way to zero.

2. Step 1: Frame the Problem

Before you design anything, say out loud what the question actually is. Two moves, and both are fast:

1. Name the family. Run the output test: does the model produce a score, content, or actions? The family decides the skeleton you will draw in step 5: a ranking funnel, a generation pipeline, or a bounded loop. Naming it early tells the interviewer exactly which map you are on.
2. Name the known system, if you can. Most questions are a system you have already seen, just wearing a new product's clothes. "This is a recommendation funnel where the items are podcasts." "This is enterprise document Q&A with higher stakes." Recognizing it buys you a proven architecture to adapt instead of one you have to invent from nothing. It also flips the interviewer from examiner into collaborator, because they now know your route. This step gets easier as the course builds up your catalog of known systems, and the final module drills it under the clock.

Then scope the problem, explicitly. Say what is in, what is out, and one sentence of why. "I'll design text-only support, no voice, English first" is what scoping sounds like. Interviewers read it as experience. Close the step with a checkpoint: "Proceeding on that scope?" That question costs you three seconds, and it prevents the worst failure you can have in this interview: designing the wrong thing, well.

3. Step 2: What Does Success Mean?

Turn the product goal into requirements on three axes, and put a number on each one:

1. Quality. The metric a dashboard could actually show. "Users find videos they enjoy" becomes "CTR on recommendations, and average watch time per session" (CTR is click-through rate, the share of shown items that get clicked). "Helpful support bot" becomes "percentage of tickets resolved without human escalation."
2. Latency. A feed has to render in a few hundred milliseconds. A chatbot can take seconds, as long as it streams. An overnight batch job has no user-facing latency at all. This one number reshapes the entire architecture, so pin it down early.
3. Scale and cost. How many users, and how many requests per second. Then the budget posture on top of that. "This is a free feature, so cost per request must be near zero" is one posture. "This replaces a $30-per-ticket human process" is a very different one.

4. Step 3: What Data Do We Have?

Every AI system is downstream of its data. So name your sources before you design anything:

• Interaction logs: clicks, watches, purchases.
• Content and catalog data.
• User profiles.
• For generative systems, the documents or knowledge the model has to draw on.

Then ask the two questions that actually expose real understanding:

1. Where do labels come from? A label is just the correct answer for one training example, the outcome the model is trying to predict. Fraud systems get their labels from chargebacks: a customer disputes a charge, and the bank reverses it. Those labels arrive weeks late. Feeds get labels from clicks: fast, but noisy. Support bots often have no labels at all, until you design a thumbs-up/down widget into the product yourself. No labels means no learning, full stop. A candidate who checks for labels before proposing a model is already ahead of most.
2. How fresh must the data be? Yesterday's data is fine for a movie recommender. A trending-topics ranker needs minutes, not days. Fraud needs the last few seconds of a user's behavior. Freshness requirements are what create streaming infrastructure, so this one answer ends up writing part of your diagram for you.

5. Step 4: What Is the Simplest Model That Works?

The word "simplest" is doing real work in that heading. Reaching for the most sophisticated model first is a junior tell. Reaching for a baseline first is how practitioners actually operate: the baseline proves the pipeline works at all, and it gives you a number to beat.

• Predictive family: "most popular items" is the baseline, and a learned ranker is the upgrade.
• Generative family: the baseline is a hosted LLM with good retrieval. (An LLM is a large language model like the one behind ChatGPT. Hosted means you call it as a paid API rather than running it yourself. Good retrieval means fetching the right documents for the model to read before it answers.) Fine-tuning, giving an existing model extra training on your own data, is a later optimization, and it should come with a stated trigger: "if prompting can't hit the quality bar," or "when volume makes small-model economics win."
• Agentic family: a single model with two or three tools is the baseline. Multi-agent orchestration, several models splitting up the task and handing off pieces to each other, has to earn its own complexity.

State the upgrade path in one sentence, then move on. The deep dives live in later modules, and in whatever the interviewer asks next.

6. Step 5: How Do We Serve It?

This is the architecture diagram, and it is where most of your hour actually goes. The shape differs by family: a two-stage ranking funnel, a retrieve-prompt-generate pipeline, or a bounded agent loop. But underneath all three, the same four serving questions apply, and this course returns to them constantly:

1. What gets computed offline, ahead of time versus online, per request? Everything precomputable should be precomputed. Per-request model calls are the most expensive thing in the entire building.
2. Where are the caches, and what is their hit rate actually worth, in dollars and milliseconds?
3. Which requests can take the cheap path, meaning a small model, a cached answer, or a heuristic (a quick rule of thumb instead of a model call)? And which requests need the expensive path?
4. What happens when the model is slow, down, or rate-limited? Every single arrow that points into a model needs a timeout and a fallback.

7. Step 6: How Does It Keep Working?

Close the loop, explicitly, right there in the diagram:

• Log the model's inputs and outputs.
• Capture outcome signals: clicks, ratings, escalations, chargebacks.
• Compute the quality metrics from step 2 on live traffic.
• Feed all of that back into retraining or prompt iteration.

Then say how changes actually ship safely: offline evaluation first, then an A/B test or a shadow deployment (running the new version on live traffic without acting on its output, purely to compare it against the current one), then a gradual rollout with a rollback trigger. Just two sentences of this, delivered without being asked, routinely separate the top candidates from everyone else. Most people simply stop at the response arrow.

8. Step 7: What Breaks, and What Does It Cost?

End with two ledgers:

• Failure ledger. List the silent failures specific to your design: wrong-but-confident outputs, drift (when the live world quietly changes and the model's training data stops matching it), and feedback loops that amplify bias (the feed shows clickbait, users click it, the model learns clickbait). Name a guardrail for each one: thresholds with human review, drift monitors, content filters, kill switches.
• Cost ledger. Give one back-of-envelope estimate: requests per day, times the expensive-path fraction, times cost per model call. "10M requests, 20% hit the LLM at roughly a cent each: about $20K a day, so caching and routing are not optional" is exactly the kind of sentence that lands in any loop.

💡 If time runs out, compress steps 6 and 7 into one closing paragraph: "I'd log outcomes against predictions, retrain weekly, ship behind an A/B test, and watch for drift and cost creep. The two risks I'd monitor first are X and Y." Never let these two steps silently vanish.

9. In the Interview

The 30-second answer (your opening after any AI design question): "I'll work through this in seven steps. Frame what kind of system this is and scope it. Define success as measurable quality, latency, and cost targets. Inventory the data and where labels come from. Pick the simplest model that could work, with an upgrade path. Design the serving architecture, which is where I expect we'll spend most time. Close the feedback loop with evaluation and safe rollout. And finish with failure modes and a cost estimate. So, framing first: this looks like..."

Likely follow-ups:

1. "Skip ahead to the architecture." Do it, but keep a one-line anchor: "Then I'll assume the quality bar is X and latency budget is Y. Here's the serving design." The framework bends. The numbers stay.
2. "You have ten minutes, not forty-five." Run steps 1, 5, and 7. One framing sentence, the diagram, then the top risk and the cost line.

10. TL;DR