What to Expect in the Modal System Design Interview

Expect a design question based on Modal's own infrastructure. Modal is a serverless platform that runs Python functions in containers on CPUs and GPUs. One candidate reports a design task in the process, though its format is not public, and Modal does not publish a question list.

Its engineering posts do describe the problems it solves every day, which are container starts in under a second, fast loading of large model files, and scheduling scarce GPUs. Prepare for questions close to those problems, because a generic social network question is unlikely here.

The Question Types

A serverless function platform. Design a system where a user submits a Python function and the platform runs it on demand. The hard parts are the cold start, autoscaling, and isolation between customers. A cold start is the delay when a new container must boot before it handles its first request. Modal's docs split that delay into two parts: time waiting in a queue for a container, and setup work inside the new container.

Fast container image loading. Design storage so that a container can start without downloading its whole image. Modal's public approach loads images lazily, so the image becomes a small index of file metadata, about five megabytes, that mounts in a few milliseconds. File contents are fetched on demand from a chain of caches, which runs from local memory to local disk, then a zone cache server, a regional content delivery network, and blob storage (large object storage in the cloud).

GPU scheduling. Design a scheduler that places jobs on machines with the right GPU type, in the right region, across several clouds. GPUs are scarce and unevenly placed, so the scheduler must balance wait time against cost, and you should expect questions about queues, priorities, and what happens when no GPU is free.

Usage metering and billing. Modal charges only for the seconds that code runs, so design the pipeline that records usage per second, per customer, without losing or double counting events.

What the Interviewer Grades

Practical judgment matters more than unusual parts, so state requirements first, including the latency target and the failure cases. Name your trade-offs out loud, such as cost against cold start time, and connect each choice to the customer, because a slow start is a slow product for every user of the platform. Show that you know where time is spent in a container start, because candidates who can name the stages of a boot score well.

A Walkthrough: Design a Serverless GPU Function Platform

Here is a high level plan for the signature question.

1. Requirements (5 minutes). Many customers, each with private code and data. A user defines a function and its dependencies, and the platform must run it on request, scale to zero when idle, and start new containers fast, with a target such as most cold starts under one second.

2. Image build and storage. Build each customer's container image once, and store it as an index plus content-addressed file chunks. Content-addressed means each chunk is named by a hash of its bytes, so identical files are stored once.

3. Lazy loading at start. When a container starts, mount the index only and fetch file contents on first read through the cache chain. Cache the most common files, such as Python packages, on every worker in advance.

4. The scheduler. Keep a pool of workers across regions and clouds, and match each request to a worker with free capacity of the right GPU type. Queue requests when no worker fits, start new workers when the queue grows, and stop idle containers after a set idle period.

5. Warm capacity and snapshots. Keep a small number of warm containers (already booted and waiting) for latency sensitive functions, and use memory snapshots for the rest. A memory snapshot saves the state of a booted container, so later starts restore it instead of repeating setup.

6. Isolation and metering. Run each customer's code in an isolated container so one customer cannot read another's data. Record start and stop events for every container, then sum them per customer for billing.

Common Mistakes in This Round

• Starting with Kubernetes. Naming a tool is not a design, so explain what the scheduler must do and then say which parts you would build or buy.
• Ignoring the cold start. A design that downloads a full image on every start fails the core requirement.
• No answer for scarce GPUs. Say what happens when demand exceeds the GPU pool: queueing, priorities, or a different region.
• Forgetting isolation. Customer code is untrusted, so mention the isolation layer early, before the interviewer asks.

How to Prepare

• Learn the standard parts. Grokking the System Design Interview covers queues, caches, and storage, which every platform design uses.
• Study the hard cases. Advanced System Design Interview, Volume II helps with replication, partitioning, and failure handling.
• Read Modal's engineering posts. The posts on lazy container loading and memory snapshots are the best public preparation for this round.
• See the full loop. The design task is one step in the Modal interview process, next to the motivation question and the wait for a decision.