System Design Master Template

Layer 1: How the Request Reaches You

Everything in this layer happens before your code runs. Its job is to get the request to a healthy machine, quickly, and to keep as much traffic away from your servers as possible.

1. Domain Name System (DNS)

DNS translates a name a human can remember into an address a machine can route to. When someone types www.designgurus.io, DNS is what turns that into an IP address so the browser knows where to send the request.

The lookup walks a chain: a resolver asks a root server, which points at the server for .io, which points at the name server that actually holds the record for your domain. The answer is then cached at several points along the way, which is why the second visit is much faster than the first.

The question it answers: where is this service?

When it matters in an interview: when you serve users in more than one region. DNS is where geographic routing happens, sending a user in Singapore to your Singapore servers rather than to Virginia. Mention it there and move on. If your system is single-region, one sentence is enough.

2. Content Delivery Network (CDN)

A CDN is a network of servers spread around the world that hold copies of your static content: images, video, stylesheets, scripts. A user's request goes to whichever CDN location is nearest to them.

If that location already has the file, it serves it directly and the request never reaches your servers. If it does not, it fetches the file from your origin once, keeps a copy, and serves everyone else from that copy.

The question it answers: how do we serve large, unchanging files without the delay that distance adds and without touching our own servers?

When you need it: any system with images, video, or downloadable files, and any system with users far from your data centers.

When you do not: an internal tool, or a system whose responses are all personalized. A CDN caches things that are the same for everybody.

3. Load Balancer

A load balancer sits in front of a group of identical servers and spreads requests across them. It also watches their health, and stops sending traffic to any server that fails its checks.

That second job is the one people forget. A load balancer is not only about splitting work evenly, it is what lets a server die without users noticing.

You will be asked how it decides where to send each request. The common answers are round robin (take each server in turn), least connections (send to whichever server is handling the fewest requests right now), and IP hash (hash the client address so the same client keeps landing on the same server).

The question it answers: how do we run more than one server?

When you need it: every time you have more than one of anything. Load balancers appear in front of web servers, in front of application servers, and in front of database replicas.

Something people often miss: a single load balancer is itself a single point of failure. Real deployments run at least two, with a failover between them.

4. API Gateway

An API gateway is a single entry point in front of many backend services. A client makes one call to the gateway, and the gateway decides which service should handle it.

It also takes on the work that every service would otherwise have to implement separately:

• Routing. Send /orders to the order service and /users to the user service.
• Authentication and authorization. Check the token once, at the edge, so the services behind it can trust the caller.
• Rate limiting. Reject abusive traffic before it reaches anything expensive.
• Response caching. Return a common response without calling a service at all.
• Request and response shaping. Combine several service calls into one client response, or translate between formats.

The question it answers: how does a client talk to twenty services without knowing about twenty services?

When you need it: any microservices design. It is close to mandatory there.

When you do not: a single service. A gateway in front of one backend is a component you have to run for no benefit yet.

Load balancer or API gateway? Both, usually. They do different jobs: the load balancer picks a machine, the gateway picks a service and applies policy. A typical arrangement is a load balancer in front of several gateway instances, and more load balancing behind the gateway. The full comparison is in Load Balancer vs. API Gateway.

8. Asynchronous Work and Message Queues

Some work does not have to finish before you answer the user. Encoding an uploaded video, sending a confirmation email, updating a follower's feed: the user does not need to wait for any of it.

A message queue lets you hand that work off. Your service writes a message and returns immediately. A separate pool of workers reads messages and does the slow work whenever it can. Apache Kafka and RabbitMQ are the usual examples.

This gives you three things:

• Fast responses, because the slow part is no longer on the request path.
• Burst absorption. If ten times the normal traffic arrives, the queue grows and the workers catch up. Without a queue, the same burst takes the system down.
• Decoupling. The producer does not need the consumer to be running.

The cost is that the work is now done eventually, not immediately. The user gets "your video is processing", not "your video is ready".

When you need it: any slow operation, any burst-prone operation, and any fan-out where one action creates many writes.

Where to put the boundary between waiting and not waiting is covered in Synchronous vs Asynchronous Communication.

9. Real-Time Delivery

Everything so far assumes the client asks and the server answers. Plenty of systems need the opposite: something happened on the server, and the client has to find out. A chat message arrives, a driver moves, a payment clears.

There are four ways to do it, and they trade freshness against how much you hold open.

Polling. The client asks again every few seconds. Simple, works everywhere, and mostly wasteful: most answers are empty, and news is up to one interval late.

Long polling. The client asks, and the server holds the request open until it has something to send. Near-instant, and it needs no new protocol. The cost is a connection held open for each waiting client.

Server-sent events. One connection stays open and the server pushes down it whenever it has new data. Clean for feeds, notifications and live counters. It only goes one way.

WebSockets. One connection, open, with both sides free to send at any moment. This is what chat and multiplayer need. The cost is that your servers now hold connection state, which makes them harder to scale and harder to restart.

Webhooks are the same idea between two servers. You register a URL with another system and it posts to you when something happens, so neither side has to poll.

The question it answers: how does the client learn that something changed?

How to choose in an interview: ask how fresh the update has to be, and which direction it flows. Seconds and one direction gets server-sent events. Instant and both directions gets WebSockets. If a delay of a minute is acceptable, polling is a perfectly good answer and cheaper than either. There is a fuller treatment in Polling vs. Long-Polling vs. WebSockets vs. Webhooks.

10. Specialized Services

Two show up so often they are worth naming as their own boxes.

A notification service sends email, push notifications, and text messages. It is almost always fed by a queue, because sending is slow and failure-prone and must never block the action that triggered it.

A full-text search service answers "find me everything containing these words". A normal database index cannot do this well, because it is built for exact lookups on a column, not for ranked matching inside text. Search engines like Elasticsearch build an inverted index instead: a map from each word to the list of documents containing it. If your requirements include search, this is a separate box with its own store, kept up to date from your primary data.

Layer 3: Where the Data Lives

This is the layer that decides whether the design works. Most systems are ultimately limited by their storage, not by their application servers.

11. Choosing the Database

Relational databases store rows in tables with a fixed schema, enforce relationships between them, and give you joins and transactions. Choose one when your data has genuine relationships you query across, or when you need multiple things to change together correctly, such as money moving between accounts.

NoSQL databases relax the schema and usually give up cross-record transactions to scale horizontally more easily. They come in four shapes:

• Key-value stores, such as Redis and DynamoDB, for simple lookups by a single key. Fast and easy to partition.
• Document stores, such as MongoDB, where each record is a self-contained document and records need not look alike.
• Wide-column stores, such as Cassandra and HBase, built for very heavy writes and for reading ranges of rows by key.
• Graph databases, such as Neo4j, for data whose value is in the connections, like a social graph.

The interview move that works: do not start from the technology. Start from the access patterns. Say what the two or three hot queries are and how much data there is, and the choice is usually clear. The full decision framework is in SQL vs. NoSQL.

12. Indexes

An index is a separate structure that lets the database find rows by a column's value without reading the whole table. Most are B-trees, which keep the values in sorted order so a lookup takes a few steps instead of a full scan.

The trade is that every index has to be updated on every write that touches it. Indexes make reads faster and writes slower, and they take up space.

What to say in an interview: name the column you would index and the query it serves. "I would index short_key because every redirect is a lookup on it" is a complete answer.

16. Blob and File Storage

Databases are bad at storing large files. A photo, a video, or a document should go into object storage such as Amazon S3, or a distributed file system, and the database should store only the metadata: who owns it, when it was created, and the path to the file.

This split shows up in almost every media-heavy design, and skipping it is a common mistake. The usual flow is that the file goes to object storage, the CDN serves it from there, and the database row just points at it.

Layer 4: How You Make It Fast

17. Caching

A cache is fast storage holding a copy of something expensive to produce. Check the cache first; on a miss, do the expensive thing and put the result in the cache for next time.

The reason caching gets its own layer is that it happens in more than one place. It appears at every step of the path you just walked:

• In the browser, so the request never leaves the device.
• At the CDN, so it never reaches your data center.
• At the API gateway, so it never reaches a service.
• In the application, in memory or in Redis or Memcached, so it never reaches the database.
• Inside the database, which keeps hot pages in memory on its own.

Every one of these is a chance to answer without doing the work.

The hard part of caching is never the reading. It is deciding what happens when the underlying data changes, and how the cache and the database are kept consistent with each other. Those strategies, and the failure modes of each, are in Cache-Aside vs Read-Through, Write-Through vs Write-Back.

Layer 5: How You Keep It Alive

This layer is what separates a design that looks senior from one that does not. Most candidates draw a system as though nothing ever fails.

18. Health Checks and Heartbeats

In a system spread across many machines, something has to notice when one of them stops working. The usual mechanism is a heartbeat: every server sends a small message at a fixed interval saying it is alive. If the messages stop for longer than some timeout, the system treats that server as dead, stops routing to it, and starts replacing it.

Load balancers do a version of this with active health checks, calling a known endpoint on each server and removing any that fail.

The number worth thinking about is the timeout. Too short and a brief network problem takes a healthy server out of rotation. Too long and users keep hitting a dead machine.

19. Checksums

Data can be corrupted in transit or on disk, by a failing drive, a network fault, or a bug. A checksum catches it. When data is stored, the system computes a short fingerprint of it, using a hash function such as SHA-256, and keeps it alongside. When the data is read back, the fingerprint is recomputed and compared. If they differ, the data is corrupt and can be fetched from another replica instead.

The point is that the system returns an error or a good copy, rather than silently handing corrupt data to a user.

20. Coordination Services

Some decisions have to be made once, by everybody, in agreement: which node is the leader, who holds a lock, what the current configuration is. Doing this correctly across machines that can fail and lose contact is genuinely difficult, so systems delegate it to a service built for the job. ZooKeeper, etcd, and Consul are the usual names.

When to mention one: leader election, distributed locking, service discovery, or holding a configuration that every node must agree on. If your design has none of those, you do not need one.

Quick Reference

Download the System Design Master Template (pdf) to keep beside you while you practice.

💡 In the interview: draw the template outward from the client, and narrate the layer you are in. "The request comes in through DNS and the load balancer, hits the gateway for auth, and reaches the feed service" gives the interviewer an overview before you explain any one part in detail. Then, before you finish, spend thirty seconds on what you deliberately left out. Candidates who name the components they did not need sound like they made choices. Candidates who draw every box sound like they memorized a diagram.

Key takeaway: the template is a menu of about twenty-two components arranged in six layers: how the request reaches you, where the work happens, where the data lives, how you make it fast, how you keep it alive, and where the data goes afterwards. Its value is not the list, it is the order. Walk it from the outside in so your explanation has a direction, attach every component you draw to a requirement you agreed at the start, and say plainly which parts of the menu this system does not need. No good design uses all of them, and the ones you leave out are worth naming.

Now let's solve our first system design problem: Designing a URL Shortening Service.