0% completed
System Design Master Template
On This Page
System Design Master Template (video)
How to Actually Use This
Layer 1: How the Request Reaches You
- Domain Name System (DNS)
- Content Delivery Network (CDN)
- Load Balancer
- API Gateway
- Rate Limiting
- A Note on Proxies
Layer 2: Where the Work Happens
- Services
- Asynchronous Work and Message Queues
- Real-Time Delivery
- Specialized Services
Layer 3: Where the Data Lives
- Choosing the Database
- Indexes
- Data Partitioning
- Replication
- Consistency Model
- Blob and File Storage
Layer 4: How You Make It Fast
- Caching
Layer 5: How You Keep It Alive
- Health Checks and Heartbeats
- Checksums
- Coordination Services
- Observability
Layer 6: Where the Data Goes Afterwards
- Analytics and the Data Pipeline
Putting It Together
Quick Reference
A system design question gives you a blank page and about forty minutes. Two things go wrong, and they are the same two things for almost everybody:
- You do not know where to start.
- You finish without knowing whether you left something important out.
A template fixes both. Not by giving you a design to memorize, but by giving you a fixed order to think in. You always start at the same place, you always move in the same direction, and at the end you can look at the parts you did not use and say out loud why you did not need them.
The picture below is that template. It is not a design for any particular system. It is the full menu of pieces that a large system might contain, arranged roughly in the order a request travels through them.
System Design Master Template (video)
Here is a video walking through the same template.
How to Actually Use This
Three rules, and the third one is the one that separates a good answer from a recited one.
1. It is a menu, not a checklist. Almost no real design uses every box, and most use well under half of them. Drawing all twenty-two does not show breadth, it shows that you are working from memory instead of from the requirements.
2. Walk it from the outside in. Start where the request enters and follow it to where the data is stored, then follow the response back. This gives your explanation a direction, and an interviewer can follow you without having to ask "where are we now?"
3. Every box needs a reason from the requirements. If you cannot say which requirement forced a component into the diagram, it does not belong there yet. "I am adding a cache because we agreed this is a hundred reads per write" is a design decision. "I am adding a cache" explains nothing.
The template splits into six layers. Each layer answers one question.
Layer 1: How the Request Reaches You
Everything in this layer happens before your code runs. Its job is to get the request to a healthy machine, quickly, and to keep as much traffic away from your servers as possible.
1. Domain Name System (DNS)
DNS translates a name a human can remember into an address a machine can route to. When someone types www.designgurus.io, DNS is what turns that into an IP address so the browser knows where to send the request.
The lookup walks a chain: a resolver asks a root server, which points at the server for .io, which points at the name server that actually holds the record for your domain. The answer is then cached at several points along the way, which is why the second visit is much faster than the first.
The question it answers: where is this service?
When it matters in an interview: when you serve users in more than one region. DNS is where geographic routing happens, sending a user in Singapore to your Singapore servers rather than to Virginia. Mention it there and move on. If your system is single-region, one sentence is enough.
2. Content Delivery Network (CDN)
A CDN is a network of servers spread around the world that hold copies of your static content: images, video, stylesheets, scripts. A user's request goes to whichever CDN location is nearest to them.
If that location already has the file, it serves it directly and the request never reaches your servers. If it does not, it fetches the file from your origin once, keeps a copy, and serves everyone else from that copy.
The question it answers: how do we serve large, unchanging files without the delay that distance adds and without touching our own servers?
When you need it: any system with images, video, or downloadable files, and any system with users far from your data centers.
When you do not: an internal tool, or a system whose responses are all personalized. A CDN caches things that are the same for everybody.
3. Load Balancer
A load balancer sits in front of a group of identical servers and spreads requests across them. It also watches their health, and stops sending traffic to any server that fails its checks.
That second job is the one people forget. A load balancer is not only about splitting work evenly, it is what lets a server die without users noticing.
You will be asked how it decides where to send each request. The common answers are round robin (take each server in turn), least connections (send to whichever server is handling the fewest requests right now), and IP hash (hash the client address so the same client keeps landing on the same server).
The question it answers: how do we run more than one server?
When you need it: every time you have more than one of anything. Load balancers appear in front of web servers, in front of application servers, and in front of database replicas.
Something people often miss: a single load balancer is itself a single point of failure. Real deployments run at least two, with a failover between them.
4. API Gateway
An API gateway is a single entry point in front of many backend services. A client makes one call to the gateway, and the gateway decides which service should handle it.
It also takes on the work that every service would otherwise have to implement separately:
- Routing. Send
/ordersto the order service and/usersto the user service. - Authentication and authorization. Check the token once, at the edge, so the services behind it can trust the caller.
- Rate limiting. Reject abusive traffic before it reaches anything expensive.
- Response caching. Return a common response without calling a service at all.
- Request and response shaping. Combine several service calls into one client response, or translate between formats.
The question it answers: how does a client talk to twenty services without knowing about twenty services?
When you need it: any microservices design. It is close to mandatory there.
When you do not: a single service. A gateway in front of one backend is a component you have to run for no benefit yet.
Load balancer or API gateway? Both, usually. They do different jobs: the load balancer picks a machine, the gateway picks a service and applies policy. A typical arrangement is a load balancer in front of several gateway instances, and more load balancing behind the gateway. The full comparison is in Load Balancer vs. API Gateway.
5. Rate Limiting
A rate limiter counts how many requests a caller has made in a window of time and rejects anything over the agreed limit, usually with a 429 Too Many Requests response.
It runs at the edge, normally inside the gateway, and that placement is the whole point. A request that is going to be rejected should be rejected before it reaches a service, a database, or anything else that costs money.
The question it answers: what stops one caller from consuming the whole system?
What to decide out loud: what you count per. Per user, per API key, or per IP address are the usual choices. A single global limit is almost always wrong, because one abusive caller then degrades everybody.
Where the state is stored. The counter has to be shared, or a caller simply spreads their traffic across your servers and gets the limit multiplied by the number of machines. In practice that means a small amount of state in something like Redis, which is worth saying out loud because interviewers listen for it.
Two ways to shape the traffic you do accept: a token bucket allows a short burst and then settles to a steady rate, while a leaky bucket smooths everything to a constant rate with no burst at all. Which you want depends on whether bursts are legitimate traffic or abuse. The full comparison is in Token Bucket vs Leaky Bucket.
6. A Note on Proxies
A forward proxy sits in front of clients and makes requests on their behalf. A corporate network that filters which sites employees can reach is running a forward proxy.
A reverse proxy sits in front of servers and receives requests on their behalf. The client thinks it is talking to one machine; the reverse proxy decides which server actually answers.
This matters because both the load balancer and the API gateway are reverse proxies. They are specialized versions of the same idea. Forward proxies rarely appear in a backend design, because the client is usually outside your control and you cannot see inside it.
Layer 2: Where the Work Happens
The request has arrived. Something now has to do the thing the user asked for.
7. Services
This is your code. In an interview it is a box labelled with a job: "URL service", "upload service", "feed service".
The important decision here is how many boxes there are.
A monolith is one deployable unit containing all the functionality. It is simpler to build, simpler to reason about, and it is the right answer far more often than interview candidates admit.
Microservices split the application into small services that each own one area and talk over the network. The properties worth naming:
- Each service can be deployed and scaled on its own, so the one service under heavy load gets more machines and the rest do not.
- Each service owns its own data, which is what makes independent deployment actually possible.
- One service failing does not necessarily take the system down.
- The cost is real: network calls between services can fail, data spread across services is hard to keep consistent, and debugging spans several systems.
How to handle this in an interview: split along the lines where the scaling requirements differ. If photo uploads are heavy and profile reads are light, those are two services. Splitting into fifteen services for a system with three features is over-engineering, and the interviewer will see it that way.
8. Asynchronous Work and Message Queues
Some work does not have to finish before you answer the user. Encoding an uploaded video, sending a confirmation email, updating a follower's feed: the user does not need to wait for any of it.
A message queue lets you hand that work off. Your service writes a message and returns immediately. A separate pool of workers reads messages and does the slow work whenever it can. Apache Kafka and RabbitMQ are the usual examples.
This gives you three things:
- Fast responses, because the slow part is no longer on the request path.
- Burst absorption. If ten times the normal traffic arrives, the queue grows and the workers catch up. Without a queue, the same burst takes the system down.
- Decoupling. The producer does not need the consumer to be running.
The cost is that the work is now done eventually, not immediately. The user gets "your video is processing", not "your video is ready".
When you need it: any slow operation, any burst-prone operation, and any fan-out where one action creates many writes.
Where to put the boundary between waiting and not waiting is covered in Synchronous vs Asynchronous Communication.
9. Real-Time Delivery
Everything so far assumes the client asks and the server answers. Plenty of systems need the opposite: something happened on the server, and the client has to find out. A chat message arrives, a driver moves, a payment clears.
There are four ways to do it, and they trade freshness against how much you hold open.
Polling. The client asks again every few seconds. Simple, works everywhere, and mostly wasteful: most answers are empty, and news is up to one interval late.
Long polling. The client asks, and the server holds the request open until it has something to send. Near-instant, and it needs no new protocol. The cost is a connection held open for each waiting client.
Server-sent events. One connection stays open and the server pushes down it whenever it has new data. Clean for feeds, notifications and live counters. It only goes one way.
WebSockets. One connection, open, with both sides free to send at any moment. This is what chat and multiplayer need. The cost is that your servers now hold connection state, which makes them harder to scale and harder to restart.
Webhooks are the same idea between two servers. You register a URL with another system and it posts to you when something happens, so neither side has to poll.
The question it answers: how does the client learn that something changed?
How to choose in an interview: ask how fresh the update has to be, and which direction it flows. Seconds and one direction gets server-sent events. Instant and both directions gets WebSockets. If a delay of a minute is acceptable, polling is a perfectly good answer and cheaper than either. There is a fuller treatment in Polling vs. Long-Polling vs. WebSockets vs. Webhooks.
10. Specialized Services
Two show up so often they are worth naming as their own boxes.
A notification service sends email, push notifications, and text messages. It is almost always fed by a queue, because sending is slow and failure-prone and must never block the action that triggered it.
A full-text search service answers "find me everything containing these words". A normal database index cannot do this well, because it is built for exact lookups on a column, not for ranked matching inside text. Search engines like Elasticsearch build an inverted index instead: a map from each word to the list of documents containing it. If your requirements include search, this is a separate box with its own store, kept up to date from your primary data.
Layer 3: Where the Data Lives
This is the layer that decides whether the design works. Most systems are ultimately limited by their storage, not by their application servers.
11. Choosing the Database
Relational databases store rows in tables with a fixed schema, enforce relationships between them, and give you joins and transactions. Choose one when your data has genuine relationships you query across, or when you need multiple things to change together correctly, such as money moving between accounts.
NoSQL databases relax the schema and usually give up cross-record transactions to scale horizontally more easily. They come in four shapes:
- Key-value stores, such as Redis and DynamoDB, for simple lookups by a single key. Fast and easy to partition.
- Document stores, such as MongoDB, where each record is a self-contained document and records need not look alike.
- Wide-column stores, such as Cassandra and HBase, built for very heavy writes and for reading ranges of rows by key.
- Graph databases, such as Neo4j, for data whose value is in the connections, like a social graph.
The interview move that works: do not start from the technology. Start from the access patterns. Say what the two or three hot queries are and how much data there is, and the choice is usually clear. The full decision framework is in SQL vs. NoSQL.
12. Indexes
An index is a separate structure that lets the database find rows by a column's value without reading the whole table. Most are B-trees, which keep the values in sorted order so a lookup takes a few steps instead of a full scan.
The trade is that every index has to be updated on every write that touches it. Indexes make reads faster and writes slower, and they take up space.
What to say in an interview: name the column you would index and the query it serves. "I would index short_key because every redirect is a lookup on it" is a complete answer.
13. Data Partitioning
When the data no longer fits on one machine, or one machine can no longer serve the traffic, you split it.
Horizontal partitioning, usually called sharding, splits the rows. Users A through M on one machine, N through Z on another. Each machine holds the same table shape with a different slice of the rows. This is the one that matters for scale.
Vertical partitioning splits the columns, moving rarely-used or very large columns into a separate table so the common queries read less.
The hard part of sharding is not doing it, it is choosing the key. A good partition key spreads both the data and the traffic evenly, and keeps the rows you read together on the same machine. A bad one gives you a hot shard that receives most of the traffic while the others sit idle.
What to say: name the key, then name what breaks. "I will shard by user id, which keeps each user's data on one machine. The cost is that a query across all users now has to hit every shard."
14. Replication
Replication keeps copies of the same data on more than one machine. Partitioning is about splitting data up; replication is about duplicating it. Real systems do both.
The usual arrangement has one primary that accepts writes and several replicas that copy from it. This gives you three things: reads can be spread across the replicas, a replica can be promoted if the primary dies, and the data survives a machine failure.
The decision to name is how the copying happens. Synchronous replication waits for the replicas to confirm before acknowledging a write, which costs latency and gives you consistency. Asynchronous replication acknowledges immediately and copies afterwards. This is faster, but a replica can serve slightly stale data, and a primary that dies at the wrong moment loses the writes that had not copied yet.
15. Consistency Model
Once the data exists in more than one place, you have to answer a question that has no default: after a write, what does the next read see?
Strong consistency means every read returns the most recent write, always. To get it, a write has to wait until the copies agree, so every write pays that waiting time and one slow replica slows everybody down.
Eventual consistency means a read is answered immediately and might briefly return the old value. The copies converge shortly afterwards, usually within milliseconds.
Neither is better. They are a trade-off: strong consistency gives you correctness and costs latency, eventual consistency gives you speed and leaves a window in which somebody can see stale data.
The question it answers: how fresh does a read have to be?
The thing candidates get wrong is treating this as one decision for the whole system. It almost never is. In the same design, an account balance needs strong consistency because showing the wrong number is unacceptable. The view counter on that page is fine a few seconds behind, and a follower count can lag by a minute without anyone noticing. Naming the parts that need strong consistency and the parts that do not is a far better answer than picking one and applying it everywhere.
How to say it: "Reads of the balance go to the primary, so they are strongly consistent. Everything else on the profile can be served from replicas, because nobody notices a few seconds of staleness there." That sentence tells the interviewer you know the trade-off you are making. There is more in Strong vs Eventual Consistency.
16. Blob and File Storage
Databases are bad at storing large files. A photo, a video, or a document should go into object storage such as Amazon S3, or a distributed file system, and the database should store only the metadata: who owns it, when it was created, and the path to the file.
This split shows up in almost every media-heavy design, and skipping it is a common mistake. The usual flow is that the file goes to object storage, the CDN serves it from there, and the database row just points at it.
Layer 4: How You Make It Fast
17. Caching
A cache is fast storage holding a copy of something expensive to produce. Check the cache first; on a miss, do the expensive thing and put the result in the cache for next time.
The reason caching gets its own layer is that it happens in more than one place. It appears at every step of the path you just walked:
- In the browser, so the request never leaves the device.
- At the CDN, so it never reaches your data center.
- At the API gateway, so it never reaches a service.
- In the application, in memory or in Redis or Memcached, so it never reaches the database.
- Inside the database, which keeps hot pages in memory on its own.
Every one of these is a chance to answer without doing the work.
The hard part of caching is never the reading. It is deciding what happens when the underlying data changes, and how the cache and the database are kept consistent with each other. Those strategies, and the failure modes of each, are in Cache-Aside vs Read-Through, Write-Through vs Write-Back.
Layer 5: How You Keep It Alive
This layer is what separates a design that looks senior from one that does not. Most candidates draw a system as though nothing ever fails.
18. Health Checks and Heartbeats
In a system spread across many machines, something has to notice when one of them stops working. The usual mechanism is a heartbeat: every server sends a small message at a fixed interval saying it is alive. If the messages stop for longer than some timeout, the system treats that server as dead, stops routing to it, and starts replacing it.
Load balancers do a version of this with active health checks, calling a known endpoint on each server and removing any that fail.
The number worth thinking about is the timeout. Too short and a brief network problem takes a healthy server out of rotation. Too long and users keep hitting a dead machine.
19. Checksums
Data can be corrupted in transit or on disk, by a failing drive, a network fault, or a bug. A checksum catches it. When data is stored, the system computes a short fingerprint of it, using a hash function such as SHA-256, and keeps it alongside. When the data is read back, the fingerprint is recomputed and compared. If they differ, the data is corrupt and can be fetched from another replica instead.
The point is that the system returns an error or a good copy, rather than silently handing corrupt data to a user.
20. Coordination Services
Some decisions have to be made once, by everybody, in agreement: which node is the leader, who holds a lock, what the current configuration is. Doing this correctly across machines that can fail and lose contact is genuinely difficult, so systems delegate it to a service built for the job. ZooKeeper, etcd, and Consul are the usual names.
When to mention one: leader election, distributed locking, service discovery, or holding a configuration that every node must agree on. If your design has none of those, you do not need one.
21. Observability
"Monitoring" is too narrow a word for this. There are three different signals and they answer three different questions.
Metrics are numbers over time: requests per second, error rate, latency at the ninety-ninth percentile, cache hit rate. They are cheap to keep and they are what alerts fire on. Metrics tell you that something is wrong.
Logs are one record per event, with detail. They tell you what exactly happened on the request that failed.
Traces follow a single request across every service it touched and show how long each hop took. They tell you where the time went, which is a question you cannot answer from one service's logs once you have more than a couple of services.
The question it answers: how do you know it is working, and how do you find out why it is not?
How to say it in an interview: name the two or three numbers you would watch for this system, not a generic list. For a URL shortener that is redirect latency at the ninety-ninth percentile, cache hit rate, and the error rate on link creation. That takes fifteen seconds and it is far better than "and we would add monitoring."
The order they get used in is worth knowing: metrics find the problem, traces find the service, logs find the cause.
Layer 6: Where the Data Goes Afterwards
There is a second path through the system, and no user is waiting on it. Reports, dashboards, recommendations and business metrics all need the same events your serving path produces, but they need them in bulk and they can tolerate being minutes or hours behind.
22. Analytics and the Data Pipeline
The rule that matters is this: analytics must never run against your production database. A report that scans a year of rows can block the store that is meant to be answering users in fifty milliseconds.
So the data takes a separate route:
- Events are emitted. Your services fire a small record for anything worth counting: a click, a view, a purchase. This is fire and forget, off the response path.
- An event stream collects them. Kafka or a managed equivalent absorbs the volume and holds the events so that several consumers can read them independently.
- Processing happens at two speeds. Stream processing produces numbers that are seconds old, which is what a live dashboard needs. Batch processing runs over much larger windows and produces the complete, accurate figures.
- The results land in a warehouse, a store built for scanning and aggregating rather than for serving single rows.
The question it answers: where do reports and metrics come from without slowing the system for users?
When it belongs in your answer: the moment the requirements mention analytics, recommendations, reporting, or anything a business person would want to look at. Whether you compute it in stream or in batch is the trade-off worth naming, and it is covered in Batch Processing vs Stream Processing.
Putting It Together
Here is the whole template applied to a photo sharing service, in the order you would draw it. Notice how short it is.
- DNS resolves the domain. One sentence.
- CDN serves every photo. This is most of the traffic, and it never reaches us.
- Load balancer in front of the application servers.
- API gateway handles authentication, with rate limiting on uploads so one account cannot flood the encoders.
- Two services, because their scaling profiles differ: an upload service and a feed service.
- Uploads go to object storage; the database stores only metadata.
- A queue handles thumbnail generation, so the upload returns immediately.
- Metadata database is a key-value or wide-column store, because the hot queries are lookups by photo id and by user id, and there are no joins.
- Sharded by photo id, replicated for availability, and eventually consistent: a photo appearing in a follower's feed a second late is not a problem, so we do not need strong consistency here.
- Cache in front of the metadata database for the hot working set.
- Observability: upload success rate, feed latency at the ninety-ninth percentile, and CDN hit rate.
- Analytics runs off an event stream, so view counts never touch the metadata database.
Twelve steps, and most of them are one line. What we did not use: full-text search (no search requirement), a coordination service (nothing needs leader election), real-time delivery (nobody needs a photo pushed to them the instant it is posted), and checksums (object storage does that for us).
Saying that last sentence out loud is worth more than drawing the boxes would have been. It tells the interviewer you considered them and rejected them for a reason.
Quick Reference
| Component | The question it answers | Skip it when |
|---|---|---|
| DNS | Where is this service? | Single region, no geographic routing |
| CDN | How do we serve static files cheaply and close to users? | No static or media content |
| Load balancer | How do we run more than one server? | Almost never skip it |
| API gateway | How does a client talk to many services? | You only have one service |
| Rate limiting | What stops one caller consuming everything? | A closed internal system with trusted callers |
| Reverse proxy | Who receives the request on the server's behalf? | Already covered by your LB or gateway |
| Services | Who does the work? | Never |
| Message queue | What can happen after we answer the user? | Everything is fast and synchronous |
| Real-time delivery | How does the client learn that something changed? | The client can just ask when it next loads |
| Notification service | How do we reach the user outside the app? | Not in the requirements |
| Full-text search | How do we find text by its contents? | No search requirement |
| SQL vs NoSQL | What shape is the data and how is it queried? | Never, this is always a decision |
| Index | How do we find a row without scanning? | Write-only workloads |
| Partitioning | What do we do when the data outgrows one machine? | The data fits comfortably |
| Replication | What happens when a machine dies? | Almost never skip it |
| Consistency model | How fresh does a read have to be? | Never, and rarely one answer for the whole system |
| Object storage | Where do large files go? | No files |
| Cache | How do we avoid doing the same work twice? | Write-heavy, low read repetition |
| Heartbeat | How do we notice a dead machine? | Single machine |
| Checksum | How do we notice corrupt data? | Rarely mentioned unless storage is the topic |
| Coordination service | How do nodes agree on one answer? | No leader, no lock, no shared config |
| Observability | How do we know it works, and why it does not? | Never |
| Analytics pipeline | Where do reports come from without slowing the system for users? | Nobody is asking for numbers |
Download the System Design Master Template (pdf) to keep beside you while you practice.
💡 In the interview: draw the template outward from the client, and narrate the layer you are in. "The request comes in through DNS and the load balancer, hits the gateway for auth, and reaches the feed service" gives the interviewer an overview before you explain any one part in detail. Then, before you finish, spend thirty seconds on what you deliberately left out. Candidates who name the components they did not need sound like they made choices. Candidates who draw every box sound like they memorized a diagram.
Key takeaway: the template is a menu of about twenty-two components arranged in six layers: how the request reaches you, where the work happens, where the data lives, how you make it fast, how you keep it alive, and where the data goes afterwards. Its value is not the list, it is the order. Walk it from the outside in so your explanation has a direction, attach every component you draw to a requirement you agreed at the start, and say plainly which parts of the menu this system does not need. No good design uses all of them, and the ones you leave out are worth naming.
Now let's solve our first system design problem: Designing a URL Shortening Service.
Jeremiah Stones
· a month ago
“Column family” is mostly legacy Cassandra terminology. The official glossary says a column family is called a table in CQL 3.
Ash Outadi
· 2 years ago
I can see it has something to do with Video/Images but I don't think it's explained in the course.
I suppose it must be related to "block" in terms of the location of the video in the distributed file storage?
Alex Zhang
· 3 years ago
I see both in a diagram, I don't know why both are linked.
Reading Progress
0%
On This Page
System Design Master Template (video)
How to Actually Use This
Layer 1: How the Request Reaches You
- Domain Name System (DNS)
- Content Delivery Network (CDN)
- Load Balancer
- API Gateway
- Rate Limiting
- A Note on Proxies
Layer 2: Where the Work Happens
- Services
- Asynchronous Work and Message Queues
- Real-Time Delivery
- Specialized Services
Layer 3: Where the Data Lives
- Choosing the Database
- Indexes
- Data Partitioning
- Replication
- Consistency Model
- Blob and File Storage
Layer 4: How You Make It Fast
- Caching
Layer 5: How You Keep It Alive
- Health Checks and Heartbeats
- Checksums
- Coordination Services
- Observability
Layer 6: Where the Data Goes Afterwards
- Analytics and the Data Pipeline
Putting It Together
Quick Reference