System Design Building Blocks: The Components Every Scalable System Uses

Almost every large system is made from the same dozen or so parts. System design is choosing which parts a problem needs and connecting them in the right order.

The parts barely change between a chat app, a video site, and a payment service. What changes is which parts are present, how many copies exist, and where data may be a little old.

This article follows one request through a system. Then it describes each block: what it does, what it costs, and when you need it.

How a request moves through a system

Say you open a news feed on your phone. The phone asks a name service (DNS) for the address of a server. The request reaches a machine that spreads traffic across many servers (a load balancer).

An entry point checks your identity and request rate (an API gateway). One of many identical machines running the app code (an application server) then handles the request.

It looks for a ready answer in fast memory (a cache). It reads the database only when the cache has nothing. Slow work, like sending notifications, goes onto a waiting list (a queue) and runs after the response is sent.

DNS

People remember names like designgurus.io, but a connection needs a numeric address. The service that maps a name to an address is the Domain Name System (DNS).

It handles change. Servers get replaced and multiplied, and no user learns a new address.

The cost is one extra round trip before the first byte. Each answer is cached for a set time (a time to live, or TTL). So address changes reach users slowly.

You always need it. Lesson: Introduction to DNS.

Load balancer

One server can handle only so many requests a second. A machine that receives every request and spreads them across many servers (a load balancer) removes that limit.

It also removes failed servers from service. A server that stops answering a small test request (a health check) gets no traffic.

The cost is one more machine on every path. It is also a part whose failure stops everything (a single point of failure), so balancers run in pairs.

You need one from the second application server onward. Lesson: Introduction to Load Balancing.

API gateway

A single entry point that checks every outside request before it reaches your code is an API gateway. It confirms who the caller is (authentication), limits the request rate, and routes the call to the right service.

It solves repeated work. Without it, every service checks identity on its own.

The cost is another machine on the path, and one place where a bad setting blocks all traffic.

You need one when several services or client types share one backend. Lesson: Introduction to API Gateway.

Rate limiter

A client can send far more requests than you planned for. A rule that caps how many requests one client may make in a time window (a rate limiter) protects the system. A common rule is 100 requests per minute per user.

One client cannot use a whole shared resource, and password guessing becomes too slow to be useful.

The cost is that some honest bursts are rejected. The counters must also be shared across every server.

You need one on any public API and any login form. Lesson: What is Rate Limiting.

Cache

Reading the same row from the database a million times a day is wasted work. A copy of recent results kept in fast memory (a cache) answers repeated reads instead. Redis and Memcached are common choices.

A read from memory takes a fraction of a millisecond, and the database never receives the request.

The cost is a copy that no longer matches the source (stale data). Every entry needs an expiry rule, and that rule is the hard part.

You need one when data is read far more often than written. Lesson: Introduction to Caching.

CDN

A user in Sydney fetching an image from Virginia waits for light to cross the Pacific. A network of servers near users that holds copies of files (a content delivery network, or CDN) removes most of that wait.

Images, video, and scripts are the same for every user, so a nearby server can serve them.

The cost is money per gigabyte and the same staleness as any cache. A changed file stays old on every CDN server until it expires.

You need one as soon as you serve media to more than one region. Lesson: What is CDN.

The six blocks so far are on the request path. The next six hold the data. System Design Fundamentals teaches every block here, one chapter per block, with flashcards and an assessment after each.

Database: SQL and NoSQL

A database that keeps data in tables with fixed columns (a relational or SQL database) is the default store. It can combine rows from two tables in one query (a join). It can also run several writes as one unit (a transaction).

A database that drops some structure to spread across many machines (a NoSQL database) is the other family. It stores documents, key-value pairs, or graphs.

SQL is harder to split across servers. NoSQL splits easily but often gives up joins and the promise that every reader sees the newest write (strong consistency).

Start with SQL unless the data will not fit on one machine or has no fixed shape. Lesson: SQL vs NoSQL.

Index

Finding one row among a billion by reading every row takes minutes. A separate structure that maps each value in a column to its rows (an index) finds it in a few disk reads.

It solves read speed. A lookup by user id stays fast no matter how large the table grows.

The cost is paid on every write. Each insert or update must also update every index on the table.

You need one on every column you filter or join on often. Lesson: What are Indexes.

Data partitioning (sharding)

One database server has a limit on disk, memory, and writes per second. Splitting one large data set across several servers (data partitioning, or sharding) removes that limit. Each part is a shard, and a key like user id decides which shard holds a row.

A query that needs several shards must ask each one and merge the answers. One very popular key can overload a single shard (a hot shard).

Do it only when one server cannot hold the data or handle the writes. Lesson: Introduction to Data Partitioning.

Replication

A single copy of your data is lost when its disk fails. Keeping the same data on two or more servers (replication) protects it. Each extra copy is a replica.

Reads spread across replicas, and when one server fails another already holds the data.

Writes go to one server and are copied to the rest. The delay before a copy has the newest write (replication lag) means a reader may see old data.

You need it for any data you cannot afford to lose. Lesson: What is Replication.

Message queue

Some work does not need to finish before the user gets a response. A buffer that holds a task until another part of the system picks it up (a message queue) separates the two.

Ten thousand orders a second can wait in the queue while workers process them steadily. Kafka and RabbitMQ are common choices.

The cost is delay and duplicates. Most queues deliver a message at least once, so a reader may see one message twice.

You need one when a request triggers work that is slow, can fail, or goes to many receivers (fan-out). Lesson: Introduction to Messaging System.

Distributed file system

Videos, backups, and logs do not fit in a database or on one disk. A store that spreads large files across many machines and shows them as one file system (a distributed file system) holds them. Each file is cut into chunks, each copied to several machines.

Storage grows by adding machines, and a lost disk loses nothing.

The cost is a directory that must know where every chunk is (a metadata server). It can become the slowest part (a bottleneck), and small files are handled badly.

You need one when files are large or many. Lesson: What is a Distributed File System.

Reliability patterns

Four smaller patterns keep the blocks above working when a machine fails.

Quorum. The minimum number of replicas that must agree before an operation counts as done is a quorum (quorum lesson). With W writes and R reads out of N copies, W plus R above N means every read sees the latest write.

Leader and follower. One replica accepts all writes (the leader) and the rest copy from it (the followers). Writes then happen in one order, and when the leader fails a follower is promoted (leader and follower lesson).

Heartbeat. A small message a server sends at a fixed interval to say it is still running is a heartbeat (heartbeat lesson). When the messages stop, the other servers treat that server as failed.

Checksum. A short value computed from a block of data and stored beside it (a checksum) detects corruption (checksum lesson). If the value computed on read differs, the reader fetches another copy.

The blocks side by side

How the blocks combine: a news feed

Take a social app's news feed. DNS returns the address of the nearest entry point. The load balancer passes the request to the API gateway, which checks identity and the rate limit.

An application server asks the cache for that user's feed, built earlier. On a miss, it reads posts from a database sharded by user id and replicated three times. Images and videos come from a CDN backed by a distributed file system.

When a user posts, the write goes to the leader of that user's shard. A message goes onto a queue, and workers add the post to the cached feed of every follower (fan-out on write).

The blocks are the same in every system. The design is in the choices: cache each feed or build it on read, shard by user or by post.

Grokking the System Design Interview makes those choices in its news feed, chat app, and URL shortener case studies. This comparison says which course to open first.

Frequently asked questions

What are the building blocks of system design? The path blocks are DNS, load balancers, API gateways, rate limiters, caches, and CDNs. The data blocks are databases, indexes, shards, replicas, message queues, and distributed file systems. Reliability patterns like heartbeat and quorum keep them working when machines fail.

What are the components of a scalable system? A scalable system handles more load by adding machines. Its main components are a load balancer, many application servers, a cache, and a sharded, replicated database. A queue holds slow work, and a CDN serves distant users.

Do I need every building block in every system? No, one server and one database are enough for a small load. Add a block only when a measured problem needs it, like a database that is too slow for its reads.

How is this different from a list of system design concepts? A concept list covers ideas like consistency, availability, and trade-offs. System Design Fundamentals: 25 Core Concepts is that list. Building blocks are the physical parts a request passes through.

Related reading

• System Design Fundamentals: 25 Core Concepts Explained for Beginners
• The 4 Basic Pillars of System Design
• Load Balancer vs Reverse Proxy vs API Gateway
• Learn System Design With No Distributed Systems Experience
• System Design Fundamentals vs Grokking the System Design Interview: Which to Take First