Grokking the System Design Interview

0% completed

Horizontal vs Vertical Scaling

On This Page

Two Ways to Add Capacity

Vertical Scaling

What Vertical Scaling Gives You

What Vertical Scaling Costs You

Horizontal Scaling

What Horizontal Scaling Gives You

What Horizontal Scaling Costs You

State Decides How Hard Scaling Out Is

Databases Are the Hard Part

The Usual Scaling Path

The Two Options Side by Side

Choosing Between Them

Using This in an Interview

Key Takeaways

Practice Questions

A photo sharing website runs on one server with 8 CPU cores. Every evening, when most users are online, the CPU reaches 90 percent, and pages start to load slowly. Traffic is also growing by about 10 percent every month.

The team has two choices. They can replace the server with a bigger one. Or they can add more servers and share the work between them.

These two choices are called vertical scaling and horizontal scaling. This lesson explains both, what each one costs, and why most real systems use both, in a specific order.

Two Ways to Add Capacity

Vertical scaling, also called scaling up, means giving one machine more resources. For example, you add more CPU cores, more memory, or faster disks.

Horizontal scaling, also called scaling out, means adding more machines and spreading the work across them.

In short: vertical scaling makes one machine bigger, and horizontal scaling adds more machines.

Vertical scaling replaces the 8-core server with a 32-core server, while horizontal scaling puts ten 8-core servers behind a load balancer

Vertical Scaling

The team replaces the 8-core server with a 32-core server. The application code does not change. The same code simply runs on stronger hardware.

What Vertical Scaling Gives You

No code changes. This is its biggest advantage. There is no load balancer to add, no shared session store to set up, and no data to split.
No distributed system problems. One machine holds the only copy of the data, so no second copy can disagree with it. Transactions, joins, and locks keep working exactly as before.
Simple operations. There is one machine to deploy, monitor, and debug, and one set of logs to read.
Low engineering cost. Engineers do not spend weeks redesigning the system. Their time is a real cost, even though it does not appear on the server bill.

What Vertical Scaling Costs You

A hard limit. There is a largest machine you can buy. Once you run on it, more money cannot buy more capacity from vertical scaling. This physical limit of a single machine is the main weakness of vertical scaling.
High prices at the top end. At common sizes, cloud prices grow roughly in proportion to machine size. But the largest and most specialized machines are very expensive, and you may pay much more for each extra unit of capacity.
No redundancy. Redundancy means having a second copy that can take over after a failure. A bigger server is still one server. It is a single point of failure, which is a part whose failure stops the whole system.
Downtime to resize. Moving to a bigger machine usually requires a restart, or at least a switch to a standby machine. Users may see an interruption.

Horizontal Scaling

Instead of one big machine, the team runs ten 8-core servers. A load balancer, which is a server that spreads incoming requests across other servers, sits in front of them. The Load Balancing lesson explains how it works.

What Horizontal Scaling Gives You

No practical limit. When more capacity is needed, the team adds more servers. Very large systems are built this way.
Built-in redundancy. If one server out of ten fails, the system loses 10 percent of its capacity, and the service keeps running.
Easy to scale down. The team can run ten servers at the evening peak and three servers at night. Paying only for the servers in use saves money.
Safer deployments. The team can update one server at a time, while the others keep serving users.
Ordinary hardware. Many standard machines are usually cheaper than one very large, specialized machine with the same total capacity.

When one big server fails the service is down, but when one of ten servers fails, 90 percent of capacity remains

What Horizontal Scaling Costs You

A load balancer to run. The load balancer must also be highly available, or it becomes the new single point of failure.
State becomes a problem. The next section explains this.
Data must be copied or split. Application servers are easy to multiply. Databases are not.
Failures become normal events. Suppose each server fails about once every three years, which is about 1,095 days. With 1,000 servers, about one server fails every day. The system must handle failures automatically.
Harder operations. A request may reach any of many servers. Deployments, monitoring, logs, and debugging all become harder.
Some work cannot be split. A single long calculation that cannot be divided runs no faster on a hundred machines than on one. Also, if every write must pass through one machine, that machine limits the whole system.

State Decides How Hard Scaling Out Is

State is data that a server remembers between requests, like a user's login session.

Suppose a user logs in, and server A stores the session in its own memory. The next request goes to server B. Server B has no session for this user, so the user appears to be logged out.

The fix is to make the application servers stateless. A stateless server keeps no user data in its own memory between requests. Instead, it stores sessions in a shared store, like Redis, which every server can read. Now any server can handle any request.

A session kept in one server memory is lost when the next request reaches another server, while a session in a shared store works on any server

Stateless application servers scale horizontally with very little extra work. You put a load balancer in front, and you run as many servers as you need. The Stateful vs Stateless Architecture lesson covers this in depth.

Databases Are the Hard Part

A database exists to hold state, so simply running more copies is not enough. Teams usually use these options, in this order.

1. Read replicas. A read replica is a copy of the database that answers read queries. All writes still go to one main database, called the primary. The primary copies each change to the replicas, usually with a small delay. This is called replication lag.

Read replicas fit read-heavy systems very well. Suppose a system serves 100 reads for every write. Each new replica adds more read capacity, so read capacity grows with every machine added. But replicas do not help with writes.

2. A bigger primary. For more write capacity, the simplest step is often to scale the primary database up. A well-tuned database on a large machine can handle a very large load, often for longer than teams expect.

3. Sharding. Sharding splits the rows of a table across several databases, based on a shard key, like the user ID. Each database is called a shard, and it holds only part of the data. Sharding is the step that finally scales writes, and it is also the most expensive step.

Queries across shards become hard. A query that needs data from every shard must ask all of them and combine the results.
Transactions across shards become much harder. Traditional SQL databases give ACID guarantees, which keep each transaction correct and complete. Keeping those guarantees across many shards is complex and slow.
A bad shard key creates uneven load. For example, sharding by country puts all users from a very large country on one shard. That shard receives much more traffic than the others.
Adding a shard moves data. The Consistent Hashing lesson shows how to keep that movement small, and Data Partitioning covers the common schemes.

Teams scale a database with read replicas first, then a bigger primary, and sharding last

The Usual Scaling Path

Real systems rarely choose only one type of scaling. They usually follow the same path.

One machine. The application and the database run on one server.
Scale up. Move to a bigger machine. This costs money, but no engineering time.
Scale the application tier out. Make the application servers stateless, put them behind a load balancer, keep sessions in a shared store, and add read replicas.
Shard the database. Do this only when the write load is too large for the biggest reasonable primary database.

The usual scaling path: one machine, then a bigger machine, then a stateless application tier, and sharding last

The rule: scale up first, because it is fast and needs no code changes. But design so that you can scale out later, because a growing system eventually must scale out. In practice, keep application servers stateless from the first day, even while you run only one. Then adding a second server is a configuration change, not a redesign.

The Two Options Side by Side

	Vertical scaling	Horizontal scaling
What changes	The size of the machine	The number of machines
Code changes	None	Stateless servers, and often more
Limit	The largest machine you can buy	No practical limit
Cost	Very high at the top end	Grows with the number of machines
Redundancy	None	Built in
Scaling down	Hard	Easy
One failure means	The whole service is down	Some capacity is lost
Best for	Databases, early growth, quick fixes	Stateless services, large scale

Choosing Between Them

The system is small, early, or needs capacity urgently: scale up. It gives months of growth with no engineering work.
A stateless service has growing traffic: scale out. This is the easy case.
The database needs more capacity: scale up first, add read replicas for reads, and shard only when writes require it.
High availability is required: one machine cannot survive its own failure. You need at least two of every important component, which means scaling out.
The work cannot be divided: scale up. More machines do not help a job that only one machine can run.

Using This in an Interview

Do not say "we scale horizontally" and move on. Talk about each tier separately. A strong answer takes two sentences:

"The application servers are stateless and scale out behind a load balancer, with sessions in a shared store like Redis. The database is harder, so I scale the primary up, add read replicas, and shard only when writes outgrow one machine."

Expect a follow-up question about the shard key. Name one, explain why it spreads the load evenly, and state its weakness. For example, sharding by user ID spreads users evenly across shards. But one extremely popular user still sends all of their traffic to one shard.

Key Takeaways

Vertical scaling (scaling up) gives one machine more resources. Horizontal scaling (scaling out) adds more machines.
Vertical scaling needs no code changes. But it has a hard physical limit, becomes very expensive at the top end, often needs downtime, and has no redundancy.
Horizontal scaling has no practical limit and has built-in redundancy. But it needs a load balancer, stateless servers, and automatic handling of failures.
State decides the difficulty. Stateless application servers scale out easily. Databases do not.
For databases, use read replicas for reads, a bigger primary for writes, and sharding last. Sharding makes cross-shard queries and transactions hard.
The usual path is: scale up first, then scale the application tier out, and shard the database last.

Most growing systems use both types of scaling: bigger machines where splitting is hard, and more machines where the work splits easily. The next lesson, Proxy vs. Reverse Proxy, compares two kinds of servers that sit between clients and the servers behind them.

Practice Questions

Try each question first, then open the answer.

1. A server runs at 85 percent CPU at peak time. The team needs twice the capacity within one week. The application stores user sessions in server memory. Should the team scale up or scale out first?