0% completed
Horizontal vs Vertical Scaling
On This Page
Vertical Scaling
Horizontal Scaling
State Is the Deciding Cost
The Usual Path
The Two Options Side by Side
Choosing
Your one server is at 90 percent CPU, and traffic is still growing. You have exactly two options. Buy a bigger machine, or add a second machine.
That is the whole decision. Vertical scaling, also called scaling up, means giving one machine more resources: more CPU cores, more memory, faster disks. Horizontal scaling, also called scaling out, means adding more machines and spreading the work across them.
The question this lesson answers: when is each one right, and what does each one cost you?
Vertical Scaling
You replace the 4-core machine with a 16-core machine. Your application does not change. The same code runs on better hardware.
What it gives you.
- No code changes. This is the real advantage, and it is easy to undervalue. No load balancer to add, no session store to introduce, no data to split.
- No distributed systems problems. One machine holds the only copy of the data, so copies cannot disagree. Transactions, joins, and locks keep working exactly as they did.
- Simple operations. One machine to deploy, monitor, and debug. One set of logs.
- Cheap in engineering time. Engineer hours are a real cost, even though they never appear on the infrastructure bill.
What it costs you.
- A hard ceiling. There is a largest machine for sale. Once you run on it, more money buys nothing.
- Price grows faster than capacity. Ordinary hardware is cheap, and top-end hardware is not. Doubling the cores usually more than doubles the price.
- No redundancy. Redundancy means having a second copy that can take over after a failure. A bigger server is still one server. When it fails, everything is down.
- Resizing usually means downtime, or at least a restart and a switch to a standby machine.
Horizontal Scaling
You keep the 4-core machines and run ten of them behind a load balancer, a server that spreads incoming requests across the machines.
What it gives you.
- No practical ceiling. Need more capacity, add more machines. Every large system is built this way.
- Redundancy comes built in. Lose one server out of ten, and you lose ten percent of capacity, not the service.
- Cheaper per unit at scale. Many ordinary machines cost less per unit of capacity than one giant machine.
- You can scale down too. Ten machines at peak and three overnight. Capacity that comes in units works in both directions.
- Safer deploys. Update one machine at a time while the others keep serving.
What it costs you.
- A load balancer to run, and it must be made highly available itself.
- State becomes a problem. State is the data a server remembers between requests, such as a user session. If a server keeps state in its own memory, every later request must return to that exact machine. So your servers must become stateless, keeping no user data between requests. Stateful vs Stateless Architecture covers this constraint in depth.
- Data must be copied or split. The application tier multiplies easily. The data tier does not, and the next section explains why.
- Machine failures become normal events. With fifty machines, something is always broken, and the network between your own machines can fail too. The system must keep working through both.
- Harder operations. A request may touch any of fifty machines. Deploys, monitoring, logs, and debugging all get harder.
- Some work does not split. A single long computation that cannot be divided runs no faster on a hundred machines than on one. If every write must pass through one machine, that machine sets the limit, no matter how many others you add.
State Is the Deciding Cost
Stateless application servers scale horizontally with almost no extra work. Put a load balancer in front and run as many as you want. If a server holds nothing the other servers need, there is no hard problem left.
Databases are where horizontal scaling gets difficult. A database exists to hold state, so running more copies is not enough. The options, in the order teams usually reach them:
- Read replicas are copies of the database that serve reads. Writes still go to one primary machine, and the replicas run slightly behind it. This scales reads only.
- A bigger primary is often the practical answer for writes, and it works longer than people expect. A well-tuned database on a large machine handles a very large load.
- Sharding splits the data itself across machines by a key, such as the user id. This is what finally scales writes, and it is the expensive step. Queries that cross shards get hard, transactions across shards get harder, and a badly chosen key piles the load onto one shard. Adding a machine also forces data to move between shards. Consistent Hashing exists to keep those moves small, and Data Partitioning covers the schemes.
The Usual Path
Real systems rarely pick one side and stop. They follow a path, and the order matters.
- One machine. Everything runs on it.
- Scale up. A bigger machine, at the cost of money and no engineering time.
- Scale the app tier out. Stateless servers behind a load balancer, sessions in a shared store, read replicas for the database.
- Shard the database. Only when write volume outgrows the biggest sensible primary.
The rule inside the path: scale up first, because it costs no engineering time. And design so you can scale out later, because eventually you must. Concretely, keep application servers stateless from the first day, even while you run only one. Then adding the second server is a configuration change, not a rewrite.
The Two Options Side by Side
| Vertical | Horizontal | |
|---|---|---|
| What changes | The size of the machine | The number of machines |
| Code changes | None | Statelessness, and usually more |
| Ceiling | The largest machine for sale | No practical limit |
| Cost curve | Rises faster than capacity | Close to linear |
| Redundancy | None | Built in |
| Scale down | Hard | Easy |
| A failure means | Total outage | Partial capacity loss |
| Best for | Databases, early growth, quick fixes | Stateless services, large scale |
Choosing
- Early, small, or urgent: scale up. It buys months of growth for zero engineering time.
- A stateless service with growing traffic: scale out. This is the easy case, so take it.
- The data tier: scale up first, add read replicas for reads, and shard only when writes demand it.
- Availability is a requirement: one machine cannot survive its own failure. You need at least two of everything that must stay up, which means scaling out.
- Work that cannot be divided: stays vertical. More machines do not help a job that only one machine can run.
💡 In the interview: do not say "we scale horizontally" once and move on. Split the tiers. The strong version is two sentences: "App servers are stateless and scale out behind a load balancer, with sessions in a shared store like Redis. The database is harder, so I scale the primary up, add read replicas, and shard only when writes outgrow one machine." Expect the follow-up on the shard key. Name one, say why it spreads load evenly, and admit its weakness. Shard by user id, and load spreads across users, but one extremely popular user still lands on one shard. A candidate who never addresses the database is leaving the exact gap the interviewer will probe.
Key takeaway: vertical scaling gives one machine more resources. It needs no code changes and creates no distributed systems problems. But it has a hard ceiling, a price that grows faster than capacity, and no redundancy. Horizontal scaling adds machines, which removes the ceiling and builds in fault tolerance. The price is stateless services, split data, and failures as routine events. State decides the difficulty: application tiers scale out easily, and databases do not. So the usual path is app tier out, database up, then read replicas, and sharding last.
On This Page
Vertical Scaling
Horizontal Scaling
State Is the Deciding Cost
The Usual Path
The Two Options Side by Side
Choosing