Horizontal vs Vertical Scaling

Your server is running at 90 percent CPU and traffic is still growing. You have two options.

Buy a bigger machine. Or add a second machine.

That is vertical scaling against horizontal scaling, and almost every other decision in this chapter eventually leads back to it.

• Vertical scaling (scaling up) means giving one machine more resources: more CPU cores, more memory, faster disks.
• Horizontal scaling (scaling out) means adding more machines and spreading the work across them.

Vertical Scaling

You replace a 4-core machine with a 16-core machine. Nothing about your application changes. The same code runs on better hardware.

What it gives you

• No code changes. This is the real advantage and it is easy to undervalue. There is no load balancer to add, no session store to introduce, no data to partition.
• No distributed systems problems. One machine cannot disagree with itself. Transactions, joins and locks all work exactly as they did.
• Simple operations. One machine to deploy, monitor and debug. One set of logs.
• Often the cheapest fix in engineering time, which is a real cost even though it never appears on the infrastructure bill.

What it costs you

• A hard ceiling. There is a largest machine, and once you are on it there is nowhere left to go. You cannot buy your way past it.
• Price rises faster than capacity. Commodity hardware is cheap. High-end hardware is not. Doubling the cores usually costs more than double, so each unit of capacity gets more expensive as you go up.
• One failure domain. A bigger server is still one server. If it fails, everything is down. Vertical scaling adds capacity but adds no redundancy at all.
• Resizing usually means downtime, or at least a restart and a failover.

Horizontal Scaling

You keep the 4-core machines and run ten of them behind a load balancer.

What it gives you

• No practical ceiling. Need more capacity, add more machines. This is how every large system is built.
• Redundancy comes free. With ten servers, losing one costs you ten percent of capacity, not the service. Fault tolerance is a side effect of the shape.
• Cheaper per unit at scale, because commodity machines have the best price to performance ratio.
• Elasticity. You can scale down as well as up. Ten machines at peak and three overnight is only possible when capacity comes in units.
• Safer deploys. Rolling updates work because you can take one machine out at a time.

What it costs you

• You now need a load balancer, and it has to be made highly available itself.
• Your servers must be stateless, or every request has to return to the machine holding its state. This is the constraint that decides whether horizontal scaling is easy or painful.
• Distributed systems problems arrive. Machines fail independently, networks partition, clocks disagree, and requests arrive out of order. None of these existed on one machine.
• Operational complexity. Deploys, monitoring, log aggregation and debugging all get harder when a request may have touched any of fifty machines.
• Some work does not parallelise. Adding machines helps when work can be split. A single long computation that cannot be divided runs no faster on a hundred servers than on one.

The Part That Actually Matters: State

Here is the honest version of this trade-off, and the thing to say in an interview.

Stateless application servers scale horizontally almost for free. Put a load balancer in front, run as many as you like, add and remove them at will. If a server holds nothing that other servers need, there is no hard problem to solve.

Databases are where horizontal scaling gets difficult. A database holds state by definition, so you cannot just run more copies and call it done. The options are all more involved:

• Read replicas scale reads horizontally. Writes still go to one primary, and replicas lag behind it.
• Sharding scales writes horizontally by splitting data across machines by some key. Adding or removing a machine is its own problem here, because hashing a key against the number of servers moves almost every key the moment that number changes, and Consistent Hashing is the standard fix. This is the real work, because cross-shard joins and transactions become hard or impossible, and choosing a key that spreads load evenly is its own problem.
• Scaling the primary vertically is frequently the pragmatic answer, and it works for far longer than people expect. A well-tuned relational database on a large machine handles a great deal of traffic.

So the common shape of a growing system is: application tier scales out, database scales up until it cannot, then shards.

Choosing

In practice you do both, and the order matters. Scale up first, because it costs almost no engineering time. Design so you can scale out later, because eventually you will have to. Concretely, that means keeping application servers stateless from the first day even while you are still running one, so that adding the second one is a configuration change rather than a rewrite.

💡 In the interview: when scale comes up, do not just say "scale horizontally" and move on. Split the tiers. The strong version is one sentence: "App servers are stateless so they scale out behind a load balancer, and sessions live in Redis rather than in memory. The database is the harder question, so I would scale the primary vertically and add read replicas first, and only shard when writes outgrow one machine." Then expect the follow-up on the shard key, and be ready to say what you would shard by and why that spreads load evenly. If a candidate mentions horizontal scaling without ever addressing the database, that is the gap an interviewer will probe.

Key takeaway: vertical scaling gives one machine more resources, needing no code changes and introducing no distributed systems problems, but it has a hard ceiling, a cost curve that rises faster than capacity, and no redundancy. Horizontal scaling adds machines, giving effectively unlimited capacity, built-in fault tolerance and elasticity, at the cost of requiring stateless services and accepting distributed systems complexity. Stateless application tiers scale out easily; the database is where the real difficulty lives, which is why systems usually scale the app tier out and the database up until sharding becomes unavoidable.