Back-of-Envelope Math

3. Calculation 1: Rate x Duration = Work in Progress

How many things are in progress at the same time? Multiply how fast they arrive by how long each one takes:

<center> things in flight = arrival rate x duration </center>

For example, say 300 requests arrive per second and each takes 2 seconds to complete. Then 300 x 2 = 600 requests are in progress at any moment. Each one holds a thread or a connection. Say your connection pool, the fixed set of database connections you keep open, has 200 slots. You now know there is a problem. You know it thirty seconds into the design, instead of during an outage.

The same formula works in reverse for queues. Say a queue holds 552,000 messages and consumers drain it, meaning process messages, at 400 per second. The backlog clears in 552,000 / 400 = about 1,400 seconds, or 23 minutes.

This is the most-used calculation in the course. It sizes thread pools, connection pools, bulkhead limits, and queue drain times. It has a formal name, Little's law, which is worth knowing for interviews.

4. Calculation 2: Availability Drops Along a Chain

When a request needs several parts to succeed, the availabilities multiply. The result is always lower than the weakest part:

<center> chain availability = availability of each part, multiplied together </center>

Five sequential dependencies at 99.9% each give 0.999^5, which is about 99.5%. There is a simpler shortcut for small failure rates: add the failure rates. Five parts x 0.1% = 0.5% failure. Same answer, no exponents needed.

Percentages are hard to interpret, so translate them into time:

Two design lessons follow directly from this math. First, every additional "nine" costs roughly 10x the engineering effort of the previous one. Second, a chain of dependencies can only lower the total. So the fix is not to make every part perfect. It is making the chain shorter, and removing parts from the must-succeed list. That is the core argument of the graceful degradation lesson, previewed here as arithmetic.

5. Calculation 3: Cache Hit Ratios

For any cache, two numbers follow from the hit ratio h, the share of requests the cache answers:

<center> average latency = h x (fast time) + (1 - h) x (fast time + slow time) </center> <center> load reaching the database = (1 - h) x request rate </center>

Run it with real numbers: 12,000 requests per second, 1 ms for a cache hit, 15 ms for a database read.

• At a 99% hit ratio: average latency is about 1.15 ms, and the database sees 1% x 12,000 = 120 queries per second.
• At a 90% hit ratio: average latency is about 2.5 ms, which is still fine. But the database sees 1,200 queries per second. That is ten times more.

💡 This is a trap worth naming in interviews: the difference between a 99% and a 90% hit ratio is not nine percent. It is 10x the database load. What matters is the miss rate, and it went from 1% to 10%. Judge caches by their misses.

This is also why the cache stampede problem is so severe. A stampede is many requests missing the cache at once. When a popular key expires, the hit ratio for that key briefly drops toward zero. The formula above shows what happens to the database.

6. Calculation 4: Sizing

Requirements arrive in per-day numbers, but systems are sized in per-second numbers. The conversion is one division:

<center> per second = per day / 100,000 </center>

For example, 10 million orders per day is about 100 orders per second, on average. Then apply the peak rule: real traffic is not flat, so plan for a peak of 3-5x the average. Here, size for 300-500 per second.

Storage is multiplication, using the sizes from Section 2:

<center> 10 million orders/day x 1 KB x 365 days = ~3.7 TB per year </center>

That fits on one ordinary database machine. Knowing this in ten seconds prevents a lot of unnecessary sharding, that is, splitting the data across machines.

7. Example: Sizing the URL Shortener

The previous lesson sketched the URL shortener design. Now size it, narrating out loud:

• "Say 100 million redirects per day. Divide by 100,000: that is 1,000 per second on average. Size for about 4,000 at peak."
• "Each mapping is two URLs plus metadata. Call it 1 KB. With 500 million total links, that is 500 GB. It fits on one database with capacity left over. We shard eventually, not today."
• "Reads outnumber writes about 100:1, and popularity is skewed. Caching the most popular 1% means 5 million rows, or about 5 GB of cache memory. At a 99% hit ratio the database sees about 10 redirect misses per second. That is a very small load."
• "In flight: 4,000 per second x 2 ms each = 8 requests in progress at once. This is a small service: one good database, a replica, a cache, and careful failure handling."

That is sixty seconds and five numbers. Every part of the design now has a number behind it. No part was added for load that does not exist.

8. Using Estimation in Interviews

1. State assumptions before calculating. "Assuming 1 KB per record and 10 million per day..." Interviewers will correct an assumption you state out loud. They cannot correct one you keep to yourself.
2. Round before multiplying, and check the units at the end. Per-second times seconds gives a count. Bytes times a count gives storage. If the units do not work out, the answer is wrong.
3. Let the numbers rule out extra components. One of the strongest sentences you can say in an interview is: "At this scale, we do not need that yet." That is the cost-awareness from the first lesson, backed by numbers.

9. TL;DR