Key System Design Patterns to Know Before a Big Tech Interview

1. Cache-Aside

The problem: Your database is serving the same expensive read thousands of times per second.

The move: The application checks the cache first. On a miss, it loads from the database and stores the result with a time-to-live (TTL). Reads that hit the cache never touch the database.

The price: Cached data goes stale, and invalidation is your job. Two numbers govern the design: the hit ratio (how often you avoid the database) and the staleness budget (how wrong the data is allowed to be). The TTL is your safety net, not your strategy.

The 30-second answer: "I would put a cache-aside layer in front of the read path. Reads check Redis first and fall back to the database on a miss. I would set the TTL from how stale this data is allowed to be, which for a product catalog might be minutes and for account balances is zero. I would also add jitter to expiry times so a thousand keys do not expire at the same instant."

Follow-up they will ask: What happens when a popular key expires and a thousand requests miss at once? The answer is cache stampede prevention: let one request rebuild while the others wait, and refresh hot keys before they expire.

2. Sharding

The problem: Your writes or your data size have outgrown one machine. Replicas do not help, because every replica takes every write.

The move: Split the rows across machines by a shard key. Each machine owns a slice of the data and handles the writes for that slice.

The price: The shard key is the entire decision. It should appear in nearly every query, spread load evenly, and keep related rows together. Get it wrong and you inherit cross-shard queries, cross-shard transactions, and hot keys that put one machine back at 100% while the rest idle.

The 30-second answer: "I would shard on user ID, because almost every query in this system is scoped to a single user, which keeps reads on one shard. I would use consistent hashing so adding a shard moves only its slice of data instead of most of it. The trade-off I am accepting is that any query spanning all users, like global analytics, has to fan out to every shard, so I would serve those from a separate read model instead."

Follow-up they will ask: What happens when one user gets huge, like a celebrity account? Answer: that is a hot key, and you handle it by splitting that key across shards or serving it from a dedicated cache.

3. Message Queues

The problem: Work is arriving faster than you can process it, and the user is waiting on work they do not need to wait for.

The move: Put the work in a queue. Workers consume at their own pace. Traffic spikes become backlog instead of outages.

The price: Delivery is at-least-once, so consumers must tolerate duplicates. Queue depth becomes a metric you have to watch and alert on, and a queue that only grows is an outage you have not noticed yet.

The 30-second answer: "Sending the confirmation email does not need to happen before the user sees their order. I would publish an event to a queue and let a worker handle it. That way a slow email provider cannot slow down checkout. I would size the consumer pool from the arrival rate, alert on backlog age rather than backlog size, and send messages that keep failing to a dead letter queue so one bad message cannot block the ones behind it."

Follow-up they will ask: What if the same message is delivered twice? That is the next pattern.

4. Idempotency

The problem: A request timed out. The client does not know whether it succeeded, so it retries. The customer gets charged twice.

The move: The client generates a unique key per operation and sends it with every attempt. The server records keys it has already processed and returns the original result instead of doing the work again.

The price: You need somewhere to store those keys, and you need to decide how long to keep them. The window has to be longer than the longest retry chain in your system.

The 30-second answer: "Every payment request carries a client-generated idempotency key. Before processing, the service checks whether it has seen that key. If it has, it returns the stored result instead of charging again. This is what makes retries safe, and a timeout is exactly the case where you need it, because a timeout means I stopped waiting, not that the work did not happen."

Why it matters: This is the pattern most candidates skip and most interviewers are listening for. Any time you mention retries, mention idempotency in the same breath.

5. Circuit Breakers

The problem: A downstream dependency is slow or failing. Your threads pile up waiting on it, and your service dies alongside it.

The move: Track the failure rate. When it crosses a threshold, stop calling the dependency and fail immediately. After a cooldown, let a probe request through to see whether it has recovered.

The price: You have to decide what to do while the circuit is open, and "return an error" is often the wrong answer. Pair it with a fallback: cached data, a smaller response, or a default.

The 30-second answer: "I would wrap the recommendations service in a circuit breaker. If more than half of calls fail over a rolling window, the breaker opens and we serve a generic popular-items list instead of waiting on a dead service. This turns a slow cascading failure into a fast contained one. The homepage still loads, just with a less personalized module."

Related pattern worth naming: Bulkheads. Separate connection pools per dependency so one slow caller cannot consume every thread in the process.

6. Sagas

The problem: A business transaction spans several services that do not share a database. You cannot wrap them in one database transaction.

The move: Break it into a sequence of local transactions, each with a designed compensating action. Book the flight, then book the hotel. If the hotel fails, run the compensation that cancels the flight.

The price: You are trading atomicity for availability. There are windows where the system is partly committed, and the compensations are real business logic you have to design, not a rollback you get for free.

The 30-second answer: "Booking spans the flight service, the hotel service, and payments, which have separate databases, so a distributed transaction is not available. I would model it as a saga: each step commits locally and publishes an event that triggers the next. If a later step fails, we run compensations in reverse. I would order the irreversible steps last, so charging the card happens after everything reversible has already succeeded."

Follow-up they will ask: Why not two-phase commit? Because 2PC blocks every participant if the coordinator dies at the wrong moment, and it does not work across organizational boundaries.

7. CQRS

The problem: Your reads and your writes want different data shapes. Normalizing for correct writes makes reads slow, and denormalizing for fast reads makes writes error-prone.

The move: Split the model. Keep a lean, validated write model, and build separate read models shaped for each question you need to answer: a pre-joined table, a search index, an aggregates store. Keep them in sync from the write side's change stream.

The price: The read models are eventually consistent, so a user can write something and not immediately see it. You also now own the synchronization pipeline.

The 30-second answer: "The write path needs to validate and store orders correctly, and the read path needs an order history joined across four tables. I would keep the normalized write model and project a denormalized read model from its change stream. The read model is eventually consistent, usually within a second, and I would handle the read-your-own-writes case by serving the user their own recent order from the write side."

The mistake to avoid: Never keep the two models in sync by writing to both from application code. Dual writes drift silently the first time one of them fails.

8. Rate Limiting

The problem: One client, script, or bad actor can consume capacity that belongs to everyone else.

The move: Cap requests per client per unit of time. Token bucket is the usual choice because it allows short bursts while enforcing a sustained average.

The price: The counter has to be shared across your whole fleet, which means a round trip to a store like Redis on every request, or approximate local counters that let clients slightly exceed the limit.

The 30-second answer: "I would rate limit per API key using a token bucket in Redis, which allows bursts up to the bucket size while capping the sustained rate. I would return a 429 with a Retry-After header so well-behaved clients back off correctly. The limit lives at the API gateway so every service behind it is protected without implementing this individually."

Follow-up they will ask: How do you keep the rate limiter itself from becoming the bottleneck? Answer: local counters synced periodically, accepting a small overshoot in exchange for removing the network hop.

How to Use These 8 in an Interview

Knowing the patterns is not the same as scoring with them. Three habits separate strong candidates:

1. Name the problem before the pattern. "Reads are overwhelming the database, so I would add a cache" scores. "I would add Redis" does not. The interviewer is checking whether you can diagnose, not whether you can list technologies.
2. Say the cost in the same breath. Every pattern is a purchase. Sharding buys write scale and costs you cross-shard queries. Caching buys speed and costs you staleness. A candidate who names the price sounds senior; a candidate who only names the pattern sounds like they read a list.
3. Handle the seams. Retries demand idempotency. Caches demand a stampede plan. Sagas demand designed compensations. The connections between patterns are where interviews are actually won.

A pattern you cannot argue against is a pattern you do not understand. Be ready to say when not to use each one: sharding before you need it, CQRS on a settings page, and event sourcing everywhere are the classic ways to pay real complexity for imaginary scale.