Sharding for System Design Interviews

01Why Sharding Is Unavoidable, and Why It's Brutal

At some point, your data exceeds what a single machine can hold. Or your write throughput exceeds what a single primary can sustain. Or your read load exceeds what replicas can absorb. When that happens, you shard: split the data across multiple machines, each holding a piece.

This is unavoidable at scale. It is also brutal. Sharding solves the resource ceiling but introduces a long list of new problems: hot keys, cross-shard queries, resharding pain, transactional limits, operational complexity. The reason sharding shows up in every senior interview is that the hard parts are subtle, and candidates who haven't operated a sharded system stumble in predictable ways when probed.

This page covers the decisions you'll make and the failure modes you'll encounter. The shard key is the center of gravity. Most other sharding decisions are downstream of choosing it well. Get the shard key right and the rest is tractable. Get it wrong and you'll spend years recovering.

02What Sharding Actually Does (and Doesn't)

Sharding does three things, and only three things:

1. Distributes storage. Each shard holds a fraction of the total data. If your dataset exceeds a single machine's disk, sharding is the resolution.
2. Distributes write load. Each shard handles writes for its portion of the data. If a single primary can't keep up with write throughput, sharding spreads it.
3. Enables horizontal scaling. When you need more capacity, you add more shards. The resource ceiling moves from "scale up the machine" to "add more machines."

Notice what's not on this list. Sharding does not, by itself:

• Improve read latency. A single read still goes to one shard, with the same disk and CPU costs. If anything, sharding adds the small overhead of routing the request.
• Fix availability. A shard that goes down still loses its share of the data. Replication fixes availability; sharding does not.
• Solve data-skew problems. If your data is naturally skewed (some users have 1000x more activity than others), sharding may concentrate the skew rather than spread it. The hot-key problem we cover in Section 5 is exactly this.
• Replace good database choice. Sharding a Postgres instance gives you sharded Postgres with all of Postgres's per-shard limits. If the underlying database is wrong for the workload, sharding doesn't fix it.

The interview move when sharding comes up is to name what it does and what it doesn't. "Sharding solves storage and write throughput. We still need replication for availability and we still need a good shard key to avoid concentration." That sentence positions you as someone who has thought about the boundaries of the technique.

03The Shard Key: The Most Important Decision

The shard key is the column or attribute that determines which shard a record lives on. Choose user_id as the shard key, and all records for a given user land on the same shard. Choose tweet_id, and tweets are scattered across shards regardless of who wrote them.

This is the single most consequential decision in sharding. Three reasons it deserves the time:

1. Changing it later is brutal. Resharding is hard enough; changing the shard key fundamentally is harder. Companies that picked the wrong shard key spend years recovering.
2. It determines query performance. Queries that filter by the shard key go to one shard. Queries that don't go to all shards (a "scatter-gather"), which is dramatically more expensive.
3. It determines concentration. If most activity in your system involves one shard key value (one user, one country, one tenant), that shard becomes a hot key and everything else slows down.

What makes a good shard key

Three properties to optimize for:

• High cardinality. Many distinct values. Sharding by gender (two values) or country (~200 values) gives you only that many possible shards. Sharding by user_id (millions of values) gives you flexibility.
• Even distribution. Each shard key value should account for roughly similar amounts of traffic and storage. A user_id is decent here; a country is awful (the US dominates traffic for many products).
• Aligned with access patterns. Most queries should filter by the shard key. If your queries usually filter by user_id, sharding by user_id keeps queries on a single shard. If your queries usually filter by date but you sharded by user_id, every query becomes scatter-gather.

Common shard key choices

• user_id for user-centric products (most consumer apps). High cardinality, generally even distribution, aligns with most product queries.
• tenant_id for multi-tenant SaaS. Each customer's data lives on a shard, queries within a tenant stay local. Risk: very large customers create hot shards.
• geo_region for products with geographic locality. Aligns with data sovereignty requirements and reduces cross-region latency. Risk: heavily uneven distribution.
• Hash of id for products without natural locality. Maximally even distribution but breaks any range queries on the original id.
• Composite keys like (user_id, timestamp) for time-ordered per-user data. Used heavily in wide-column databases like Cassandra.

The depth probe

"Why did you choose user_id as the shard key?" is a question with a real answer. The strong response: "Because most of our queries filter by user_id, the cardinality is high enough to support hundreds of shards, and the distribution is even enough that no single user dominates a shard. The risk is celebrity users with disproportionate activity, which we'd handle separately rather than letting it dictate the global shard key."

That sentence does four things at once: it commits to a choice, justifies it on three dimensions, names the failure mode, and indicates how the failure mode will be handled. That's the senior bar.

04The Four Sharding Strategies

Once you have a shard key, you need a strategy to map shard key values to specific shards. Four common strategies, each with tradeoffs.

Consistent hashing: the resharding-aware variant

The naive hash-mod-N strategy has a brutal property: when you change N (add or remove a shard), almost every key gets remapped. If you go from 4 shards to 5, roughly 80% of your data needs to move. This makes resharding effectively impossible at scale.

Consistent hashing solves this. Instead of hash % N, you place shards on a hash ring (positions 0 to 2^32). Each key hashes to a position on the ring and lives on the next shard clockwise. When you add a shard, only the keys whose ring position falls between the new shard and its predecessor need to move. Roughly 1/N of the data, not (N-1)/N.

Consistent hashing shows up in almost every distributed system at scale: Cassandra, DynamoDB, Redis Cluster, content delivery networks, distributed caches. If sharding is the topic and you don't mention consistent hashing during the deep-dive, you've left a depth signal on the table.

Virtual nodes (the practical refinement)

Pure consistent hashing has a problem: with a small number of physical shards, the ring positions can produce uneven distribution by chance. Virtual nodes fix this. Each physical shard owns hundreds or thousands of virtual nodes scattered around the ring. The averaging effect produces near-uniform distribution and lets you weight shards differently (a stronger machine can own more virtual nodes).

05The Hot-Key Problem

Sharding distributes data across shards. It does not distribute access patterns. If 10% of your users generate 90% of your traffic, sharding by user_id gives you a few extremely busy shards and many idle ones. This is the hot-key problem and it shows up in nearly every senior interview that touches sharding.

The classic case: celebrity users

The canonical example is Twitter. Most users have hundreds of followers. Some celebrities have tens of millions. If you shard tweets by author, the celebrity's shard receives traffic for every read of any of their tweets, plus every write fan-out, plus every follower interaction. The celebrity shard burns; the other shards idle.

Variants of the same problem appear everywhere. The big tenant in a multi-tenant SaaS. The trending product in an e-commerce catalog. The popular video on a streaming platform. The viral post on any social product. Whenever access patterns are power-law distributed (which they almost always are), naive sharding concentrates the load.

Three ways to handle hot keys

The depth probe

"What about the celebrity user case?" is the most common follow-up after any feed-shaped sharding discussion. The strong response acknowledges the problem, picks one of the three strategies, defends the choice, and notes the operational implications. The weak response either ignores the problem or hedges across strategies without committing.

Sharding distributes data. It does not distribute access patterns. The interview move is to name this gap and state how you'd handle it.

06Cross-Shard Queries: The Pain You Didn't Ask For

Sharding is wonderful when every query filters by the shard key. The query goes to one shard, runs locally, returns. Latency stays low. The architecture stays clean.

Most production systems aren't this lucky. Some queries don't filter by the shard key. Maybe an admin dashboard counts users by signup date. Maybe a search feature looks for a string across all records. Maybe a report aggregates revenue by region. None of these queries can be served by one shard.

Scatter-gather: the technique and its costs

The standard solution is scatter-gather: send the query to every shard in parallel, collect the partial results, merge them in the application or in a coordinator. This works but it has costs that compound at scale.

• Latency floor. The query is as slow as the slowest shard. With 16 shards, p99 latency on the slowest shard becomes p99 latency on the whole query. The "tail latency amplification" is dramatic.
• Resource cost. Every shard does work for every cross-shard query. If your scatter-gather queries are 10% of total traffic, every shard now does 10% of "their" work plus 10% of every other shard's work.
• Coordinator complexity. Some database systems do this for you (Vitess, Citus, MongoDB's mongos). Others require you to build the coordinator at the application layer. The coordinator itself becomes a system component to operate.

Avoiding cross-shard queries when you can

Three patterns to consider:

• Denormalization. Duplicate data so that every common query has a path to a single shard. Write amplification is the cost; read simplicity is the gain.
• Secondary indexes. Maintain a separate index sharded by a different key. Searches by the secondary key go to the index, then resolve to the primary store. Adds complexity to writes.
• OLAP offloading. Stream changes from your sharded OLTP store to a separate analytics store (BigQuery, Snowflake, ClickHouse). Cross-cutting analytical queries run there, away from the operational shards. Change data capture is the bridge.

The depth probe

"How would you support [admin query that doesn't filter by shard key]?" is a question candidates often handle poorly. The strong response picks one of the patterns above and explains the tradeoff. "I'd stream changes to a BigQuery warehouse and run admin queries there. The trade is some staleness, but admin queries don't need real-time freshness. The OLTP shards stay focused on user-facing traffic."

07Resharding Without Downtime

You picked your shard key well. You chose a good strategy. Distribution is even. And then your traffic doubles, or your dataset outgrows your shards, or you discover a hot key that needs to be split. You need to reshard.

Resharding is one of the most operationally painful procedures in distributed systems. The constraints make it hard:

• The system must stay online during the transition. Customers don't accept "scheduled maintenance."
• Writes are happening continuously. The data being moved is also being changed.
• The new shards have to be consistent with the old shards at some cutover moment. Mistakes cause data loss or duplication.
• The application's understanding of "which shard owns which key" has to change without breaking in-flight requests.

The four-phase resharding pattern

Most resharding procedures, regardless of database, follow some variant of this pattern:

1. Provision new shards. Stand up the destination capacity. No data flowing yet. The system runs on the old shards.
2. Backfill historical data. Copy existing data from the old shards to the new ones. This can take hours or days for large datasets. The old shards keep serving traffic during this phase.
3. Dual-write phase. Writes go to both old and new shards. The two stay in sync as the backfill catches up. Reads still serve from the old shards. This is the riskiest phase: any write that lands on one but not the other is a divergence bug.
4. Cutover. Reads switch to the new shards. The old shards become read-only as a safety net for some period, then are decommissioned. The cutover itself should be a flag-flip, not a deploy.

Most of the operational work is in phase 3. Detecting divergence between dual-writes, handling reconciliation, deciding when the new shards are caught up enough to cut over.

Why consistent hashing makes this dramatically easier

If you used consistent hashing from the start, the resharding pattern is much smaller in scope. Adding a new shard only requires moving roughly 1/N of the data (the keys whose ring positions fall in the new shard's range). The rest of the system is undisturbed. Compare with naive hash-mod-N where adding one shard remaps almost every key. The difference is between an operationally sane procedure and an operationally infeasible one.

08When Not to Shard

The senior signal in sharding interviews is knowing when to push back on sharding rather than reaching for it. Many candidates assume sharding is the answer to any scale concern. It usually isn't, until the requirements actually demand it.

Postgres on a single primary handles more than candidates think

A well-tuned Postgres instance on modern hardware (say, 64 cores, 256GB of RAM, NVMe storage) can comfortably handle:

• Tens of TB of data on a single primary
• Tens of thousands of writes per second on hot tables
• Many tens of thousands of reads per second from the primary, plus more from read replicas

Adding read replicas extends read capacity further. Adding query optimization, partitioning (within a single Postgres instance, not sharding across instances), and caching extends the ceiling further still. Many companies that "need to shard" actually need a better Postgres configuration first.

The threshold to actually shard

Reasonable triggers to consider sharding:

• Storage approaches a single machine's limit. Or compute on the primary saturates and read replicas can't absorb the read load.
• Write throughput exceeds what a single primary can sustain. Roughly 30K-50K writes per second on a hot table is where most teams start to feel pain.
• Operational considerations. Backups take days. Maintenance windows are too long. The blast radius of a single primary failure is unacceptable.

Below those thresholds, the cost of sharding (operational complexity, application changes, cross-shard query pain) usually exceeds the benefit. Right-size the database to the realistic workload.

The interview move

When asked "would you shard this?" the strong default is "not yet." Then explain at what point you would shard. "At our current scale of 10K writes per second, Postgres on a single primary with read replicas handles this comfortably. I'd reconsider sharding at roughly 50K writes per second on the same table, or when storage approaches the single-machine limit. Until then, sharding adds operational debt without enough benefit."

Reaching for sharding before you need it is over-engineering. The senior signal is naming the threshold at which sharding becomes the right answer, not assuming it's the answer now.

09How Sharding Interacts With Other Concepts

Sharding doesn't live alone. The interesting depth probes often cross concept boundaries.

• Sharding × Replication. Sharding gives you horizontal scale. Replication gives you availability. You usually need both. The interaction matters: replicate each shard, or shard your replicas, or both. The replication deep-dive covers the choices.
• Sharding × Caching. A hot-key problem in your sharded store often becomes a job for caching. Cache the hot keys and the shard concentration disappears. The caching deep-dive covers placement strategies for this.
• Sharding × Database selection. Some databases shard natively (Cassandra, DynamoDB). Some require manual sharding (Postgres, MySQL). The "how hard is sharding later?" question is one of the inputs to database selection.
• Sharding × Load balancing. The load balancer needs to know which shard owns which key. With consistent hashing, this knowledge can be pushed to the routing layer. With other strategies, the application or a coordinator does the routing.
• Sharding × Search. Cross-shard search queries are notoriously hard. Most teams solve this by streaming data to a separate search index (Elasticsearch, OpenSearch) sharded differently from the primary store.

For more on cross-concept interactions, see the concepts library hub.

10Practice Scenarios

Three scenarios. Read the setup. Decide your sharding approach before opening the reveal. The reveal is one defensible answer, not the only correct answer.

11Sharding FAQ