Consistent Hashing

1. The Incident

The sharding worked. Four shards, keyed by hash(customer_id) % 4, each one a comfortable 625 writes per second. Two years later, growth says you need a fifth.

So you change the routing rule: % 4 becomes % 5. Before deploying, someone on the team does the arithmetic on what that innocent-looking edit means, and goes pale. A key stays on its current shard only if hash % 4 and hash % 5 happen to agree, and for random hashes they agree only 20% of the time. Read that again: **you added one shard, and 80% of all your data is now on the wrong machine