Cache Stampede Prevention

1. The Incident

It is flash sale night. The bestseller's page is served from cache at 12,000 requests per second. The hit ratio is 99.9%. The database handles a gentle 150 queries per second. This is the architecture from this module, working as designed.

At 8:00:00 PM, the bestseller's cache entry reaches the end of its one-hour TTL and expires. A TTL (time to live) is the timer that removes an entry from the cache. This is routine. It happens to every key. Here is what the next 50 milliseconds look like.

Rebuilding the entry takes one database query, about 50 ms