Cache Stampede Prevention

1. The Incident

Flash sale night. The bestseller's page is served from cache at 12,000 requests per second, hit ratio 99.9%, database humming at a gentle 150 queries per second. The architecture from this module, working perfectly.

At 8:00:00 PM, the bestseller's cache entry reaches the end of its one-hour TTL and expires. Routine. Happens to every key. Here's what the next 50 milliseconds look like:

Rebuilding the entry takes one database query: about 50 ms. But the key is receiving 12,000 requests per second, and every request that arrives during those 50 ms misses. 12,000 × 0