Cache Stampede Prevention

1. The Incident

During a flash sale, the bestseller's page serves 12,000 requests per second from cache with a 99.9% hit ratio. The database receives only 150 queries per second, so the architecture from this module is operating as intended.

At 8:00:00 PM, that product's cached entry reaches its one-hour TTL and expires. A TTL (time to live) is the timer that removes an entry from the cache, so this routine event eventually affects every key. The next 50 milliseconds reveal the problem.

Rebuilding the value requires one database query that normally takes about 50 ms