Bulkhead

1. The Incident

Your API service handles three features from one deployment: checkout, search, and order history. They share a pool of 200 threads and a pool of 100 database connections. A pool is a fixed set of workers or connections that requests borrow and return. Everything works.

Then marketing sends 2 million emails: "Your year in orders!" with a link to the order history page. Order history is your heaviest query, about 500 ms each. Traffic to it jumps from 50 requests per second to 1,500.

Use the module intro's arithmetic: things in progress = arrival rate x time each takes