Retry with Exponential Backoff

1. The Incident

Your checkout service calls a healthy payment provider over the network, and both applications are implemented correctly.

Even so, 1 out of every 200 calls fails because networks drop packets, reset connections, and route requests to unhealthy instances. The overall provider can remain available while individual calls fail.

At 100,000 daily orders, that rate produces errors for 500 customers a day after they enter payment details. The same request would often succeed if repeated 100 milliseconds later.