System Design Patterns: From Fundamentals to Real Systems
Vote

0% completed

​

GPU Auto-Scaling

1. The Incident

The support assistant's five-GPU fleet is running well. It sits at 70% utilization, batches are full, and finance is satisfied. Then marketing lands a morning-show mention. This is the scaling module's TV spike, now with GPUs; traffic quadruples in fifteen minutes.

The autoscaler was configured web-style, because that is what the team knew. It fails three separate ways, in order.

The signal lies, twice: the scaler watches CPU, but the work is on the GPU

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%

Vote for new content