System Design Patterns: From Fundamentals to Real Systems
Vote

0% completed

​

GPU Auto-Scaling

1. The Incident

The support assistant's five-GPU fleet is running well. It sits at 70% utilization, batches are full, and finance is satisfied. Then marketing lands a morning-show mention. This is the scaling module's TV spike, now with GPUs. Traffic quadruples in fifteen minutes.

The autoscaler was configured web-style, because that is what the team knew. It fails three separate ways, in order.

The signal lies, twice. The scaler watches CPU, but the work is on the GPU

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%