System Design Patterns: From Fundamentals to Real Systems

0% completed

​

GPU Auto-Scaling

1. The Incident

The support assistant's five-GPU fleet is humming: 70% utilized, batches full, finance mollified. Then marketing lands a morning-show mention (the scaling module's TV Tuesday, now with GPUs), and traffic quadruples in fifteen minutes. The autoscaler: configured web-style, because that's what the team knew: fails three separate ways, in order:

  • The signal lies, twice. The scaler watches CPU: but the work is on the GPU, so host CPU idles at 12% while requests queue

.....

.....

.....

Like the course? Get enrolled and start learning!