Grokking DevOps and SRE Interviews
0% completed
Probes, Rolling Updates, and Shutdown
At 18:00, traffic to a product search service doubled within five minutes. The autoscaler requested 10 more pods, but each new pod needed about 90 seconds to load its search index into memory. Its liveness probe began at 30 seconds and allowed three failures 10 seconds apart, so the kubelet restarted every new pod at about 50 seconds, before it could serve a request.
Meanwhile, the original 10 pods carried all the traffic. Their health endpoint now answered in more than 1 second, which was the probe's limit, so the kubelet restarted 4 of them as well. Capacity fell while demand rose.
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%