Grokking Concurrency in Python
Vote

0% completed

​

Sizing, Backpressure and Shutdown

A site that serves 8 requests at a time, and needs 0.1 seconds for each one, can answer at most 8 / 0.1 = 80 requests per second. Eight workers can reach that limit, and 32 workers cannot raise it, because the other 24 requests only wait at the site for a free slot.

Besides its worker count, a pool has two settings whose defaults do not suit a crawler: how much work it accepts before the first result comes back, and what happens to queued work when the program has to stop.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content