Grokking Concurrency in Python
0% completed
Failure Drills
The final design of Harvest makes three promises about failure:
- If an index worker process dies, no page is lost and no page is written twice.
- If one site stops answering in time, the crawl still ends, and the other sites are still crawled.
- If the process is told to stop, it finishes the pages in flight, saves the rest, and a later run continues from the saved state.
It makes no promise that a slow site leaves the crawl time unchanged, and nothing survives SIGKILL sent to the main process
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%