0% completed
CPU Bound and IO Bound Work
On This Page
Timing the Two Jobs
IO Bound and CPU Bound
Both Jobs on One Core
CPU Bound Work on Several Cores
Comparing the Two Kinds of Work
A Common Mistake
Key Takeaways
Consider two jobs. The first job calls fetch three times, and each call sleeps for 100 ms, so the job takes 3 x 100 ms = 300 ms. The second job adds up every whole number from 0 up to, but not including, 1.2 billion, which needs 1.2 billion additions and no waiting at all.
On the computer used to test this course, the second job takes about 260 ms. The two jobs look alike on a stopwatch, yet they spend their time in opposite ways. The first job spends almost all of it waiting, while the second job keeps a CPU core busy from start to finish.
Whether goroutines make a job faster, and by how much, depends on that difference. Below, both jobs are timed alone, then with goroutines on one core, and then on several cores.
Timing the Two Jobs
The program below runs both jobs one step after another, with no goroutines. The sum function adds the numbers in a range. The sum job calls it three times on three equal ranges, so both jobs are made of three calls.
One possible output (measured on one computer):
fetch job: 300ms sum job: 260ms total 719999999400000000
IO Bound and CPU Bound
Work whose running time is set mostly by waiting for something outside the CPU is called IO bound (limited by input and output). Calls over a network, reads from a disk, and queries to a database are all IO bound. In each case, the program spends most of its time waiting for an answer. The fetch job is IO bound: each call uses very little CPU time and then sleeps for 100 ms.
Work whose running time is set by how fast the CPU executes instructions is called CPU bound (limited by the processor). Adding numbers, compressing data, resizing images, and hashing passwords are CPU bound. The sum job is CPU bound, because nothing in it waits.
The distinction matters because a waiting goroutine does not need a core. While it waits, the runtime can run a different goroutine on the same core. A computing goroutine, on the other hand, needs a core for every instruction it executes.
Both Jobs on One Core
The next program runs each job as goroutines, one goroutine per call, with runtime.GOMAXPROCS(1) so that only one core runs Go code. Each goroutine sends its result into a channel, and main receives once per goroutine so that it waits for all of them.
One possible output (measured on the same computer):
fetch job, 3 goroutines: 100ms sum job, 3 goroutines: 260ms total 719999999400000000
The fetch job dropped from 300 ms to about 100 ms on a single core. All three goroutines start their sleep at almost the same moment, and the core runs whichever goroutine has something to do. The total time is the length of the longest wait, not the sum of the waits.
The sum job did not get faster. The three goroutines take turns on the one core, and each turn executes additions that would have been executed anyway. Splitting CPU bound work into goroutines on one core only changes the order in which the instructions run.
CPU Bound Work on Several Cores
CPU bound work speeds up only when more cores run it. The program below splits the same 1.2 billion additions into four goroutines and runs the job with GOMAXPROCS set to 1, 2, 4, and 8.
One possible output (measured on the same computer, which has 16 logical CPUs):
GOMAXPROCS=1: 260ms (total 719999999400000000) GOMAXPROCS=2: 130ms (total 719999999400000000) GOMAXPROCS=4: 70ms (total 719999999400000000) GOMAXPROCS=8: 70ms (total 719999999400000000)
Going from one core to two roughly halves the time, and going to four roughly halves it again. Going to eight changes nothing, because the job has only four goroutines and four of the eight cores have nothing to run. The speedup of CPU bound work is limited by two numbers: the cores available and the number of goroutines with work to do.
There is a third limit. A real machine runs other programs, the cores share memory and caches, and some cores may run at lower speeds than others. For these reasons, a measured speedup can fall below the number of cores. The only way to know the real figure is to measure it.
Comparing the Two Kinds of Work
| IO bound | CPU bound | |
|---|---|---|
| Time spent | Waiting for a network, disk, or database | Executing instructions |
| Examples | fetch, file reads, database queries | Sums, compression, image resizing, hashing |
| Goroutines on one core | Faster: waits overlap | No faster |
| Goroutines on many cores | Faster, but cores are rarely the limit | Faster, up to the number of cores |
| Useful number of goroutines | Can far exceed the number of cores | About the number of cores |
A Common Mistake
A common mistake is to start one goroutine per item for CPU bound work, such as one goroutine for each of 100,000 images to resize. The machine still has the same number of cores, so the job finishes no sooner than with one goroutine per core. Meanwhile, every extra goroutine holds its own memory while it waits for a turn.
The reverse mistake is to limit IO bound work to the number of cores. A program on an 8 core machine that makes 1,000 service calls, at most 8 at a time, wastes most of its time waiting. Because waiting goroutines need no core, IO bound work can run many more goroutines than there are cores.
Most real programs mix both kinds of work. A request handler that queries a database and then builds a large response is a typical case. It is IO bound while it waits for the database and CPU bound while it builds the response. Timing each step separately tells you which kind dominates.
Key Takeaways
- IO bound work spends its time waiting, and CPU bound work spends its time executing instructions.
- Goroutines speed up IO bound work even on one core, because waiting goroutines need no core.
- Goroutines speed up CPU bound work only when more than one core is available.
- The speedup of CPU bound work cannot exceed the number of cores or the number of busy goroutines.
- Measure a job before splitting it, because the kind of work determines how many goroutines help.
Goroutines shorten a job by overlapping the parts that wait and by spreading the parts that compute across cores. Waiting overlaps on any number of cores, while computing spreads only across the cores that the machine actually has.
Reading Progress
0%
On This Page
Timing the Two Jobs
IO Bound and CPU Bound
Both Jobs on One Core
CPU Bound Work on Several Cores
Comparing the Two Kinds of Work
A Common Mistake
Key Takeaways