0% completed
Concurrency and Parallelism in Python
On This Page
Two Words for Two Ideas
Meet Harvest
Threads Overlap the Waiting
Why the Threads Did Not Speed Up Indexing
The Free-Threaded Build
Matching the Tool to the Work
In Production
Key Takeaways
Practice Questions
Harvest, the crawler this course builds, downloads eight pages from a test site in 2.06 seconds. With eight threads, the same downloads take 0.28 seconds. Harvest then indexes those pages, which takes 1.51 seconds with one thread and 1.54 seconds with eight.
The same eight threads made one job seven times faster and left the other one unchanged. The explanation lies in two words that are often used as if they meant the same thing: concurrency and parallelism.
Two Words for Two Ideas
Concurrency means that a program has several tasks in progress during the same period of time. The tasks do not have to run at the same instant. One CPU can run task A until A has to wait, switch to task B, and return to A later.
Parallelism means that several tasks execute at the same instant, each on its own CPU. The operating system counts each stream of instructions that the hardware can execute at once as a logical CPU. A machine with a single logical CPU can be concurrent, but it can never be parallel.
Concurrency is about structure: how a program is organized so that waiting in one task does not stop the others. Parallelism is about execution: how many instructions run at the same moment.
Meet Harvest
Every lesson in this course works on one program. Harvest is a small web crawler and search indexer: it downloads pages and builds an index of the words on each page. Its two halves behave very differently:
- Fetching a page means sending a request and waiting for the reply, while the CPU has nothing to do.
- Indexing a page means reading and counting every word, which keeps a CPU busy from start to finish.
The test site runs on the same machine, answers each request after 250 milliseconds, and serves pages of 500,000 words. These two functions do the work:
def fetch(n): with urllib.request.urlopen(f"{BASE}{n}") as reply: return reply.read().decode() def index(text): counts = {} for word in text.split(): key = 0 for ch in word: key = (key * 31 + ord(ch)) % 1_000_003 counts[key] = counts.get(key, 0) + 1 return counts
The index function is written in plain Python on purpose, so that every step of it runs as Python code.
A process is a running program with its own memory, which other processes cannot see unless they arrange to share it. A thread is a sequence of instructions inside a process that shares all of that process's memory with its other threads. Harvest runs each job one page at a time and with a pool of eight threads, and runs indexing with a pool of eight processes as well. Both pools come from the standard library module concurrent.futures:
with ThreadPoolExecutor(max_workers=PAGES) as pool: timed(f"fetch, {PAGES} threads", lambda: list(pool.map(fetch, numbers)))
The call pool.map(fetch, numbers) runs fetch once per page number across the pool's threads and returns the results in order. Each reported time is the median of three runs.
Threads Overlap the Waiting
The runs used Python 3.15.2 on an 8-core Windows desktop that reports 16 logical CPUs. The standard build of Python printed this:
Output:
$ python3.15 harvest.py Python 3.15.2, GIL on, 16 logical CPUs fetch, one at a time 2.06 s fetch, 8 threads 0.28 s index, one at a time 1.51 s index, 8 threads 1.54 s index, 8 processes 0.28 s
Fetching one page at a time took 2.06 seconds: eight waits of 250 milliseconds, plus transfer time. With eight threads, all eight waits happened during the same quarter of a second. The threads did not need to run at the same instant, only to wait at the same time, and that is concurrency.
Indexing behaved differently. Eight threads took as long as one, while eight processes were 5.4 times faster. The work was identical, and only the kind of worker changed.
Why the Threads Did Not Speed Up Indexing
CPython is the standard Python interpreter. A lock is an object that one thread owns at a time, while every other thread that wants it waits. The global interpreter lock (GIL) is a lock inside CPython that a thread must hold while it runs Python code.
The output says GIL on, so the eight indexing threads took turns. All eight were in progress, but only one ran at any instant: they were concurrent, not parallel. Fetching was not affected, because a thread releases the GIL while it waits for the network.
Each worker process runs its own interpreter with its own GIL, so eight processes can run Python code on eight cores at once. They missed the ideal of 0.19 seconds partly because each page had to be copied into a worker process and each result copied back.
The Free-Threaded Build
The free-threaded build is a separate, optional CPython installation, run as python3.15t, in which the GIL is disabled by default, so threads can run Python code at the same time. The standard build still has the GIL. The same program on the free-threaded build printed this:
Output:
$ python3.15t harvest.py Python 3.15.2, GIL off, 16 logical CPUs fetch, one at a time 2.05 s fetch, 8 threads 0.27 s index, one at a time 1.61 s index, 8 threads 0.38 s index, 8 processes 0.36 s
Eight threads now indexed the pages 4.2 times faster than one, so they were parallel as well as concurrent. A single thread was slower here, 1.61 seconds against 1.51, because this build does extra work to keep objects safe without the GIL.
A second program, busy.py, measured how many CPUs the eight indexing threads kept busy. CPU time is the processor time a program uses, added up over all of its threads, and wall time is the time that passes on a clock. CPU time divided by wall time gives the average number of busy CPUs.
Output:
$ python3.15 busy.py standard build: CPU time 1.64 s, wall time 1.65 s, 1.0 CPUs busy $ python3.15t busy.py free-threaded build: CPU time 2.27 s, wall time 0.32 s, 7.0 CPUs busy
With the GIL, eight threads kept one CPU busy, and without it they kept seven busy. The speedup was less than sevenfold because the threads used 2.27 seconds of CPU time in total, more than one thread needed.
Matching the Tool to the Work
Work that mostly waits gains from concurrency, and threads provide it on either build. Work that runs Python code on the CPU gains only from parallelism, which the standard build provides through processes, not threads.
| Kind of work | Standard build | Free-threaded build |
|---|---|---|
| Waiting for a network, disk, or database | Threads overlap the waits | Threads overlap the waits |
| Python code that keeps a CPU busy | Threads take turns; processes run in parallel | Threads and processes both run in parallel |
A third tool, asyncio, overlaps many waits on a single thread, which gives concurrency without parallelism.
In Production
| What you see | What it means | How to check |
|---|---|---|
| A Python process with many threads never uses more than one CPU | Its threads run Python code and take turns on the GIL | Compare CPU time with wall time, as busy.py does; a ratio near 1.0 means one busy CPU |
| Adding threads made a service that calls other services much faster | The threads overlap waiting, not computing | The process uses little CPU while its threads wait for replies |
| You are not sure whether a service runs without the GIL | Importing an extension module that is not marked as safe without the GIL turns it back on, with a warning | Call sys._is_gil_enabled() inside the running service, after its imports |
Key Takeaways
- Concurrency means several tasks are in progress during the same period. Parallelism means they execute at the same instant on different CPUs.
- Threads overlapped Harvest's network waits and cut fetching from 2.06 to 0.28 seconds on either build.
- On the standard build, the GIL lets only one thread run Python code at a time, so indexing threads gave no speedup while processes did.
- The free-threaded build,
python3.15t, lets threads run Python code in parallel. - Before choosing a tool, find out whether the work mostly waits or mostly computes.
Real code is rarely divided as clearly as Harvest's two halves. The next lesson, IO Bound and CPU Bound Work, shows how to measure which kind of work a program does.
Practice Questions
Try each question first, then open the answer.
1. A machine has a single logical CPU. Can a Python program on it be concurrent? Can it be parallel?
<details> <summary>Show answer</summary>It can be concurrent, but it cannot be parallel. Tasks can take turns on the one CPU, for example while one of them waits for a network reply. Two tasks can execute at the same instant only with a second logical CPU.
</details>2. On the standard build, eight threads fetched Harvest's pages seven times faster than one thread. Why did the GIL not stop this speedup?
<details> <summary>Show answer</summary>A thread releases the GIL while it waits for the network. Almost all of the fetch time was waiting for replies, and waiting runs no Python code. The threads only needed to wait at the same time, not to compute at the same time.
</details>3. A teammate adds 16 threads to a report generator that spends all its time in Python loops, and the report is no faster. On the standard build, what would you change?
<details> <summary>Show answer</summary>Use a pool of processes instead of threads. The 16 threads take turns on the GIL and keep only one CPU busy. Each process has its own GIL, so processes can run the loops on several cores at once, although copying data to them makes the speedup smaller than the number of processes.
</details>4. A service on the free-threaded build stops running its CPU-heavy threads in parallel after a new library is added. What is a likely cause, and how can you confirm it?
<details> <summary>Show answer</summary>The library probably includes an extension module that is not marked as safe without the GIL, so importing it turned the GIL back on. Calling sys._is_gil_enabled() in the running service confirms it by returning True.
Reading Progress
0%
On This Page
Two Words for Two Ideas
Meet Harvest
Threads Overlap the Waiting
Why the Threads Did Not Speed Up Indexing
The Free-Threaded Build
Matching the Tool to the Work
In Production
Key Takeaways
Practice Questions