Grokking Concurrency in Python
Vote

0% completed

​

When the GIL Is Released

Holding and Releasing the Lock

Which Steps Run in Parallel

The Switch Interval

In Production

Key Takeaways

Practice Questions

When the GIL Is Released
When the GIL Is Released
def compress(data): return zlib.compress(data, 9)

Harvest compresses every page before it stores it. On the standard build of Python, with the GIL on, compressing eight pages took 1,463 milliseconds with one thread and 186 milliseconds with eight threads. That is 7.9 times faster, on a build where threads supposedly cannot run in parallel.

The indexing step did not get faster at all. The difference is in what each step does while it holds the GIL, and whether it gives the lock up.

Holding and Releasing the Lock

A thread must hold the GIL to run Python code, because Python code changes interpreter state such as reference counts. C code that works on its own memory and touches no Python object does not need the GIL.

A thread can release the GIL before it runs code that does not touch Python objects, and it must take the lock back before it touches them again. CPython's C API gives extension authors two macros for this, Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS, placed around the slow part of a function.

Two kinds of work release the GIL in practice:

Plain Python code never releases the GIL by itself. A thread running it gives the GIL up only when the interpreter asks, at the end of the switch interval described below.

Which Steps Run in Parallel

The program steps.py runs four steps on eight of Harvest's pages, with one thread and then with a pool of eight threads. It ran on Python 3.15.2 on an 8-core Windows desktop that reports 16 logical CPUs. Each time is the median of three runs:

Output:

$ python3.15 steps.py GIL on, NumPy 2.5.4 step, 8 pages 1 thread 8 threads speedup index (Python code) 2455 ms 2437 ms 1.0x compress (zlib, level 9) 1463 ms 186 ms 7.9x fingerprint (hashlib.sha256) 12 ms 2 ms 7.2x rank 2,000,000 scores (NumPy sort) 273 ms 62 ms 4.4x

The indexing threads took turns on the GIL and gained nothing. Both indexing times are higher than in the first lesson's run, so compare the two columns of this run with each other. The other three steps spend almost all of their time in C code that runs without the GIL:

  • zlib.compress ran eight compressions on eight cores at once.
  • hashlib.sha256 releases the GIL for data larger than 2,047 bytes, as the hashlib documentation states. Its speedup is real, but 12 milliseconds is too small to matter in Harvest.
  • NumPy's sort ran each sort of 2,000,000 numbers in C without the GIL. NumPy is a third-party library for arrays of numbers.
Steps with 1 and 8 threads
Steps with 1 and 8 threads

On the standard build, threads run in parallel only while the code they run has released the GIL. Each thread still needs the GIL briefly to start the call and build the result, so a step is only as parallel as the share of its time spent without the lock. A C extension is not required to release the GIL, so check the library's documentation or measure it as steps.py does.

The Switch Interval

The switch interval is the time a thread waits for the GIL before it asks the thread holding the GIL to release it. The default is 5 milliseconds. sys.getswitchinterval() returns it, and sys.setswitchinterval() changes it. The running thread then releases the GIL at its next opportunity, and the operating system decides which waiting thread runs next.

This matters most for a thread that needs only a moment of CPU, but needs it on time. The program heartbeat.py starts a health-check thread that sleeps for 1 millisecond, again and again, and records how late it wakes:

def heartbeat(stop, delays): while not stop.is_set(): start = time.perf_counter() time.sleep(0.001) delays.append(time.perf_counter() - start - 0.001)

Meanwhile, the main thread indexes two pages. This part ran on a cloud virtual machine running Linux with 2 virtual CPUs (one physical core). The column p99 is the 99th percentile, the delay that 99 percent of the wake-ups did not exceed:

Output:

$ python3.15 heartbeat.py GIL on, switch interval 5 ms idle (sleep 1 s) work 1.00 s late by: median 0.1 ms, p99 0.1 ms indexing 2 pages work 0.29 s late by: median 5.1 ms, p99 18.6 ms indexing 2 pages, 1 ms interval work 0.29 s late by: median 1.1 ms, p99 14.5 ms

While the main thread indexed, the heartbeat's median delay grew from 0.1 to 5.1 milliseconds, which is the switch interval. It woke on time, then waited for the GIL until the interval ran out. With a 1 millisecond interval, the median delay fell to 1.1 milliseconds, and indexing took the same 0.29 seconds.

Waiting for the GIL
Waiting for the GIL

The free-threaded build has no GIL to wait for:

Output:

$ python3.15t heartbeat.py GIL off, switch interval 5 ms idle (sleep 1 s) work 1.00 s late by: median 0.1 ms, p99 0.1 ms indexing 2 pages work 0.31 s late by: median 0.1 ms, p99 0.1 ms indexing 2 pages, 1 ms interval work 0.30 s late by: median 0.1 ms, p99 0.1 ms

The heartbeat ran on the second virtual CPU, so it woke as promptly as when idle.

In Production

What you seeWhat it meansHow to check
A thread pool doing compression, hashing, or array work uses several CPUs on the standard buildThe library releases the GIL inside those callsCompare CPU time with wall time for the pool, as in IO Bound and CPU Bound Work; a ratio above 1 means several busy CPUs
Health checks or timers in a threaded service are several milliseconds late when the service is busyThose threads wait for a thread running Python code to give up the GILMeasure the delay with and without load, as heartbeat.py does
One request stalls every other thread for a long timeA long call in a C extension holds the GILTime the call alone; if threads do not overlap it, it does not release the GIL

A shorter switch interval makes the interpreter ask for switches more often, so measure throughput before you change it on a real service.

Key Takeaways

  • A thread releases the GIL during blocking I/O and during C code that chose to release it.
  • On the standard build, eight threads compressed pages 7.9 times faster and sorted NumPy arrays 4.4 times faster, while Python indexing stayed at 1.0 times.
  • hashlib releases the GIL for data over 2,047 bytes; for other libraries, check their documentation or measure.
  • Plain Python code never releases the GIL by itself. A waiting thread asks for it after the switch interval, 5 milliseconds by default.
  • A heartbeat thread woke a median 5.1 milliseconds late while another thread indexed, and 1.1 milliseconds late with a 1 millisecond interval.

The next lesson, The Four Models and How to Size Them, compares the four ways of running concurrent Python and shows how to choose the number of workers for each one.

Practice Questions

Try each question first, then open the answer.

1. On the standard build, eight threads made zlib.compress 7.9 times faster but did not speed up index. What is the difference between the two functions?

<details> <summary>Show answer</summary>

zlib.compress runs most of its work in C code that releases the GIL, while index is Python code that needs the GIL for every step.

</details>

2. A service on the standard build runs a CPU-heavy Python loop in one thread and a health check in another. The health check is about 5 ms slower when the service is busy. What causes this, and what setting controls it?

<details> <summary>Show answer</summary>

The health check thread waits for the GIL. The busy thread keeps it until the waiting thread asks for it, after the switch interval. sys.setswitchinterval() sets that interval; the default is 5 ms.

</details>

3. You need to hash 10,000 files of about 1 MB each with hashlib.sha256 on the standard build. Will a thread pool help with the hashing itself?

<details> <summary>Show answer</summary>

Yes. hashlib releases the GIL while it hashes data larger than 2,047 bytes, so several threads can hash on several cores at once.

</details>

4. A colleague says that a 1 ms switch interval will make a CPU-bound thread pool on the standard build finish faster. What did the heartbeat run show?

<details> <summary>Show answer</summary>

It will not. With a 1 ms interval, indexing took the same 0.29 seconds; only the waiting thread's median delay fell, from 5.1 ms to 1.1 ms. The interval changes how quickly threads take turns, not how many run Python code at once.

</details>

Reading Progress

0%


Vote for new content

On This Page

Holding and Releasing the Lock

Which Steps Run in Parallel

The Switch Interval

In Production

Key Takeaways

Practice Questions