Grokking Concurrency in Python
Vote

0% completed

​

Race Conditions in Python Code

Shared State and Race Conditions

Check-Then-Act

A Smaller Window Is Still a Race

Read-Modify-Write

Making the Steps One Step

In Production

Key Takeaways

Practice Questions

Race Conditions in Python Code
Race Conditions in Python Code

If every function in a threaded program is correct on its own, can the program still produce wrong results? It can, whenever a thread reads shared data, makes a decision, and writes back while another thread changes the same data in between. Such bugs raise no exception. They produce work that is done twice and totals that are too low, and only in some runs.

Shared State and Race Conditions

Harvest's crawler now follows links. Each page on the test site ends with four links to other pages, and the crawler fetches every page at the current depth with eight threads, collects their links, and repeats. All eight threads share visited, the set of pages already fetched, and stats, a dictionary that holds the running word count.

Shared state is data that more than one thread can read and change. A race condition is a bug in which the result depends on the order in which threads happen to run. In one run the threads take their steps in a harmless order, and in the next run they do not, so race conditions pass tests and then fail in production.

The demos ran with Python 3.15.2 on both builds, on a Linux virtual machine with 2 virtual CPUs (one physical core). The test site has 40 pages of 20,000 words, answers each request after 250 milliseconds, and counts the requests for each page.

Check-Then-Act

The first version of the visit function checks the set, fetches the page, and then records it:

def visit_mark_late(n): if n in visited: return [] text = fetch(n) visited.add(n) return links(text)

A check-then-act race happens when a thread checks a condition and then acts on it, while another thread changes the condition between the two steps. Thread A checks page 12, finds it missing, and starts a 250 millisecond fetch. Thread B checks page 12 during that fetch and also finds it missing, because A adds it only after its fetch returns. Both threads fetch the same page.

Two threads check before either one adds
Two threads check before either one adds

Each crawl ran three times on each build. Every request beyond the first for a page is a duplicate. The first round printed this:

Output:

$ python3.15 race.py (round 1) GIL on visit_mark_late 55 requests for 38 pages, 17 duplicates visit_mark_early 38 requests for 38 pages, 0 duplicates visit_locked 38 requests for 38 pages, 0 duplicates count_words counted 320,064 words, true total 800,160, lost 480,096 count_words_locked counted 800,160 words, true total 800,160, lost 0 $ python3.15t race.py (round 1) GIL off visit_mark_late 55 requests for 38 pages, 17 duplicates visit_mark_early 38 requests for 38 pages, 0 duplicates visit_locked 38 requests for 38 pages, 0 duplicates count_words counted 180,036 words, true total 800,160, lost 620,124 count_words_locked counted 800,160 words, true total 800,160, lost 0

The late version made 16 to 20 duplicate requests in every round, with the GIL and without it. The GIL did not help, because a thread releases it while it waits for the network, and the gap between the check and the add contains exactly such a wait.

A Smaller Window Is Still a Race

The second version moves the add before the fetch:

def visit_mark_early(n): if n in visited: return [] visited.add(n) return links(fetch(n))

It made no duplicate requests in any of the six runs. The race window, the time between the check and the add during which another thread can make the same check, is now a few bytecode instructions instead of 250 milliseconds. A smaller window is hit less often, but the check and the add are still two steps.

A second program removes the network so that only the window remains. Eight threads each try to claim the same 200,000 URLs with the same check and add, as fast as they can:

def claim(n): if n in seen: return False seen.add(n) return True

Every URL should be claimed exactly once:

Output:

$ python3.15 claim.py; python3.15t claim.py (round 1) GIL on: 200,000 claims for 200,000 URLs, 0 claimed twice GIL off: 200,000 claims for 200,000 URLs, 0 claimed twice $ python3.15 claim.py; python3.15t claim.py (round 2) GIL on: 200,000 claims for 200,000 URLs, 0 claimed twice GIL off: 200,025 claims for 200,000 URLs, 25 claimed twice $ python3.15 claim.py; python3.15t claim.py (round 3) GIL on: 200,000 claims for 200,000 URLs, 0 claimed twice GIL off: 200,000 claims for 200,000 URLs, 0 claimed twice

On the free-threaded build, two threads ran at the same instant and claimed 25 URLs twice in one round of three. The standard build made no double claims, but that does not make it safe: as What the GIL Protects explained, the language does not promise where the standard build switches threads. A race that appears once in three runs, on one build, is the kind that reaches production.

Read-Modify-Write

The word counter has the second pattern:

def count_words(text): stats["words"] += word_count(text)

A read-modify-write race happens when a thread reads a value, computes a new value from it, and writes the result back, while another thread writes the same value in between. Python reads stats["words"], then calls word_count, which runs plain Python code for several milliseconds, then stores the sum.

On the standard build, the GIL moves to another thread every few milliseconds while word_count runs. That thread can store its own sum. When the first thread continues, it stores its old value plus its own count, and the other thread's addition is gone.

In round 1, the standard build counted 320,064 of the 800,160 words. Across the three rounds it lost between 480,096 and 540,108 words, and the free-threaded build lost 620,124 words in every round.

A lost update in the word counter
A lost update in the word counter

Making the Steps One Step

Both races have the same repair: no other thread may touch the shared data between the read and the write. As Concurrency and Parallelism in Python described, a lock is an object that one thread owns at a time, while every other thread that wants it waits. The with statement takes the lock and releases it at the end of the block:

def visit_locked(n): with visited_lock: if n in visited: return [] visited.add(n) return links(fetch(n)) # ... def count_words_locked(text): n = word_count(text) with stats_lock: stats["words"] += n

With the locks, every round on both builds made 38 requests for 38 pages and counted all 800,160 words. The slow parts, the fetch and word_count, touch no shared data, so they stay outside each lock. A lock held during them would make the eight threads fetch and count one page at a time.

In Production

What you seeWhat it meansHow to check
A downstream service receives the same request twiceA check-then-act race on a "seen" set or a "sent" flagLook for if x not in seen: followed by slow work before the add
Counters in metrics are lower than the real number of eventsA read-modify-write race on a shared counterCompare with an independent count; look for += on shared data outside a lock
A bug appears only under load, or only on the free-threaded buildThe race window is small and is hit only when threads run at the same instantRun with more threads and on the free-threaded build

Key Takeaways

  • A race condition is a bug whose result depends on the order in which threads run, so it can pass every test and still fail.
  • Check-then-act caused Harvest's 17 duplicate fetches, and read-modify-write lost 480,096 words.
  • The GIL prevented neither race: one waited on the network, and the other ran Python code long enough for the GIL to change hands.
  • Moving the steps closer together only makes the race rarer, as the 25 double claims showed.
  • A lock makes the read and the write one step. Keep slow work outside it.

The two locks in this lesson each protected one piece of data, and no thread held both at once. Harvest soon needs threads that hold two locks at the same time. The next lesson, Locks and Lock Ordering, covers Python's two kinds of lock and the rule that keeps two locks from stopping a program.

Practice Questions

Try each question first, then open the answer.

1. In visit_mark_late, the GIL is on, yet two threads fetched the same page. Why?

<details> <summary>Show answer</summary>

The gap between the check and the add contains a network wait, and a thread releases the GIL while it waits. While thread A waits, thread B checks the same page and also finds it missing.

</details>

2. A teammate says visit_mark_early is correct because it made no duplicates in six runs. What would you answer?

<details> <summary>Show answer</summary>

The check and the add are still two steps, so the race is only rarer. claim.py ran the same steps under heavy load, and the free-threaded build claimed 25 URLs twice in one round of three.

</details>

3. Why does stats["words"] += word_count(text) lose updates even on the standard build?

<details> <summary>Show answer</summary>

Python reads the counter before it calls word_count and stores the sum after. During the call the GIL passes to other threads, which store their sums, and the first thread then overwrites them with its old value plus its own count.

</details>

4. A service sends a welcome email if user_id not in welcomed, then adds the user after the email provider replies. Some users get two emails. Name the pattern and a fix.

<details> <summary>Show answer</summary>

This is a check-then-act race with a slow network call between the check and the act. Check and add the user in one locked block, then send the email outside the lock.

</details>

Reading Progress

0%


Vote for new content

On This Page

Shared State and Race Conditions

Check-Then-Act

A Smaller Window Is Still a Race

Read-Modify-Write

Making the Steps One Step

In Production

Key Takeaways

Practice Questions