Grokking Multithreading and Concurrency for Coding Interviews

0% completed

Programs, Processes, and Threads

On This Page

Know the Vocabulary

Part 1: From a File to a Running Server

The Program on Disk

The Running Process

Three Words, Three Things

Part 2: What a Process Is

The Definition

What a Process Is Not

Part 3: What a Thread Is

The Definition

What Each Thread Keeps for Itself

What All Threads Share

What a Thread Is Not

Part 4: Seeing Shared and Private Memory in Code

The Code

What Happens in Memory, Step by Step

Part 5: Sharing Is the Benefit and the Risk

Part 6: When One Thread Fails

A Crash Ends the Whole Process

An Uncaught Exception Depends on the Language

Part 7: Processes Versus Threads

Separate Memory, Seen in Code

The Comparison

FAQs

Key Takeaways

Check Your Understanding

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

Program: A file of instructions stored on disk. It does nothing until someone starts it.
Process: A running copy of a program. The operating system gives each process its own memory.
Thread: One path of work inside a process. A process can run many threads at the same time.
Main thread: The first thread of a process. The operating system creates it when the process starts.
Operating system: The software that runs the machine, such as Linux or Windows. It starts processes and decides which thread runs on which core.
Core: One part of the processor that runs one thread at a time.
Address space: All the memory that one process can use. Other processes cannot see it.
Heap: The part of a process's memory where the program keeps its objects, such as the seat map. Every thread in the process can reach it.
Stack: The block of memory that each thread keeps for its own function calls and local variables.
Local variable: A variable created inside a function. It lives on the stack of the thread that runs the function.
Program counter: A small storage place in the core that holds the address of the next instruction a thread will run.
Crash: An error that ends a program before its normal end.

Part 1: From a File to a Running Server

The Why Concurrency Matters lesson showed why the ticket server must do many things at once, and it does them with threads. This lesson looks at where those threads live, and at what they share.

The Program on Disk

The ticket website runs on a server program. Before anyone starts it, the program is only a file on disk, named ticket-server. The file holds instructions, such as "send the ticket" and "check the seat count" for each fan. A file on disk cannot serve a fan, because it uses no memory and no processor time.

The Running Process

When an engineer starts the program, the operating system does four things:

It creates a process. It gives the process an ID number, called the process ID.
It gives the process memory. It loads the program's instructions into that memory.
It creates the main thread. This thread starts at the program's entry point, such as the main function.
It runs the main thread on a core. Only now does the program begin to do its work.

The main thread loads the seat map into memory and opens a network connection point where fans can connect. Then it starts more threads, and each of them serves fans. A moment later, the server is one process with many threads, all running the same program.

Three Words, Three Things

A program is a file of instructions. It describes the work but does none of it.
A process is a program that is running. It owns memory and other resources from the operating system.
A thread is one path of work inside a process. It runs the program's instructions, one after another.

One program can become many processes, for example when the site starts the same ticket-server file 3 times. That gives 3 processes, and each one has its own memory and its own threads. They come from the same file, but they share no data in memory.

A kitchen works the same way. A recipe in a book is the program, because it describes a dish but cooks nothing. A kitchen where people cook that recipe is a process. It has its own counter, its own food, and its own tools. The cooks in that kitchen are the threads. They all use the same counter and the same food. Each cook also keeps a small notepad with their own notes and the step they are on. In a program, the counter and the food are the process's shared memory, and the tools are its open files and connections. Each notepad is one thread's own stack and program counter. A second kitchen is a second process, and its cooks cannot touch the food in the first kitchen.

Part 2: What a Process Is

The Definition

A process is a running copy of a program, with its own memory and its own resources. The operating system creates it, gives it an ID, and keeps track of it until it ends.

Each process owns the four things below.

Its address space, which holds the program's instructions, its global variables, its heap, and the stack of every thread.
Its resources, such as open files and the network connection point where fans connect.
Its process ID, the number the operating system uses to name the process.
At least one thread, because every process starts with its main thread.

The operating system keeps processes apart, so one process cannot read or change another process's memory. If a bug in the ticket server writes wrong values into its memory, other processes on the machine are not affected. Processes can still talk to each other through tools that the operating system provides, such as files, network connections, and pipes. A pipe is a channel that carries bytes from one process to another.

What a Process Is Not

A process is not the program. The program is the file, and the process is one run of it. The same file can run as many processes at once, and each process has its own data.

A process is not a thread. A process holds the memory and the resources, and its threads run the instructions. The operating system runs threads on cores, not whole processes.

A process does not keep every resource apart from other processes. Two processes can still change the same file on disk or the same row in a database, because only their memory is kept apart.

Part 3: What a Thread Is

The Definition

A thread is one path of work inside a process. It has its own place in the code and its own stack. It shares everything else with the other threads of the same process.

When a core is free, the operating system picks a thread that is ready to run and runs it there. The Thread Life Cycle and Scheduling lesson explains how it picks.

What Each Thread Keeps for Itself

Its stack. Each function call adds a block to the stack with that call's local variables. When the function returns, the block is removed. Two threads can run the same function at once, and each gets its own local variables.
Its registers, including the program counter. Registers are small, very fast storage places inside the core. When the operating system takes a core away from a thread, it saves the thread's registers. When the thread runs again, they are put back, so the thread continues at the exact place it stopped.
Its thread ID and its state, such as running or waiting.

The heap. Any thread that has a reference to an object on the heap can read it and change it.
Global and static variables, with one copy for the whole process, not one copy per thread.
The program's instructions, so many threads can run the same function at the same moment.
Open files, network connections, and the process ID. They belong to the process, so every thread can use them.

The table below puts the two lists side by side.

Shared by all threads of a process	Kept by each thread
The heap and every object on it	Its own stack and local variables
Global and static variables	Its own registers and program counter
The program's instructions	Its own place in those instructions
Open files and network connections	Its own thread ID and state

There is one exception to the global row. A thread-local variable looks like a global variable, but each thread gets its own copy. Java has ThreadLocal, Python has threading.local, C++ has thread_local, and C# has ThreadLocal<T>.

In Java, Python, and C#, a local variable that refers to an object holds only a reference, and the object itself lives on the heap. If a thread hands that reference to another thread, both threads now use the same object.

Each stack also sits inside the process's memory. It is private because each thread uses only its own stack, not because the operating system blocks the others. In C++, a thread that passes the address of a local variable lets another thread reach it.

The diagram below shows the ticket server process, its shared memory, and three of its threads.

The ticket server program starts as one process, and its three threads share the code, the heap, and the network connections while each keeps its own stack and program counter

What a Thread Is Not

A thread is not a process. It has no memory of its own beyond its stack, and it cannot outlive its process. When the process ends, every thread in it ends too.

A thread is not a core. A core is hardware, and a thread is work for a core to run. A ticket server with 200 threads can run on 4 cores, because the operating system lets the threads take turns.

A thread is not a copy of the data. Starting a thread does not copy the process's data, so the new thread works on the same objects as every other thread in the process.

Part 4: Seeing Shared and Private Memory in Code

The Code

The program below runs one process with a seat map of 10 seats. First, a box office thread sells seat 1 by changing the shared seat map. Then three worker threads serve fans, and each fan takes 20 ms. Worker 1 serves 2 fans, Worker 2 serves 3, and Worker 3 serves 4. Each worker counts its fans in a local variable named fans, and each one reads seat 1 from the shared map.

At the same time, a checkout thread holds seat 2 and then tries to charge a card, but the charge fails. The main thread waits for every thread and then prints what they left behind.

Before you look at the results, predict what the three workers report.

Quiz

Three worker threads run the same function in one process. The function counts fans in a local variable named fans, which starts at 0. Worker 1 serves 2 fans, Worker 2 serves 3, and Worker 3 serves 4. What count does each worker report?

Click an option to answer

Python3

. . . .

Here are two real runs of the Python version on one test machine:

1 process, 3 worker threads, 10 seats, 20 ms per fan
Box office thread : sold seat 1
Worker 1          : served 2 fans, saw seat 1 as sold
Worker 2          : served 3 fans, saw seat 1 as sold
Worker 3          : served 4 fans, saw seat 1 as sold
Checkout thread   : failed (the card was declined)
Seat 2 afterwards : held
Main thread       : 9 fans served in 84 ms

1 process, 3 worker threads, 10 seats, 20 ms per fan
Box office thread : sold seat 1
Worker 1          : served 2 fans, saw seat 1 as sold
Worker 2          : served 3 fans, saw seat 1 as sold
Worker 3          : served 4 fans, saw seat 1 as sold
Checkout thread   : failed (the card was declined)
Seat 2 afterwards : held
Main thread       : 9 fans served in 85 ms

Java, C++, and C# printed the same lines, and only the time changed. Java took 109 to 110 ms, C++ took 115 to 119 ms, and C# took 129 to 130 ms. On that test machine, each 20 ms sleep lasted a little longer in those three languages.

Notice the counts of 2, 3, and 4. All three workers ran the same function with the same local variable, fans. Yet each worker had its own copy on its own stack, so no worker changed another worker's count.

Notice also that every worker saw seat 1 as sold. The box office thread changed the seat map on the heap, and the other threads saw the change at once. Nobody copied the map or sent a message. That is sharing, and it is the main benefit of threads.

Notice seat 2 as well. The checkout thread changed shared memory and then failed before it finished, so its half-done change stayed behind. Every other thread now sees seat 2 as held, even though nobody bought it.

Notice the time last. One thread serving all 9 fans would need about 180 ms. The workers ran at the same time, so the program took about as long as Worker 3 needed for its 4 fans.

What Happens in Memory, Step by Step

Step 1: The process starts. The operating system creates the process and its main thread. The main thread creates the server object on the heap, with 10 free seats.

Step 2: The box office thread changes the heap. It writes "sold" into seat 1. The main thread waits for it with join, which waits until a thread ends.

Step 3: Four threads run together. Each worker gets its own stack with its own fans variable. All of them read the same seat map on the heap. The checkout thread writes "held" into seat 2 and then fails.

Step 4: The main thread reads the results. After join, it reads each worker's count and the seat map from the heap.

Each worker writes its count into its own slot of a shared array. No two threads write to the same slot, so the program needs no lock. The main thread reads the slots only after join, which also makes the finished thread's changes visible to it. The Race Conditions and Critical Sections lesson shows what goes wrong when threads change the same place.

In the program, seat 1 showed the benefit of sharing, and seat 2 showed the risk.

What sharing gives you:

Fast communication. A thread passes data to another thread by writing it into an object. The other thread reads it directly, with no copy and no help from the operating system.
Less memory. One seat map in memory serves every thread. Separate processes would each need their own copy.
Cheaper creation. A new thread needs only a stack and a few records. A new process needs a whole new address space.

What sharing costs you:

Any thread can change shared data at any moment. Two threads that change the same data at the same time can break it, and this bug is called a race condition.
A mistake in one thread can damage data for every thread, as the failed checkout did to seat 2.
One crash can end the whole process, with every thread in it, as Part 6 explains.

A simple rule follows from both lists: share only what the threads must share, and keep everything else local. The demo followed this rule, because each worker counted in a local variable and wrote to its own slot.

Part 6: When One Thread Fails

A Crash Ends the Whole Process

Some errors end the entire process at once. One example is C++ code that uses a pointer to memory the process does not own. The operating system stops the process, and every thread in it stops too. The other threads get no warning, and any work they had not saved is lost.

On the ticket site, this means one bad request can stop the server for every fan who is connected to it.

An Uncaught Exception Depends on the Language

An exception is an error that a program can catch and handle. If no code catches it, the result depends on the language. In a small test, one thread threw an exception that nothing caught. A second thread waited 100 ms and then printed a line.

Language	What happened on one test machine
Java	The failing thread printed the error and ended. The second thread printed its line, and the program reached its end.
Python	The same as Java. Only the failing thread ended.
C++	The program called `std::terminate`, and the whole process ended. The second thread never printed its line.
C#	The whole process ended with "Unhandled exception." The second thread never printed its line.

Notice that surviving is not the same as being correct. In Java and Python, the failing thread stops halfway through its work. Shared data can stay half-changed, as seat 2 did in Part 4.

So a server catches errors at the top of each thread's work, as the checkout thread did. The catch block should also undo the half-done change. In the ticket server, it would put seat 2 back to "free."

Part 7: Processes Versus Threads

Separate Memory, Seen in Code

The short Python program below changes a seat count once in a separate process and once in a thread. Python's multiprocessing module starts a new process with almost the same code that starts a thread.

from multiprocessing import Process
from threading import Thread

seats_left = 10


def sell_one_seat():
    global seats_left
    seats_left -= 1
    print("Inside the worker:", seats_left)


if __name__ == "__main__":
    process = Process(target=sell_one_seat)
    process.start()
    process.join()
    print("Main, after the process:", seats_left)

    thread = Thread(target=sell_one_seat)
    thread.start()
    thread.join()
    print("Main, after the thread:", seats_left)

Here is a real run on one test machine:

Inside the worker: 9
Main, after the process: 10
Inside the worker: 9
Main, after the thread: 9

The child process lowered its own copy of seats_left to 9. The main process still had 10, because the two processes do not share memory. The thread changed the one and only seats_left, so the main thread saw 9.

The diagram below shows the same two cases.

A child process changes only its own copy of seats_left, while a thread changes the one copy that the main thread also reads

Processes also cost more to start. On one test machine, starting and joining 20 threads in Python took about 2 ms, while 20 processes took 419 to 450 ms. That machine ran Windows, where each new Python process starts a fresh Python interpreter. The gap is smaller on Linux, but a process still costs more to start than a thread.

The Comparison

	Process	Thread
Memory	Its own. Other processes cannot see it.	Shares its process's memory with the other threads.
Cost to start	Higher. The system sets up a new address space.	Lower. It needs a stack and a few records.
Talking to others	Through files, pipes, or network connections	By reading and writing the same objects
After a crash	Other processes keep running.	The whole process can end, with every thread.
Switching a core between them	Usually costs more, because the address space changes too	Usually costs less inside one process

Use processes when:

A crash in one part must not stop the others. Many web browsers run each site in its own process for this reason.
The parts do not need to share much data.
In standard Python, the work is CPU-bound, which means it spends its time computing. Python's Global Interpreter Lock lets only one thread of a process run Python code at a time. Separate processes each have their own lock.

Use threads when:

The parts must share a lot of data, such as one seat map that every request reads.
The program starts many short pieces of work, and start time matters.

Many real servers use both. The ticket site can run 4 copies of ticket-server on one machine, as 4 processes with many threads each. A crash in one process leaves the other three serving fans. But the 4 processes do not share the seat count in memory. They must keep it in a place they all reach, such as a database.

FAQs

Q: Can two processes share memory?

Not by default. A process can ask the operating system for a block of shared memory, and other processes can then use the same block. Python offers this in multiprocessing.shared_memory. Data in that block needs the same care as data shared by threads.

Q: Does a process end when its main thread ends?

Not always. In Java, Python, and C#, the process keeps running while any normal thread is still alive. Threads marked as daemon or background threads do not keep it alive. In C++, returning from main ends the whole process, along with every other thread. So join your threads before main returns.

Q: In Python, should I always use processes instead of threads?

No. Threads work well in Python for I/O-bound work, which spends its time waiting, such as calls to the payment provider. A waiting thread lets the other threads run. For speed, processes help only with CPU-bound work. The Creating, Starting, and Joining Threads lesson explains the Global Interpreter Lock.

Key Takeaways

A program is a file, a process is a running copy of it, and a thread is one path of work inside the process. One program can run as many processes, and one process can run many threads.
Each process has its own memory. Other processes cannot read or change it, so they talk through files, pipes, or network connections.
Threads share the heap, global variables, code, and open connections. Each thread keeps its own stack, local variables, registers, and program counter.
Sharing is both the benefit and the risk. In the demo, every worker saw seat 1 as sold at once, and every worker also saw the failed checkout's seat 2 stuck as held.
One thread's failure can end the whole process. A real crash, such as a bad memory access in C++, ends it. An uncaught exception ends the process in C++ and C#, and only the thread in Java and Python.
Processes give isolation, and threads give cheap sharing. In the test, 20 Python threads started in about 2 ms, and 20 processes needed 419 to 450 ms.
In a coding interview, say what your threads share and what each one keeps local. Then say why you chose threads over processes, or processes over threads.

Every thread in the ticket server is always in some state, such as running on a core or waiting for the payment provider. The next lesson, The Thread Life Cycle and Scheduling, follows a thread through those states.

Check Your Understanding

1. Two threads run the same function, which has a local variable named count. One thread adds 1 to count. Does the other thread see the change?

<details> <summary>Show answer</summary>

No. Each thread has its own stack, so each call has its own count. The demo showed this when three workers reported 2, 3, and 4 fans.

</details>

2. The box office thread writes "sold" into the shared seat map. Does a worker thread need a copy of the map to see the change?

<details> <summary>Show answer</summary>

No. The seat map lives on the heap, and every thread in the process uses the same object. The worker reads the changed map directly.

</details>

3. In a C# program, a worker thread throws an exception that no code catches. What happens to the other threads?

<details> <summary>Show answer</summary>

They all stop, because the whole process ends. C++ does the same. In Java and Python, only the failing thread ends, but the shared data it was changing can stay half-changed.

</details>

4. The ticket site wants a bug in one server copy to never stop the other copies. Should it run them as threads or as processes?

<details> <summary>Show answer</summary>

As processes. Each process has its own memory, so a crash in one leaves the others running. The processes must then share data, such as the seat count, through a database.

</details>

Flashcards Review

What is the difference between a program and a process?

1 / 12

Mark as Completed

Reading Progress

Vote for new content

On This Page

Know the Vocabulary

Part 1: From a File to a Running Server

The Program on Disk

The Running Process

Three Words, Three Things

Part 2: What a Process Is

The Definition

What a Process Is Not

Part 3: What a Thread Is

The Definition

What Each Thread Keeps for Itself

What All Threads Share

What a Thread Is Not

Part 4: Seeing Shared and Private Memory in Code

The Code

What Happens in Memory, Step by Step

Part 5: Sharing Is the Benefit and the Risk

Part 6: When One Thread Fails

A Crash Ends the Whole Process

An Uncaught Exception Depends on the Language

Part 7: Processes Versus Threads

Separate Memory, Seen in Code

The Comparison

FAQs

Key Takeaways

Check Your Understanding