Grokking Operating Systems and Networking for Backend Engineers
Vote

0% completed

​

A Container Is Still a Process

What a Container Is

One Process, Two Views

Namespaces: What the Process Can See

Cgroups: What the Process May Use

What Stays the Same

In Production

Using This in an Interview

Key Takeaways

Practice Questions

A Container Is Still a Process
A Container Is Still a Process

Checkout was started in a Docker container, and its status file in /proc was read twice. Read from inside the container, the file said that Checkout is process 1. Read from the host, it gave two numbers for the one process:

Output:

Name: python Pid: 2736 PPid: 2713 NSpid: 2736 1

Both answers are correct. The host sees an ordinary Python process with PID 2736, and the container sees the same process as PID 1. Nothing was copied, emulated, or virtualized.

This page shows what a container changes about the process studied in this chapter, and, just as important, what it leaves exactly as it was.

What a Container Is

A container is one or more ordinary processes that the kernel runs with their own namespaces and inside their own cgroup, using a file tree unpacked from an image. The container runtime, which is Docker here together with its helpers containerd and runc, sets up those namespaces and the cgroup and then starts the program. After that, the runtime is not involved in the program's system calls.

Writing container configuration and running containers in a cluster are outside this course. This page looks only at the kernel features underneath.

The demo ran on a cloud virtual machine with 2 CPUs and Linux kernel 7.0. It started Checkout from the python:3.12-slim image with a memory limit of 256 MB and a CPU limit of half a CPU:

docker run -d --name checkout --memory 256m --memory-swap 256m --cpus 0.5 -p 8080:8080 -v $PWD:/app python:3.12-slim python /app/checkout.py

One Process, Two Views

Inside the container, the status file of PID 1 says that Checkout has no parent:

Output:

Name: python Pid: 1 PPid: 0 NSpid: 1

On the host, NSpid listed both numbers, and ps shows the real parent:

Output:

PID PPID USER COMMAND 2713 1 root containerd-shim 2736 2713 root python

Checkout's parent is containerd-shim, a small process that the runtime keeps for each container to start it, hold its exit code, and collect it when it ends. Inside the container, that parent is invisible, so the kernel reports a parent of 0.

One process, two views
One process, two views

Namespaces: What the Process Can See

A namespace is a kernel feature that gives a group of processes their own separate set of one kind of resource, such as process IDs or network interfaces, so that they see only the members of their own set. Each process belongs to exactly one namespace of each kind, and /proc/<pid>/ns holds one link per kind. Two processes in the same namespace show the same number:

Output:

namespace host container same? pid pid:[4026531836] pid:[4026532280] no net net:[4026531833] net:[4026532282] no mnt mnt:[4026531832] mnt:[4026532273] no uts uts:[4026531838] uts:[4026532277] no ipc ipc:[4026531839] ipc:[4026532279] no cgroup cgroup:[4026531835] cgroup:[4026532281] no user user:[4026531837] user:[4026531837] yes

The container's process differs from the host shell in six namespaces:

  • pid. Its own process numbers, starting at 1.
  • net. Its own network interfaces, addresses, ports, and routing table.
  • mnt. Its own file system tree, built from the image.
  • uts. Its own hostname.
  • ipc. Its own set of inter-process communication objects, such as shared memory segments.
  • cgroup. Its own view of the cgroup tree, in which its group appears as the root.

The user namespace is the same, so user ID 0 inside this container is the host's root user. Docker leaves the user namespace shared by default.

Docker is not needed to create namespaces. The unshare command asks the kernel for new ones directly, and here it started ps in a new PID namespace:

Output:

PID PPID COMMAND 1 0 ps

ps saw only itself, as process 1. This is the same mechanism Docker used, without the image or the cgroup. The --mount-proc option also gave ps a new mount namespace, so that /proc showed the new PID namespace.

Cgroups: What the Process May Use

A control group (cgroup) is a kernel feature that places processes in a group and limits the resources the whole group may use, such as memory and CPU time. Each cgroup is a folder under /sys/fs/cgroup, and its files hold the limits:

Output:

0::/system.slice/docker-<container id>.scope

Output:

268435456 50000 100000

memory.max is 268,435,456 bytes, which is 256 MB. cpu.max means 50,000 microseconds of CPU time in every period of 100,000 microseconds, which is half of one CPU. Docker wrote these values from --memory 256m and --cpus 0.5, and from then on the kernel enforces them.

A second container, greedy, had the same memory limit and tried to build a 400 MB string:

Output:

137

Output:

OOMKilled=true ExitCode=137

The group reached memory.max, and the kernel ended the process with SIGKILL. Exit code 137 is 128 plus 9, the same code that Signals, Graceful Shutdown and Exit Codes explained, and OOMKilled=true records that the kernel's out-of-memory handling chose it.

What a container changes
What a container changes

What Stays the Same

The container shares the host's kernel, which uname -r printed both outside and inside:

Output:

7.0.0-1017-azure 7.0.0-1017-azure

Because the kernel is shared, every mechanism from this chapter applies unchanged. Checkout's system calls cost the same, its threads are tasks for the same scheduler, and signals reach it in the same way. Host tools also work on it. strace ran on the host against PID 2736 while an order arrived:

Output:

accept4(3, {sa_family=AF_INET, sin_port=htons(<client port>), sin_addr=inet_addr("<docker bridge address>")}, [16], SOCK_CLOEXEC) = 4 recvfrom(4, "POST /orders HTTP/1.1\r\nHost: 127"..., 8192, 0, NULL, NULL) = 179 sendto(4, "HTTP/1.0 201 Created\r\nServer: Ba"..., 118, 0, NULL, 0) = 118 sendto(4, "{\"order_id\": 1042, \"item\": \"coff"..., 64, 0, NULL, 0) = 64

These are the same calls that One Request, Step by Step traced without a container. The only visible difference is the client's address, which belongs to Docker's bridge network, because Docker forwards the published port 8080 into the container's network namespace.

In Production

SymptomCauseHow to check
Stopping the container always takes the full grace period and ends with exit code 137The program is PID 1 and has no SIGTERM handler; the kernel ignores signals that PID 1 has not set a handler fordocker inspect for the exit code, then look for a handler in the code
Exit code 137 with OOMKilled=trueThe cgroup reached memory.maxcat memory.events in the container's cgroup folder: the oom_kill count
The service is slow while its CPU use looks moderateThe cgroup used its cpu.max quota early in each period and had to waitcat cpu.stat in the cgroup folder: nr_throttled
Zombie processes pile up inside the containerPID 1 is the application, and it never calls wait for orphansps -eo pid,ppid,stat,comm inside the container
You need strace or /proc on a containerized processThe tools are missing from the imageFind the host PID with docker inspect -f '{{.State.Pid}}' <name> and run the tools on the host

The first row surprises many teams. The kernel protects the first process of a PID namespace: a signal whose action is the default is not delivered to it, except SIGKILL and SIGSTOP sent from outside the namespace. The rule is described in the pid_namespaces manual page. It was not demonstrated in this lesson's run. A program that runs as PID 1 must therefore install its own SIGTERM handler, which is also why container tools offer a small init program to run as PID 1 in front of the application.

Using This in an Interview

A common question asks how a container differs from a virtual machine. A clear answer is: "A virtual machine runs its own kernel on emulated hardware. A container is an ordinary process on the host's kernel, isolated with namespaces and limited with a cgroup, so it starts as fast as a process and every container on the host shares one kernel."

  • "Can a container run a different kernel version from its host?" No. uname -r printed the host's kernel inside the container. Only the files of the image, such as the C library and the Python interpreter, differ.
  • "How is a container's memory limit enforced?" By the kernel, through memory.max in the container's cgroup. When the group exceeds it, the kernel ends a process in the group with SIGKILL, which appears as exit code 137.

Key Takeaways

  • A container is an ordinary process with its own namespaces and a cgroup; the runtime sets them up and then takes no part in the program's system calls.
  • The same process had PID 2736 on the host and PID 1 inside the container, and NSpid listed both.
  • Namespaces change what a process can see: process IDs, network, files, hostname, and IPC. Docker left the user namespace shared.
  • A cgroup limits what the group may use; memory.max was 256 MB and cpu.max was half a CPU, and exceeding the memory limit gave exit code 137.
  • The kernel is shared, so system calls, scheduling, signals, and host tools such as strace work exactly as they do outside a container.
  • PID 1 in a container ignores signals it has no handler for, so it must handle SIGTERM itself.

Every process in this chapter, inside a container or not, competes for the same few CPUs. The next lesson, How the Scheduler Shares the CPU, explains how the kernel decides which task runs next and for how long.

Practice Questions

Try each question first, then open the answer.

1. Inside a container, ps shows Checkout as PID 1. On the host, ps shows no process with PID 1 named python. Is Checkout running on the host at all?

<details> <summary>Show answer</summary>

Yes. It is one process with two numbers, one in each PID namespace. On the host it had PID 2736, and its NSpid line listed 2736 1. The host's PID 1 is the host's own init process.

</details>

2. A container's cpu.max file contains 50000 100000. How much CPU may the container use?

<details> <summary>Show answer</summary>

Half of one CPU. The group may run for 50,000 microseconds in every period of 100,000 microseconds. If it uses its share early in a period, its tasks wait until the next period starts, even when the machine has idle CPUs.

</details>

3. A team says their service cannot be traced because the image has no strace. What can they do instead?

<details> <summary>Show answer</summary>

Run strace on the host against the container's host PID. The container's process is an ordinary process on the host's kernel. docker inspect -f '{{.State.Pid}}' <name> gives the host PID, and strace, ps, and /proc work on it as on any other process.

</details>

4. A Python service stops at once on SIGTERM when it runs on a VM, but in a container it takes 10 seconds to stop and exits with code 137. What is the likely reason?

<details> <summary>Show answer</summary>

In the container, the service is PID 1, and it probably relied on the default action of SIGTERM. The kernel does not apply default signal actions to the first process of a PID namespace, so the signal was ignored. After the 10-second grace period, docker stop sent SIGKILL, which gave exit code 137. Installing an explicit SIGTERM handler fixes it.

</details>

5. Which namespace kept Checkout inside the container from seeing the host's network interfaces, and which feature kept it from using more than 256 MB?

<details> <summary>Show answer</summary>

The net namespace hid the interfaces, and the cgroup's memory.max limited the memory. Namespaces decide what a process can see, while a cgroup decides how much it may use. The demo showed a different net namespace number for the container and 268435456 in memory.max.

</details>

Reading Progress

0%


Vote for new content

On This Page

What a Container Is

One Process, Two Views

Namespaces: What the Process Can See

Cgroups: What the Process May Use

What Stays the Same

In Production

Using This in an Interview

Key Takeaways

Practice Questions