0% completed
1.2 Threads, Pipelining, and Scaling
On This Page
I/O Threads
The Network Bottleneck
How I/O Threading Works
Configuration and Gains
Other Background Threads
Other Contributors to Speed
Limits and Scaling Beyond One Core
Using This in an Interview
FAQs
Key Takeaways
One thread that executes every command against data in memory, with no locks on the keyspace, is the core of Redis speed. The previous lesson followed that chain from in-memory storage through the event loop to predictable latency, and it showed why a single slow command delays every client.
This lesson covers the work that does not run on that thread, and the limits of the thread itself. It explains how I/O threads take over socket reading, parsing and writing, which background threads handle slow system jobs, which other design choices such as pipelining add speed, and how Redis scales beyond the single core that executes commands. The example is still the online shop, with its page view counter page:views and user profiles such as user:42.
I/O Threads
The Network Bottleneck
As Redis matured, profiling showed that for small commands most CPU time was not spent executing commands. It was spent in the kernel and in protocol handling: system calls to read from sockets, parsing the requests, and system calls to write the replies. Command execution is fast, but moving bytes through the network stack is relatively expensive. On one thread, that cost caps throughput long before the CPU runs out of capacity for executing commands.
How I/O Threading Works
Redis 6 introduced threaded I/O, and Redis 8.0 replaced it with a new implementation. Both keep command execution on the main thread and move the socket work to a pool of I/O threads. In the current design, each request for user:42 follows this sequence:
- When a client connects, the main thread assigns it to the I/O thread that has the fewest clients. Each I/O thread runs its own event loop.
- The I/O thread reads the request from the socket and parses it into a command.
- The I/O thread hands the parsed command to the main thread, which executes it against the data and builds the reply.
- The main thread returns the client to its I/O thread, which writes the reply to the socket.
What did not change is the important part. Only the main thread touches the data, so there are still no locks on the keyspace, and every command is still atomic and ordered. Parallel work is applied only to the part that is naturally independent per connection: reading, parsing and writing. A few kinds of clients, such as replicas and Pub/Sub subscribers, stay on the main thread.
Configuration and Gains
Threaded I/O is off by default: the io-threads setting defaults to 1, which means only the main thread. The redis.conf guidance is to enable it only on machines with at least 4 cores, leaving at least one core spare, for example 3 I/O threads on a 4-core machine or 7 on an 8-core machine. It also advises enabling threads only when the instance actually uses a large share of a CPU core. Since Redis 8.0, the threads always handle both reads and writes, and the older io-threads-do-reads setting has no effect.
Redis reports that the 8.0 implementation gives up to 112 percent more throughput with io-threads 8 on a multi-core Intel CPU, depending on the command mix. Redis 8.4 reports more than 30 percent higher throughput for a caching workload of 10 percent SET and 90 percent GET on 4 cores. The gains are largest with many clients and small payloads. They are smallest when the commands themselves are expensive, because then the main thread's execution is the bottleneck.
Valkey, an open source fork of Redis maintained under the Linux Foundation, took the same approach in its version 8.0. It keeps command execution on the main thread and moves reading, parsing, writing, connection polling and memory freeing to I/O threads. In both projects the principle is the same: scale the network work, not the data access.
Other Background Threads
Redis was never strictly one thread. Background threads handle three kinds of slow jobs: closing files, running fsync for the append-only file, and freeing memory lazily. Lazy freeing serves UNLINK and FLUSHALL ASYNC, and also the optional lazyfree-* settings, which make eviction, expiration and other deletions free memory in the background. Those settings are all off by default in Redis. Moving these system operations off the main loop keeps the loop predictable.
Other Contributors to Speed
- A simple protocol. Clients talk to Redis in RESP (the Redis Serialization Protocol), a simple, human-readable format. A client sends each command as an array of strings, and each string carries its length in front. The server therefore knows how many bytes to read without scanning the data for delimiters, so parsing costs little. Redis 6 added RESP3, a newer version with more reply types, which a client chooses with the HELLO command.
- Pipelining. A client can send many commands without waiting for each reply. This removes the network round trip between commands, which often dominates latency, and it can raise throughput many times for batch workloads. Since Redis 8.4, the server also parses up to 16 pipelined commands ahead (the
lookaheadsetting), which lets it prepare their data in advance. - Compact encodings. Small hashes, lists, sets and sorted sets use listpacks, and small sets of integers use intsets, which are sorted arrays of integers. Data that sits close together in memory makes good use of the CPU caches, and this matters when a whole command takes a microsecond.
- Fork-based persistence. The main thread keeps serving while a child writes the snapshot. The price is extra memory when many pages change during the snapshot, because each changed page is copied.
- A small C codebase. There is no query planner, no join engine and no transaction log on the request path, and the command set is small, with documented complexity for each command.
Limits and Scaling Beyond One Core
A single Redis instance uses one core for command execution, so that core sets its throughput ceiling. Memory capacity is limited by RAM, and one very large instance has long fork times and slow recovery after a restart.
Redis therefore scales out rather than up. Redis Cluster divides the keyspace into 16,384 hash slots, assigns each key to one slot by hashing its name, and spreads the slots across several primary instances. Each shard still executes commands on one thread, but many shards run in parallel, and replicas add read capacity and failover. Running several instances on one multi-core machine is also common.
Latency can still degrade for reasons outside the design. Common causes are swapping to disk when memory runs out, Linux transparent huge pages causing stalls during fork, a neighboring process competing for the same CPU, and slow persistence settings. The latency monitor, the slow log and the INFO command reveal these problems.
Using This in an Interview
Say that for small commands the network work, not command execution, became the bottleneck, so I/O threads now read, parse and write while execution stays serial.
Do I/O threads make commands run in parallel? No: they only read, parse and write, and the main thread still executes every command, so the data needs no locks.
How do you use more than one core? You shard with Redis Cluster so that many single-threaded primaries run in parallel, and you can enable I/O threads for network-heavy workloads.
FAQs
Is Redis really single-threaded?
Command execution is, and that is the part that matters for correctness. Redis also uses background threads for lazy freeing, closing files and fsync, and since version 6 it can use I/O threads for reading and writing sockets. Only the main thread ever reads or changes the data.
What do I/O threads speed up?
They speed up reading requests from sockets, parsing them and writing replies. They do not speed up command execution. They help most with many connections, small payloads and high request rates, where network system calls take most of the CPU time.
Key Takeaways
- I/O threads parallelize only the network work. Reading, parsing and writing run on I/O threads, while command execution stays on the main thread, so the data still needs no locks.
- Background threads handle slow system jobs. Closing files, fsync for the append-only file and lazy freeing run off the main loop, so the loop stays predictable.
- Pipelining removes round trips. A client that sends many commands without waiting for each reply avoids the network round trip between them, which often dominates latency.
- Scale out, not up. One instance uses one core for commands, so Redis Cluster and replicas add throughput and capacity.
Reading Progress
0%
On This Page
I/O Threads
The Network Bottleneck
How I/O Threading Works
Configuration and Gains
Other Background Threads
Other Contributors to Speed
Limits and Scaling Beyond One Core
Using This in an Interview
FAQs
Key Takeaways