Grokking Redis
Vote

0% completed

​

2.5 Lists

What a List Is

Core Commands

Queues and Stacks

Queue (FIFO)

Stack (LIFO)

Blocking Commands

Why Blocking Matters

The Reliability Problem

LMOVE and BLMOVE

Capped Lists with LTRIM

The Pattern

Atomicity

Reading the Feed

Performance and Pitfalls

Lists or Streams

Using This in an Interview

FAQs

Key Takeaways

Two commands are enough to build a working job queue in Redis. A producer appends a job to the tail of a list, and a worker removes the oldest job from the head:

redis> RPUSH jobs:email "send-welcome:1001"
(integer) 1
redis> LPOP jobs:email
"send-welcome:1001"

A Redis list is an ordered sequence of strings. Adding or removing an element at either end takes constant time, which makes lists a natural fit for queues, stacks, and "most recent N" feeds. With blocking commands, a list also becomes a light job queue where workers wait for work instead of asking repeatedly. This lesson follows two examples: the email queue jobs:email, and an activity feed activity:42 that keeps the latest posts for user 42.

What a List Is

A list holds strings in insertion order, and the same value can appear many times. The left end is the head and the right end is the tail. Commands that start with L act on the head, and commands that start with R act on the tail.

Internally, a small list is stored as one listpack, a single contiguous block of memory that stores entries one after another with small headers and no pointers (the Hashes lesson describes it in more detail). When the list grows past the list-max-listpack-size limit (default -2, meaning about 8 KB per block), Redis converts it to a quicklist: a doubly linked list whose nodes are listpacks of up to 8 KB each. Since Redis 7.2, a quicklist that shrinks enough converts back to a single listpack, and OBJECT ENCODING reports listpack or quicklist.

This design gives O(1) operations at the head and tail with good memory efficiency. Access by index in the middle is O(N), because Redis must walk the nodes to reach the position. A list can hold up to 2^32 elements (about four billion), but a huge list is a big key that is slow to delete and cannot be spread across cluster nodes.

Redis List Structure
Redis List Structure

Core Commands

  • LPUSH and RPUSH add one or more elements to the head or tail, create the list if it does not exist, and return the new length. LPUSH with several values inserts them one at a time, so the last argument ends up first.
  • LPOP and RPOP remove and return an element from the head or tail, or return (nil) if the list is empty. Since Redis 6.2 they accept a count to pop several elements in one call, and since Redis 7.0 LMPOP pops from the first non-empty list among several keys.
  • LRANGE key start stop returns a slice. Negative indexes count from the end, so LRANGE key 0 -1 returns everything. It costs O(S+N), where S is the distance of the start from the nearer end and N is the number of elements returned, so request bounded ranges on large lists.
  • LLEN returns the length in O(1). LINDEX reads by position, LSET replaces an element by position, LPOS finds the position of a value, LREM removes occurrences of a value, and LINSERT inserts before or after a given element. These are all O(N).

Redis deletes a list automatically when its last element is removed, so empty lists never remain in the keyspace.

Redis List Commands
Redis List Commands

Queues and Stacks

Queue (FIFO)

For a first-in, first-out queue, producers RPUSH to the tail and workers LPOP from the head, as in the opening example. LPUSH with RPOP works equally well. Both ends are O(1), so throughput is high, and each pop is atomic, so two workers never receive the same element.

Stack (LIFO)

For a last-in, first-out stack, push and pop on the same end: LPUSH with LPOP, or RPUSH with RPOP. The most recent element comes out first, which suits undo histories and depth-first work lists.

Redis Queue vs. Stack
Redis Queue vs. Stack

Blocking Commands

Why Blocking Matters

A worker that calls LPOP in a loop and sleeps after each empty reply has two problems. It sends many useless commands to the server, and every job waits up to one sleep interval before anyone notices it. Blocking commands remove both problems.

BLPOP and BRPOP wait until an element is available or until a timeout passes. The timeout is in seconds, accepts decimals since Redis 6.0, and 0 means wait forever. They can watch several lists and pop from the first non-empty one in the order given, which gives simple priority queues: list the high-priority queue first. The reply names the list the element came from:

redis> BLPOP jobs:email:high jobs:email 5
1) "jobs:email"
2) "send-welcome:1002"

A blocking call blocks only the calling client. Redis parks the waiting connection, keeps serving other clients, and wakes the connection when data arrives. When several clients wait on the same list, Redis serves them in the order they started waiting, so work is spread fairly. Use a finite timeout, so workers regularly regain control to notice shutdown signals and replace dead connections.

The Reliability Problem

A plain pop removes the job from Redis before the worker finishes it. If the worker crashes halfway, the job is gone. This is at-most-once delivery: each job runs once or not at all, which is acceptable for some work, such as refreshing a cache, and unacceptable for others, such as charging a card.

LMOVE and BLMOVE

LMOVE source destination LEFT|RIGHT LEFT|RIGHT pops an element from one end of one list and pushes it to an end of another list, atomically, and returns it. BLMOVE is the blocking version. Both arrived in Redis 6.2 and replace RPOPLPUSH and BRPOPLPUSH, which are deprecated. The reliable queue pattern uses BLMOVE in four steps:

  1. A worker runs BLMOVE jobs:email jobs:email:processing:w1 LEFT RIGHT 5, which moves the oldest job to that worker's processing list. Because the move is atomic, the job is always in exactly one of the two lists.
  2. The worker processes the job.
  3. On success, the worker removes the job from its processing list with LREM jobs:email:processing:w1 1 <job>.
  4. A separate recovery process checks the processing lists for jobs that stayed too long and moves them back to jobs:email. To know how long a job has waited, store a start time for it, for example in a hash keyed by job ID.

This gives at-least-once delivery: no job is lost, but a job can run twice if a worker finishes it and crashes before step 3, so job handlers must be idempotent. Redis does not track how long an element has been in a list, so the recovery logic is yours to build. Redis 8.10 added LMOVEM and BLMOVEM, which move up to a given number of elements (COUNT), or exactly that number (EXACTLY), in one call, so a worker can take a batch of jobs at once. The count must be followed by OBO, which places the elements one by one as repeated LMOVE calls would, or BULK, which keeps the batch in its original order. For example, LMOVEM jobs:email jobs:email:processing:w1 LEFT RIGHT COUNT 10 OBO moves up to ten jobs, and BLMOVEM takes its timeout before the count: BLMOVEM jobs:email jobs:email:processing:w1 LEFT RIGHT 5 COUNT 10 OBO.

Redis Reliable Queue
Redis Reliable Queue

LMOVE can also use the same list as source and destination, taking from one end and pushing to the other. That rotates the list, which gives round-robin processing over a fixed set of items, such as a list of servers to health-check.

Capped Lists with LTRIM

The Pattern

LTRIM key start stop keeps only the elements in the given range and deletes the rest. Paired with a push, it bounds the list size. For the activity feed, LPUSH activity:42 <post> followed by LTRIM activity:42 0 99 keeps the newest 100 posts. The result is a rolling window of recent activity, such as the latest notifications, recently viewed items, or the tail of a log, and its memory stays bounded however long the system runs.

Atomicity

The push and the trim are two commands, so other clients can run commands between them, and a crash between them skips the trim. If the bound must always hold, send both inside MULTI and EXEC or in a Lua script. Pipelining alone sends them in one round trip but does not make them atomic. For most feeds, a brief overshoot is acceptable, because the next push and trim restore the bound.

Reading the Feed

LRANGE activity:42 0 19 returns the newest 20 posts for the first page, and further ranges page through the window. Because the window holds at most 100 entries, paging stays cheap.

Redis Capped Lists
Redis Capped Lists

Performance and Pitfalls

  • Middle access is slow. LINDEX, LSET, LPOS, LREM, and LINSERT are O(N). If you need random access or ordering by a score, use a sorted set.
  • Big lists are big keys. An unbounded list grows memory without limit. Cap it or make sure workers keep up, and delete large lists with UNLINK, which frees memory in a background thread.
  • Large LRANGE calls keep the main thread busy. Fetch bounded pages.
  • No acknowledgment or visibility timeout. Lists have no built-in retry, dead-letter handling, or view of in-flight work. Reliable queues need the processing list pattern and a recovery process.
  • Blocking and connection pools. A blocked client holds its connection until it returns. Do not run BLPOP on a pooled connection that other code shares; give each blocking worker its own connection.
  • Cluster. LMOVE and BLMOVE need both keys in the same hash slot, so use hash tags such as {jobs:email} and {jobs:email}:processing:w1.
  • Persistence. Queued jobs are only as durable as the persistence settings, so a crash can lose the most recent pushes unless the append-only file is configured for it.

Lists or Streams

Use a list for simple, high-throughput queues where occasional reprocessing is acceptable and you want little machinery, and for capped feeds of recent items. Use a stream when you need consumer groups, per-message acknowledgment, tracking of pending entries, reclaiming of stalled work, replay of history, or several independent readers of the same data. Use Pub/Sub only for live broadcast that stores nothing, and use a sorted set for delayed jobs and priority by score. The Reliable Job Queues with Lists lesson and the Redis Streams lesson cover both designs in depth.

Using This in an Interview

Describe a list as an ordered sequence with O(1) operations at both ends, stored as a listpack when small and a quicklist of listpacks when large. Show the queue (RPUSH and LPOP), blocking pops that remove polling, and BLMOVE with a processing list for at-least-once delivery.

What happens if a worker crashes mid-job? With a plain pop the job is lost, while with BLMOVE it stays in the processing list until a recovery process moves it back.

When would you switch to a stream? When you need acknowledgments, automatic reclaiming of stalled messages, replay, or several consumer groups reading the same data.

FAQs

What is the difference between LPOP and BLPOP?

LPOP returns immediately, with (nil) if the list is empty. BLPOP waits until an element arrives or the timeout passes, so workers do not need to poll.

How do I build a reliable queue with lists?

Use BLMOVE to move each job atomically from the pending list to a processing list, remove it with LREM when done, and run a recovery process that moves stale jobs back. Make handlers idempotent, because a job can run more than once.

How do I keep only the latest N items?

LPUSH the new item and LTRIM the list to positions 0 through N minus 1, inside MULTI and EXEC or a Lua script if the bound must never be exceeded.

Are list operations atomic?

Each command is atomic, so a pop never gives the same element to two clients. A sequence such as a push and then a trim needs a transaction or a script to be atomic as a whole.

Can several workers share one list?

Yes. Each element goes to exactly one worker, and blocked workers are served in the order they started waiting, which spreads work fairly.

Key Takeaways

  • Lists are ordered sequences with fast ends. Push and pop at the head or tail are O(1), while access in the middle is O(N).
  • A queue is RPUSH plus LPOP, and a stack uses one end for both. Pops are atomic, so workers never receive the same element.
  • Blocking pops remove polling. BLPOP and BRPOP wait for data, serve waiting clients in order, and give simple priority by listing queues in order.
  • BLMOVE enables reliable queues. Move jobs to a processing list, remove them on success, and move stale jobs back, accepting at-least-once delivery.
  • LTRIM bounds size. A push followed by a trim keeps a rolling window, atomically when wrapped in MULTI and EXEC or a script.
  • Avoid big lists and operations in the middle. Cap lists, read in pages, and use sorted sets when you need ordering by score.
  • Choose streams for richer messaging. Consumer groups, acknowledgments, and replay are built in.

Reading Progress

0%

Vote for new content

On This Page

What a List Is

Core Commands

Queues and Stacks

Queue (FIFO)

Stack (LIFO)

Blocking Commands

Why Blocking Matters

The Reliability Problem

LMOVE and BLMOVE

Capped Lists with LTRIM

The Pattern

Atomicity

Reading the Feed

Performance and Pitfalls

Lists or Streams

Using This in an Interview

FAQs

Key Takeaways