Grokking Redis
Vote

0% completed

​

1.6 Memory Limits and Eviction

maxmemory and Eviction

The Setting

What Happens at the Limit

Choosing a Policy

Memory as a Budget

Where the Memory Goes

Per-Key Overhead

Measuring

Forecasting

Fragmentation

Reducing Memory

Common Failure Modes

Using This in an Interview

FAQs

Key Takeaways

Key names and TTLs decide which keys exist and how long they stay, but they do not decide what happens when memory runs out. The previous lesson followed the shop's session key shop:session:abc123 through naming, expiry options and the two ways Redis removes expired keys.

This lesson turns to the limit itself. It covers the maxmemory setting and the ten eviction policies, and then treats memory as a budget: where it goes, how to measure it, how to forecast it for the shop's sessions, and how to reduce it.

maxmemory and Eviction

The Setting

The maxmemory setting caps the memory that Redis may use for data. On 64-bit systems its default is 0, which means no limit, so Redis grows until the operating system intervenes. On Linux that can mean the out-of-memory killer terminates the process. Always set maxmemory in production, below physical RAM, and leave room for client buffers, replication buffers, the fork during persistence and the operating system.

What Happens at the Limit

When usage reaches maxmemory, Redis applies the eviction policy chosen with maxmemory-policy before it accepts a write that needs more memory. Redis has ten policies:

  • noeviction (the default) rejects writes that need more memory with an error, while reads keep working. It is right when data must not disappear silently, such as a queue or a primary store.
  • allkeys-lru evicts an approximately least recently used key among all keys. It is the standard choice for a pure cache.
  • allkeys-lfu evicts the least frequently used keys. It works better when a stable set of popular keys should survive an occasional scan of rarely used data.
  • allkeys-lrm (Redis 8.6) evicts the least recently modified keys. Only writes update its timestamp, so a key that is read often but rarely written can still be evicted.
  • volatile-lru, volatile-lfu, volatile-lrm and volatile-random apply the same rules, but only to keys that have a TTL.
  • volatile-ttl evicts, among keys with a TTL, the ones closest to expiring.
  • allkeys-random evicts random keys among all keys.

If no key has a TTL, the volatile policies have nothing to evict and behave like noeviction.

LRU, LFU, LRM and volatile-ttl are approximations. Redis samples a small number of keys, set by maxmemory-samples (default 5), and evicts the best candidate among them, keeping a small pool of good candidates between rounds. A higher sample count comes closer to exact LRU at slightly more CPU cost, and the approach avoids keeping all keys in a global order.

Redis maxmemory: What Happens When Memory Is Full?
Redis maxmemory: What Happens When Memory Is Full?

Choosing a Policy

Use an allkeys policy when Redis is a cache and every key can be recomputed. Use a volatile policy when the instance mixes durable keys, which have no TTL, with cache keys that have one, so only the cache part is evicted. Use noeviction when Redis holds data that cannot be lost, and monitor memory and add capacity before the limit is reached.

Two configurations cause most problems. A cache left on noeviction makes the application start failing writes once memory fills. A volatile policy on an instance where few keys have TTLs leaves almost nothing to evict.

Eviction also has a cost. Under constant memory pressure, each write may trigger eviction work, and the hit rate falls as useful keys are evicted. Evictions are a signal that the frequently used data no longer fits, so track the evicted_keys counter in INFO stats.

Redis Eviction Policies: LRU vs. LFU vs. TTL
Redis Eviction Policies: LRU vs. LFU vs. TTL

Memory as a Budget

Where the Memory Goes

Treat memory as a fixed budget: every key spends some, and you should know where it goes. The budget has five parts: the dataset itself with its per-key overhead, fragmentation, client and replication buffers, the extra memory used during a persistence fork, and headroom for growth.

Per-Key Overhead

Every key carries overhead beyond its value: the key string, an entry in the main hash table, an object header, and expiry metadata if it has a TTL. For small values, this overhead can be larger than the value itself. The Redis documentation reports that 1 million small string keys with small values use about 85 MB on a 64-bit instance, which is about 85 bytes for each key and value together. On a Redis 8.10 test server, one million keys such as key:1 holding values such as value:1 used about 70 MB, or about 70 bytes per pair. The exact figure depends on the memory allocator and on the lengths of the keys and values. At scale this overhead dominates, which is why storing many tiny items as fields inside hashes, with their compact encoding, can save a large amount of memory compared with one key per item.

Redis Memory Budget: Where RAM Goes
Redis Memory Budget: Where RAM Goes

Measuring

INFO memory reports used_memory, the resident set size (used_memory_rss, the RAM the operating system has given the process), the peak, and the fragmentation ratio. MEMORY USAGE key estimates one key's footprint including overhead, and MEMORY STATS gives a breakdown. The redis-cli --bigkeys and --memkeys options scan the keyspace to find the largest keys by element count and by memory, and offline tools that analyze snapshot files produce a full report by key prefix, which is where good naming helps. INFO keyspace shows, for each database, the number of keys and how many of them have an expiry.

Forecasting

To forecast memory, estimate the average bytes per entry, multiply by the expected key count, and add room for growth. Suppose the shop expects 10 million sessions, each a 45-byte key such as shop:session: followed by a 32-character ID, holding a 200-byte value with a TTL. On a Redis 8.10 test server, MEMORY USAGE reports about 280 bytes for one such key. After loading 100,000 of them, however, INFO memory shows about 340 bytes per session, because MEMORY USAGE does not count the key's share of the main hash table and of the index of expiry times. A forecast from MEMORY USAGE alone would therefore be about 20 percent too low, so load a sample batch and divide the growth in used_memory by the key count. With 340 bytes, the dataset is 10,000,000 × 340 bytes, or about 3.4 GB. A maxmemory of about 4.1 GB leaves 0.7 GB, which lets the dataset grow by roughly 20 percent.

The forecast must also include memory outside the dataset. During a snapshot, the fork shares pages with the parent, but every page the parent modifies is copied, so on a write-heavy instance memory could almost double in the worst case. Replication buffers and client output buffers, especially for slow subscribers and replicas, also use memory and should be limited by configuration. Redis also recommends setting the Linux vm.overcommit_memory option to 1 so that fork does not fail when free RAM is smaller than the dataset.

A common sizing guideline sets maxmemory to about 60 to 70 percent of RAM on instances that fork for persistence, and higher on pure caches without persistence, then adjusts from observed behavior. For the shop, 4.1 GB at 65 percent needs about 6.3 GB of RAM, so an 8 GB machine is the practical choice.

Fragmentation

The fragmentation ratio (mem_fragmentation_ratio) is used_memory_rss divided by used_memory. A ratio well above 1.5 suggests memory wasted by the allocator, often after many deletes of values with varied sizes. The ratio also counts other process overhead, so INFO reports allocator_frag_ratio as the direct measure of allocator fragmentation. Active defragmentation (activedefrag yes) can reclaim some of it while the server runs. A slower but reliable remedy is to restart a replica, which loads the data without fragmentation, and then fail over to it. A ratio clearly below 1 means part of Redis has been swapped to disk, which is a severe problem because reads then run at disk speed instead of RAM speed.

Reducing Memory

Several techniques reduce memory, roughly in order of how often they help:

  • Add TTLs to everything that can be recomputed, and remove big or abandoned keys found by scanning.
  • Store many small objects as hashes, and use integers instead of long strings and short field names when the volume is huge.
  • Compress large values in the client and choose a compact serialization format.
  • Trim lists, streams and sorted sets to bounded sizes, for example with capped streams.
  • Use approximate structures such as HyperLogLog and bitmaps where exact answers are not needed.
  • Use UNLINK rather than DEL for large keys, so freeing them does not block the main thread.
  • Split data across a cluster when one instance cannot hold it.

Common Failure Modes

  • A cache without maxmemory or eviction. Memory grows until the process is killed. Set both.
  • Wrong eviction policy. noeviction on a cache, or a volatile policy when few keys have TTLs.
  • Ignoring memory outside the dataset. Buffers and fork copies push usage over the limit even when the dataset fits.
  • Monitoring only used memory. Also watch evictions, hit rate, fragmentation, and the gap between maxmemory and system RAM.

Using This in an Interview

Say that memory is the hard budget in Redis: you set maxmemory below RAM with room for buffers and forks, and you choose an eviction policy that matches the role of the data.

Which policy suits a cache? Usually allkeys-lru, or allkeys-lfu when a stable set of keys is read far more often than the rest.

How do you size the instance? Load a sample batch, divide the growth in used_memory by the key count, multiply by the expected keys, and set maxmemory to about 60 to 70 percent of RAM on instances that fork.

FAQs

What happens when Redis reaches maxmemory?

It applies the configured eviction policy. With noeviction, it rejects write commands that need more memory. With the other policies, it evicts keys until enough memory is free and then runs the write.

Why does Redis use more memory than my data size?

Per-key overhead, fragmentation, replication and client buffers, expired keys that have not been deleted yet, and copy-on-write during snapshots all add to the dataset. Check INFO memory and the fragmentation ratio.

How do I find what is using memory?

Use INFO keyspace and INFO memory for totals, redis-cli --bigkeys or --memkeys to find large keys, MEMORY USAGE for individual keys, and offline analysis of a snapshot grouped by key prefix. Consistent naming makes the last method effective.

Key Takeaways

  • Always set maxmemory. Leave room for buffers, copy-on-write during forks, and the operating system.
  • Match the eviction policy to the data. allkeys-lru or allkeys-lfu for caches, volatile policies for mixed instances, noeviction for data that must not disappear.
  • Treat memory as a budget. Measure per-key overhead, forecast growth with real numbers, watch fragmentation, and reduce cost with hashes, compact types, bounded structures and compression.
  • Monitor the right signals. Evictions, hit rate, the fragmentation ratio, big keys and keyspace expiry counts reveal trouble before an outage.

Reading Progress

0%

Vote for new content

On This Page

maxmemory and Eviction

The Setting

What Happens at the Limit

Choosing a Policy

Memory as a Budget

Where the Memory Goes

Per-Key Overhead

Measuring

Forecasting

Fragmentation

Reducing Memory

Common Failure Modes

Using This in an Interview

FAQs

Key Takeaways