DynamoDB Hot Partitions: Why They Happen and How to Fix Them

A hot partition is a single DynamoDB partition that receives far more traffic than the others, and it throttles your requests even when the table as a whole has capacity to spare. The cause is almost always the partition key, because the key you chose sends a disproportionate share of reads or writes to one key value.

This happens because DynamoDB provisions capacity at the table level but enforces its limits per partition. A table configured for 10,000 writes per second can still reject writes if they all target the same key value.

This article explains what a partition is, why the throttling happens with capacity left over, how the two capacity modes differ, and the five designs that cause the problem. A worked example sets out the arithmetic. The rest covers detection, six fixes with the cost each one carries, and how to pick a shard count. It closes on read problems against write problems, index hot partitions, and changing a partition key on a table that is already serving traffic.

What is a partition in DynamoDB?

A partition is the unit of storage and throughput behind a DynamoDB table. Each partition is stored on solid state drives and replicated across three availability zones within a region.

DynamoDB hashes the partition key of every item and uses that hash to choose the partition the item lives in. Items that share a partition key value are always stored together. That single fact is the root of the whole problem.

Every partition carries three hard limits:

• 3,000 read capacity units per second. One read capacity unit, or RCU, is one strongly consistent read per second of an item up to 4 KB. An eventually consistent read of the same item costs half an RCU.
• 1,000 write capacity units per second. One write capacity unit, or WCU, is one write per second of an item up to 1 KB.
• 10 GB of data.

A hot partition is a partition that reaches one of those limits while the rest of the table sits mostly idle.

A closely related term is the hot key. A hot key is one partition key value that attracts the traffic, like a single product id during a sale. A hot key produces a hot partition, because that key value can only be stored in one place.

Storage has a second limit worth knowing. All the items that share a partition key value are called an item collection. If your table has a local secondary index, which is an index that keeps the same partition key but adds a different sort key, then that whole item collection has to fit inside 10 GB. A write that breaches that limit fails with ItemCollectionSizeLimitExceededException. Without a local secondary index, DynamoDB can split a large item collection across partitions as it grows.

Why you get throttled with capacity left over

Your provisioned capacity is a table-level number, but the enforcement is local. When one partition passes its own limit, DynamoDB rejects the request and your client receives a throttling error, even though the table total is nowhere near the configured ceiling.

Two mechanisms soften this, and you should know the limits of both.

Burst capacity lets a partition draw on up to 300 seconds of its own unused capacity. It absorbs a short spike and does nothing for sustained traffic. Once the saved capacity is spent, the per-partition limits apply again.

Adaptive capacity shifts throughput toward a busy partition automatically, and it can isolate a frequently accessed item onto a partition of its own. It is on by default and it works within seconds. What it cannot do is lift a single partition key value above the per-partition limits. This is the distinction interviewers listen for: adaptive capacity fixes an uneven spread across many keys, and it does not fix one key that is too busy on its own.

When a request is rejected, the AWS SDK retries it with exponential backoff, which means it waits a little longer before each attempt. That retry is useful during a brief spike and harmful during a sustained one, because the retries themselves add load. Watch for a throttling rate that stays flat for minutes rather than seconds.

Provisioned and on-demand capacity behave differently

DynamoDB offers two capacity modes, and candidates often assume one of them removes the problem. It does not.

Provisioned capacity means you set the read and write units for the table. You pay for what you reserve, and you can attach auto scaling that adjusts the reservation as traffic changes. Auto scaling reacts over minutes, so it is too slow for a spike that arrives in seconds.

On-demand capacity means you pay per request and plan no capacity. A new on-demand table sustains 4,000 writes and 12,000 reads per second straight away. From there it accommodates up to double your previous traffic peak, and that peak ratchets upward as traffic grows. Growth beyond double the previous peak within 30 minutes can still be throttled, so AWS advises spreading a large increase over at least that long.

On-demand is not unbounded, despite the name. You can set a maximum read or write throughput per table to keep cost predictable, and each account has a default table-level ceiling of 40,000 read and 40,000 write units per second. Both are limits on request rate rather than reservations you pay to hold.

Neither mode changes the per-partition limits. The table-level ceiling moves, and the 3,000 RCU and 1,000 WCU boundaries on a single partition stay where they are. The difference that matters for diagnosis is that provisioned mode gives you a number to compare against, which makes an uneven spread obvious. On-demand offers no such number, so you work from the throttling reason instead, which the next section covers.

The five designs that create hot partitions

Cardinality is the number of distinct values a key can take. High cardinality is necessary but not sufficient, because the traffic also has to be spread evenly across those values. A user id has millions of distinct values and still produces a hot key when one account is a very large customer.

The first row is the most common mistake in interviews, and the second is the most common in production. Storing a time series under a date key feels natural until every write for today competes for one partition.

The last row is the one that misleads people, because the key design is sound and the access pattern is not. It returns under detection, since it is the case where no culprit key exists to find.

A worked example: an orders table at 2,400 writes per second

Take an orders table that receives 2,400 writes per second at peak, where each order record is about 2 KB.

An item up to 1 KB costs one write capacity unit, and the cost rounds up per kilobyte. A 2 KB item therefore costs 2 WCUs, so the real write demand is 4,800 WCUs per second.

Spread evenly, that is well within what DynamoDB can serve. The table needs at least five partitions to stay under the 1,000 WCU per partition limit, and DynamoDB creates partitions as capacity and storage grow.

Now give the table a partition key of order_date. Every order written today shares one key value, so all 4,800 WCUs target one partition. That partition serves 1,000 and rejects the rest. Roughly 79 percent of writes fail, and the table reports consuming a fraction of its provisioned capacity while doing so.

The arithmetic is what makes this concrete in an interview. Say the item size, convert it to capacity units, divide by the per-partition limit, and you have the minimum number of distinct key values the design needs. Here that is 4,800 divided by 1,000, which rounds up to five. A key with only one active value per day can never satisfy that.

How to tell whether you actually have one

DynamoDB names the cause in the error itself, which makes this more direct than it used to be.

Read the throttling reason. Every throttling exception carries a ThrottlingReasons list, and each entry holds a reason string and the ARN of the table or index affected. The reason string follows the pattern resource type, then operation type, then limit type, as in TableWriteProvisionedThroughputExceeded. The limit type is the part that answers this question.

A limit type of KeyRangeThroughputExceeded means one partition passed the internal per-partition limit. So TableWriteKeyRangeThroughputExceeded is a hot partition, stated outright, and IndexWriteKeyRangeThroughputExceeded is a hot partition on an index. A limit type of ProvisionedThroughputExceeded is a different problem, because it means the table or index as a whole is under-provisioned. Separating those two cases used to require inference, and it no longer does.

Which exception you catch depends on the mode. Provisioned tables raise ProvisionedThroughputExceededException, on-demand tables raise ThrottlingException, and an account-level ceiling raises RequestLimitExceeded. All three carry the same reason fields.

Watch the matching metric. Each reason maps to a CloudWatch metric of its own. Per-partition throttling appears in ReadKeyRangeThroughputThrottleEvents and WriteKeyRangeThroughputThrottleEvents, published for the table and for each index. The older metrics, ThrottledRequests, ReadThrottleEvents, and WriteThrottleEvents, aggregate every kind of throttling together, so they report that requests are being rejected without saying why.

Compare throttling against provisioned capacity. On a provisioned table, compare ConsumedWriteCapacityUnits against what you reserved. If writes fail while the table consumed a third of its allowance, the problem is distribution rather than volume. This remains a useful cross-check, and an on-demand table has no reservation to compare against.

Name the culprit. CloudWatch Contributor Insights for DynamoDB reports the most accessed partition keys and the most throttled keys, for the table and for its indexes. Its throttled keys mode processes events only when throttling occurs, which makes it cheap enough to leave running. This moves you from knowing that a partition is hot to naming the key value responsible, which is what you need before choosing a fix.

One result deserves care. If the key range metric is high and Contributor Insights shows no key standing out, the cause is usually sequential access rather than one busy key. A continued scan, or writes issued in the order a scan returned them, loads one key range at a time. The answer there is to rate-limit that traffic below the per-partition limits, or to use parallel segmented scans that spread reads across partitions.

Check the storage signal too. If a single item collection is approaching 10 GB on a table with a local secondary index, you have a size problem that will stop writes regardless of throughput.

The data layer decides the outcome of most design rounds. Grokking the System Design Interview works through the storage choices behind the problems that come up most often.

Six fixes, and what each one costs

The two sharding rows are the same idea with different read costs, and choosing between them is a real design decision rather than a detail.

With a random suffix, a write picks a number between 1 and N at random, so user#842 becomes user#842#7. Writes spread evenly across N partitions. A read then has to query all N shard keys and merge the results, because you cannot know which shard holds a given item.

With a calculated suffix, the number comes from something you will know when you read. If you always read an order by its order id, then hash(order_id) % N gives a suffix you can recompute later, and the read stays one query. The constraint is that the access pattern has to be fixed in advance. A second access pattern that does not know the order id is back to querying every shard.

The fifth row deserves a note. Per period tables suit a workload where old data is queried rarely and deleted on a schedule, because dropping a table is far cheaper than deleting millions of items one at a time.

The sixth row is the only one that leaves your keys alone. The request rate a table can serve the moment traffic arrives is called its warm throughput, and DynamoDB raises it automatically as usage grows. Ahead of a launch or a sale, you can raise it yourself, which is known as pre-warming the table. It is the right answer for a spike you can predict, and the wrong answer for a hot key, because the per-partition limits still apply.

How to choose the shard count

Pick N from the peak write rate for the busiest key, not the average, and leave room to grow.

Take the worked example above. The busiest key needs 4,800 WCUs per second, and one partition serves 1,000. Four shards leaves each one at 1,200, which still throttles. Eight shards leaves each at 600, which is comfortable and allows the traffic to grow before the design has to change again.

Two constraints pull in opposite directions. A larger N spreads writes better. A larger N also makes every scatter read more expensive, because a random suffix scheme queries all N shards and merges them.

Changing N later is the part people underestimate. The suffix is part of the key, so existing items stay where the old scheme put them. Raising the count means rewriting the affected items into their new keys, which is a migration rather than a configuration change. Choose a number you can live under for a year of growth.

Read hot keys need different fixes from write hot keys

Write sharding solves write concentration. It does nothing useful for a popular item that millions of people are reading, because the reads still target whichever shard holds that item.

For read concentration, three answers are worth naming.

A cache in front of the table. DynamoDB Accelerator, or DAX, is a managed in-memory cache that sits in front of a table and serves repeated reads in microseconds. Any shared cache works for the same purpose. The cost is staleness, so you have to say how long a stale read is acceptable.

Replicate the hot item. Write the same item under several keys, then have readers pick one at random. Reads spread across those copies. The cost is that every update now writes several items, and they can diverge briefly.

Collapse duplicate requests. If a thousand readers ask for the same item in the same moment, one request can fetch it and serve them all. This reduces load without adding staleness beyond the time of a single fetch.

Say which problem you are solving before you name a fix. Candidates lose points by proposing write sharding for a celebrity read problem, where it adds complexity and changes nothing.

Hot partitions on a global secondary index

A global secondary index, or GSI, is an index with its own partition key, an optional sort key, and its own partitions. It can be hot even when the base table is spread perfectly.

This happens whenever the index key has lower cardinality than the table key. A table keyed by order id is spread well, and an index keyed by order status has only a handful of distinct values, so writes to the index concentrate on that handful of partitions.

The consequence is easy to miss. On a table with provisioned capacity, every write to the base table that also updates the index consumes index capacity. If the index does not have the capacity, those writes are throttled, and that throttling reaches back to the base table. An index you query rarely can therefore stop your main write path. An index inherits its capacity mode from the base table, so this coupling applies wherever that base table is provisioned.

Two moves help. The first is to raise the cardinality of the index partition key. A global secondary index accepts a partition key composed of up to four attributes, and a sort key of up to four more, so you can combine status with a tenant or a bucketed timestamp without concatenating strings yourself. The constraint is that a query has to supply every partition key attribute by equality. The second move is to make the index sparse, meaning only items that carry the indexed attribute appear in it, which shrinks both its size and its write cost.

Changing a partition key on a table that is already live

You cannot change the partition key of an existing DynamoDB table. The key determines placement, so a new key means a new table and a migration. Interviewers like this question because it tests whether you have operated a system rather than only designed one.

A workable sequence has four steps.

Create the new table with the corrected key design, and size it for the backfill as well as for live traffic.

Write to both tables from the application, so every new record exists in both while the old table still serves reads.

Backfill the history. Export the old table to object storage and import it into the new one, which avoids spending table capacity on a full scan. A scan with a worker pool also works for a smaller table, and it competes with live traffic for capacity.

Move reads over, then retire the old table. Shift a small share of reads first, compare results, then move the rest. Keep the dual write until you are confident, because it is the only cheap way back.

The order matters. Starting the dual write before the backfill means the backfill copies a set of records that is no longer growing.

What is expected at each level

Four moves separate a confident answer from a vague one. Name the per-partition limit with a number, state your partition key and defend its cardinality, say what adaptive capacity does and does not fix, and choose a sharding scheme while naming its read cost.

How far you take those moves is what changes with seniority.

Mid-level. Recognize that the partition key determines placement, and that a low cardinality key concentrates traffic. Propose a better key and explain why it spreads the load.

Senior. Produce the arithmetic without prompting. Convert item size to capacity units, divide by the per-partition limit, and state the minimum key cardinality the design needs. Choose between a random and a calculated suffix on the basis of the read pattern, and say what the choice costs.

Staff and above. Treat it as an operational problem as well as a design one. Say how you would detect the hot key in a running system, what the migration to a new key looks like, how the index interacts with the base table, and what you would monitor afterward. Naming the key range throttling reason, rather than an aggregate throttle count, is the detail that signals you have debugged this in production. At this level the interesting answer is usually about the path from the current system to the better one.

Frequently asked questions

Does on-demand capacity prevent hot partitions? No. On-demand removes the job of guessing table capacity, but the per-partition limits of 3,000 RCUs and 1,000 WCUs still apply. On-demand also accommodates up to double your previous peak, so traffic that more than doubles inside 30 minutes can still be throttled. Pre-warming the table raises the rate it serves immediately, which covers a spike you can predict.

Does adaptive capacity make hot partitions a non issue? No, it reduces them. DynamoDB shifts throughput toward a busy partition and can isolate a frequently accessed item onto its own partition. It cannot raise one partition key value above the per-partition limits, so a single very busy key still needs a key design change.

What is the difference between a hot partition and a hot key? A hot key is one partition key value that attracts a disproportionate share of traffic. A hot partition is the storage and throughput unit that key is stored on. The hot key is the cause and the hot partition is the effect.

Can a global secondary index have a hot partition? Yes. A global secondary index has its own partition key and its own partitions, so a low cardinality index key creates a hot partition there. On a table with provisioned capacity, a throttled index can also throttle writes to the base table.

How many shards should a write sharding scheme use? Enough that the peak write rate for the busiest key, divided by the shard count, sits well under 1,000 writes per second. Start from the measured peak rather than the average, and leave headroom, because changing the count later requires rewriting existing items.

Which error tells me a hot partition is the cause? Any throttling exception whose reason string ends in KeyRangeThroughputExceeded. A reason of TableWriteKeyRangeThroughputExceeded means one partition of the base table passed the per-partition write limit. A reason ending in ProvisionedThroughputExceeded points instead at a table or index that is under-provisioned, which calls for a different fix.

What is pre-warming a DynamoDB table? Warm throughput is the request rate a table can serve the moment traffic arrives. DynamoDB raises it as usage grows, and you can raise it yourself before a peak you know about, which is pre-warming. The value cannot be lowered afterward, and it does not move the per-partition limits.

Why am I throttled per partition when Contributor Insights shows no hot key? Sequential access is the usual answer. A continued scan, or writes issued in the order a scan returned them, loads one key range at a time without any single key standing out. Rate-limit that traffic below the per-partition limits, or use parallel segmented scans to spread reads across partitions.

How do I find the key that is causing the problem? Turn on CloudWatch Contributor Insights for the table. It reports the most accessed and most throttled partition keys, which names the culprit instead of leaving you to guess from aggregate metrics.

Does a hot partition affect storage as well as throughput? It can. A partition stores up to 10 GB, and on a table with a local secondary index every item sharing a partition key value has to fit within that limit together. A key with far more items than the others reaches the size limit even when its request rate is modest.

Is a hot partition the same problem in Cassandra? The shape is the same and the numbers are not. Any system that places data by hashing a key concentrates traffic when the key has low cardinality. The per-partition limits, the detection tooling, and the automatic mitigation differ by system, so the fix has to be chosen against the real limits of the one you are using.

Advanced System Design Interview, Volume II goes further into the storage layer, including a walkthrough of Amazon's Dynamo design and the partitioning and replication ideas it introduced.

Related reading

• What is a hot partition and how to avoid it, the version of this question that covers distributed databases in general
• PostgreSQL vs MongoDB vs DynamoDB, for choosing between them in the first place
• Is NoSQL read or write heavy?
• Vector clocks explained, the conflict resolution idea that came out of the same Dynamo design
• System design failure modes