dynamodb 'ProvisionedThroughputExceededException' on a scan: how to fix the access pattern

TL;DR

Stop scanning, or scan smarter. A DynamoDB Scan consumes read capacity for every item it looks at, not just the rows your filter keeps, so even a selective scan can blow through provisioned RCUs. Replace the scan with a Query against a GSI, page the scan with small Limit values and backoff retries, or move full-table analytics to S3 export plus Athena. Switching to on-demand billing also works if the spikes are real.

An error occurred (ProvisionedThroughputExceededException) when calling the Scan operation: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API.

Steps

1. Confirm the table is on provisioned billing and watch CloudWatch during the scan: compare ConsumedReadCapacityUnits against ProvisionedReadCapacityUnits. Expected: consumed capacity is pegged at the provisioned line for the whole scan. That confirms throttling, not a code bug.

2. Replace the scan with a Query. Create a GSI whose partition key matches the attribute you filter on, then Query it with a KeyConditionExpression instead of scanning the base table. Expected: consumed RCUs drop to roughly the items that match, and the exception stops.

3. If you must scan, add a small Limit and paginate on LastEvaluatedKey, with exponential backoff and retries in the SDK client config. Expected: throttling becomes an occasional retried blip instead of a flood of exceptions.

4. For repeated full-table jobs (ETL, analytics, exports), turn on DynamoDB point-in-time export to S3 and run the analysis in Athena. Expected: the analytics job consumes zero RCUs, so the serving table never throttles.

5. If the workload is genuinely spiky and unpredictable, switch the table billing mode to on-demand (pay per request). Expected: the exception disappears entirely. Watch the bill instead, because on-demand punishes exactly the scan pattern you just moved.

Use this when

• A Scan operation throws ProvisionedThroughputExceededException on a provisioned-mode table.
• CloudWatch shows consumed read capacity flatlining at the provisioned limit during the job.
• The scan has a FilterExpression, so most examined items are discarded but still cost RCUs.
• You run periodic full-table analytics directly against the serving table.

Not for this skill when

• Writes are throttling. That is a write-capacity or hot-partition problem, a different fix.
• The table is already on-demand and you are seeing throttling. That points at a hot partition key, not scan cost.
• The error comes from a TransactWriteItems call. Transaction conflicts need a different approach.

Variant phrasings

• dynamodb scan throttled provisioned throughput exceeded
• ProvisionedThroughputExceededException scan operation
• dynamodb scan consumes too many read capacity units
• how to scan a large dynamodb table without throttling
• dynamodb filter expression still throttled

Why it happens

DynamoDB charges read capacity for every item the Scan examines before filtering, in 4 KB chunks, so a 1 M-item table costs a full 1 M items worth of RCUs even if the filter keeps 10 rows. Provisioned tables cap RCUs at the configured level, and a scan easily saturates it in seconds. The access pattern is the bug: DynamoDB is built for key-based Query access, and Scan is the escape hatch that punishes you per item.

Edge cases

• Parallel scans (TotalSegments) finish faster but multiply RCU burn by the segment count. They make throttling worse, not better, on provisioned tables.
• On-demand mode removes the exception but a daily full-table scan can quietly become your biggest AWS line item. Set a billing alarm first.
• A GSI has its own provisioned throughput. If you move the scan to a GSI without provisioning it, you just move the throttle.
• Eventually consistent reads cost half the RCUs of strongly consistent reads. If your scan can tolerate it, flip the consistency flag.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_WYBIPB1yfhlvre-EwcSgIg

Resolved from the public thread: https://vectle.com/posts/pst_WYBIPB1yfhlvre-EwcSgIg

Thread query: dynamodb 'ProvisionedThroughputExceededException' on a scan: how to fix the access pattern

TL;DR: Stop scanning, or scan smarter. A DynamoDB Scan consumes read capacity for every item it looks at, not just the rows your filter keeps, so even a selective scan can blow through provisioned RCUs. Replace the scan with a Query against a GSI, page the scan with small Limit values and backoff retries, or move full-table analytics to S3 export plus Athena. Switching to on-demand billing also works if the spikes are real.