wrangler kv bulk put failed: bulk write limit of 5000 keys exceeded

TL;DR

The KV bulk API accepts at most 5000 keys per request. Larger uploads must be chunked into 5000-key batches with a small pause between them. Split the input file, loop the bulk command over chunks, and verify the final key count matches.

The query

wrangler kv bulk put failed: bulk write limit of 5000 keys exceeded

Use this when

• wrangler kv:key bulk put or kv bulk put fails naming the key limit.
• Seeding KV with a large dataset (feature flags, redirects, cached pages).
• A migration script builds one giant JSON array of keys.

Not for

• Single kv:key put failures (value size or key format problems).
• 'binding is not defined' at runtime (binding config problem).
• KV reads returning stale data (consistency behavior, separate topic).

Steps

Step 1: Count the keys in the upload file

python3 -c "import json; print(len(json.load(open('keys.json'))))"

Expected result: a number above 5000. That confirms batch size, not content, is the blocker.

Step 2: Split into 5000-key chunks

python3 -c "
import json
keys = json.load(open('keys.json'))
for i in range(0, len(keys), 5000):
    json.dump(keys[i:i+5000], open(f'keys_chunk_{i//5000}.json','w'))
    print(f'wrote chunk {i//5000}: {len(keys[i:i+5000])} keys')
"

Expected result: several chunk files, each with at most 5000 keys.

Step 3: Upload chunks sequentially with pacing

for f in keys_chunk_*.json; do
  npx wrangler kv:key bulk put --namespace-id [namespace-id] "$f"
  sleep 2
done

Expected result: each chunk uploads successfully. The pause keeps you under the write rate limit too; back-to-back bulk calls can trip the per-second throttle.

Step 4: Verify the total

npx wrangler kv:key list --namespace-id [namespace-id] | python3 -c "import json,sys; print(len(json.load(sys.stdin)))"

Expected result: the key count matches the input total. KV list paginates, so count across pages or trust the per-chunk success plus a spot check.

Step 5: Make the pipeline chunk-aware

Expected result: the seed script chunks automatically from now on. Hard-coding one bulk call guarantees this failure returns the next time the dataset grows.

Why it happens

Bulk endpoints exist to reduce round trips, not to accept unbounded payloads. The 5000-key cap bounds request size and server work per call. Seed scripts written against small dev datasets hit the wall the first time they run against production-sized data.

Edge cases

• Values have their own limits (25 MiB per value). Chunking keys does not fix oversized values.
• Bulk delete has the same 5000-key cap. Chunk deletes the same way.
• Keys with special characters must be valid UTF-8 and within the 512-byte key limit; bulk mode reports these per key, so check the output for partial failures.
• If the upload is recurring (nightly sync), consider writing through the worker at request time instead of bulk seeding; bulk is for bootstrapping.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_nc-Ae9l5u1V1bTNp2zUHjw