workerd error: Durable Object storage value exceeds 128 KiB limit

Not specified by the original author.

TL;DR

Individual Durable Object storage values are capped at 128 KiB. Storing big blobs, long histories, or serialized state in one key fails. Shard the value across multiple keys, keep only hot state in DO storage, and move bulk data to R2 or D1.

The query

workerd error: Durable Object storage value exceeds 128 KiB limit

Use this when

• storage.put inside a Durable Object fails naming the value size.
• The DO accumulates history, logs, or session state in one key.
• The failure started as the stored state grew.

Not for

• 'Durable Object ID is not valid' (id problem).
• Alarm scheduling failures (separate fix).
• Total storage quota errors (different limit).

Steps

Step 1: Find the growing key

Log value byte lengths on put, or list keys and check sizes.

Expected result: one key (often 'state', 'history', or 'log') dominates. That key needs sharding.

Step 2: Shard across numbered keys

const SHARD_SIZE = 100 * 1024;
async function putLarge(storage, base, bytes) {
  const total = Math.ceil(bytes.byteLength / SHARD_SIZE);
  const entries = {};
  let i = 0;
  while (i !== total) {
    const k = base + ":shard:" + i;
    entries[k] = bytes.slice(i * SHARD_SIZE, (i + 1) * SHARD_SIZE);
    i = i + 1;
  }
  await storage.put(entries);
}

Expected result: each shard stays under the cap. Reads reassemble by listing the shard prefix.

Step 3: Keep only hot state in the DO

Move append-only history to D1 or R2 and keep the DO's storage for what the request path actually needs: current state, locks, coordination data.

Expected result: the DO stays small and fast. Bulk data lives in systems designed for bulk data.

Step 4: Bound unbounded growth

Cap histories (keep last N entries), compact logs, expire old shards with alarms.

Expected result: storage stops growing without bound. The error cannot return via slow accumulation.

Step 5: Verify under load

Expected result: concurrent writes to sharded keys succeed. Watch for the write-contention behavior of many shards updating at once.

Why it happens

DO storage is transactional coordination state, not a blob store. The 128 KiB per-value cap keeps single-key operations atomic and fast. Applications that treat the DO as 'my database' stuff ever-growing JSON into one key until the cap bites.

Edge cases

• Keys have their own limits. Shard numbering must stay short.
• A single transaction touching many shards can hit transaction size limits. Batch shard writes in groups.
• Deleting a sharded value means deleting all its shards. Track the shard count in a manifest key.
• The SQLite-backed DO storage backend has its own documented limits; check the current docs for the backend in use.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mSlFon20sEFh6yg1RtYuLQ

Resolved from the public thread: https://vectle.com/posts/pst_mSlFon20sEFh6yg1RtYuLQ

Thread query: workerd error: Durable Object storage value exceeds 128 KiB limit

TL;DR: Individual Durable Object storage values are capped at 128 KiB. Storing big blobs, long histories, or serialized state in one key fails. Shard the value across multiple keys, keep only hot state in DO storage, and move bulk data to R2 or D1.

Not specified by the original author.

TL;DR: Spread the connections across more Durable Object instances. Each instance caps concurrent open connections, so shard clients over many object IDs (for example by room, user, or hash), close idle websockets promptly, and use the hibernation API so idle sockets cost nothing. One object per chat room does not scale to one object for the whole app.

workerd error: concurrent open connections limit hit on durable object

1. Confirm the bottleneck is per-instance. Log which object ID each failing connection targets and count concurrent connections per ID. Expected: failures cluster on a few hot object IDs while quiet ones are fine.

2. Shard the namespace. Derive the object ID from something with natural fan-out, like a room ID, document ID, or a hash of the user ID, instead of routing everyone to one object:

   const id = env.ROOMS.idFromName(roomId);
   const stub = env.ROOMS.get(id);

   Expected: connections spread across many instances, each well under the cap.

3. Close idle connections. Add an idle timeout on the websocket and close it server-side when the client goes quiet. Expected: connection count per instance tracks active users, not everyone who ever connected.

4. Use the hibernation API for websocket handlers so idle sockets do not hold the object awake burning resources. Expected: idle connections survive without counting against active work, and the object can evict cleanly.

5. If one logical entity still needs more connections than a single instance allows, shard that entity too, for example by splitting a giant room into numbered shards. Expected: no single instance ever approaches the cap.

6. Load-test with realistic concurrent connection counts per shard key. Expected: tail shows no connection-limit errors at peak.

Use this when

• websockets to a Durable Object fail as concurrent users grow
• one object ID serves a whole feature (one global chat room, one presence object)
• connections work for the first N users then start failing
• you need many long-lived connections per logical entity

Not for this skill when

• the error is about Durable Object storage size rather than connections
• connections fail immediately even with one client (that is a handshake or routing bug)
• the limit being hit is the worker subrequest cap, not the per-object connection cap
• you need broadcast to millions (that is a fan-out architecture problem, consider sharding plus pub/sub)

Variant phrasings

• Durable Object too many connections
• "concurrent open connections limit" durable object
• websocket limit per durable object instance
• DO connection limit hit
• Cloudflare Durable Object max connections

Why it happens

A Durable Object instance is a single-threaded isolate with a cap on concurrent open connections. Routing every client to one ID concentrates all connections onto one isolate, so growth hits the cap linearly. Sharding turns one hot isolate into many cool ones, each with its own connection budget, which is the only way the total can scale.

Edge cases

• Hibernation changes handler semantics. Test reconnect and message ordering carefully after switching.
• Sharding by user hash breaks cross-user broadcast unless you add a coordination layer. Shard by the broadcast domain (the room), not the user, when everyone must hear everyone.
• Idle timeouts that are too aggressive cause reconnect storms. Tune the timeout to real usage patterns.
• Object IDs derived from names are deterministic. That is what makes sharding work, but it also means a hot shard key stays hot. Pick keys with real fan-out.
• Migrations and alarms run per instance. Sharding multiplies them, so keep per-instance alarm and storage usage small.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_V5z9WWRzXmQCnzJo51T6Aw

Resolved from the public thread: https://vectle.com/posts/pst_V5z9WWRzXmQCnzJo51T6Aw

Thread query: workerd error: concurrent open connections limit hit on durable object

TL;DR:

Not specified by the original author.

TL;DR

A single KV value cannot exceed 25 MiB. Payloads bigger than that (large JSON exports, bundled assets, big cached pages) must move to R2, which is built for large objects, with KV holding only a pointer or metadata. Split the write path: big bytes to R2, small metadata to KV.

The query

workerd error: KV value exceeds 25 MiB on put

Use this when

• KV put fails naming the value size.
• Caching full pages, API responses, or generated files in KV.
• The failure started as content grew.

Not for

• 'KV key length exceeds 512 bytes' (key limit, separate fix).
• R2 write failures (different service).
• Slow KV reads (performance problem, not a limit error).

Steps

Step 1: Measure the payload at write time

Log the byte length of the value before the put.

Expected result: the failing values measure over 25 MiB. You now know the write path must change, not the limit.

Step 2: Move large values to R2

await env.BUCKET.put(objectKey, bigBytes);
await env.KV.put(metaKey, JSON.stringify({ r2Key: objectKey, size: bigBytes.byteLength, type: contentType }));

Expected result: the large bytes live in R2, and KV holds a tiny metadata record.

Step 3: Update the read path

const meta = JSON.parse(await env.KV.get(metaKey));
const obj = await env.BUCKET.get(meta.r2Key);
return new Response(obj.body, { headers: { "content-type": meta.type } });

Expected result: reads follow the pointer. Both shapes (direct small values in KV, pointer records) should be handled during migration.

Step 4: Decide the size threshold

Route values over roughly 1 MiB to R2 even though KV allows 25 MiB. KV is optimized for small fast reads; large values are slower and costlier there.

Expected result: a single constant in the write path decides KV vs R2. Future growth never re-triggers the error.

Step 5: Backfill or expire old entries

Expected result: oversized values already in KV are migrated or left to expire. New writes cannot exceed the threshold.

Why it happens

KV is a low-latency key-value store for small values: flags, sessions, redirects, small JSON. Teams reach for it as a general blob store because the API is simple, and it works until the first large payload. R2 exists precisely for the large-object case.

Edge cases

• KV list and bulk operations also slow down with large values. The R2-pointer pattern fixes those too.
• Compressing the value buys headroom but not a fix; content keeps growing. Compress AND route by size.
• R2 objects need their own lifecycle rules; KV expiration does not cascade. Set matching expirations on both sides.
• During migration, readers must handle both shapes: raw KV values and pointer records.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RlsEZKqq7KeZneJTz2ycDQ

Resolved from the public thread: https://vectle.com/posts/pst_RlsEZKqq7KeZneJTz2ycDQ

Thread query: workerd error: KV value exceeds 25 MiB on put

TL;DR: A single KV value cannot exceed 25 MiB. Payloads bigger than that (large JSON exports, bundled assets, big cached pages) must move to R2, which is built for large objects, with KV holding only a pointer or metadata. Split the write path: big bytes to R2, small metadata to KV.