workerd error: Durable Object storage value exceeds 128 KiB limit

TL;DR

Individual Durable Object storage values are capped at 128 KiB. Storing big blobs, long histories, or serialized state in one key fails. Shard the value across multiple keys, keep only hot state in DO storage, and move bulk data to R2 or D1.

The query

workerd error: Durable Object storage value exceeds 128 KiB limit

Use this when

• storage.put inside a Durable Object fails naming the value size.
• The DO accumulates history, logs, or session state in one key.
• The failure started as the stored state grew.

Not for

• 'Durable Object ID is not valid' (id problem).
• Alarm scheduling failures (separate fix).
• Total storage quota errors (different limit).

Steps

Step 1: Find the growing key

Log value byte lengths on put, or list keys and check sizes.

Expected result: one key (often 'state', 'history', or 'log') dominates. That key needs sharding.

Step 2: Shard across numbered keys

const SHARD_SIZE = 100 * 1024;
async function putLarge(storage, base, bytes) {
  const total = Math.ceil(bytes.byteLength / SHARD_SIZE);
  const entries = {};
  let i = 0;
  while (i !== total) {
    const k = base + ":shard:" + i;
    entries[k] = bytes.slice(i * SHARD_SIZE, (i + 1) * SHARD_SIZE);
    i = i + 1;
  }
  await storage.put(entries);
}

Expected result: each shard stays under the cap. Reads reassemble by listing the shard prefix.

Step 3: Keep only hot state in the DO

Move append-only history to D1 or R2 and keep the DO's storage for what the request path actually needs: current state, locks, coordination data.

Expected result: the DO stays small and fast. Bulk data lives in systems designed for bulk data.

Step 4: Bound unbounded growth

Cap histories (keep last N entries), compact logs, expire old shards with alarms.

Expected result: storage stops growing without bound. The error cannot return via slow accumulation.

Step 5: Verify under load

Expected result: concurrent writes to sharded keys succeed. Watch for the write-contention behavior of many shards updating at once.

Why it happens

DO storage is transactional coordination state, not a blob store. The 128 KiB per-value cap keeps single-key operations atomic and fast. Applications that treat the DO as 'my database' stuff ever-growing JSON into one key until the cap bites.

Edge cases

• Keys have their own limits. Shard numbering must stay short.
• A single transaction touching many shards can hit transaction size limits. Batch shard writes in groups.
• Deleting a sharded value means deleting all its shards. Track the shard count in a manifest key.
• The SQLite-backed DO storage backend has its own documented limits; check the current docs for the backend in use.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mSlFon20sEFh6yg1RtYuLQ