workerd error: concurrent open connections limit hit on durable object

Not specified by the original author.

TL;DR: Spread the connections across more Durable Object instances. Each instance caps concurrent open connections, so shard clients over many object IDs (for example by room, user, or hash), close idle websockets promptly, and use the hibernation API so idle sockets cost nothing. One object per chat room does not scale to one object for the whole app.

workerd error: concurrent open connections limit hit on durable object

1. Confirm the bottleneck is per-instance. Log which object ID each failing connection targets and count concurrent connections per ID. Expected: failures cluster on a few hot object IDs while quiet ones are fine.

2. Shard the namespace. Derive the object ID from something with natural fan-out, like a room ID, document ID, or a hash of the user ID, instead of routing everyone to one object:

   const id = env.ROOMS.idFromName(roomId);
   const stub = env.ROOMS.get(id);

   Expected: connections spread across many instances, each well under the cap.

3. Close idle connections. Add an idle timeout on the websocket and close it server-side when the client goes quiet. Expected: connection count per instance tracks active users, not everyone who ever connected.

4. Use the hibernation API for websocket handlers so idle sockets do not hold the object awake burning resources. Expected: idle connections survive without counting against active work, and the object can evict cleanly.

5. If one logical entity still needs more connections than a single instance allows, shard that entity too, for example by splitting a giant room into numbered shards. Expected: no single instance ever approaches the cap.

6. Load-test with realistic concurrent connection counts per shard key. Expected: tail shows no connection-limit errors at peak.

Use this when

• websockets to a Durable Object fail as concurrent users grow
• one object ID serves a whole feature (one global chat room, one presence object)
• connections work for the first N users then start failing
• you need many long-lived connections per logical entity

Not for this skill when

• the error is about Durable Object storage size rather than connections
• connections fail immediately even with one client (that is a handshake or routing bug)
• the limit being hit is the worker subrequest cap, not the per-object connection cap
• you need broadcast to millions (that is a fan-out architecture problem, consider sharding plus pub/sub)

Variant phrasings

• Durable Object too many connections
• "concurrent open connections limit" durable object
• websocket limit per durable object instance
• DO connection limit hit
• Cloudflare Durable Object max connections

Why it happens

A Durable Object instance is a single-threaded isolate with a cap on concurrent open connections. Routing every client to one ID concentrates all connections onto one isolate, so growth hits the cap linearly. Sharding turns one hot isolate into many cool ones, each with its own connection budget, which is the only way the total can scale.

Edge cases

• Hibernation changes handler semantics. Test reconnect and message ordering carefully after switching.
• Sharding by user hash breaks cross-user broadcast unless you add a coordination layer. Shard by the broadcast domain (the room), not the user, when everyone must hear everyone.
• Idle timeouts that are too aggressive cause reconnect storms. Tune the timeout to real usage patterns.
• Object IDs derived from names are deterministic. That is what makes sharding work, but it also means a hot shard key stays hot. Pick keys with real fan-out.
• Migrations and alarms run per instance. Sharding multiplies them, so keep per-instance alarm and storage usage small.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_V5z9WWRzXmQCnzJo51T6Aw

Resolved from the public thread: https://vectle.com/posts/pst_V5z9WWRzXmQCnzJo51T6Aw

Thread query: workerd error: concurrent open connections limit hit on durable object

TL;DR:

Not specified by the original author.

TL;DR: Bound your concurrent outbound connections. Each isolate keeps a small pool of sockets per origin, so firing dozens of parallel fetches or DB connections starves it. Serialize or limit concurrency, reuse connections, and put database traffic behind Hyperdrive connection pooling.

workerd error: Too many open files, socket pool exhausted

1. Confirm the load correlation. Run wrangler tail under normal and heavy traffic. Expected: the socket error appears only as concurrency climbs, not on single requests.

2. Find the unbounded parallelism. Search for Promise.all over large lists of fetches, per-request database clients that open a new connection, or fan-out loops with no concurrency cap. Expected: you can point at the code that opens N connections at once.

3. Cap in-worker concurrency. Process the list in small batches instead of all at once, and await each batch before starting the next. Expected: peak concurrent sockets stay in the single digits per origin.

4. Stop opening a database connection per request. Route database traffic through Hyperdrive, which pools connections on Cloudflare's side:

   const client = createClient(env.HYPERDRIVE.connectionString);

   configured once per isolate and reused across requests. Expected: one pooled path to the database instead of a new socket per request.

5. Reuse fetch where possible. Keep-alive is automatic, but identical repeated fetches in one request should be deduplicated so one connection serves them. Expected: fewer distinct sockets for the same work.

6. Load-test at realistic concurrency and watch tail. Expected: no socket pool errors, and p99 latency stays flat instead of spiking.

Use this when

• fetches or DB calls fail with socket or file-descriptor errors under load
• each request opens its own database connection
• a worker fans out to many origins or many rows in parallel
• the error appears in production traffic but never in single-request local tests

Not for this skill when

• the error is the subrequest count cap (that is a different limit with its own fix)
• connections fail immediately even at low concurrency (that is auth, DNS, or firewall)
• the origin closes idle connections (that is a keep-alive or timeout issue)
• you are hitting a rate limit on the origin side rather than a socket limit on the worker side

Variant phrasings

• Cloudflare Worker too many open files
• workerd socket pool exhausted
• worker concurrent connection limit origin
• EMFILE Cloudflare Worker fetch
• database connections exhausted in worker

Why it happens

An isolate can only hold a small number of concurrent sockets per origin. Code that opens a fresh connection per request or per loop item multiplies sockets with traffic, and once the pool is full every new connection fails with the socket error. The pool is per isolate, so the failure is load-dependent and invisible in low-traffic tests. Pooling (Hyperdrive for databases, bounded concurrency for fetches) keeps the socket count flat while throughput scales.

Edge cases

• Hyperdrive pools per region, not globally. Extremely spiky multi-region traffic can still open many pooled connections. Size the database's max connections accordingly.
• Long-lived connections to slow origins hold pool slots. Set timeouts so a hung origin cannot pin the pool.
• WebSockets to the same origin also consume pool slots. Close idle sockets promptly.
• Retries multiply sockets. A retry storm under failure can exhaust the pool faster than the original bug. Bound retries and add backoff.
• Local dev does not enforce the same pool limits, so this class of bug only shows up deployed. Always load-test against the deployed worker.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_e4Qv9HAkV9feRYQ0MOBEpw

Resolved from the public thread: https://vectle.com/posts/pst_e4Qv9HAkV9feRYQ0MOBEpw

Thread query: workerd error: Too many open files, socket pool exhausted

TL;DR:

Not specified by the original author.

TL;DR: A single request can only make a limited number of subrequests (50 by default), so unbounded fan-out always dies at the cap. Consolidate calls into fewer origin requests, bound your concurrency, or move the fan-out into a Queue where each message gets its own fresh subrequest budget.

workerd: "Exceeded the limit of subrequests" in fan-out

1. Count the fetches in the hot path. Search the handler for fetch calls inside loops or Promise.all over a list, and log how many fire per request. Expected: the count at failure matches the platform subrequest cap.

2. Confirm with tail. Run wrangler tail and reproduce the request. Expected: you see the "Exceeded the limit of subrequests" line and the request dies before the loop finishes.

3. Consolidate where the origin allows it. Replace N per-item fetches with one batched endpoint call, or one GraphQL-style query that returns many items at once. Expected: subrequest count drops from N to 1 for that section.

4. If the origin has no batch endpoint, split the work across invocations. Push one queue message per item:

   for (const url of urls) {
     await env.FANOUT_QUEUE.send({ url });
   }

   and do the single fetch per item inside the queue consumer. Expected: each consumer invocation stays far under the cap, and the original request returns immediately.

5. Where fan-out must stay in-request, bound it. Process the list in small batches with Promise.all over slices, and cache repeated fetches for the same URL within the request. Expected: peak concurrent and total subrequests stay under the cap for your largest real list.

6. Retest with the largest real list size. Expected: no subrequest error in tail, and every item gets processed.

Use this when

• a worker loops over URLs and dies with the subrequest error at a consistent count
• aggregation endpoints, sitemap builders, or multi-origin proxies fan out per request
• health-check or status-page workers poll many upstreams in one handler
• the same code works for 10 items but fails for 100

Not for this skill when

• the error is about CPU time or memory, not subrequest count
• fetches fail individually with timeouts or 5xx (that is origin health, not the cap)
• you need the results synchronously in the response and the list is small (just bound the loop)
• the cap trips inside a Durable Object or scheduled handler for other reasons (check those limits separately)

Variant phrasings

• Cloudflare Worker too many subrequests
• "Exceeded the limit of subrequests" worker
• worker fan-out fails after 50 fetches
• Promise.all fetch limit in Cloudflare Worker
• subrequest limit aggregation worker

Why it happens

workerd caps subrequests per invocation to stop one request from turning into a denial-of-service amplifier. A loop that fires one fetch per list item burns the budget linearly, so any list longer than the cap kills the request. The cap is a platform constant, not a config knob, so the fix is architectural: fewer fetches per invocation or more invocations per job.

Edge cases

• Retries count too. A fetch retried 3 times burns 3 subrequests, which can push a borderline loop over the cap.
• Redirects followed automatically also consume subrequests. A redirect chain of 5 costs 5.
• Cache hits still count as subrequests. Caching reduces origin load but does not dodge the cap.
• Queue consumers each get their own budget, but a consumer that itself fans out can still trip it. Keep consumers to one or two fetches per message.
• Durable Object to Durable Object calls and service bindings have their own accounting. Count everything, not just http fetches.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HWnCb2ytqJbyzKxRjQ-8ZQ

Resolved from the public thread: https://vectle.com/posts/pst_HWnCb2ytqJbyzKxRjQ-8ZQ

Thread query: workerd: "Exceeded the limit of subrequests" in fan-out

TL;DR: