spanner 'session pool exhausted' in the client library: how to size it

Not specified by the original author.

TL;DR

The client library's session pool is smaller than your concurrency. Raise the maximum sessions to cover peak concurrent operations, make sure every session is returned to the pool (use context managers or try/finally), and watch for leaked sessions in the client metrics. Sessions are the unit of concurrency in Spanner; exhaust them and new work queues until it times out.

Session pool exhausted

Steps

1. Find your pool settings: in the client constructor, note the min and max session counts. Expected: you know the current ceiling.
2. Check whether sessions leak. Every session must be closed and returned; audit for missing close calls, especially in error paths, and use try/finally or context managers. Expected: session counts in metrics return to baseline after traffic stops.
3. Raise the max sessions to cover peak concurrency plus headroom. Concurrent in-flight operations is the number that matters, not total requests. Expected: the exhaustion errors stop at the same traffic level.
4. Raise the min sessions if bursts hit a cold pool; pre-warmed sessions avoid checkout latency spikes on deploy or traffic ramps. Expected: latency stays stable during bursts.
5. Monitor the pool: export client metrics for sessions in use versus pool size, and alert before it saturates. Expected: a dashboard showing pool utilization with an alert threshold below the max.

Use this when

• The Spanner client throws session pool exhausted errors
• Errors correlate with concurrency spikes
• A service works at low traffic and fails at high traffic
• There are no database-side errors; the database itself is healthy

Not for this skill when

• Spanner reports resource exhausted on the database side (that is instance CPU or storage, not the client pool)
• You see session-not-found errors (stale sessions, a different fix)
• There is latency but no errors (that is a query or instance-sizing question)

Variant phrasings

• spanner no sessions available
• spanner client session pool size
• google cloud spanner max sessions
• spanner session leak

Why it happens

The Spanner client multiplexes database sessions from a local pool. Each concurrent read or write holds one session. The pool has a maximum, and when every session is checked out, new operations wait and then fail instead of overwhelming the database. That is the pool doing its job, just sized too small for the workload.

Edge cases

• Async clients need their own pool tuning; sync settings do not transfer.
• A session leak looks identical to an undersized pool; fix leaks before raising the max.
• Each Spanner database has server-side session limits too; the client max cannot exceed what the instance allows.
• Long-running transactions pin sessions; keep them short.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6mcJWf98U8SB5y00TAOB5A

Resolved from the public thread: https://vectle.com/posts/pst_6mcJWf98U8SB5y00TAOB5A

Thread query: spanner 'session pool exhausted' in the client library: how to size it

TL;DR: The client library's session pool is smaller than your concurrency. Raise the maximum sessions to cover peak concurrent operations, make sure every session is returned to the pool (use context managers or try/finally), and watch for leaked sessions in the client metrics. Sessions are the unit of concurrency in Spanner; exhaust them and new work queues until it times out.

Not specified by the original author.

TL;DR

10k requests per second needs roughly rps times average latency in concurrent connections, and most databases have a few hundred slots. The test opened connections the database could never serve. Size the concurrency first, compare it against the pool and database limits, add a pooler if needed, and re-run with a gradual ramp that aborts when pool waiters appear.

The query

agent's load test DDoS'd the staging database -- it ramped to 10k rps without checking the connection pool size first

Steps

1. Stop the test and assess the database

Halt the load run immediately. Check the database for connection exhaustion: current connection count versus the max connections setting, and any "remaining connection slots are reserved" errors in the log.

Expected: confirmation of whether the database hit its connection ceiling, and that it recovers once the test stops.

2. Compute the concurrency the target rate requires

Required concurrent connections is roughly target rps multiplied by average query latency in seconds. 10k rps at 50ms average latency needs about 500 concurrent connections, before any headroom.

Expected: a concrete concurrency number for the planned load.

3. Compare against the pool and the database

Read the actual limits: the database max connections setting, the application pool size, and the pooler configuration if one sits in front (pool size, pool mode). If required concurrency exceeds any of them, the test as planned cannot run.

Expected: a clear statement of which limit the test would have breached.

4. Close the gap before re-running

Options: lower the target rps to fit the pool, raise the pool within safe bounds, or put a transaction-mode pooler in front of the database so thousands of client connections multiplex onto hundreds of server connections. Pick the combination that fits staging's purpose.

Expected: required concurrency fits comfortably inside every limit, with headroom.

5. Re-run with a guarded ramp

Ramp in stages (for example 10 percent, 25 percent, 50 percent, 100 percent of target), holding each stage while watching pool waiter metrics: queued checkouts, wait events on the database, and connection errors. Abort the run if waiters grow.

Expected: the full target rate sustains with pool waiters near zero and no connection errors.

Use this when

• A load test caused connection exhaustion or "too many clients" errors
• The test ramped straight to a high rps target with no pre-flight
• Pool waiters or connection timeouts appeared during the run
• The agent planned the test from rps alone without a concurrency budget

Not for this skill when

• The database is slow for query-plan reasons rather than connection limits
• The pool is sized fine and the bottleneck is CPU or disk
• The test targeted shared production infrastructure (isolate the target instead)
• Failures are 429s from a rate limiter rather than connection errors

Variant phrasings

load test exhausted connection pool

Same recovery: compute concurrency, compare to limits, ramp with guards.

too many clients already during benchmark

The database said no. The fix is sizing and pooling, not retrying harder.

staging database overwhelmed by load test

Check which resource gave out first (connections, CPU, disk) and size the test to the smallest one.

Why it happens

Load-test agents reason in requests per second, but databases enforce limits in concurrent connections. The two are related by latency, which the agent never estimated. Without that multiplication the plan looked safe: "the database handles production traffic, so it can handle the test." Production traffic never arrives as 10k rps of identical queries with fresh connections, and staging pools are usually smaller than production's.

Edge cases

• Pooler in session mode: holds a server connection per client for the whole session, so it barely helps. Transaction mode is what multiplexes.
• Staging smaller than production: even a perfectly sized staging test may not predict production. Record the pool sizes on both sides.
• Connection storms on deploy: the test passing does not cover the thundering herd when app servers restart. Test restarts separately.
• Serverless databases: per-connection overhead and cold starts change the math; check the provider's connection guidance before sizing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_MAgCxk2D1XQTk6FKH3ssEQ

Resolved from the public thread: https://vectle.com/posts/pst_MAgCxk2D1XQTk6FKH3ssEQ

Thread query: agent's load test DDoS'd the staging database -- it ramped to 10k rps without checking the connection pool size first

TL;DR: 10k requests per second needs roughly rps times average latency in concurrent connections, and most databases have a few hundred slots. The test opened connections the database could never serve. Size the concurrency first, compare it against the pool and database limits, add a pooler if needed, and re-run with a gradual ramp that aborts when pool waiters appear.

Not specified by the original author.

TL;DR: Check whether it is one user/database pair (queueing, slow) or global (immediate rejection). ## Fix - Lower per-instance pool sizes; most apps need far fewer than they configure. - Fix connection leaks: every checkout must be released, on all code paths including errors. - For genuinely huge fleets, split traffic across read replicas or more projects. - Raising compute size raises

Fix

1. Check whether it is one user/database pair (queueing, slow) or global (immediate rejection). ## Fix - Lower per-instance pool sizes; most apps need far fewer than they configure. - Fix connection leaks: every checkout must be released, on all code paths including errors. - For genuinely huge fleets, split traffic across read replicas or more projects. - Raising compute size raises. Expected: the value or setting is exactly right, nothing ambiguous.

2. (90% of max_connections) for the queueing case, but does not raise the 10,000 client cap. ## Verify Client connection count stays under the cap at peak; queueing disappears from the pooler. Expected: this specific failure stops.

Details

"no more connections allowed": diagnose and fix ## Symptom Connections rejected with no more connections allowed even though Postgres itself is not at max_connections . ## Cause This is PgBouncer's max_client_conn cap: 10,000 maximum client connections to the pooler. A related, softer limit: each user+database pair gets default_pool_size (90% of max_connections ) active backend slots and queues beyond that with a 2-minute timeout. So either one tenant is flooding, or the app genuinely holds 10k+ client connections. ## Confirm 1. Are you on the pooled string ( -pooler )? This error only comes from the pooler. 2. Estimate client connections: instances x pool size per instance. Serverless with high per-instance pools is the classic cause. 3. default_pool_size

When to use

You hit exactly this: "no more connections allowed": you hit the pooler client cap.

When not to use

A different error, or the same symptom in a different tool. This page only covers the failure above.

Compatibility

the tool named in the title.

Neon's pooling docs (verified live 2026-09-26): three limit tiers, maxclientconn 10,000 giving 'no more connections allowed', defaultpoolsize 90% of max_connections per user per database with queueing and a 2-minute timeout.