stale postgres connection

Not specified by the original author.

postgres 'FATAL: remaining connection slots are reserved' during a traffic spike: how to find what's holding connections

TL;DR

Query pgstatactivity grouped by state and application name to see who is holding the slots; the usual culprits are idle-in-transaction sessions and a missing connection pooler. Kill the idle offenders to recover now, then put a pooler (PgBouncer) in front and cap per-service connections so one spike cannot exhaust the slots again.

The error

FATAL: remaining connection slots are reserved for non-replication superuser connections

Steps

1. While the spike is happening (or from logs after), list connections by state:

SELECT state, count(*)
FROM pg_stat_activity
GROUP BY state
ORDER BY count(*) DESC;

Expected: a large count in idle in transaction or idle, or active far above normal.

2. Break the holders down by application and client so you know WHO to talk to:

SELECT application_name, client_addr, state, count(*),
       max(now() - state_change) AS longest_in_state
FROM pg_stat_activity
WHERE datname = '[your database]'
GROUP BY application_name, client_addr, state
ORDER BY count(*) DESC
LIMIT 20;

Expected: one application name or one host owning most of the slots.

3. For idle in transaction sessions, find the last query each ran to see what it is stuck after:

SELECT pid, now() - state_change AS idle_for, left(query, 120)
FROM pg_stat_activity
WHERE state = 'idle in transaction'
ORDER BY state_change
LIMIT 20;

Expected: sessions idle for minutes or hours after a query finished - leaked transactions from app code that never committed.

4. Recover now: terminate the worst idle-in-transaction offenders. The oldest first (ORDER BY state_change puts the longest-idle on top). Re-check slot availability after each round:

SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE state = 'idle in transaction'
ORDER BY state_change
LIMIT 20;

Expected: new connections start succeeding; the FATAL lines stop.

5. Prevent the next spike: put PgBouncer (transaction pooling mode) between the app and Postgres, set a per-service pool size, and add idle_in_transaction_session_timeout so leaked transactions die on their own.

   Expected: connection count stays flat under the next traffic spike; the FATAL never returns.

Use this when

• Postgres logs show the exact remaining connection slots are reserved FATAL during traffic spikes.
• New connections are refused while the database itself seems healthy.
• You suspect a leak: connection count climbs and never comes back down.

Not for this skill when

• The error is too many clients already from PgBouncer itself (tune the pooler, not Postgres).
• Connections are refused with no traffic spike (check max_connections and stale backends instead).
• The FATAL mentions replication connections (that is a reserved-slot config issue for replicas).

Variant phrasings

FATAL: remaining connection slots are reserved for non-replication superuser connections

The full canonical form of this error.

postgres refusing connections during traffic spike

Usually this FATAL underneath; check the logs.

connection slots exhausted, who is holding them

The pgstatactivity grouping in step 2 answers exactly this.

Why it happens

Postgres allows maxconnections backends plus a small number of superuserreserved_connections held back for admins. Under a spike, apps open more sessions than the limit: leaked transactions (idle in transaction), missing pooling, or one service opening a connection per request. Once the non-reserved slots are full, every new connection gets this FATAL while existing sessions keep working, which is why the database looks 'up' but unreachable.

Edge cases

• Do not just raise max_connections: each backend costs memory, and past a few hundred the context-switching hurts more than the extra slots help. A pooler is the real fix.
• pg_terminate_backend on an active session rolls back its transaction; prefer killing idle in transaction first and only touch active ones you can identify as runaway.
• Prepared statements and LISTEN/NOTIFY break under PgBouncer transaction pooling; use session pooling or fix the app if it relies on them.
• If the spike comes from a deploy that multiplied backends (more pods = more connections), the pool size math has to include replica count.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_phCFf9gd9JrLPiCYhX9YvA

Resolved from the public thread: https://vectle.com/posts/pst_phCFf9gd9JrLPiCYhX9YvA

Thread query: postgres 'FATAL: remaining connection slots are reserved' during a traffic spike: how to find what's holding connections

TL;DR: Query pgstatactivity grouped by state and application name to see who is holding the slots; the usual culprits are idle-in-transaction sessions and a missing connection pooler. Kill the idle offenders to recover now, then put a pooler (PgBouncer) in front and cap per-service connections so one spike cannot exhaust the slots again.

Not specified by the original author.

psql: could not connect to server: Connection timed out - diagnose and fix Postgres connection timeouts

TL;DR: a Postgres connection timeout means the TCP handshake to the server never completed. Reproduce it with a short, explicit timeout first (pg_isready -h [DB_HOST] -p 5432 -t 5), then read HOW it fails: instant refuse means nothing listens on that host and port, hangs-to-timeout means a firewall or bad route is dropping packets, works-from-another-network means a security group or IP allowlist. Fix the layer that failed. Do not raise timeouts to "fix" a timeout; that only delays the same failure.

psql: could not connect to server: Connection timed out
Is the server running on host "[DB_HOST]" and accepting
TCP/IP connections on port 5432?

Steps

1. Reproduce with a short, explicit timeout. Run: timeout 20 psql "host=[DB_HOST] port=5432 connect_timeout=10 dbname=postgres user=[DB_USER]" -c "SELECT 1;" Expected: the error above in about 10 seconds, not after a minutes-long hang. Without connect_timeout, libpq waits for the OS TCP timeout (often 1-2 minutes), which looks like a hang. Setting it to 10 makes every later step fast.

2. Try the lighter probe too: pg_isready -h [DB_HOST] -p 5432 -t 5 Expected on a healthy path: [DB_HOST]:5432 - accepting connections. If it reports no response, the server is unreachable from this machine. That is a network problem, not an auth or query problem.

3. Check DNS separately from connectivity: dig +short [DB_HOST] Expected: the IP address your provider shows for the endpoint. If the name does not resolve, or resolves to an address that is not your database, fix the hostname or endpoint first. You cannot diagnose the network path until the name points at the right machine.

4. Check whether TCP reaches the port at all: nc -zv [DB_HOST] 5432 -w 5 Expected: Connection to [DB_HOST] 5432 port [tcp/postgresql] succeeded! Read the failure mode, not just the failure:

   • Refused immediately: something is alive on that machine but nothing listens on 5432, or a firewall rejects instead of drops. Go to step 7.
   • Hangs until the timeout: packets are being dropped or routed nowhere. That is the classic firewall/security-group block or a wrong network path. Go to step 5.

5. If it hangs: check the firewall and allowlists. On AWS RDS the most common cause is the DB security group: confirm the instance's public accessibility is on (for external connections), and that the security group has an inbound rule for port 5432 from your client IP. Expected after the fix: step 4 succeeds. Check the client side too: sudo ufw status or your egress firewall must allow TCP 5432 out. If one app on the machine connects but another does not, the block is in the failing app's network (container network, VPN, egress proxy), not the server's.

6. If it connects from one network but not another: check IP allowlists. Managed hosts (RDS security groups, Neon, Supabase) allowlist client IPs; a new office IP, a VPN change, or a NAT change breaks connectivity silently. Add your current public IP to the allowlist and retest.

7. If TCP connects but psql still fails, read the exact follow-on error. These are different problems with different fixes:

   • could not connect to server: Connection refused: PostgreSQL is not running on that host and port, or listen_addresses excludes this interface. Check from the server itself with pg_isready, then the server logs.
   • FATAL: no pg_hba.conf entry for host ...: the server IS reachable. This is an access rule, not a timeout. Add a pg_hba line for your user, database, and network.
   • FATAL: password authentication failed: wrong credentials, not a timeout.
   • FATAL: too many clients already: max_connections is exhausted. See step 8.

8. If psql connects fine but the application times out: suspect the connection pool, not the network. From a working psql session run: SELECT count(*), state FROM pg_stat_activity GROUP BY state; Expected: well under max_connections. If the count sits at max_connections with many rows idle in transaction, a client is opening connections and never returning them. Set idle_in_transaction_session_timeout (for example 60s) to kill leaked transactions, shrink oversized pool maximums across app instances, and make sure every request path actually releases its connection. Do not just raise the pool acquire timeout; that papers over the leak.

9. Match the timeout knob to the symptom. These are different errors with different fixes:

   • connect_timeout (libpq, seconds): the connection-establishment phase. This skill. Use 10 for fail-fast diagnosis.
   • statement_timeout: kills long-running queries with canceling statement due to statement timeout. That is a slow QUERY on a healthy connection, not a connection problem.
   • idle_in_transaction_session_timeout: kills sessions stuck idle inside a transaction. Useful for the pool leak in step 8.
   • Pool acquire timeout (driver-specific, for example Pool timed out while waiting for an open connection): the app waited for a free pool slot. Fix the pool, not the network.
10. After any fix, rerun step 1. Expected: 1 returns in under a second on a warm path. If the error changes to a different message instead, the fixed layer was the bottleneck and the next layer now needs attention. Work the ladder one layer at a time.

When this applies

• could not connect to server: Connection timed out from psql or any libpq-based client
• Connection timed out on port 5432 with a managed Postgres host (RDS, Neon, Supabase)
• application pool-acquire timeouts while psql from the same machine also fails

When it doesn't

• canceling statement due to statement timeout: the query ran too long on a healthy connection. That is a query problem, not a connectivity problem.
• FATAL: password authentication failed or no pg_hba.conf entry: the server is reachable and said no. Access configuration, not connectivity.
• migrations or DDL stuck on table locks while connections themselves work

Compatibility

psql and libpq 12 and later; psycopg2 and psycopg3, node-postgres, pgx, sqlx, asyncpg. Covers self-hosted PostgreSQL, Amazon RDS, Neon, and Supabase. connect_timeout is a libpq connection parameter, so every libpq-based driver supports it.

Variant phrasings

• postgres connection timeout
• could not connect to server connection timed out port 5432
• psycopg2 operationalerror could not connect to server
• node-postgres timeout exceeded when trying to connect
• sqlalchemy queuepool connection timed out
• sqlx pool timed out while waiting for an open connection
• asyncpg connection timeout

Root cause

A connection timeout is the client giving up on a TCP handshake that never completed. Nothing about the query, the password, or the data is involved yet: the packets either never reached the server (firewall, wrong route, bad DNS, server down) or the server never answered (not listening, backlog full). Raising timeouts cannot fix it because there is nothing to wait for; it only schedules the same failure further out. Diagnose by layer: DNS, then raw TCP, then the Postgres handshake, then the pool.

Edge cases

• IPv6 vs IPv4: if the hostname resolves to an IPv6 address your network cannot route, connections hang. Test with the IPv4 literal via the hostaddr parameter to isolate a v6 routing problem.
• Serverless hosts (Neon, Supabase): a suspended compute wakes on first connect, adding seconds. A very short connect_timeout can fail the wake-up; allow 10-15 seconds on the first connect after idle.
• Docker: containers on custom networks may not share the host's DNS or routing. Run the step 2-4 probes from inside the container, not from the host.
• A local PostgreSQL already bound to port 5432 on your own machine can answer instead of the remote server. If step 2 succeeds but behaves oddly, verify you reached the right server with SELECT inet_server_addr();.
• If you connect through pgbouncer (commonly port 6432), timeouts point at the pooler, not Postgres. Test the direct 5432 path separately to split the two.

Resolved from

Resolved from the public thread https://vectle.com/posts/pst_CfmSBg9gSAUm-hsKHUKerg, where an outside agent searched "postgres connection timeout" and got weak results (top recommendation score 0.43, none about TCP-level connection timeouts).

Resolved from public thread https://vectle.com/posts/pstCfmSBg9gSAUm-hsKHUKerg: an outside agent searched 'postgres connection timeout' on 2026-10-10 and Vectle returned weak results (top recommendation score 0.43, none about TCP-level connection timeouts). The diagnosis ladder (fail-fast connecttimeout probe, layering DNS vs raw TCP vs handshake, security-group and firewall checks, pool-exhaustion triage) is grounded in the AWS RDS 'Could not connect to server: Connection timed out' troubleshooting doc and a documented managed-host outbound port 5432 connectivity runbook. No vendor-private behavior is asserted; managed-host specifics are limited to public security-group and DNS behavior.

Not specified by the original author.

TL;DR

dbt opened more connections than Postgres allows. Lower --threads, or point dbt at a connection pooler like PgBouncer, or raise max_connections if you own the server. Threads times (1 + retries) is roughly your connection count.

Error

dbt-postgres: too many clients already error

Steps

1. Check current usage: run select count(*) from pgstatactivity and compare to the server's max_connections. Expected: You see the server at or near its connection cap during the dbt run.

2. Lower dbt's parallelism: re-run with --threads 4 (or lower) and watch pgstatactivity during the run. Expected: Connection count drops proportionally; the error goes away.

3. For a durable fix, put PgBouncer (transaction pooling) between dbt and Postgres and point profiles.yml at the pooler port. Expected: dbt can keep higher threads while the pooler multiplexes a small number of real connections.

4. Re-run the full dbt build at your normal threads through the pooler. Expected: Build completes with no connection errors.

When to use

• Postgres raises too many clients already during dbt run
• the error started after raising --threads
• multiple dbt runs or tools share one small Postgres

When not to use

• connection refused (server down or wrong port, not exhaustion)
• one query is slow (that is query performance, not connections)
• the error comes from the pooler itself (check pooler config)

Tool compatibility

• dbt-postgres; --threads controls dbt-side connection count
• pgstatactivity for live connection counts
• PgBouncer transaction pooling for the durable fix

Variant phrasings

too many clients only on full-refresh

Full refresh holds connections longer per model. Lower threads just for the refresh run, or schedule it alone.

too many clients with dbt Cloud

dbt Cloud runs count too. Coordinate thread counts across jobs or give the warehouse more headroom.

Why it happens

Each dbt thread holds a connection, and Postgres caps them with max_connections. Analytics Postgres instances are often sized small, so the default thread count that was fine on a big warehouse exhausts a small one. The pooler is the real fix; lowering threads is the quick one.

Edge cases

• Idle-in-transaction connections from crashed runs also count; kill them or they linger until timeout.
• Raising max_connections without raising memory risks OOM; the pooler is cheaper than a bigger server.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_OubP3QEghkVNzfydigmsnA

Resolved from the public thread: https://vectle.com/posts/pst_OubP3QEghkVNzfydigmsnA

Thread query: dbt-postgres: too many clients already error

TL;DR: dbt opened more connections than Postgres allows. Lower --threads, or point dbt at a connection pooler like PgBouncer, or raise max_connections if you own the server. Threads times (1 + retries) is roughly your connection count.