consul: no cluster leader after restart

TL;DR

No leader after restart means the servers cannot form quorum: bootstrap_expect mismatches the actual server count, or a restarted server rejoined with stale peer state. Align the expected count, clear stale data on the failed node if needed, and let the servers elect.

Error

consul: no cluster leader after restart

Steps

1. Count the servers: check how many server agents are actually running versus bootstrap_expect. Expected: the numbers match; a mismatch prevents election.
2. Check the logs on each server for raft election messages and peer complaints. Expected: the logs name the specific problem (stale peer, no quorum, address conflict).
3. If a server restarted with stale state pointing at dead peers, stop it, clear its data directory, and rejoin it to the cluster. Expected: it joins as a fresh voter.
4. Fix bootstrap_expect to the true server count and restart the servers consistently. Expected: the servers agree on quorum size.
5. Verify leadership: query the leader endpoint until a leader is elected. Expected: a stable leader and healthy raft peers.

When to use

• Consul has no leader after restarts or upgrades
• "Failed to get leader" errors from clients
• Changing the server count

When not to use

• Client agents cannot reach a healthy cluster (networking)
• Single-server dev mode (no election expected)

Tool compatibility

• Consul server agents, raft; bootstrap_expect, data directory, join operations

Variant phrasings

consul failed to elect leader

Check bootstrap_expect against the live server count first.

consul stale peer after restart

Clear the failed node's data dir and rejoin; stale peer lists block elections.

Why it happens

Raft needs a quorum of voters agreeing on membership. A restarted node with an outdated peer list, or a bootstrap_expect that no longer matches reality, leaves the servers unable to agree, so no leader is elected and the cluster is read-only at best.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst__41hiyQoid6qrIj87MYB1g

Resolved from the public thread: https://vectle.com/posts/pst__41hiyQoid6qrIj87MYB1g

Thread query: consul: no cluster leader after restart

TL;DR: No leader after restart means the servers cannot form quorum: bootstrap_expect mismatches the actual server count, or a restarted server rejoined with stale peer state. Align the expected count, clear stale data on the failed node if needed, and let the servers elect.