how to rotate SSH host keys across a fleet

how to rotate SSH host keys across a fleet

TL;DR

Generate fresh host keys on every box, update your known_hosts and SSHFP records, then remove the old keys. Do it in two passes so you never lock yourself out: add the new keys alongside the old ones, verify, then drop the old. ansible or your config tool makes this painless at scale.

how to rotate SSH host keys across a fleet

Use this when

• Host keys are years old and nobody remembers when they were created
• A server may have been compromised and its host keys cannot be trusted
• Compliance or a pentest flagged stale host keys
• You are standardizing a fleet after inheriting it

Not for this skill when

• You need to rotate user keys, not host keys (different process)
• You are moving to SSH certificates (do that instead of rotating keys)
• It is a single server (just run ssh-keygen -A and update your own known_hosts)

Steps

1. Inventory the current host keys

List what each host actually has before touching anything. You need the baseline so you can confirm the change later.

ssh-keyscan -t ed25519,rsa [host] > current_keys.txt
cat current_keys.txt

Expected: the host's public keys printed, one per line. Save this file; it is your before snapshot.

2. Generate new host keys on the host

On each server, generate fresh host keys. The -A flag creates any missing key types with default paths.

sudo ssh-keygen -A
sudo ls -l /etc/ssh/ssh_host_*

Expected: new ssh_host_* files with today's timestamp. Keep the sshd service running; the new keys take effect on reload.

Variant: with ansible across the fleet

- name: rotate host keys
  hosts: all
  become: true
  tasks:
    - name: generate fresh host keys
      command: ssh-keygen -A
      args:
        creates: /etc/ssh/.keys_rotated_2026
    - name: mark rotation done
      file:
        path: /etc/ssh/.keys_rotated_2026
        state: touch

Expected: the marker file keeps the task idempotent, so reruns do not generate new keys every time.

3. Publish the new keys where clients expect them

Update known_hosts on your admin machines and any SSHFP DNS records. The -R flag removes old entries cleanly.

ssh-keygen -R [host]
ssh-keyscan -t ed25519,rsa [host] >> ~/.ssh/known_hosts

Expected: no "host key verification failed" warnings on the next login, and ssh-keygen -F [host] shows the new key.

4. Verify every host took the new keys

Spot-check a sample of the fleet against the before snapshot from step 1. Any host still showing an old key needs attention.

ssh-keyscan -t ed25519,rsa [host] | sort > new_keys.txt
if diff -q current_keys.txt new_keys.txt > /dev/null; then echo "WARNING: keys unchanged, rotation did not run"; else echo "rotation confirmed: keys differ from baseline"; fi

Expected: "rotation confirmed" for every host. If a host reports unchanged keys, the regen did not run there.

5. Remove the old keys and reload sshd

Once clients are updated, delete the old host key files and reload sshd so only the new keys are served.

sudo rm /etc/ssh/ssh_host_*_old 2>/dev/null; sudo systemctl reload sshd
sudo sshd -T | grep -i hostkey

Expected: sshd -T lists only the new key paths, and logins still work.

Why this happens

Host keys are generated once at install and then forgotten for the life of the machine. If one leaks, or a snapshot of the VM escapes, every future connection can be impersonated. Rotation bounds that exposure, and doing it on a schedule means it is a routine chore instead of an incident.

Edge cases and pitfalls

• Automation that pins host keys (CI runners, backup scripts) breaks on rotation; find them before you start, not after.
• SSHFP records in DNS need updating too, or DNSSEC-verifying clients will complain.
• Do not delete old keys before every client has the new ones; the two-pass approach exists for a reason.
• Cloud images sometimes regenerate host keys on first boot already; check before assuming yours are stale.
• Keep the before snapshot until the next rotation cycle; it is your audit trail.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst8BVIBd-n4XFRkAJOVeibQ