Rotating an SSH key sounds simple until you’re the one holding the terminal when a deploy script suddenly can’t authenticate. How to rotate SSH keys without downtime comes down to one rule: never remove access before the replacement is proven to work. Skip that order and a routine security task turns into an outage, an emergency console session, or a page to whoever is on call.
Most teams put off rotating keys until someone leaves or a key leaks, because the fear of locking themselves out outweighs the security upside of doing it on a schedule. That fear is solvable with a process, not a hope. This guide walks through the exact sequence that keeps a server reachable at every step, how to do it across more than one host, and what to do if something still goes wrong.
What zero-downtime SSH key rotation actually means
Zero downtime doesn’t mean the key changes without anyone noticing. It means there is never a moment where the old key has been removed and the new key isn’t trusted yet. Every rotation method that avoids downtime follows the same shape: add the new key, prove it works, then remove the old one. Reverse steps two and three, and that’s the version that locks people out.
Why removing the old key first breaks access
It’s tempting to treat rotation as a swap: delete the old public key from authorized_keys, drop in the new one, done. The problem is the gap between those two edits. If the connection drops mid-edit, a script errors out, or a typo lands in the file, the server is left with no valid key and no open session to fix it. SSH has no built-in undo for a broken authorized_keys file, so the safer move is to never create that gap in the first place.
Before you rotate: audit what actually needs a new key
A rotation that only touches your own laptop key is the easy case. Most real environments have several kinds of keys, and each needs different handling. This is one piece of the larger practice covered in our guide to SSH key management for teams: rotation only goes smoothly if you already know which keys exist, where they’re used, and who owns them.
User keys vs. host keys vs. service account keys
- User keys: the personal key pair a developer uses to log in. Rotate on a schedule, or immediately after someone leaves the team.
- Host keys: the server’s own identity key, the one that triggers a “host key changed” warning on every connecting client if it changes. Rotate these rarely, and warn everyone before you do.
- Service account and CI keys: whatever a deploy pipeline or automation script uses to connect. Highest priority for a short rotation cycle, since a leaked automation key is usually the most damaging kind.
Building a quick inventory across more than one server
Before generating anything new, check what’s actually authorized on each host. A short loop over a list of hosts will surface every key comment currently in use, which tells you what to expect once you start swapping keys out.
for host in $(cat hosts.txt); do
echo "== $host =="
ssh "$host" "cat ~/.ssh/authorized_keys" | awk '{print $NF}'
doneIf a developer connects from a laptop and a couple of cloud workstations, their personal key needs updating everywhere they connect from, not just on the servers. See how to manage SSH credentials across multiple devices for that side of the problem.
The overlap method: how to rotate SSH keys without losing access
This is the core sequence. Every step below assumes you keep the current key working right up until the new one is confirmed, and never skip ahead.
Step 1: generate the new key pair
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new -C "rotated-$(date +%Y%m%d)"Prefer Ed25519 over RSA: the keys are smaller, generate faster, and offer stronger security at a shorter length. If a legacy system requires RSA, use at least 4096 bits.
Step 2: authorize the new key alongside the old one
cat ~/.ssh/id_ed25519_new.pub | ssh user@host "cat >> ~/.ssh/authorized_keys"The important word here is append. Add the new public key as a new line without touching the old one, then confirm permissions are still correct: chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys. A file with the wrong permissions gets silently ignored by sshd, which looks identical to a bad key from the client side.
Step 3: verify the new key from a fresh session, not the one you’re already in
Don’t test the new key inside the terminal tab that’s already connected with the old one. Open a new terminal window and connect specifically with the new key, so you’re testing the exact path a locked-out session would need.
ssh -i ~/.ssh/id_ed25519_new user@hostIf automation or a CI pipeline also connects to this host, confirm that path works with the new key too before moving on. A rotation that only checks the interactive login and skips the deploy pipeline is only half tested.
Step 4: only then revoke the old key
Back up the file before editing it, then remove the old key’s line. Keeping a dated backup means you can restore in seconds if a later cleanup goes wrong.
cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys.bak-$(date +%Y%m%d)
grep -v "old-key-comment" ~/.ssh/authorized_keys > /tmp/ak && mv /tmp/ak ~/.ssh/authorized_keysHow do you rotate keys across many servers at once?
Doing this by hand for one server is manageable. Doing it for a fleet, one host at a time, is where rotations get skipped, forgotten, or done inconsistently.
Scripting the rollout instead of going host by host
The same add-verify-remove order applies, just run as two separate passes across the whole inventory instead of one host at a time. Add the new key everywhere first:
ansible all -i inventory.ini -m authorized_key \
-a "user=deploy key='{{ lookup('file', 'id_ed25519_new.pub') }}' state=present"Verify a sample of hosts, including at least one from every environment (staging, production, whatever automation touches), then run a second pass that removes the old key, again across the whole inventory, only after every host has confirmed the new one.
ansible all -i inventory.ini -m authorized_key \
-a "user=deploy key='{{ lookup('file', 'id_ed25519_old.pub') }}' state=absent"Why manual rotation stops scaling past a handful of hosts
With p people and s servers, manually keeping every person’s key current on every server they touch is roughly a p times s problem: each new hire, departure, or leaked key means touching every host that person had access to. Missing even one host during a manual rotation is indistinguishable, from the outside, from an intentional backdoor, which is exactly why this step benefits from a script and a checklist instead of memory.
What if you rotate a key and lock yourself out anyway?
Keep a break-glass session open for the whole rotation window
Keep at least one already-authenticated session open to the host for the entire duration of the rotation, not just while editing the file. If step 3 fails, or an edit goes wrong during step 4, that open session is what lets you fix authorized_keys without needing SSH to already be working.
Recovering access out-of-band when SSH itself is broken
If every session does close and the new key doesn’t work, you need a path into the box that doesn’t go through sshd. Cloud providers expose exactly this: AWS EC2 Instance Connect or Systems Manager Session Manager, a web-based console on DigitalOcean or Linode, or IPMI and KVM access on bare metal. Know which one applies to your hosts before you need it, not while you’re locked out.
Should key rotation run on a schedule or only after an incident?
Both. A scheduled cadence handles the keys that are fine today but shouldn’t stay valid forever. An incident-driven rotation, someone leaving the team, a laptop lost, a key accidentally pushed to a public repository, happens immediately and outside the schedule, using the same overlap method above.
A sane cadence for regular keys vs. high-privilege and service keys
As a starting point: rotate personal user keys every 90 to 180 days, and rotate high-privilege or service account keys every 30 to 90 days, since those cause more damage if they leak and are touched by fewer humans who’d notice something odd. For the full breakdown of rotation frequency, offboarding triggers, and policy design, see SSH key rotation best practices.
Are SSH certificates a better fix than rotating keys by hand?
SSH certificates, issued by a certificate authority with a built-in expiry, sidestep a lot of this. A certificate simply stops being valid at a set time, and revoking one person’s access doesn’t mean editing authorized_keys on every host they’ve ever touched. That turns the p-times-s management problem into a p-plus-s one. It’s a real infrastructure investment though, standing up a CA and getting every host to trust it, so most teams still need the manual overlap process above for the keys they haven’t migrated yet.
Keeping a record of who rotated what, and when
On a shared bastion or any host more than one person connects to, a rotation is only as trustworthy as the record of who did it. termique’s SSH key vault generates and stores keys in the OS keychain rather than a plaintext file, so the new half of a rotation never sits unencrypted on disk, and its per-command audit log timestamps who ran which command against which host. Pair that with the process above, and a key rotation stops being a manual leap of faith. It becomes a recorded, checkable event, the same way it’s covered in our complete guide to SSH security.
Recap: a zero-downtime SSH key rotation checklist
- Generate the new key before touching anything old.
- Append the new public key. Never overwrite the file.
- Verify from a brand-new session, using only the new key.
- Confirm automation and CI paths work with the new key too, not just interactive logins.
- Only then remove the old key, after keeping a dated backup of authorized_keys.
- For a fleet, script the add-verify-remove sequence instead of doing it host by host.
- Never close your last authenticated session until the new key is confirmed working.
- Rotate service and CI keys on a shorter cycle than personal ones.
termique’s SSH key vault and per-command audit log are free on every plan, so this workflow doesn’t require upgrading anything to start doing rotations properly.
