SSH Key Management at Fleet Scale: Rotation and Certificates on VPS

Disabling password authentication and switching to SSH keys is step one of any VPS hardening checklist — and the moment you manage more than a handful of servers, the keys themselves become the problem. Unlabeled keys accumulate in authorized_keys, nobody knows which laptop a key belongs to, and rotation means manually editing files on every host. This guide covers the practices that keep key management sane at fleet scale. If you are still choosing where to run this, our VPS comparison table is a useful starting point for hosts that give you full control over sshd.

Audit What You Have Before Changing Anything

You cannot manage what you cannot see. On every server, list the keys that are authorized today, with their fingerprints and comments:

# Human-readable inventory of authorized keys
awk '{print $3, $1}' ~/.ssh/authorized_keys | sort | uniq -c | sort -rn

# Fingerprint each key so you can match it to a person/role
ssh-keygen -lf ~/.ssh/authorized_keys



Two things to enforce while you are in there: every key must carry a comment identifying its owner (ssh-keygen -t ed25519 -C "alice-laptop-2026"), and keys without comments should be removed or re-issued. The awk one-liner above instantly reveals duplicates and orphans — keys that appear on ten servers but belong to someone who left the team two years ago.



Rotation: A Procedure, Not an Emergency



Key rotation is often treated as a crisis response, but it should be a scheduled, boring operation:



Generate the new keypair locally: ssh-keygen -t ed25519 -a 100 -C "alice-laptop-2026b".
Add the public key to every target server's authorized_keys — do not remove the old key yet.
Verify the new key works from a fresh session: ssh -i ~/.ssh/id_ed25519_new user@server 'hostname'.
Remove the old key from all servers once you have confirmed the new one authenticates.



For a fleet, script step 2 and 4 with a small loop over your inventory file rather than editing files by hand:



for host in $(cat /etc/ansible/hosts); do
  ssh-copy-id -i ~/.ssh/id_ed25519_new.pub deploy@$host
done



The discipline that makes rotation painless: keep the overlap window short (hours, not weeks), and keep a record of which key fingerprint maps to which person and period. A spreadsheet works; an inventory file in Git works better.



Agent Forwarding Is a Liability — Use ProxyJump



ssh -A (agent forwarding) lets a compromised intermediate server use your agent to authenticate onward — an attacker with root on the jump host can hijack your key's signing while the session is live. The safer pattern for multi-hop setups is ProxyJump, where your local machine does the authentication for the final hop and the intermediate server never touches your private key:



# ~/.ssh/config
Host internal-*
  ProxyJump bastion.example.com
  User deploy



Disable forwarding server-side too: AllowAgentForwarding no in sshd_config on every host that does not strictly need it.



SSH Certificates: The Fleet-Scale Upgrade



When you manage dozens of servers, maintaining authorized_keys on each one stops scaling. SSH certificates flip the model: one CA keypair, servers trust the CA's public key, and users get short-lived, CA-signed certificates instead of permanent keys. Setting it up takes minutes:



# On a dedicated, offline CA host
ssh-keygen -t ed25519 -f /etc/ssh/ca_user_key -C "user-ca"

# Sign a user's public key for 30 days
ssh-keygen -s /etc/ssh/ca_user_key -I "alice" -n alice,deploy \
  -V +30d ~alice/.ssh/id_ed25519.pub

# On each server: trust the CA
echo "TrustedUserCAKeys /etc/ssh/ca_user_key.pub" >> /etc/ssh/sshd_config



Revocation is the killer feature: instead of editing authorized_keys on every host when someone leaves, you sign a revocation list once and push it — ssh-keygen -k -s /etc/ssh/ca_user_key -u /etc/ssh/revoked_keys. Servers refuse any certificate listed there, instantly, fleet-wide.



Verify, Then Verify Again



ssh -v user@server — confirm the authentication method actually used: publickey, not password or keyboard-interactive.
ssh-keygen -L -f ~/.ssh/id_ed25519-cert.pub — inspect a certificate's principals and validity window.
sshd -T | grep -E 'trusteduserca|allowagent' — confirm effective sshd settings after your config edits.



Good key hygiene is a habit, not a project: audit quarterly, rotate on a schedule, keep agent forwarding off, and move to certificates as soon as the fleet outgrows manual authorized_keys editing. The same "control what you expose" logic applies to choosing a host — our security and control features comparison shows which providers let you manage SSH access cleanly, and the provider ranking is a good place to start if you are consolidating a fleet onto fewer, better-managed servers.


Related Articles：


VPS Automated Backup Strategies: Comparing rsync, BorgBackup, and Restic for Server Data

Fail2ban on a VPS: Blocking Brute-Force Attacks Automatically

Fail2ban on a VPS: Block SSH Brute-Force Attacks With Custom Jails

Setting Up nftables on a Linux VPS: A Practical Firewall Guide