{"id":532,"date":"2026-06-28T07:03:21","date_gmt":"2026-06-28T07:03:21","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=532"},"modified":"2026-08-17T22:13:22","modified_gmt":"2026-08-17T22:13:22","slug":"vps-ssh-key-management-best-practices-multi-server","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/vps-ssh-key-management-best-practices-multi-server\/","title":{"rendered":"SSH Key Management at Fleet Scale: Rotation and Certificates on VPS"},"content":{"rendered":"\n

Disabling password authentication and switching to SSH keys is step one of any VPS hardening checklist \u2014 and the moment you manage more than a handful of servers, the keys themselves become the problem. Unlabeled keys accumulate in authorized_keys<\/code>, nobody knows which laptop a key belongs to, and rotation means manually editing files on every host. This guide covers the practices that keep key management sane at fleet scale. If you are still choosing where to run this, our VPS comparison table<\/a> is a useful starting point for hosts that give you full control over sshd<\/code>.<\/p>\n\n\n\n

Audit What You Have Before Changing Anything<\/h2>\n\n\n\n

You cannot manage what you cannot see. On every server, list the keys that are authorized today, with their fingerprints and comments:<\/p>\n\n\n\n

# Human-readable inventory of authorized keys\nawk '{print $3, $1}' ~\/.ssh\/authorized_keys | sort | uniq -c | sort -rn\n\n# Fingerprint each key so you can match it to a person\/role\nssh-keygen -lf ~\/.ssh\/authorized_keys<\/code>\n\n\n\n
Two things to enforce while you are in there: every key must carry a comment identifying its owner (ssh-keygen -t ed25519 -C \"alice-laptop-2026\"<\/code>), and keys without comments should be removed or re-issued. The awk<\/code> one-liner above instantly reveals duplicates and orphans \u2014 keys that appear on ten servers but belong to someone who left the team two years ago.<\/p>\n\n\n\n
Rotation: A Procedure, Not an Emergency<\/h2>\n\n\n\n
Key rotation is often treated as a crisis response, but it should be a scheduled, boring operation:<\/p>\n\n\n\n
  1. Generate the new keypair locally: ssh-keygen -t ed25519 -a 100 -C \"alice-laptop-2026b\"<\/code>.<\/li>
  2. Add the public key to every target server's authorized_keys<\/code> \u2014 do not remove the old key yet.<\/li>
  3. Verify the new key works from a fresh session: ssh -i ~\/.ssh\/id_ed25519_new user@server 'hostname'<\/code>.<\/li>
  4. Remove the old key from all servers once you have confirmed the new one authenticates.<\/li><\/ol>\n\n\n\n
    For a fleet, script step 2 and 4 with a small loop over your inventory file rather than editing files by hand:<\/p>\n\n\n\n
    for host in $(cat \/etc\/ansible\/hosts); do\n  ssh-copy-id -i ~\/.ssh\/id_ed25519_new.pub deploy@$host\ndone<\/code>\n\n\n\n
    The discipline that makes rotation painless: keep the overlap window short (hours, not weeks), and keep a record of which key fingerprint maps to which person and period. A spreadsheet works; an inventory file in Git works better.<\/p>\n\n\n\n
    Agent Forwarding Is a Liability \u2014 Use ProxyJump<\/h2>\n\n\n\n
    ssh -A<\/code> (agent forwarding) lets a compromised intermediate server use your agent to authenticate onward \u2014 an attacker with root on the jump host can hijack your key's signing while the session is live. The safer pattern for multi-hop setups is ProxyJump<\/code>, where your local machine does the authentication for the final hop and the intermediate server never touches your private key:<\/p>\n\n\n\n
    # ~\/.ssh\/config\nHost internal-*\n  ProxyJump bastion.example.com\n  User deploy<\/code>\n\n\n\n
    Disable forwarding server-side too: AllowAgentForwarding no<\/code> in sshd_config<\/code> on every host that does not strictly need it.<\/p>\n\n\n\n
    SSH Certificates: The Fleet-Scale Upgrade<\/h2>\n\n\n\n
    When you manage dozens of servers, maintaining authorized_keys<\/code> on each one stops scaling. SSH certificates flip the model: one CA keypair, servers trust the CA's public key, and users get short-lived, CA-signed certificates instead of permanent keys. Setting it up takes minutes:<\/p>\n\n\n\n
    # On a dedicated, offline CA host\nssh-keygen -t ed25519 -f \/etc\/ssh\/ca_user_key -C \"user-ca\"\n\n# Sign a user's public key for 30 days\nssh-keygen -s \/etc\/ssh\/ca_user_key -I \"alice\" -n alice,deploy \\\n  -V +30d ~alice\/.ssh\/id_ed25519.pub\n\n# On each server: trust the CA\necho \"TrustedUserCAKeys \/etc\/ssh\/ca_user_key.pub\" >> \/etc\/ssh\/sshd_config<\/code>\n\n\n\n
    Revocation is the killer feature: instead of editing authorized_keys<\/code> on every host when someone leaves, you sign a revocation list once and push it \u2014 ssh-keygen -k -s \/etc\/ssh\/ca_user_key -u \/etc\/ssh\/revoked_keys<\/code>. Servers refuse any certificate listed there, instantly, fleet-wide.<\/p>\n\n\n\n
    Verify, Then Verify Again<\/h2>\n\n\n\n