How to Add SSH Keys to a New or Existing VPS

SSH keys are cryptographic key pairs — a public key stored on the server and a private key kept on your local machine — that authenticate your identity without transmitting a password over the network. When you connect, the server issues a cryptographic challenge that only your private key can solve, granting access if the response is valid. This mechanism makes SSH key authentication fundamentally more resistant to brute-force attacks, credential stuffing, and man-in-the-middle interception than any password-based scheme.

This guide covers the complete process of generating, deploying, and hardening SSH key authentication on both new and existing VPS instances — including edge cases, permission pitfalls, and multi-key management that most tutorials skip entirely.

Why SSH Keys Are Architecturally Superior to Passwords

Password authentication sends a secret (or its hash) across the wire and relies on the server storing a verifiable credential. SSH public-key authentication never exposes the private key — not during generation, not during login, not ever. The private key never leaves your local machine.

Beyond raw security, SSH keys unlock operational capabilities that passwords cannot:

• Unattended automation — CI/CD pipelines, Ansible playbooks, and cron-driven rsync jobs require non-interactive authentication. Passwords break this entirely.
• Granular access control — Each authorized_keys entry can carry command=, from=, and no-pty restrictions, limiting exactly what a given key is permitted to do.
• Auditability — Individual keys can be revoked without changing a shared password and disrupting every other user or script.
• Resistance to phishing — There is no password to steal via a fake login page.

Prerequisites

Before proceeding, confirm the following:

• You have a running VPS or are in the process of provisioning one.
• Your local machine runs Linux, macOS, or Windows with OpenSSH installed (Windows 10/11 ships OpenSSH by default; verify with ssh -V).
• You have either root or sudo access to the target server.
• You understand that losing your private key without an alternative login method (console access, recovery key) can lock you out permanently.

Choosing the Right Key Algorithm

The original ssh-keygen -t rsa -b 4096 command is solid but not the only option. Understanding the tradeoffs helps you make the right choice for your environment.

Ed25519 is the recommended algorithm for any server running OpenSSH 6.5 or later (released 2014). Use RSA 4096 only when connecting to legacy systems that do not support elliptic-curve keys.

Part 1: Adding SSH Keys to a New VPS

Many hosting control panels allow you to inject a public key at provisioning time, before the instance ever boots. This is the cleanest approach — the key is baked into the image and password authentication may never need to be enabled.

Step 1: Generate Your SSH Key Pair

Open a terminal on your local machine. Generate an Ed25519 key pair:

ssh-keygen -t ed25519 -C "your_email@example.com"

If you need RSA for compatibility reasons:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

When prompted for a file location, press Enter to accept the default (~/.ssh/id_ed25519 or ~/.ssh/id_rsa). When prompted for a passphrase, set one — it encrypts the private key on disk, so even if your laptop is stolen, the key is useless without the passphrase. Your SSH agent (ssh-agent) will cache the decrypted key in memory so you only type the passphrase once per session.

Verify the generated files:

ls -la ~/.ssh/

You will see:

• id_ed25519 — your private key (permissions must be 600; never share this file)
• id_ed25519.pub — your public key (safe to copy anywhere)

Display the public key to copy it:

cat ~/.ssh/id_ed25519.pub

The output looks like:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... your_email@example.com

Copy this entire string — you will paste it into your hosting panel.

Step 2: Add the Public Key During VPS Provisioning

Log into your hosting control panel. During the VPS creation workflow:

1. Select your operating system (Ubuntu 22.04 LTS, Debian 12, Rocky Linux 9, etc.).
2. Locate the SSH Key or Authentication section.
3. Paste the full contents of id_ed25519.pub into the provided field.
4. Complete the remaining configuration (plan, region, hostname).

Once the VPS boots, the provisioning system writes your public key to /root/.ssh/authorized_keys automatically. No password login is required.

Step 3: Connect to the Newly Provisioned VPS

ssh root@your_vps_ip

If you used a non-default key filename, specify it explicitly:

ssh -i ~/.ssh/id_ed25519 root@your_vps_ip

A successful connection with no password prompt confirms the key was injected correctly. If you set a passphrase, your SSH agent or a passphrase prompt will handle it locally.

Part 2: Adding SSH Keys to an Existing VPS

If your VPS is already running with password authentication, you need to manually deploy the public key. This is a two-phase process: copy the key, then optionally harden the SSH daemon.

Step 1: Generate a Key Pair (If You Do Not Have One)

Follow the same process as Step 1 in Part 1 above. If you already have a key pair, retrieve your public key:

cat ~/.ssh/id_ed25519.pub

Step 2: Copy the Public Key to the Server

Method A — ssh-copy-id (Recommended)

The ssh-copy-id utility handles directory creation, file appending, and permission setting automatically:

ssh-copy-id -i ~/.ssh/id_ed25519.pub root@your_vps_ip

Enter your password when prompted. The tool appends the key to ~/.ssh/authorized_keys on the remote server and sets correct permissions. This is the safest method because it prevents accidental overwrites of existing keys.

Method B — Manual Deployment

Use this when ssh-copy-id is unavailable (e.g., on some Windows environments or restricted networks):

cat ~/.ssh/id_ed25519.pub | ssh root@your_vps_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

This single pipeline is safer than opening a text editor on the remote server because it appends rather than overwrites.

Method C — Manual via Text Editor (Fallback)

Log into the server with your password:

ssh root@your_vps_ip

Create the .ssh directory if it does not exist:

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Open authorized_keys in a text editor:

nano ~/.ssh/authorized_keys

Paste your public key on a new line. Save with Ctrl+X, then Y, then Enter. Set permissions:

chmod 600 ~/.ssh/authorized_keys

Exit the session:

exit

Step 3: Verify Key-Based Login

Test the connection before making any further changes to the SSH daemon. Open a new terminal window (do not close your existing session yet):

ssh -i ~/.ssh/id_ed25519 root@your_vps_ip

If you connect without a password prompt, the key is working. Only proceed to the hardening steps below after confirming this.

Critical pitfall: If you disable password authentication before verifying key access, and the key deployment failed for any reason, you will lock yourself out. Always test first.

Step 4: Harden the SSH Daemon Configuration

Once key-based access is confirmed, open the SSH daemon configuration file:

nano /etc/ssh/sshd_config

Apply the following settings:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password
AuthorizedKeysFile .ssh/authorized_keys
ChallengeResponseAuthentication no
UsePAM yes

Key notes on these directives:

• PermitRootLogin prohibit-password allows root login via key but blocks root password login — a better middle ground than PermitRootLogin no when you are still setting up a non-root sudo user.
• ChallengeResponseAuthentication no disables keyboard-interactive authentication, which can bypass PasswordAuthentication no on some PAM configurations.
• On Ubuntu 22.04+, the file /etc/ssh/sshd_config.d/50-cloud-init.conf may override your settings. Check and edit it if necessary.

Validate the configuration syntax before restarting:

sshd -t

If no errors are reported, restart the SSH service:

systemctl restart sshd

On systems using ssh.service instead of sshd.service:

systemctl restart ssh

Managing Multiple SSH Keys and Users

Real-world environments rarely involve a single key and a single user. Here is how to handle more complex scenarios.

Adding Keys for Multiple Users

Each user account has its own ~/.ssh/authorized_keys file. To add a key for a non-root user named deploy:

mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
echo "ssh-ed25519 AAAAC3... deploy@ci-server" >> /home/deploy/.ssh/authorized_keys
chmod 600 /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh

The chown step is frequently missed and causes authentication failures — SELinux and OpenSSH both verify that the authorized_keys file is owned by the authenticating user.

Restricting What a Key Can Do

You can prefix any line in authorized_keys with options to limit its capabilities. This is especially useful for deployment keys or backup automation:

command="/usr/bin/rsync --server --daemon .",no-pty,no-agent-forwarding,no-X11-forwarding ssh-ed25519 AAAAC3... backup@monitoring

This key can only run rsync in daemon mode — nothing else.

Using ~/.ssh/config for Cleaner Connections

Instead of typing long ssh commands, define host aliases on your local machine:

Host myserver
    HostName 203.0.113.45
    User root
    IdentityFile ~/.ssh/id_ed25519
    Port 22

After saving this to ~/.ssh/config, connect with simply:

ssh myserver

Using ssh-agent to Cache Passphrases

If your private key has a passphrase (it should), add it to the agent so you are not prompted repeatedly:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

On macOS, add --apple-use-keychain to persist across reboots:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Common Pitfalls and Troubleshooting

Problem: Still prompted for a password after adding the key

Run the connection with verbose output to diagnose:

ssh -vvv root@your_vps_ip

Look for lines like Offering public key and Server accepts key. If the key is offered but rejected, the issue is almost always file permissions on the server.

Check permissions on the server:

ls -la ~/.ssh/
stat ~/.ssh/authorized_keys

Required permissions:

• ~/.ssh/ — 700 (drwx——)
• ~/.ssh/authorized_keys — 600 (-rw——-)
• Home directory ~/ — must not be world-writable (755 or 750)

Problem: SELinux blocking authentication on RHEL/Rocky/AlmaLinux

restorecon -Rv ~/.ssh

Problem: authorized_keys file has Windows line endings (CRLF)

If you edited the file on Windows and transferred it, line endings may be corrupted. Fix with:

sed -i 's/r//' ~/.ssh/authorized_keys

Problem: Wrong key being offered

If you have multiple keys, specify the correct one explicitly:

ssh -i ~/.ssh/id_ed25519 root@your_vps_ip

Or configure it in ~/.ssh/config as shown above.

SSH Keys in the Context of Your Hosting Infrastructure

SSH key authentication is not just a single-server concern — it scales across your entire infrastructure. If you manage a fleet of dedicated servers, a centralized approach using tools like Ansible, Puppet, or a secrets manager (HashiCorp Vault, AWS Secrets Manager) to distribute and rotate authorized keys is essential.

For teams running web applications on VPS with cPanel, cPanel's SSH Access interface under the Security section provides a GUI for generating and managing key pairs — useful for developers who are not comfortable with the command line but still need secure access.

If you are running GPU-intensive workloads on GPU hosting infrastructure, SSH key authentication is especially critical because these instances often run long-lived, unattended jobs. A compromised credential on a GPU server can result in significant unauthorized compute costs in addition to data exposure.

Pairing SSH hardening with a valid SSL certificate on any web-facing services running on the same server closes the two most common attack vectors simultaneously — unauthenticated shell access and unencrypted HTTP traffic.

For projects using shared web hosting that also need SSH access, check whether your provider supports SSH key injection through the hosting panel, as shared environments often restrict SSH to specific users with limited shell access.

Key Rotation and Long-Term Key Management

SSH keys do not expire automatically. Without a rotation policy, a key compromised years ago may still grant access. Establish these practices:

• Rotate keys annually or immediately upon suspicion of compromise.
• Audit authorized_keys files on all servers regularly. Remove keys belonging to former employees or decommissioned services.
• Use separate keys per device — do not copy the same private key to multiple laptops or servers. If one device is lost, you revoke only that key.
• Use separate keys per role — your personal access key, your CI/CD deployment key, and your backup automation key should all be distinct.
• Document key ownership — maintain a registry mapping each public key fingerprint to its owner, purpose, and expiry date.

To display the fingerprint of a key for auditing:

ssh-keygen -lf ~/.ssh/id_ed25519.pub

Technical Decision Checklist

Use this matrix before finalizing your SSH key setup:

• Algorithm: Use Ed25519 unless connecting to systems older than OpenSSH 6.5; use RSA 4096 as a fallback.
• Passphrase: Always set one on private keys used by humans; service keys used by automation may omit it if stored in a secrets manager.
• Deployment method: Use ssh-copy-id for interactive setups; use configuration management (Ansible, Puppet) for fleet deployments.
• Permission verification: Confirm 700 on ~/.ssh/, 600 on authorized_keys, and correct ownership before disabling password auth.
• Test before locking down: Always verify key login in a separate terminal session before setting PasswordAuthentication no.
• Daemon config validation: Run sshd -t before restarting the SSH service to catch syntax errors.
• Disable password auth: Set PasswordAuthentication no and ChallengeResponseAuthentication no together — one alone is insufficient on PAM-enabled systems.
• Root login policy: Prefer PermitRootLogin prohibit-password over PermitRootLogin yes; migrate to a non-root sudo user and set PermitRootLogin no as the final hardened state.
• Key rotation: Schedule annual rotation; revoke immediately on device loss or personnel change.
• Audit: Periodically run grep -r "" /home/*/.ssh/authorized_keys /root/.ssh/authorized_keys to review all authorized keys across the system.

FAQ

Can I add multiple SSH keys to the same server?

Yes. Each line in ~/.ssh/authorized_keys represents one authorized public key. You can add as many keys as needed — one per line. This allows multiple administrators or multiple devices to authenticate independently, and you can revoke any single key by deleting its line without affecting the others.

What happens if I lose my private key after disabling password authentication?

You will be locked out of SSH. Recovery requires accessing the server through an out-of-band method: your hosting provider's web console, VNC, or a rescue/recovery boot environment. From there you can re-enable password authentication or add a new public key. This is why keeping console access credentials secure and separate is essential.

Is Ed25519 supported on all servers?

Ed25519 requires OpenSSH 6.5 or later, released in January 2014. Any server running a modern Linux distribution (Ubuntu 18.04+, Debian 9+, CentOS 7+, Rocky Linux 8+) supports it. The only scenario requiring RSA is connecting to genuinely legacy systems or embedded devices with outdated OpenSSH builds.

Does adding an SSH key automatically disable password login?

No. Adding a key to authorized_keys enables key-based login but does not disable password authentication. You must explicitly set PasswordAuthentication no in /etc/ssh/sshd_config and restart the SSH daemon to enforce key-only access.

Can I use the same SSH key pair for multiple servers?

Technically yes, but it is not recommended for production environments. Using a single key across many servers means that compromising one private key grants access to all of them. The better practice is to use per-device keys (one key per workstation or laptop) so that losing a device requires revoking only that key across your server fleet, not replacing keys everywhere simultaneously.