How to Upload an SSH Public Key to an Existing VPS

An SSH public key is a cryptographic credential stored in ~/.ssh/authorized_keys on a remote server that grants access to any client holding the corresponding private key — without transmitting a password over the network. Uploading your public key to an existing VPS replaces or supplements password-based authentication with asymmetric cryptography, eliminating the attack surface exploited by brute-force and credential-stuffing campaigns.

This guide covers every stage of the process: key generation, manual and automated upload methods, permission hardening, sshd_config tuning, multi-key management, and common failure modes that trip up even experienced administrators.

Why SSH Key Authentication Outperforms Passwords

Before touching a single command, it is worth understanding the cryptographic mechanics. When you authenticate with a key pair, the server issues a challenge encrypted with your public key. Only the holder of the matching private key can decrypt and sign the response. No secret is ever transmitted — contrast this with password authentication, where the credential itself travels across the wire (even if TLS-wrapped).

The practical upshot: on any VPS Hosting environment exposed to the public internet, SSH keys are not optional hardening — they are the baseline.

Prerequisites

• An existing VPS reachable via SSH (password or existing key login currently works)
• A local machine running Linux, macOS, or Windows with OpenSSH or PuTTY
• Sufficient privileges on the VPS to write to the target user's home directory
• Basic comfort with a terminal and a text editor such as nano or vim

Step 1: Generate an SSH Key Pair

If you already have a key pair at ~/.ssh/id_ed25519 or ~/.ssh/id_rsa, skip ahead. Otherwise, generate one now.

Choosing the Right Algorithm

Ed25519 is the modern standard. Use RSA 4096 only when connecting to legacy servers that do not support Ed25519.

Generate an Ed25519 key pair:

ssh-keygen -t ed25519 -C "your_email@example.com"

Generate an RSA 4096 key pair (legacy systems):

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

During key generation you will be prompted for a save path and an optional passphrase. Accepting the default path (~/.ssh/id_ed25519) is fine for most users. Always set a passphrase — it encrypts the private key on disk using AES-256, so a stolen laptop does not automatically mean a compromised server.

The process produces two files:

~/.ssh/id_ed25519 — your private key. Never share this, never copy it to a server, never commit it to version control.
~/.ssh/id_ed25519.pub — your public key. This is safe to distribute freely.

Display the public key so you can copy it:
cat ~/.ssh/id_ed25519.pub
The output will look similar to:
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... your_email@example.com
Copy the entire single-line string including the algorithm prefix and the comment at the end.
Step 2: Log Into Your VPS
Connect to the VPS using your current authentication method:
ssh root@your_vps_ip
Replace your_vps_ip with the actual IPv4 or IPv6 address of your server. If you are managing a non-root user account, substitute root with the appropriate username. On Dedicated Servers where you may have multiple user accounts, always prefer deploying keys to a non-root user and using sudo for privilege escalation.
Step 3: Prepare the .ssh Directory
Once logged in, verify or create the .ssh directory for the target user:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
The 700 permission (rwx------) is mandatory. OpenSSH will silently refuse to use authorized_keys if the .ssh directory is group- or world-writable. This is one of the most common reasons key authentication fails after an otherwise correct setup.
Step 4: Add the Public Key to authorized_keys
Method A: Manual Paste (Universal)
Open or create the authorized_keys file:
nano ~/.ssh/authorized_keys
Paste your public key on a new line. Each line in this file represents one authorized key. Save with Ctrl+X, then Y, then Enter.
Set the correct permissions:
chmod 600 ~/.ssh/authorized_keys
The 600 permission (rw-------) ensures only the file owner can read or write it. OpenSSH enforces this strictly.
Method B: ssh-copy-id (Recommended for Speed)
If your local machine has ssh-copy-id available (standard on Linux and macOS), this single command handles directory creation, key appending, and permission setting automatically:
ssh-copy-id -i ~/.ssh/id_ed25519.pub root@your_vps_ip
You will be prompted for the current SSH password once. After that, key-based login is active. The -i flag explicitly specifies which public key to upload, preventing accidental uploads of the wrong key.
Method C: One-Liner via cat and Pipe (Scriptable)
Useful in automation pipelines or when ssh-copy-id is unavailable:
cat ~/.ssh/id_ed25519.pub | ssh root@your_vps_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
This approach is idempotent-safe in the sense that it appends rather than overwrites, preserving any existing authorized keys.
Step 5: Verify Correct File Ownership
A frequently overlooked pitfall: if you created the .ssh directory or authorized_keys file as root while setting up another user's account, the ownership will be wrong and SSH will reject the key silently.
Check ownership:
ls -la ~/.ssh/
The output should show the target username as both owner and group for both the directory and the file:
drwx------ 2 alice alice 4096 Jan 15 10:00 .ssh
-rw------- 1 alice alice  571 Jan 15 10:00 .ssh/authorized_keys
If ownership is incorrect, fix it (run as root):
chown -R alice:alice /home/alice/.ssh
Step 6: Test the SSH Key Login
Exit the current session:
exit
Reconnect using explicit key specification to confirm the setup:
ssh -i ~/.ssh/id_ed25519 root@your_vps_ip
If the login succeeds without a password prompt (or prompts only for your local key passphrase), the configuration is correct. If it still asks for the server password, proceed to the troubleshooting section below.
Step 7: Harden the SSH Daemon Configuration
Once key-based login is confirmed working, disable password authentication to eliminate the password brute-force attack vector entirely.
Open the SSH daemon configuration file:
nano /etc/ssh/sshd_config
Locate and set the following directives. If a line is commented out with #, remove the # and set the value:
PasswordAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PermitRootLogin prohibit-password
ChallengeResponseAuthentication no
UsePAM yes
A note on PermitRootLogin prohibit-password: this setting allows root login exclusively via key, blocking password-based root access while still permitting key-authenticated root sessions. For maximum hardening, set it to no and use a non-root user with sudo.
On some distributions, a secondary configuration drop-in may override your settings. Check for overrides:
grep -r "PasswordAuthentication" /etc/ssh/sshd_config.d/
If any file in that directory sets PasswordAuthentication yes, edit or remove it.
Validate the configuration syntax before restarting:
sshd -t
A clean output (no errors) means it is safe to reload. Apply the changes:
systemctl restart sshd
Critical warning: Do not close your existing SSH session before opening a second terminal and confirming you can still log in with your key. Restarting sshd with a misconfigured file or before your key is working will lock you out. Most VPS Control Panels provide an emergency console (KVM/VNC access) for recovery, but it is far better to avoid the situation entirely.
Step 8: Manage Multiple Keys and Servers with ~/.ssh/config
When managing several servers — common in staging/production environments or when administering multiple Dedicated Servers — the SSH client configuration file eliminates the need to remember IP addresses, usernames, and key paths.
Create or edit ~/.ssh/config on your local machine:
nano ~/.ssh/config
Example configuration for multiple hosts:
Host production-vps
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    Port 22

Host staging-vps
    HostName 203.0.113.20
    User deploy
    IdentityFile ~/.ssh/id_ed25519_staging
    Port 2222

Host legacy-server
    HostName 203.0.113.30
    User admin
    IdentityFile ~/.ssh/id_rsa_legacy
    PubkeyAcceptedKeyTypes +ssh-rsa
Set correct permissions on the config file:
chmod 600 ~/.ssh/config
You can now connect with clean aliases:
ssh production-vps
ssh staging-vps
The PubkeyAcceptedKeyTypes +ssh-rsa directive on the legacy entry is important: newer OpenSSH clients (8.8+) disable RSA-SHA1 by default. Without this override, connections to older servers will fail with a cryptic "no matching host key type" error.
Troubleshooting: Why SSH Key Authentication Fails
Even with a correct setup, several environmental factors can cause key authentication to silently fall back to password prompts:
Wrong permissions on .ssh or authorized_keys:
Run ls -la ~/.ssh/ on the server. The directory must be 700 and the file must be 600. Any looser permissions cause OpenSSH to ignore the file.
SELinux or AppArmor context mismatch:
On RHEL/CentOS/AlmaLinux systems with SELinux enforcing, the authorized_keys file may have the wrong security context. Restore it with:
restorecon -Rv ~/.ssh
Wrong user's home directory:
If the user's home directory is not writable only by the owner, SSH will refuse key auth. Check with:
ls -ld ~
The home directory itself must not be group- or world-writable.
AuthorizedKeysFile directive pointing elsewhere:
Some distributions configure AuthorizedKeysFile to use a non-standard path (e.g., /etc/ssh/authorized_keys/%u). Verify the active setting:
sshd -T | grep authorizedkeysfile
Multiple keys and ssh-agent conflicts:
If ssh-agent is running with several loaded keys, the server may reject the connection after too many failed key attempts before trying the correct one. Use -i to specify the key explicitly, or configure IdentitiesOnly yes in ~/.ssh/config.
Firewall or fail2ban blocking your IP:
If you have previously failed multiple login attempts, fail2ban or ufw/iptables rules may have temporarily banned your IP. Check with:
fail2ban-client status sshd
Rotating and Revoking SSH Keys
Key rotation is a security hygiene practice that is often neglected. To revoke a specific key, open ~/.ssh/authorized_keys on the server and delete the corresponding line. Each line is one key — removing it immediately revokes access for the holder of that private key without affecting any other authorized keys.
For auditing purposes, use distinct comments on each key (the part after the key material, e.g., alice@workstation-2024) so you can identify which key belongs to which person or device. When an employee leaves or a device is decommissioned, locate their key by comment and remove it.
To rotate your own key, generate a new pair, add the new public key to authorized_keys, verify login works with the new key, then remove the old key entry.
Practical Key-Takeaway Checklist

Generate Ed25519 keys by default; use RSA 4096 only for legacy server compatibility
Always protect your private key with a strong passphrase
Use ssh-copy-id for fast, error-free key deployment when possible
Verify .ssh directory permissions are 700 and authorized_keys is 600

FAQ

Can I add multiple SSH public keys to the same VPS user account?

Yes. Each line in ~/.ssh/authorized_keys is an independent authorized key. Add one key per line. This is the standard approach for granting access to multiple administrators or from multiple devices — each person or device holds a unique private key, and revocation is per-line.

What happens if I lose my private key after disabling password authentication?

You will be locked out of SSH. Recovery requires using your hosting provider's out-of-band console access (KVM/VNC), which is available through most VPS Hosting control panels. From the console, re-enable PasswordAuthentication yes in /etc/ssh/sshd_config, restart sshd, and upload a new key. This is why testing key login before disabling passwords is non-negotiable.

Is Ed25519 supported on all servers?

Ed25519 requires OpenSSH 6.5 or later (released 2014). Any modern Linux distribution — Ubuntu 18.04+, Debian 9+, CentOS 7+, AlmaLinux 8+ — supports it natively. Only genuinely ancient systems require RSA fallback.

Should I use the same SSH key pair for every server I manage?

It is operationally convenient but creates a single point of compromise. A better practice is to use one key pair per role or environment (e.g., one key for personal servers, one for client infrastructure). This limits blast radius if a private key is ever exposed.

Does uploading an SSH key affect my SSL Certificates or web server configuration?

No. SSH keys govern terminal access to the operating system and are entirely separate from TLS/SSL certificates used by web servers (Apache, Nginx) for HTTPS. They use different ports (22 vs. 443), different key formats, and different trust chains. Changing one has zero effect on the other.