SSH Keys for Cloud Servers: The Complete Setup & Security Guide

SSH (Secure Shell) key authentication is the gold standard for securing access to cloud servers. Whether you're managing a single VPS Hosting instance or an entire fleet of Dedicated Servers, replacing password-based logins with cryptographic key pairs dramatically reduces your attack surface and streamlines administrative workflows. This comprehensive guide covers everything you need to know — from the underlying mechanics to step-by-step configuration and hardening best practices.

What Are SSH Keys?

SSH keys are asymmetric cryptographic key pairs used to authenticate a client to an SSH server. Unlike a username/password combination — which is susceptible to brute-force attacks, credential stuffing, and phishing — SSH keys rely on mathematical relationships between two distinct components:

• Private Key: Stored exclusively on your local machine. This file must never be shared, transmitted, or exposed. It is the proof of your identity.
• Public Key: Deployed to the remote server. It can be shared freely without compromising security.

When you initiate an SSH connection, the server checks whether your public key exists in its ~/.ssh/authorized_keys file. If it does, the server issues a cryptographic challenge that only the holder of the corresponding private key can solve. A successful response grants access — no password required.

Why Use SSH Keys for Cloud Servers?

SSH key authentication offers concrete, measurable advantages over traditional password logins:

Key Benefits in Detail

Enhanced Security

SSH keys use 2048-bit to 4096-bit RSA encryption (or Ed25519 elliptic-curve cryptography), making them computationally infeasible to crack. There is no shared secret transmitted over the network, eliminating interception risks entirely.

Operational Convenience

Once configured, SSH keys enable passwordless logins. For administrators managing multiple servers — including environments running VPS Control Panels — this eliminates repetitive credential entry and dramatically speeds up workflows.

Automation-Ready

CI/CD pipelines, deployment scripts, configuration management tools (Ansible, Puppet, Chef), and backup jobs all rely on non-interactive SSH authentication. Key-based auth is the only practical solution for these use cases.

Granular Access Control

Each user or service receives a unique key pair. Revoking access for a specific user requires nothing more than deleting their public key from the server — no password resets, no account lockouts.

How SSH Key Authentication Works: Step-by-Step

Understanding the authentication handshake helps you troubleshoot issues and appreciate why this method is so secure:

1. Connection Request: Your SSH client sends a connection request to the server, advertising which public key it intends to use.
2. Key Lookup: The server searches ~/.ssh/authorized_keys for a matching public key.
3. Challenge Issuance: If a match is found, the server generates a random challenge and encrypts it using your public key.
4. Challenge Response: Your SSH client decrypts the challenge using your private key and sends back a cryptographic signature derived from the decrypted data.
5. Verification & Access: The server verifies the signature using the public key. If valid, access is granted — without a single password being transmitted.

This entire exchange happens in milliseconds and is resistant to man-in-the-middle attacks when host key verification is properly configured.

How to Generate SSH Keys

On Linux or macOS

Open your terminal and run the following command:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Parameter breakdown:

• -t rsa — Specifies the RSA algorithm
• -b 4096 — Generates a 4096-bit key (stronger than the 2048-bit default)
• -C "your_email@example.com" — Embeds an identifying comment in the key

Modern Alternative — Ed25519 (Recommended):

ssh-keygen -t ed25519 -C "your_email@example.com"

Ed25519 keys are shorter, faster, and considered more secure than RSA-4096 for most modern use cases.

During key generation, you will be prompted to:

1. Choose a save location — Press Enter to accept the default (~/.ssh/id_rsa or ~/.ssh/id_ed25519)
2. Set a passphrase — Strongly recommended. This encrypts your private key on disk, providing a second layer of protection if your machine is compromised

After generation, you will have two files:

• ~/.ssh/id_rsa — Your private key (never share this)
• ~/.ssh/id_rsa.pub — Your public key (safe to distribute)

On Windows

Windows 10 and Windows 11 include OpenSSH natively. Open PowerShell or Command Prompt and run:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

The process is identical to Linux/macOS. Your keys will be saved to C:UsersYourUsername.ssh.

Alternative: PuTTYgen

If you use PuTTY as your SSH client:

1. Open PuTTYgen
2. Select RSA and set the bit count to 4096
3. Click Generate and move your mouse to create entropy
4. Save the private key (.ppk format) and copy the public key text

Adding Your SSH Public Key to the Cloud Server

Once your key pair is generated, the public key must be installed on the target server.

Method 1: Using ssh-copy-id (Linux/macOS — Recommended)

ssh-copy-id user@your-server-ip

This command automatically appends your public key to ~/.ssh/authorized_keys on the remote server. You will be prompted for your password once — after that, password-based access is no longer needed.

To specify a particular key file:

ssh-copy-id -i ~/.ssh/id_rsa.pub user@your-server-ip

Method 2: Manual Installation (All Platforms)

Use this method when ssh-copy-id is unavailable or when you need precise control.

Step 1: Display your public key:

cat ~/.ssh/id_rsa.pub

Copy the entire output — it will look similar to:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ... your_email@example.com

Step 2: Connect to your server using password authentication:

ssh user@your-server-ip

Step 3: Create the .ssh directory if it does not exist:

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Step 4: Add your public key to the authorized_keys file:

nano ~/.ssh/authorized_keys

Paste the public key, save, and exit (Ctrl+X, then Y, then Enter).

Step 5: Set the correct file permissions:

chmod 600 ~/.ssh/authorized_keys

> Critical: Incorrect permissions will cause SSH to silently reject your key. The .ssh directory must be 700 and authorized_keys must be 600.

Step 6: Test the connection before closing your current session:

ssh -i ~/.ssh/id_rsa user@your-server-ip

Disable Password Authentication (Strongly Recommended)

Once SSH key authentication is confirmed working, disabling password logins eliminates the most common attack vector against SSH servers. This is an essential hardening step for any production environment.

Step 1: Open the SSH daemon configuration file:

sudo nano /etc/ssh/sshd_config

Step 2: Locate and modify the following directives:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no
PermitRootLogin prohibit-password

Step 3: Save the file and restart the SSH service:

sudo systemctl restart sshd

> Warning: Before restarting sshd, verify that your SSH key login works in a separate terminal session. Locking yourself out of a remote server is a serious operational risk.

After this change, only clients presenting a valid, authorized SSH key will be able to connect.

Advanced SSH Key Management

Managing Multiple Users

To grant multiple users access to a server, simply append each user's public key on a new line in the authorized_keys file:

nano ~/.ssh/authorized_keys
# Add one public key per line

Revoking Access

To revoke a specific user's access, open authorized_keys, locate their key (identifiable by the comment at the end), and delete that line:

nano ~/.ssh/authorized_keys

No service restart is required — the change takes effect immediately.

Using an SSH Config File for Multiple Servers

If you manage multiple servers, an SSH config file (~/.ssh/config) simplifies connections:

Host alexhost-vps
    HostName your-server-ip
    User root
    IdentityFile ~/.ssh/id_rsa_alexhost
    Port 22

Host alexhost-dedicated
    HostName your-dedicated-ip
    User admin
    IdentityFile ~/.ssh/id_rsa_dedicated

With this configuration, connecting is as simple as:

ssh alexhost-vps

Using ssh-agent for Passphrase Management

If your private key is protected by a passphrase, ssh-agent caches it in memory so you only enter it once per session:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_rsa

SSH Key Security Best Practices

SSH Keys and AlexHost Services

SSH key authentication is supported across all AlexHost server products. Whether you're deploying a lightweight application on Shared Web Hosting, scaling with a fully managed VPS with cPanel, or running compute-intensive workloads on GPU Hosting, SSH key-based access provides the security foundation your infrastructure requires.

For complete server security, consider pairing SSH hardening with an SSL Certificate to encrypt all web-facing traffic — ensuring end-to-end protection for both your server administration and your users.

Frequently Asked Questions

Can I use multiple SSH keys on the same server?

Yes. Each public key occupies one line in authorized_keys. There is no practical limit to the number of authorized keys.

What happens if I lose my private key?

You lose access via that key pair. If password authentication is disabled and you have no other access method, you may need to use your hosting provider's out-of-band console access (such as AlexHost's VPS control panel) to regain entry and add a new key.

Is Ed25519 better than RSA?

For most modern use cases, yes. Ed25519 offers equivalent or superior security with shorter keys and faster operations. RSA-4096 remains acceptable but is considered legacy by many security professionals.

Should I use the same SSH key for all servers?

No. Using unique key pairs per server limits blast radius — if one private key is compromised, only that server is at risk.

Conclusion

SSH key authentication is not merely a best practice — it is the baseline security requirement for any serious cloud server deployment. By replacing vulnerable password logins with cryptographic key pairs, you eliminate entire categories of attacks including brute-force, credential stuffing, and password interception.

The setup process requires a modest one-time investment: generate a key pair, deploy the public key to your server, disable password authentication, and implement the management practices outlined in this guide. The return on that investment is a dramatically more secure, more manageable, and more automation-friendly server infrastructure.

Start securing your servers today with AlexHost's range of hosting solutions — from entry-level VPS Hosting to high-performance Dedicated Servers — all built to support modern security standards from day one.