SSH Access: The Complete Technical Guide to Secure Remote Server Management

SSH (Secure Shell) is a cryptographic network protocol that establishes an encrypted tunnel between two networked hosts, enabling authenticated command execution, file transfer, and port forwarding over untrusted networks. It operates on TCP port 22 by default and replaces plaintext predecessors — Telnet, rsh, and FTP — with a protocol that provides confidentiality, integrity, and mutual authentication in a single handshake.

For any administrator managing a VPS or dedicated server, SSH is not optional infrastructure — it is the primary control plane. Every configuration decision you make around SSH directly affects your server's attack surface, operational reliability, and compliance posture.

How SSH Works: The Protocol Architecture

Understanding SSH at the protocol level is what separates administrators who configure it correctly from those who leave exploitable gaps.

The Three-Layer Model

SSH is defined by RFC 4251–4254 and operates across three distinct sublayers stacked on top of TCP:

• SSH Transport Layer Protocol — handles server authentication, key exchange, encryption negotiation, and MAC (message authentication code) setup. This is where the cryptographic handshake occurs.
• SSH User Authentication Protocol — runs on top of the transport layer and handles client-to-server authentication using methods such as publickey, password, keyboard-interactive, or gssapi-with-mic.
• SSH Connection Protocol — multiplexes the encrypted tunnel into logical channels, each carrying a shell session, an SFTP subsystem, a forwarded port, or an agent connection.

The Handshake in Detail

When you run ssh user@host, the following sequence executes before you see a prompt:

1. TCP connection is established to the server on the configured port.
2. Version exchange — client and server exchange protocol version strings (SSH-2.0-OpenSSH_9.x).
3. Algorithm negotiation (SSH_MSG_KEXINIT) — both sides advertise supported key exchange algorithms, host key types, ciphers, MACs, and compression methods. The first mutually supported option in each list wins.
4. Key exchange (KEX) — typically Diffie-Hellman or Elliptic Curve Diffie-Hellman (ECDH). Both sides derive a shared secret without transmitting it. This produces session keys.
5. Server host key verification — the server signs a value with its private host key. The client checks this signature against its ~/.ssh/known_hosts file. A mismatch triggers a warning and blocks connection by default.
6. Encryption activated — all subsequent traffic is encrypted and integrity-protected using the negotiated cipher (e.g., chacha20-poly1305) and MAC.
7. User authentication — the client attempts authentication using the negotiated method. With publickey auth, the client signs a challenge with its private key; the server verifies using the stored public key.
8. Channel open — a shell, SFTP subsystem, or exec channel is opened and the session begins.

This entire process typically completes in under 100 milliseconds on a local network.

Symmetric vs. Asymmetric Cryptography in SSH

A common misconception is that SSH "uses public-key encryption" for all traffic. It does not. The roles are distinct:

SSH vs. Legacy Remote Access Protocols

Generating SSH Key Pairs

SSH key pairs are the foundation of secure, scalable server access. Password authentication is vulnerable to brute-force attacks and credential stuffing; key-based authentication is not.

Choosing the Right Key Algorithm

Ed25519 is the current best practice. It uses Curve25519 elliptic curve cryptography, produces compact 256-bit keys, is faster than RSA at equivalent security levels, and is supported by OpenSSH 6.5+ (released 2014).

ssh-keygen -t ed25519 -C "admin@yourhost.example.com"

Use RSA only when you need compatibility with legacy systems that do not support Ed25519:

ssh-keygen -t rsa -b 4096 -C "admin@yourhost.example.com"

Do not use DSA (limited to 1024 bits, broken) or ECDSA with NIST curves (concerns about NIST curve parameter provenance). Ed25519 is the unambiguous choice for new deployments.

Key Generation Walkthrough

ssh-keygen -t ed25519 -C "ops-team-key-2025"

You will be prompted for:

• File location — default is ~/.ssh/id_ed25519. Accept the default or specify a custom path for multi-key environments.
• Passphrase — always set one. A passphrase encrypts the private key at rest using AES-256-CBC (or bcrypt with newer OpenSSH). If your private key file is stolen, the passphrase is the last line of defense.

This produces two files:

~/.ssh/id_ed25519 — the private key. Never share this. Permissions must be 600.
~/.ssh/id_ed25519.pub — the public key. This is what you distribute to servers.

Managing Multiple Keys with ~/.ssh/config
When managing multiple servers or accounts, use an SSH config file to avoid specifying flags on every connection:
# ~/.ssh/config

Host prod-web
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519_prod
    Port 2222

Host staging
    HostName 203.0.113.20
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_staging
With this in place, ssh prod-web expands to the full connection parameters automatically.
Deploying Your Public Key to a Server
Method 1: ssh-copy-id (Recommended for Initial Setup)
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your_server_ip
This appends the public key to ~/.ssh/authorized_keys on the remote host and sets correct permissions automatically.
Method 2: Manual Deployment (When ssh-copy-id Is Unavailable)
cat ~/.ssh/id_ed25519.pub | ssh user@your_server_ip 
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && 
   cat >> ~/.ssh/authorized_keys && 
   chmod 600 ~/.ssh/authorized_keys"
Method 3: Cloud Provider Console or API
Most cloud providers and hosting control panels allow you to inject public keys during instance provisioning. This is the correct approach for automated infrastructure — the key is present before the instance boots, eliminating the chicken-and-egg problem of needing SSH to deploy SSH keys.
The authorized_keys File Format
Each line in ~/.ssh/authorized_keys is one public key, optionally prefixed with options:
restrict,command="/usr/local/bin/backup.sh" ssh-ed25519 AAAA... backup-key
from="192.168.1.0/24" ssh-ed25519 AAAA... ops-key
The restrict option disables port forwarding, agent forwarding, and PTY allocation for that key — useful for deployment keys or backup automation accounts that should not have interactive shell access.
Hardening the SSH Server: /etc/ssh/sshd_config
A default OpenSSH installation is functional but not hardened. The following configuration represents a production-grade baseline. Apply changes to /etc/ssh/sshd_config, then validate and reload:
sshd -t && systemctl reload sshd
Always run sshd -t before reloading — a syntax error in sshd_config will not crash the running daemon but will prevent it from restarting after a reboot, locking you out.
Recommended sshd_config Hardening Block
# /etc/ssh/sshd_config - Production hardening baseline

Port 2222
AddressFamily inet
ListenAddress 0.0.0.0

# Host keys - prefer Ed25519
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

# Cryptographic hardening
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yes

# Session hardening
LoginGraceTime 30
MaxAuthTries 3
MaxSessions 5
ClientAliveInterval 300
ClientAliveCountMax 2
TCPKeepAlive no

# Disable unused features
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
PermitTunnel no
GatewayPorts no

# Restrict access
AllowUsers deploy ops-user
Banner /etc/ssh/banner.txt
Critical Hardening Decisions Explained
PermitRootLogin no — Root login over SSH is a high-value target. Use a regular user and escalate with sudo. If you absolutely need root-equivalent access via a specific key (e.g., for automation), use PermitRootLogin prohibit-password to allow key-only root login while blocking password attempts.
AllowTcpForwarding no — If your server is not a bastion or jump host, disable TCP forwarding. An attacker with a valid SSH session could otherwise use your server as a proxy to reach internal network resources.
TCPKeepAlive no with ClientAliveInterval — TCPKeepAlive operates at the TCP layer and is visible to network intermediaries. ClientAliveInterval sends keepalive messages through the encrypted SSH channel, which is both more reliable and more private.
LoginGraceTime 30 — Reduces the window during which an unauthenticated connection holds a server slot. The default of 120 seconds is excessive.
AllowUsers — Whitelist only the accounts that legitimately need SSH access. This is a hard gate that operates before any authentication attempt.
Changing the Default SSH Port
Moving SSH off port 22 does not improve security against a targeted attacker — any port scan will find it. What it does do is eliminate the enormous volume of automated, opportunistic brute-force bots that exclusively hammer port 22. This meaningfully reduces log noise and server load.
# In /etc/ssh/sshd_config
Port 2222
Before reloading, open the new port in your firewall:
# UFW
ufw allow 2222/tcp
ufw delete allow 22/tcp

# firewalld
firewall-cmd --permanent --add-port=2222/tcp
firewall-cmd --permanent --remove-service=ssh
firewall-cmd --reload

# iptables
iptables -A INPUT -p tcp --dport 2222 -j ACCEPT
iptables -D INPUT -p tcp --dport 22 -j ACCEPT
If your server runs SELinux, you must also update the SELinux port context:
semanage port -a -t ssh_port_t -p tcp 2222
Connecting to a Server
Basic Connection
ssh user@your_server_ip
Connecting to a Non-Default Port
ssh -p 2222 user@your_server_ip
Connecting with a Specific Key
ssh -i ~/.ssh/id_ed25519_prod user@your_server_ip
Verbose Mode for Debugging
ssh -vvv user@your_server_ip
The -vvv flag prints every step of the handshake, authentication attempt, and channel negotiation. It is the first tool to reach for when a connection fails unexpectedly.
Secure File Transfers Over SSH
SCP (Secure Copy Protocol)
SCP is a simple, non-interactive file copy tool. It is fast and widely available but has no resume capability and limited error handling.
Copy a local file to a remote server:
scp -P 2222 /local/path/file.tar.gz user@your_server_ip:/remote/path/
Copy a remote file to local:
scp -P 2222 user@your_server_ip:/remote/path/file.tar.gz /local/path/
Recursive directory copy:
scp -rp -P 2222 /local/directory/ user@your_server_ip:/remote/directory/
Note: The OpenSSH project deprecated the legacy SCP protocol in OpenSSH 9.0. Modern scp now uses the SFTP protocol under the hood by default. The command interface remains the same, but the underlying transport is more robust.
SFTP (SSH File Transfer Protocol)
SFTP is a full-featured file transfer subsystem with directory listing, permission management, and resume support. It is the correct choice for interactive file management.
sftp -P 2222 user@your_server_ip
Common SFTP commands within the interactive session:
sftp> ls -la                          # List remote directory
sftp> lls                             # List local directory
sftp> put localfile.txt /remote/path/ # Upload file
sftp> get /remote/file.txt ./         # Download file
sftp> mput *.log /remote/logs/        # Upload multiple files
sftp> mkdir /remote/newdir            # Create remote directory
sftp> chmod 640 /remote/file.txt      # Change remote permissions
sftp> bye                             # Exit
rsync over SSH (Production Recommendation)
For synchronizing directories, incremental backups, or large data sets, rsync over SSH is significantly more efficient than SCP or SFTP. It transfers only changed blocks, not entire files.
rsync -avz --progress -e "ssh -p 2222 -i ~/.ssh/id_ed25519" 
  /local/data/ user@your_server_ip:/remote/data/
The -z flag enables compression in transit. For already-compressed data (archives, images), omit it — compression of compressed data wastes CPU without reducing transfer size.
SSH Port Forwarding and Tunneling
Port forwarding is one of SSH's most powerful and underused capabilities. It allows you to securely access services that are not directly exposed to the internet.
Local Port Forwarding
Forward a local port to a remote service. Example: access a remote MySQL instance (port 3306) that is bound to localhost on the server:
ssh -L 3307:127.0.0.1:3306 user@your_server_ip -N
Now mysql -h 127.0.0.1 -P 3307 on your local machine connects through the encrypted tunnel to the remote MySQL.
Remote Port Forwarding
Expose a local service to the remote server. Useful for testing webhooks or sharing a local development server:
ssh -R 8080:127.0.0.1:3000 user@your_server_ip -N
Dynamic Port Forwarding (SOCKS Proxy)
Turn the SSH connection into a SOCKS5 proxy, routing arbitrary TCP traffic through the server:
ssh -D 1080 user@your_server_ip -N
Configure your browser or application to use SOCKS5 127.0.0.1:1080.
SSH Agent and Agent Forwarding
ssh-agent holds decrypted private keys in memory so you do not have to re-enter your passphrase on every connection.
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
Agent forwarding (ssh -A) allows a remote server to use your local agent to authenticate to a third server. This is useful for bastion host architectures but carries risk: a root user on the intermediate server can use your forwarded agent socket. Prefer ProxyJump instead:
ssh -J bastion.example.com user@internal-server.example.com
ProxyJump routes the TCP connection through the bastion without exposing your agent to it.
Troubleshooting Common SSH Issues
Connection Refused (ssh: connect to host ... port 22: Connection refused)
Systematic diagnosis:

Verify the SSH daemon is running: systemctl status sshd

Permission Denied (publickey)

This is the most common SSH error. Work through this checklist:

# On the server, check authorized_keys permissions
ls -la ~/.ssh/
stat ~/.ssh/authorized_keys

# Fix permissions if wrong
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh

# Verify the public key is actually present
cat ~/.ssh/authorized_keys

# Check sshd logs for the specific rejection reason
journalctl -u sshd -n 50 --no-pager
# or
tail -50 /var/log/auth.log

Common causes beyond permissions:

• The authorized_keys file contains the wrong key (e.g., you copied the private key instead of the .pub file).

StrictModes yes in sshd_config (the default) rejects connections if home directory permissions are too open — chmod 755 ~ is the maximum allowed.
AllowUsers or AllowGroups in sshd_config excludes the connecting user.
SELinux is blocking access to ~/.ssh/ — check ausearch -m avc -ts recent.

SSH Connection Timeout
# In /etc/ssh/sshd_config (server side)
ClientAliveInterval 60
ClientAliveCountMax 3

# In ~/.ssh/config (client side)
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
ClientAliveInterval sends a null packet through the encrypted channel every 60 seconds. After ClientAliveCountMax consecutive missed responses, the server terminates the connection. This prevents zombie sessions from accumulating.
Host Key Verification Failed
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
This warning means the server's host key no longer matches what is stored in ~/.ssh/known_hosts. Legitimate causes include server reinstallation or IP address reassignment. Malicious causes include a man-in-the-middle attack. Investigate before proceeding.
If you have confirmed the server was legitimately reinstalled:
ssh-keygen -R your_server_ip
Then reconnect and verify the new host key fingerprint against the server console or provider dashboard before accepting it.
Multi-Factor Authentication for SSH
Key-based authentication is strong, but adding a TOTP (Time-based One-Time Password) second factor creates defense in depth. Even if a private key and its passphrase are compromised, an attacker cannot authenticate without the TOTP token.
Install and configure libpam-google-authenticator on the server:
apt install libpam-google-authenticator
google-authenticator
Then configure PAM and sshd_config:
# /etc/pam.d/sshd - add this line
auth required pam_google_authenticator.so

# /etc/ssh/sshd_config
UsePAM yes
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive
With AuthenticationMethods publickey,keyboard-interactive, a user must provide a valid key AND a TOTP code. This is the correct production configuration for high-value servers.
SSH Certificate Authority: Scaling Key Management
In environments with dozens of servers and multiple administrators, managing individual authorized_keys entries becomes operationally unsustainable. SSH certificates solve this.
An SSH Certificate Authority (CA) signs user and host keys. Servers trust the CA's public key rather than individual user keys. Adding a new administrator requires only signing their public key — no changes to any server's authorized_keys file.
# Create a CA key pair (store the private key offline or in a secrets manager)
ssh-keygen -t ed25519 -f /etc/ssh/ca_key -C "infrastructure-ca-2025"

# Sign a user's public key, valid for 7 days, for specific principals
ssh-keygen -s /etc/ssh/ca_key 
  -I "alice-ops-cert" 
  -n alice,deploy 
  -V +7d 
  ~/.ssh/id_ed25519.pub
On each server, configure trust for the CA:
# /etc/ssh/sshd_config
TrustedUserCAKeys /etc/ssh/ca_key.pub
This is how large-scale infrastructure (including cloud providers and enterprises) manages SSH access without per-server key management.
Practical Decision Matrix: SSH Configuration by Use Case



Use Case
Key Type
Port
Password Auth
Root Login
Forwarding
MFA








—
—
—
—
—
—
—








Personal VPS
Ed25519
2222+
Disabled
Prohibited
Disabled
Optional








Production web server
Ed25519
Non-default
Disabled
No
Disabled
Required








Bastion / jump host
Ed25519
22 or custom
Disabled
No
Controlled
Required








CI/CD automation
Ed25519 (deploy key)
Non-default
Disabled
No
Disabled
No (key only)








Database server
Ed25519
Non-default
Disabled
No
Local only
Required








Development server
Ed25519
Default or custom
Optional
No
Optional
Optional





Setting Up SSH on AlexHost Infrastructure
When you provision a VPS or dedicated server through AlexHost, SSH access is configured by default. The initial root password is delivered via the provisioning email, and the recommended first action is to:

Log in as root, create a non-root administrative user, and add your public key to that user's ~/.ssh/authorized_keys.
Apply the sshd_config hardening baseline documented above.
Disable password authentication and root login.
Configure your firewall to restrict SSH access to known IP ranges where operationally feasible.

If you prefer a graphical management layer alongside SSH, AlexHost's VPS with cPanel and VPS Control Panels options provide a web interface for common tasks while leaving SSH available for full administrative control.
For environments where you need to secure web applications running on the same server, pairing SSH hardening with a properly configured SSL certificate covers both the administrative and application transport layers.
Technical Key-Takeaway Checklist
Before considering your SSH configuration production-ready, verify each of the following:
Key Management

Private keys use Ed25519 or RSA-4096 minimum
All private keys are protected with a strong passphrase
The ~/.ssh/ directory has 700 permissions; authorized_keys has 600

Server Configuration

PasswordAuthentication no is set and active
PermitRootLogin no or prohibit-password is enforced
SSH is running on a non-default port with firewall rules updated
AllowUsers or AllowGroups restricts access to named accounts
LoginGraceTime is set to 30 seconds or less
sshd -t passes without errors after every configuration change

Cryptographic Hardening

KexAlgorithms excludes diffie-hellman-group1-sha1 and diffie-hellman-group14-sha1

Operational Security

• SSH authentication logs are monitored (/var/log/auth.log or journalctl -u sshd)
• fail2ban or equivalent is configured to block repeated authentication failures
• Host key fingerprints are recorded and stored out-of-band for verification
• MFA is enabled for all interactive user sessions on production systems
• SSH CA is in use for environments with more than five servers or three administrators

FAQ

What is the difference between SSH keys and SSH certificates?

SSH keys require each server to store the user's public key in authorized_keys. SSH certificates are signed by a Certificate Authority; servers trust the CA rather than individual keys. Certificates scale to large fleets without per-server key management and support expiry times, making revocation straightforward.

Why does Permission denied (publickey) appear even when the key is correct?

The most common causes are incorrect file permissions on ~/.ssh/ (must be 700) or authorized_keys (must be 600), a home directory that is world-writable (blocked by StrictModes), or an AllowUsers directive in sshd_config that excludes the connecting account. Check journalctl -u sshd on the server for the specific rejection reason.

Is changing the SSH port from 22 a real security measure?

It eliminates automated opportunistic attacks targeting port 22, which meaningfully reduces log noise and failed authentication attempts. It does not protect against a targeted attacker who performs a port scan. It should be combined with fail2ban, key-only authentication, and firewall IP allowlisting for meaningful security improvement.

Can I use SSH without a static IP address on the client side?

Yes. Key-based authentication does not require a fixed client IP. If you want to restrict by IP, use the from= option in authorized_keys or firewall rules. For dynamic IPs, consider a VPN to establish a stable network identity before connecting, rather than opening SSH to the entire internet.

What is the safest way to recover SSH access if I am locked out?

Access the server through your hosting provider's out-of-band console (VNC, IPMI, or KVM over IP). From there, mount the filesystem, correct the sshd_config or authorized_keys issue, and restart the SSH daemon. On AlexHost VPS and dedicated server plans, the provider console is available through the client portal and does not depend on SSH being functional.