How to Install and Configure SSH on Linux: A Complete Security Guide for 2025

SSH is the single most critical access point on your server. A misconfigured SSH setup can be compromised in under five minutes by automated bots scanning the internet. Whether you're managing a VPS Hosting environment, a bare-metal machine, or a cloud instance, locking down SSH correctly from day one is non-negotiable.

In this guide, you'll learn how to install OpenSSH, configure it securely, implement key-based authentication, and apply production-grade hardening techniques — all on a Linux server in 2025.

What Is SSH and Why Does It Matter?

SSH (Secure Shell) is a cryptographic network protocol that allows users to securely connect to a remote system over an unsecured network such as the internet. All data transmitted between the client and the server is fully encrypted, making it the industry standard for remote server management.

By default, SSH operates on port 22 and supports:

• Remote login to servers and virtual machines
• Secure file transfer via SCP and SFTP
• Remote command execution and scripted automation
• Port forwarding and tunneling for secure traffic routing
• X11 forwarding for graphical application access

SSH is your primary administrative interface. Treat it accordingly.

Step 1: Installing OpenSSH Server

Most modern Linux distributions ship with OpenSSH pre-installed. If it's missing, install it using the appropriate package manager for your distribution.

Ubuntu / Debian

sudo apt update
sudo apt install openssh-server -y

CentOS / RHEL / AlmaLinux / Rocky Linux

sudo yum install openssh-server -y

Fedora

sudo dnf install openssh-server -y

Arch Linux

sudo pacman -S openssh

After installation, both the OpenSSH server (sshd) and OpenSSH client are available, enabling you to accept incoming connections and connect to other remote servers.

Step 2: Starting and Enabling the SSH Service

Once installed, you need to start the SSH daemon (sshd) and configure it to launch automatically on system boot.

Start the SSH Service

sudo systemctl start ssh

Enable SSH to Start on Boot

sudo systemctl enable ssh

Verify the Service Is Running

sudo systemctl status ssh

A healthy output will show active (running) in green. If you see any errors, check your system logs with journalctl -xe for diagnostic details.

Step 3: Understanding the SSH Configuration File

SSH's behavior is governed by a single primary configuration file:

/etc/ssh/sshd_config

This file controls everything — the listening port, authentication methods, allowed users, login restrictions, and more. Always create a backup before making changes:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

Open the file with your preferred text editor:

sudo nano /etc/ssh/sshd_config

After making any changes, always validate the configuration syntax before restarting the service to avoid locking yourself out:

sudo sshd -t

If no errors are returned, apply the changes:

sudo systemctl restart ssh

Step 4: Essential SSH Configuration Changes

4.1 Change the Default SSH Port

Port 22 is the first port that automated scanners and brute-force bots target. Changing it to a non-standard port dramatically reduces noise in your logs and lowers exposure to opportunistic attacks.

Locate this line in sshd_config:

#Port 22

Uncomment it and set a custom port (choose a number between 1024 and 65535 that isn't already in use):

Port 2222

Save the file, validate, and restart SSH:

sudo sshd -t && sudo systemctl restart ssh

> Important: Update your firewall rules immediately after changing the port (see Step 6). Failing to do so will lock you out of your server.

4.2 Disable Root Login via SSH

Permitting direct root login over SSH is one of the most common and dangerous misconfigurations. If an attacker guesses or brute-forces the root password, they have full, unrestricted control of your system.

Find this directive:

PermitRootLogin yes

Change it to:

PermitRootLogin no

Users should log in with a standard account and escalate privileges using sudo when necessary. This creates an essential audit layer — every privileged action is logged against a named user account.

4.3 Enforce Key-Based Authentication and Disable Passwords

Password-based authentication is inherently vulnerable to brute-force attacks. SSH key pairs — a mathematically linked public/private key combination — are exponentially more secure.

Locate and set the following directives:

PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no

> Warning: Only disable password authentication after you have successfully tested key-based login. Disabling passwords without a working key will lock you out permanently.

4.4 Additional Hardening Directives

Add or modify these settings in sshd_config for a hardened production configuration:

# Restrict login to specific users (replace 'youruser' with actual usernames)
AllowUsers youruser

# Disconnect idle sessions after 5 minutes
ClientAliveInterval 300
ClientAliveCountMax 2

# Limit authentication attempts per connection
MaxAuthTries 3

# Disable empty passwords
PermitEmptyPasswords no

# Disable X11 forwarding if not needed
X11Forwarding no

# Use only modern, secure protocol version
Protocol 2

# Restrict SSH to specific network interface (optional, replace with your IP)
ListenAddress 0.0.0.0

Step 5: Generating and Deploying SSH Key Pairs

SSH key authentication replaces passwords with cryptographic proof of identity. Here's how to set it up correctly.

Step 5.1: Generate a Key Pair on Your Local Machine

Run this command on your local workstation (not the server):

ssh-keygen -t ed25519 -C "your_email@example.com"

> Why Ed25519? Ed25519 is a modern elliptic-curve algorithm that is faster, more secure, and produces shorter keys than the older RSA algorithm. It is the recommended choice in 2025.

If your system or tooling requires RSA for compatibility reasons, use a 4096-bit key:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

You'll be prompted to:

1. Choose a file path (press Enter to accept the default ~/.ssh/id_ed25519)
2. Set an optional passphrase (strongly recommended — this encrypts your private key at rest)

This generates two files:

~/.ssh/id_ed25519 — Your private key. Never share this. Never copy it to a server.
~/.ssh/id_ed25519.pub — Your public key. This is what gets installed on servers.

Step 5.2: Copy the Public Key to the Server
Use ssh-copy-id to securely transfer your public key to the remote server:
ssh-copy-id -p 2222 username@your_server_ip
Replace username with your server account name and your_server_ip with the actual IP address.
This command appends your public key to ~/.ssh/authorized_keys on the server with correct permissions automatically.
Manual method (if ssh-copy-id is unavailable):
cat ~/.ssh/id_ed25519.pub | ssh username@your_server_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
Step 5.3: Test Key-Based Login
Before disabling password authentication, verify that key-based login works:
ssh -p 2222 username@your_server_ip
If you connect successfully without being prompted for a password (or only for your key passphrase), key-based authentication is working correctly. Now you can safely disable password authentication in sshd_config.
Step 6: Configuring Your Firewall for SSH
A firewall is your first line of defense. Always configure it to allow only the SSH port you're using.
Using UFW (Ubuntu / Debian)
# Allow your custom SSH port
sudo ufw allow 2222/tcp

# Enable the firewall
sudo ufw enable

# Verify the rules
sudo ufw status verbose
Using firewalld (CentOS / RHEL / Fedora)
# Add the custom SSH port
sudo firewall-cmd --permanent --add-port=2222/tcp

# Remove the default port 22 (optional, after confirming new port works)
sudo firewall-cmd --permanent --remove-service=ssh

# Reload firewall rules
sudo firewall-cmd --reload
Restrict SSH Access by IP Address
For maximum security, limit SSH access to known, trusted IP addresses only:
# UFW example: allow SSH only from a specific IP
sudo ufw allow from 203.0.113.10 to any port 2222 proto tcp
This is especially effective on Dedicated Servers where you control all administrative access from a fixed office or VPN IP range.
Step 7: Advanced Security Measures
7.1 Install and Configure Fail2Ban
Fail2Ban monitors SSH log files and automatically bans IP addresses that show signs of brute-force activity.
# Install Fail2Ban
sudo apt install fail2ban -y   # Ubuntu/Debian
sudo yum install fail2ban -y   # CentOS/RHEL

# Create a local configuration file
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Edit /etc/fail2ban/jail.local and configure the SSH jail:
[sshd]
enabled = true
port    = 2222
filter  = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime  = 3600
findtime = 600
Start and enable Fail2Ban:
sudo systemctl start fail2ban
sudo systemctl enable fail2ban
7.2 Use SSH Config File for Client-Side Management
On your local machine, create or edit ~/.ssh/config to simplify connections and enforce security settings:
Host myserver
    HostName your_server_ip
    User youruser
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3
With this configuration, you can connect simply by typing:
ssh myserver
7.3 Two-Factor Authentication (2FA) for SSH
For environments requiring the highest level of access security — such as production databases, financial systems, or compliance-regulated infrastructure — consider adding TOTP-based two-factor authentication using Google Authenticator or Authy alongside SSH keys.
sudo apt install libpam-google-authenticator -y
google-authenticator
Then configure PAM and sshd_config to require both a key and a one-time password. This creates a true multi-factor authentication flow.
Step 8: Testing and Troubleshooting SSH
After completing your configuration, systematically verify that everything works as expected.
Connection Test
ssh -p 2222 -v username@your_server_ip
The -v flag enables verbose mode, printing detailed debugging information about each stage of the connection — invaluable for diagnosing authentication failures.
Common Issues and Solutions




Problem
Likely Cause
Solution




Connection refused
SSH not running or wrong port
Check systemctl status ssh and firewall rules


Permission denied (publickey)
Key not in authorized_keys or wrong permissions
Verify ~/.ssh/authorized_keys exists with chmod 600


Host key verification failed
Server fingerprint changed
Remove old entry from ~/.ssh/known_hosts


Connection timed out
Firewall blocking the port
Check UFW/firewalld rules and cloud security groups


Locked out after config change
Misconfiguration in sshd_config
Use console/KVM access to revert changes




Diagnostic Commands
# Check SSH service status
sudo systemctl status ssh

# View real-time SSH logs
sudo journalctl -u ssh -f

# Check which port SSH is listening on
sudo ss -tlnp | grep sshd

# Test configuration file syntax
sudo sshd -t
Complete Hardened sshd_config Reference
Here is a production-ready sshd_config incorporating all the security recommendations from this guide:
# Network
Port 2222
ListenAddress 0.0.0.0
Protocol 2

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
MaxAuthTries 3
LoginGraceTime 30

# Session Management
ClientAliveInterval 300
ClientAliveCountMax 2
MaxSessions 5

# Access Control
AllowUsers youruser

# Features (disable what you don't need)
X11Forwarding no
AllowTcpForwarding no
GatewayPorts no
PermitUserEnvironment no

# Logging
SyslogFacility AUTH
LogLevel VERBOSE
Why Your Hosting Infrastructure Matters for SSH Security
The security of your SSH configuration doesn't exist in isolation — it depends heavily on the underlying infrastructure. A properly hardened SSH setup is most effective when deployed on a server that already provides:

DDoS protection to absorb volumetric attacks before they reach your SSH port
NVMe storage for fast log writes and rapid Fail2Ban response times
1 Gbps network ports to ensure connection stability under load
KVM/console access as an out-of-band recovery method if you accidentally lock yourself out

AlexHost's VPS Hosting plans include all of the above, with full root access and support for custom firewall configurations — making them ideal for deploying the hardened SSH setup described in this guide.
If you need a control panel to manage your server alongside SSH, explore VPS Control Panels for options including cPanel, Plesk, and DirectAdmin. For teams managing multiple web properties, Shared Web Hosting provides a managed environment where SSH hardening is handled at the infrastructure level.
And if you're running SSL-secured web services alongside your SSH-hardened server, pairing your setup with a trusted SSL Certificates solution ensures end-to-end encryption across all your public-facing services.
SSH Security Checklist
Use this checklist before considering your SSH configuration production-ready:

[ ] OpenSSH server installed and running
[ ] Default port 22 changed to a custom port
[ ] Root login disabled (PermitRootLogin no)
[ ] Ed25519 or RSA-4096 key pair generated
[ ] Public key deployed to authorized_keys on server
[ ] Key-based login tested successfully
[ ] Password authentication disabled
[ ] Firewall configured to allow only the new SSH port
[ ] Fail2Ban installed and configured
[ ] MaxAuthTries set to 3 or fewer
[ ] ClientAliveInterval configured to terminate idle sessions
[ ] sshd_config syntax validated with sudo sshd -t

• [ ] Out-of-band console access confirmed (KVM/VNC)

Conclusion

Installing and configuring SSH correctly is one of the most fundamental skills in Linux system administration. A default SSH installation is a liability; a hardened SSH configuration is an asset. By following this guide, you have:

1. Installed and started the OpenSSH server
2. Changed the default port and disabled root login
3. Implemented cryptographically strong key-based authentication
4. Configured firewall rules to restrict access
5. Deployed Fail2Ban to block brute-force attempts
6. Established a complete troubleshooting methodology

SSH, when properly configured, becomes a powerful, reliable, and secure gateway for remote management, automation pipelines, and encrypted communication between systems. Combined with robust hosting infrastructure — such as AlexHost's Dedicated Servers with hardware-level DDoS mitigation — you have a foundation that is genuinely difficult to compromise.

Revisit your SSH configuration periodically. Rotate your keys annually. Monitor your logs. Security is not a one-time task — it's an ongoing practice.