Harden a Fresh Linux VPS in 30 Minutes: SSH Keys, UFW, and Fail2ban

What You'll Build

By the end of this tutorial, your fresh Ubuntu VPS will reject password-based SSH logins, block all non-essential network ports, and automatically ban IPs that probe your server — all before any application code is deployed.

Prerequisites

• Ubuntu 22.04 LTS VPS with initial root access (DigitalOcean, Linode, Vultr, or similar)
• A local machine running macOS, Linux, or Windows WSL2
• ~30 minutes

⚠️ Lock-out warning: Keep two terminal windows open throughout this guide. Never close your first session until you've verified key-based login works in the second.

Step 1 — Create a Non-Root Sudo User

Log in as root, then create a limited account you'll use from here on:

adduser deploy
usermod -aG sudo deploy

Replace deploy with any username you prefer. adduser will prompt for a password — set a strong one even though password SSH auth is getting disabled shortly.

Step 2 — Set Up SSH Key Authentication

On your local machine, generate an Ed25519 key pair (skip if ~/.ssh/id_ed25519 already exists):

ssh-keygen -t ed25519 -C "your@email.com"

Copy the public key to your server:

ssh-copy-id deploy@YOUR_SERVER_IP

Windows WSL2 users: ssh-copy-id is available in WSL2. If it's missing, use this fallback:

cat ~/.ssh/id_ed25519.pub | ssh deploy@YOUR_SERVER_IP \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Open a second terminal and verify key login works right now:

ssh deploy@YOUR_SERVER_IP

You should land in a shell with no password prompt. Do not continue until this succeeds.

Step 3 — Harden SSH Configuration

On the server, edit the SSH daemon config:

sudo nano /etc/ssh/sshd_config

Find and update these three lines (uncomment them if they start with #):

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

Cloud VPS users: Many providers drop a cloud-init override at /etc/ssh/sshd_config.d/50-cloud-init.conf containing PasswordAuthentication yes. Because sshd_config processes the Include directive at the top of the file first, and the first occurrence of a keyword wins, this override will silently defeat your change. Check for it before reloading:

grep -r "PasswordAuthentication" /etc/ssh/sshd_config.d/

If you see PasswordAuthentication yes in any file there, change it to no in that file (or delete the file entirely).

Test for syntax errors, then reload — reload keeps existing sessions alive while applying changes:

sudo sshd -t
sudo systemctl reload ssh

Confirm your second (key-based) terminal still connects before closing anything.

Step 4 — Configure the UFW Firewall

UFW ships with Ubuntu but is inactive by default. Set rules before enabling:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw enable

Type y when prompted about disrupting SSH connections. Then verify:

sudo ufw status verbose

You should see 22/tcp ALLOW IN Anywhere. Only open additional ports when you actually need them (e.g., sudo ufw allow 80/tcp for HTTP).

Step 5 — Install and Configure Fail2ban

Fail2ban monitors auth logs and temporarily bans IPs that fail authentication repeatedly.

sudo apt update
sudo apt install fail2ban -y

Create a local override file — never edit jail.conf directly, as package upgrades overwrite it:

sudo nano /etc/fail2ban/jail.local

Paste the following:

[sshd]
enabled  = true
port     = ssh
maxretry = 5
bantime  = 1h
findtime = 10m
backend  = systemd

Why backend = systemd? Minimal Ubuntu 22.04 cloud images often omit rsyslog, so /var/log/auth.log doesn't exist. Without this setting, Fail2ban's sshd jail will refuse to start with the error Have not found any log file for sshd jail. Setting backend = systemd tells Fail2ban to read the systemd journal directly instead of looking for a log file — which works on every Ubuntu 22.04 image regardless of whether rsyslog is installed.

Enable and start the service in one command:

sudo systemctl enable --now fail2ban

Verify It Works

Confirm all three defenses are active:

# Firewall rules
sudo ufw status verbose

# Fail2ban is watching SSH
sudo fail2ban-client status sshd

# Passwords are now rejected
ssh -o PreferredAuthentications=password deploy@YOUR_SERVER_IP
# Expected output: "Permission denied (publickey)"

Fail2ban output should show Currently banned: 0 initially; Total banned will increment once real-world scanners find your server.

Troubleshooting

Locked out after disabling password auth Use your provider's emergency console (DigitalOcean Recovery Console, Linode Lish, etc.). Re-enable PasswordAuthentication yes, run sudo systemctl reload ssh, fix your key issue, then re-disable it.

ssh-copy-id fails with "Permission denied" You haven't successfully authenticated as the new user yet. Confirm the password set in Step 1, or use the manual copy command shown in the WSL2 fallback.

UFW blocks a port you need later Run sudo ufw allow PORT/tcp (e.g., sudo ufw allow 443/tcp). List rules with sudo ufw status numbered and remove one with sudo ufw delete NUMBER.

Fail2ban fails to start Use systemd to surface startup errors directly:

sudo systemctl status fail2ban
sudo journalctl -u fail2ban

Common causes: a syntax mistake in jail.local (look for stray characters or misformatted lines), or a missing backend = systemd line if you're on a minimal cloud image without rsyslog (see Step 5).

Next Steps

• Automatic security updates: sudo dpkg-reconfigure --priority=low unattended-upgrades
• Reduce SSH scan noise: change the default SSH port in sshd_config, then update ufw and the port = value in jail.local to match
• Two-factor authentication: add libpam-google-authenticator as a second factor for SSH logins
• Dive deeper with the Ubuntu Server Security Guide for OS-level hardening beyond the network layer