A new Linux VPS is reachable on port 22 from the whole internet the moment it boots. This checklist walks through hardening SSH layer by layer, testing each change before you lock the door behind you.
Your VPS faces hundreds of SSH brute-force attempts daily. This checklist hardens SSH against the common attacks — key-only auth, non-root user, fail2ban, firewall, optional port change, 2FA and a jump host — with commands for AlmaLinux, Rocky Linux and Ubuntu.
The attack surface
Scanners probe every public IP's port 22 continuously, and a new VPS usually logs its first failed SSH logins within minutes of going online. A default configuration that accepts root passwords is a ticking bomb.
The nine layers of SSH defence:
- Generate a strong SSH key
- Add the public key to the VPS
- Create a non-root sudo user
- Harden the sshd config (keys only, no root login)
- Install fail2ban
- Firewall rules
- (Optional) Change the SSH port
- (Optional) Two-factor authentication
- (Optional) Jump host
Layer 1 — Generate strong SSH key (on your laptop)
ssh-keygen -t ed25519 -C "[email protected]" -f ~/.ssh/domainindia_vps -N 'strong-passphrase'ed25519is faster and more secure than RSA- Passphrase protects the key if your laptop is stolen
Add to SSH agent:
ssh-add ~/.ssh/domainindia_vpsLayer 2 — Add public key to VPS
# From laptop
ssh-copy-id -i ~/.ssh/domainindia_vps.pub root@your-vps-ip
# Or manually — paste contents of .pub file into:
# /root/.ssh/authorized_keys on VPSTest before continuing:
ssh -i ~/.ssh/domainindia_vps root@your-vps-ip
# Should succeed without prompting for passwordCritical — don't proceed until key login works.
Layer 3 — Create non-root user
# On VPS as root
adduser admin # or your name
passwd admin # set a password; sudo asks for it
usermod -aG wheel admin # AlmaLinux (sudo group)
# or on Ubuntu:
usermod -aG sudo admin
# Copy authorized_keys
mkdir -p /home/admin/.ssh
cp /root/.ssh/authorized_keys /home/admin/.ssh/
chown -R admin:admin /home/admin/.ssh
chmod 700 /home/admin/.ssh
chmod 600 /home/admin/.ssh/authorized_keysTest login as new user:
ssh -i ~/.ssh/domainindia_vps admin@your-vps-ip
sudo whoami # should return "root" after password promptConfigure passwordless sudo (optional, for automation):
echo 'admin ALL=(ALL) NOPASSWD:ALL' | sudo tee /etc/sudoers.d/adminLayer 4 — Harden sshd config
Put your settings in a drop-in file rather than editing the main /etc/ssh/sshd_config. AlmaLinux and Rocky Linux 9 and later, and current Ubuntu releases, read /etc/ssh/sshd_config.d/*.conf through an Include line at the top of the main file, and for most settings the first value sshd reads wins. Naming your file 00-hardening.conf makes it load before files such as 50-cloud-init.conf or 50-redhat.conf, which can otherwise switch password login back on.
grep -i '^Include' /etc/ssh/sshd_config # should list sshd_config.d/*.conf
sudo vi /etc/ssh/sshd_config.d/00-hardening.confIf the grep prints nothing (AlmaLinux or Rocky Linux 8, for example), add Include /etc/ssh/sshd_config.d/*.conf as the first line of /etc/ssh/sshd_config after taking a backup copy of it.
Set these:
# Disable root login entirely
PermitRootLogin no
# Keys only
PasswordAuthentication no
PubkeyAuthentication yes
KbdInteractiveAuthentication no
UsePAM yes
# Limit to specific users
AllowUsers admin deploy
# Disable empty passwords
PermitEmptyPasswords no
# Disable X11 forwarding if not needed
X11Forwarding no
# Reduce login grace time
LoginGraceTime 30
# Max auth tries per connection
MaxAuthTries 3
# Session timeout (idle disconnect)
ClientAliveInterval 300
ClientAliveCountMax 2
# Modern ciphers only
Ciphers [email protected],[email protected],aes256-ctr
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512
MACs [email protected],[email protected]Validate, confirm the values sshd will actually use, then reload:
sudo sshd -t # validate syntax — must return no output
sudo sshd -T | grep -Ei '^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|maxauthtries|allowusers)'
sudo systemctl reload sshd # on Ubuntu the service is called sshsshd -T prints the effective configuration after every file is merged. If it still shows passwordauthentication yes, another file in sshd_config.d is being read first: rename yours so it sorts earlier.
ChallengeResponseAuthentication is the deprecated name for KbdInteractiveAuthentication, and the Protocol line is obsolete (OpenSSH supports only protocol 2), so leave both out.
Critical: keep your current SSH session open. Open a NEW terminal to test. If locked out, use the old session to revert.
Layer 5 — fail2ban (auto-ban brute-forcers)
# AlmaLinux
sudo dnf install -y epel-release
sudo dnf install -y fail2ban
# Ubuntu
sudo apt install -y fail2banConfigure /etc/fail2ban/jail.local:
[DEFAULT]
bantime = 24h
findtime = 10m
maxretry = 3
ignoreip = 127.0.0.1/8 YOUR_HOME_IP/32
[sshd]
enabled = true
port = ssh
backend = systemdStart:
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd
# Currently banned: 2
# IP list: 45.123.x.x 62.45.x.xLayer 6 — Firewall
firewalld (AlmaLinux / Rocky)
sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reloadUFW (Ubuntu)
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enableRestrict SSH to specific IP
If you have a static IP at office:
# firewalld
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="YOUR.OFFICE.IP/32" service name="ssh" accept'
sudo firewall-cmd --reload
# ufw
sudo ufw delete allow 22/tcp
sudo ufw allow from YOUR.OFFICE.IP to any port 22Layer 7 — Change SSH port (security through obscurity)
Scanners mostly hit port 22. Moving to 2222 or 22022 cuts most of the log noise (not a real defense, just log hygiene).
In your drop-in file, /etc/ssh/sshd_config.d/00-hardening.conf:
Port 2222
# or add alongside 22 during transition:
# Port 22
# Port 2222Open firewall for new port, reload sshd, test, then close port 22:
# AlmaLinux: SELinux may block new port
sudo semanage port -a -t ssh_port_t -p tcp 2222
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload
sudo systemctl reload sshd
ssh -p 2222 admin@your-vpsOn Ubuntu 24.04, SSH is started by systemd socket activation, so after changing the port run sudo systemctl daemon-reload and sudo systemctl restart ssh.socket, then test with ss -tlnp | grep ssh before you close the old port. On Ubuntu, open the port with sudo ufw allow 2222/tcp instead of the firewalld and SELinux lines.
Tell team. Update ~/.ssh/config on laptops:
Host domainindia-vps
HostName your-vps-ip
User admin
Port 2222
IdentityFile ~/.ssh/domainindia_vpsNow ssh domainindia-vps just works.
Layer 8 — Two-factor auth (Google Authenticator)
For extra paranoia:
sudo dnf install -y google-authenticator
# Ubuntu: sudo apt install libpam-google-authenticator
# As user (admin), run:
google-authenticator
# Scan QR with Authy/Google Auth on phone
# Save backup codes!Edit /etc/pam.d/sshd — add at top:
auth required pam_google_authenticator.soOn AlmaLinux and Rocky Linux the package comes from EPEL (installed in Layer 5). So that sshd asks only for the code and not also for the account password, comment out the password line in the same file: auth substack password-auth on AlmaLinux/Rocky, or @include common-auth on Ubuntu.
In your drop-in file, /etc/ssh/sshd_config.d/00-hardening.conf, replace the earlier KbdInteractiveAuthentication no line with:
KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactiveRun sudo sshd -t, reload sshd and test from a new terminal while the old session stays open. Login now requires the SSH key AND a 6-digit code.
Layer 9 — Jump host / bastion
For multiple VPS, expose SSH only on one bastion host. All others only accept SSH from bastion's internal IP.
~/.ssh/config:
Host bastion
HostName bastion.yourcompany.com
User admin
Host internal-*
User admin
ProxyJump bastionUsage:
ssh internal-db # routes via bastion automaticallyAttack surface: only bastion is public.
Monitoring
Watch SSH logs
sudo journalctl -u sshd --follow # Ubuntu: -u ssh
# or
sudo tail -f /var/log/secure # Ubuntu: /var/log/auth.logAlert on successful root-like sudo
Set up a simple cron that monitors and emails. The mail command needs a working mail setup on the VPS, for example a relay through an email provider's SMTP service:
# /etc/cron.hourly/ssh-monitor
#!/bin/bash
LAST_HOUR=$(date -d '1 hour ago' +'%Y-%m-%d %H')
SUDO_COUNT=$(journalctl --since "$LAST_HOUR:00:00" | grep -c 'sudo.*COMMAND')
if [ "$SUDO_COUNT" -gt 20 ]; then
mail -s "High sudo activity on $(hostname)" [email protected] <<EOF
Sudo commands in past hour: $SUDO_COUNT
Review: journalctl --since "$LAST_HOUR:00:00" | grep sudo
EOF
fiAudit login sessions
# Last 20 logins
last -20
# Failed attempts
lastb -20
# Who is currently logged in
whoSSH config management
Keep your ~/.ssh/config organized:
# Default settings
Host *
IgnoreUnknown UseKeychain
AddKeysToAgent yes
UseKeychain yes # macOS only; ignored elsewhere
IdentityFile ~/.ssh/id_ed25519
ServerAliveInterval 60
# Domain India VPS
Host domainindia-prod
HostName 203.0.113.10
User admin
Port 2222
IdentityFile ~/.ssh/domainindia_vps
Host domainindia-staging
HostName 203.0.113.11
User admin
Port 2222Common pitfalls
Running this on Domain India
A Domain India VPS is self-managed with full root access, and there is no VPS console page in the client area. If a change to SSH or the firewall locks you out, open a support ticket and give the VPS IP; console access and reinstalls are done by support. Snapshots and backups are not included with a VPS, so copy anything important off the server before large changes.
- The checklist applies to your own VPS, where you control
sshd, the firewall and fail2ban. - On Domain India shared hosting (cPanel, DirectAdmin, Webuzo) you cannot change the SSH server. SSH there is a jailed shell that is off by default and enabled on request, with key-only login. See Enabling and accessing jailed SSH.
FAQ
Do I need fail2ban if I disabled password auth?
It is still useful. It blocks scanning bots that waste CPU and fill logs with attempts that would never succeed, and it can protect other services, such as a web login, with their own jails.
Should I change port from 22?
Security benefit is marginal (scanners scan other ports too). Main benefit: cleaner logs. If you want it, fine; don't rely on it as defense.
SSH key compromised — what do I do?
1) Remove pub key from authorized_keys on every server. 2) Generate new key, add to authorized_keys. 3) Rotate any service account keys. 4) Review logs for unauthorized access since compromise.
Can I use Cloudflare Zero Trust for SSH?
Yes. It removes the public SSH port and adds an audit trail and multi-factor login in front of it. See our Cloudflare Zero Trust article.
Can I disable SSH entirely?
On a VPS, no: SSH is how you manage the server, and a Domain India VPS has no console page in the client area, so recovery from a lockout goes through a support ticket. On Domain India shared hosting (cPanel, DirectAdmin, Webuzo), SSH is a jailed shell that is off by default and switched on only when you ask support, so there is nothing to disable unless you requested it.
Ready to put this into practice? Compare VPS plans, then set up your VPS firewall.
A self-managed Domain India VPS gives you full root SSH access, so you can apply every layer in this checklist.
See VPS plans