SSH Key Management

SSH Security Hardening Checklist for DomainIndia VPS

By Domain India Team · DomainIndia EngineeringPublished · Updated 7 min read
Knowledge base article
Contents (20 sections)
TL;DR

Your VPS faces hundreds of SSH brute-force attempts daily. This checklist hardens SSH against every common attack — key-only auth, non-root user, fail2ban, port hopping, 2FA, jump host — all tested on DomainIndia AlmaLinux / Ubuntu VPS.

The attack surface

Scanners probe every public IP's port 22 continuously. Logs on a new VPS show 500-2000 failed SSH attempts per day within the first hour. Default configuration is a ticking bomb.

The eight layers of SSH defense:

  1. Disable password auth entirely
  2. Require SSH key authentication
  3. Disable root login
  4. Create a non-root sudo user
  5. Install fail2ban
  6. Firewall rules
  7. (Optional) Change SSH port
  8. (Optional) Two-factor auth + jump host

Layer 1 — Generate strong SSH key (on your laptop)

bash
ssh-keygen -t ed25519 -C "[email protected]" -f ~/.ssh/domainindia_vps -N 'strong-passphrase'
  • ed25519 is faster and more secure than RSA
  • Passphrase protects the key if your laptop is stolen

Add to SSH agent:

bash
ssh-add ~/.ssh/domainindia_vps

Layer 2 — Add public key to VPS

bash
# From laptop
ssh-copy-id -i ~/.ssh/domainindia_vps.pub root@your-vps-ip

# Or manually — paste contents of .pub file into:
# /root/.ssh/authorized_keys on VPS

Test before continuing:

bash
ssh -i ~/.ssh/domainindia_vps root@your-vps-ip
# Should succeed without prompting for password

Critical — don't proceed until key login works.

Layer 3 — Create non-root user

bash
# On VPS as root
adduser admin   # or your name
usermod -aG wheel admin   # AlmaLinux (sudo group)
# or on Ubuntu:
usermod -aG sudo admin

# Copy authorized_keys
mkdir -p /home/admin/.ssh
cp /root/.ssh/authorized_keys /home/admin/.ssh/
chown -R admin:admin /home/admin/.ssh
chmod 700 /home/admin/.ssh
chmod 600 /home/admin/.ssh/authorized_keys

Test login as new user:

bash
ssh -i ~/.ssh/domainindia_vps admin@your-vps-ip
sudo whoami   # should return "root" after password prompt

Configure passwordless sudo (optional, for automation):

bash
echo 'admin ALL=(ALL) NOPASSWD:ALL' | sudo tee /etc/sudoers.d/admin

Layer 4 — Harden sshd config

Edit /etc/ssh/sshd_config:

bash
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
sudo vi /etc/ssh/sshd_config

Set these:

code
# Disable root login entirely
PermitRootLogin no

# Keys only
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM yes

# Limit to specific users
AllowUsers admin deploy

# Disable empty passwords
PermitEmptyPasswords no

# Limit protocol
Protocol 2

# Disable X11 forwarding if not needed
X11Forwarding no

# Reduce login grace time
LoginGraceTime 30

# Max auth tries per connection
MaxAuthTries 3

# Session timeout (idle disconnect)
ClientAliveInterval 300
ClientAliveCountMax 2

# Modern ciphers only
Ciphers [email protected],[email protected],aes256-ctr
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512
MACs [email protected],[email protected]

Validate + reload:

bash
sudo sshd -t   # validate syntax — must return no output
sudo systemctl reload sshd

Critical: keep your current SSH session open. Open a NEW terminal to test. If locked out, use the old session to revert.

Layer 5 — fail2ban (auto-ban brute-forcers)

bash
# AlmaLinux
sudo dnf install -y epel-release
sudo dnf install -y fail2ban

# Ubuntu
sudo apt install -y fail2ban

Configure /etc/fail2ban/jail.local:

ini
[DEFAULT]
bantime = 24h
findtime = 10m
maxretry = 3
ignoreip = 127.0.0.1/8 YOUR_HOME_IP/32

[sshd]
enabled = true
port = ssh
logpath = /var/log/secure
backend = systemd

Start:

bash
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd
# Currently banned: 2
# IP list: 45.123.x.x 62.45.x.x

Layer 6 — Firewall

firewalld (AlmaLinux / Rocky)

bash
sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

UFW (Ubuntu)

bash
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

Restrict SSH to specific IP

If you have a static IP at office:

bash
# firewalld
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="YOUR.OFFICE.IP/32" service name="ssh" accept'
sudo firewall-cmd --reload

# ufw
sudo ufw delete allow 22/tcp
sudo ufw allow from YOUR.OFFICE.IP to any port 22

Layer 7 — Change SSH port (security through obscurity)

Scanners mostly hit port 22. Moving to 2222 or 22022 cuts 95% of noise (not a real defense, just log hygiene).

/etc/ssh/sshd_config:

code
Port 2222
# or add alongside 22 during transition:
# Port 22
# Port 2222

Open firewall for new port, reload sshd, test, then close port 22:

bash
# AlmaLinux: SELinux may block new port
sudo semanage port -a -t ssh_port_t -p tcp 2222
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload

sudo systemctl reload sshd
ssh -p 2222 admin@your-vps

Tell team. Update ~/.ssh/config on laptops:

code
Host domainindia-vps
    HostName your-vps-ip
    User admin
    Port 2222
    IdentityFile ~/.ssh/domainindia_vps

Now ssh domainindia-vps just works.

Layer 8 — Two-factor auth (Google Authenticator)

For extra paranoia:

bash
sudo dnf install -y google-authenticator
# Ubuntu: sudo apt install libpam-google-authenticator

# As user (admin), run:
google-authenticator
# Scan QR with Authy/Google Auth on phone
# Save backup codes!

Edit /etc/pam.d/sshd — add at top:

code
auth required pam_google_authenticator.so

Edit /etc/ssh/sshd_config:

code
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Reload sshd. Now login requires: SSH key AND 6-digit code.

Layer 9 — Jump host / bastion

For multiple VPS, expose SSH only on one bastion host. All others only accept SSH from bastion's internal IP.

~/.ssh/config:

code
Host bastion
    HostName bastion.yourcompany.com
    User admin

Host internal-*
    User admin
    ProxyJump bastion

Usage:

bash
ssh internal-db    # routes via bastion automatically

Attack surface: only bastion is public.

Monitoring

Watch SSH logs

bash
sudo journalctl -u sshd --follow
# or
sudo tail -f /var/log/secure

Alert on successful root-like sudo

Set up a simple cron that monitors and emails:

bash
# /etc/cron.hourly/ssh-monitor
#!/bin/bash
LAST_HOUR=$(date -d '1 hour ago' +'%Y-%m-%d %H')
SUDO_COUNT=$(journalctl --since "$LAST_HOUR:00:00" | grep -c 'sudo.*COMMAND')
if [ "$SUDO_COUNT" -gt 20 ]; then
    mail -s "High sudo activity on $(hostname)" [email protected] <<EOF
Sudo commands in past hour: $SUDO_COUNT
Review: journalctl --since "$LAST_HOUR:00:00" | grep sudo
EOF
fi

Audit login sessions

bash
# Last 20 logins
last -20

# Failed attempts
lastb -20

# Who is currently logged in
who

SSH config management

Keep your ~/.ssh/config organized:

code
# Default settings
Host *
    AddKeysToAgent yes
    UseKeychain yes
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60

# DomainIndia VPS
Host domainindia-prod
    HostName 65.108.x.x
    User admin
    Port 2222
    IdentityFile ~/.ssh/domainindia_vps

Host domainindia-staging
    HostName 65.108.y.y
    User admin
    Port 2222

Common pitfalls

Locking yourself out
always keep an open SSH session while testing changes. Use sshd -t before reload.
Disabling password auth before adding SSH key
you're locked out. Always test key login first.
Weak passphrase on SSH key
if laptop stolen, key is effectively plain text. Use 20+ character random passphrase.
Same key across all machines
compromise of one = all. Per-machine keys isolate risk.
Forgetting to allow port in firewall
changed SSH to 2222, still only 22 is open. Test before closing old port.
SELinux blocking custom port
sudo semanage port -a -t ssh_port_t -p tcp 2222.
Shared authorized_keys
everyone's keys in one file. Use per-user home dirs.

FAQ

Do I need fail2ban if I disabled password auth?

Yes — still useful. It blocks scanning bots wasting CPU on auth attempts even if they'd never succeed. Also catches misconfigured services exposing other ports.

Should I change port from 22?

Security benefit is marginal (scanners scan other ports too). Main benefit: cleaner logs. If you want it, fine; don't rely on it as defense.

SSH key compromised — what do I do?

1) Remove pub key from authorized_keys on every server. 2) Generate new key, add to authorized_keys. 3) Rotate any service account keys. 4) Review logs for unauthorized access since compromise.

Cloudflare Zero Trust for SSH?

Excellent option — no public SSH port, audit trail, MFA built in. See our Cloudflare Zero Trust article.

Can I disable SSH entirely?

On a VPS — no, you lose access. On a cPanel/DirectAdmin shared server — yes, use the panel. VPS admins always need some recovery path.

Harden your DomainIndia VPS from day one. Get a VPS

Was this article helpful?

Your answer helps us decide what to improve next.

Still need help? Open a support ticket and our team will reply.

Related articles

Prefer an app? Add this site to your home screen.Get the app
Need Help?
Live chatNew TicketTrack TicketKnowledge BaseContact UsWhatsApp