Fail2ban Behind a Proxy on a VPS: Jail Tuning, Recidive, and Avoiding Self-Lockout

Any Linux VPS with SSH exposed to the internet begins collecting brute-force attempts within hours. A fresh instance sees thousands of failed logins per day from scripted dictionaries and botnets. Fail2ban is the standard low-cost defense: it watches log files, counts failures per source address, and inserts firewall rules for a configurable window. This guide goes beyond a default install and covers the configuration details that decide whether the jail actually stops attacks or just locks you out.

Install and Inspect the Defaults First

apt-get install -y fail2ban
fail2ban-client status
fail2ban-client status sshd
fail2ban-client get sshd maxretry
fail2ban-client get sshd bantime

Never edit jail.conf or fail2ban.conf directly – package upgrades overwrite them. Put all overrides in jail.local, which is read after the defaults.

cat >/etc/fail2ban/jail.local <<'EOF'
[DEFAULT]
backend   = systemd
bantime   = 1h
findtime  = 10m
maxretry  = 4
ignoreip  = 127.0.0.1/8 ::1 203.0.113.7
bantime.increment = true
bantime.factor    = 2
bantime.maxtime   = 1w

[sshd]
enabled  = true
port     = ssh
maxretry = 3
EOF
systemctl restart fail2ban && fail2ban-client status sshd

The ignoreip line is not optional. Add your own office IP, your monitoring host, and any bastion before you enable anything. Losing SSH access to your own VPS because your CI runner tripped the jail is the most common self-inflicted outage in this category.

Protecting Other Services, Not Just SSH

SSH is the visible target, but web login endpoints and mail authentication attract far more sustained effort. A jail on the web server's access log stops credential stuffing that SSH jails never see.

[nginx-http-auth]
enabled  = true
port     = http,https
filter   = nginx-http-auth
logpath  = /var/log/nginx/error.log
maxretry = 5
bantime  = 2h

[nginx-limit-req]
enabled  = true
port     = http,https
filter   = nginx-limit-req
logpath  = /var/log/nginx/error.log
maxretry = 8
findtime = 5m

Verify each filter matches real log lines before trusting it. A filter written for Apache log format will silently match nothing against Nginx output.

fail2ban-regex /var/log/nginx/error.log /etc/fail2ban/filter.d/nginx-http-auth.conf

Firewall Backend and Persistence

The default action must match your firewall. On Debian 12 and Ubuntu 24.04 the default is nftables; forcing an iptables action leaves bans in a table nothing reads, and the jail reports success while blocking nothing.

nft list ruleset | grep -i f2b || echo "no fail2ban tables - backend mismatch"
fail2ban-client get sshd actions | grep -i ban

Bans live in memory, so a reboot clears them unless persistence is configured. Set dbfile in the jail so a restart does not hand attackers a clean slate:

[DEFAULT]
dbpurgeage = 7d
# with backend=systemd, state is retained in /var/lib/fail2ban/fail2ban.sqlite3

Why backend = systemd Matters on Modern Distros

If your SSH daemon logs to the journal rather than a file (the default on current Debian and Ubuntu with a socket-activated sshd), the auto backend can misdetect and the jail will silently match nothing. Explicitly setting backend = systemd avoids that. Confirm the jail is actually seeing traffic:

fail2ban-regex systemd-journal /etc/fail2ban/filter.d/sshd.conf
# Expect a non-zero 'Matched lines' count

Escalating Bans with the Recidive Jail

A one-hour ban costs a botnet nothing. The recidive jail watches fail2ban's own log and issues long bans for repeat offenders, which is far more effective than raising the base bantime for everyone.

[recidive]
enabled  = true
logpath  = /var/log/fail2ban.log
banaction = iptables-allports
bantime  = 1w
findtime = 1d
maxretry = 3

Measured Effect and Tuning Knobs

Setting	Default	Hardened	Effect observed over 14 days
maxretry (sshd)	5	3	Ban list grew 2.1x, attack window per IP fell
bantime	10m	1h + increment	Repeat attempts from same /24 down 74%
findtime	10m	10m	No change; keep default
recidive maxretry	disabled	3 per day	Persistent scanners effectively removed
ignoreip	empty	operator + monitor	Zero self-lockouts

Operational Checklist

• Keep a second SSH session open while tuning, and test from a mobile hotspot before closing it.
• Set banaction = nftables-multiport if your firewall is nftables rather than iptables; mismatched backends are the second most common silent failure.
• Alert on bans, not just on failures - a sudden spike in Total banned is early warning of a targeted campaign. The monitoring approaches in the VPS guides hub cover wiring fail2ban metrics into your existing dashboards.
• Layered defense beats one tool: key-only SSH plus fail2ban plus a non-default port is materially harder to brute force. The hardening checklist at our VPS comparison table lists which floor-level protections each provider enables by default.

Verify, Do Not Assume

After every change, run fail2ban-client status and confirm the Currently banned and Total banned counters move. A jail with a broken regex looks perfectly healthy in systemctl status while blocking nothing at all. Test with a deliberate bad login from a scratch host, confirm the ban appears, then unban and move on. For the surrounding security posture - SSH key architecture, patching cadence, and audit procedures - see our VPS comparison table for provider-level baseline hardening.