Fail2ban integration with CSF to counter DDoS on nginx

The set of ConfigServer Security & Firewall (CSF) scripts initially has a fairly rich ability to organize the protection of a Web hosting server using the iptables packet filter. In particular, it can counteract flooding of the attacked host with TCP SYN, UDP, and ICMP packets of weak and medium strength. CSF is complemented by the built-in Login Failure Daemon (lfd), which monitors logs for multiple failed login attempts in various network services in order to find a password. Such attempts are blocked by adding the attacker's IP address to the CSF blacklist.

There is another third-party tool that implements similar functionality: Fail2ban. Despite the similarity of the tasks being solved, there is a cardinal difference between lfd and Fail2ban. The first has a closed architecture and supports a limited set of services. While the second allows you to independently develop filters for almost any task. However, coexisting CSF and Fail2ban within the same server are bad, because they treat the iptables rules somewhat unceremoniously. We will try to solve this problem using the example of Linux OS Debian v7.XX amd64 so that we can extract the maximum from the capabilities of both tools. And as an example, we organize protection against DDoS attacks on nginx.

In my configuration, CSF was initially installed and configured on the server. I will not dwell on this question in detail; there is enough material on it. I recommend to make the following settings in the file "/etc/csf/csf.conf":

 SYNFLOOD = "1"
 SYNFLOOD_RATE = "100 / s"
 SYNFLOOD_BURST = "10"

 CONNLIMIT = "80; 110,443; 110"

 PORTFLOOD = "80; tcp; 20; 1,443; tcp; 20; 1"

 CT_LIMIT = "300"
 CT_INTERVAL = "60"

This configuration will allow the CSF to counteract those hosts that are involved in the flooding attack with TCP SYN packets or open a large number of connections to the TCP ports of the HTTP server.
We proceed to the installation of Fail2ban, after which it must be stopped and the autostart disabled:

  service fail2ban stop
 update-rc.d -f fail2ban remove

')
We will launch Fail2ban using CSF. To do this, create a script "/etc/csf/csfpost.sh":

  #! / bin / sh
 /etc/init.d/fail2ban reload

It will also provide automatic download of Fail2ban rules to iptables if the CSF reboots its own, for example, in the case of an update.
The idea of integrating Fail2ban with CSF is based on the fact that to block the IP addresses of the attackers, the first one will use the black list of the second, and not directly iptables rules. However, we do not completely refuse iptables in Fail2ban.
Disable all filters in Fail2ban. Most of them overlap lfd. Fail2ban will be used only for the fact that it does not support lfd.

  sed -i "s | enabled = true | enabled = false | g" /etc/fail2ban/jail.conf

Add CSF support to Fail2ban. To do this, create a configuration file "/etc/fail2ban/action.d/csf-ip-deny.conf" with the following content:

  [Definition]
 actionstart =
 actionstop =
 actioncheck =
 actionban = csf -d <ip> Added by Fail2Ban for <name>
 actionunban = csf -dr <ip>

For all filters Fail2ban, replace the blocking action with the created “csf-ip-deny”:

 sed -i -e "s | banaction = | banaction = csf-ip-deny \ n # banaction = |"  /etc/fail2ban/jail.conf

For particularly annoying intruders, long-term blocking is provided. This mechanism is implemented by monitoring the log of Fail2ban itself. Create the configuration file "/etc/fail2ban/filter.d/fail2ban.conf":

 [Definition]
 # Count all bans in the logfile
 failregex = fail2ban.actions: WARNING \ [(. *) \] Ban <HOST>
 # Ignore our own bans, to keep our counts exact.
 # In your config, name your jail 'fail2ban', or change this line!
 ignoreregex = fail2ban.actions: WARNING \ [fail2ban \] Ban <HOST>

Add the following lines to "/etc/fail2ban/jail.conf":

 ## fail2ban with CSF to block repeat offenders
 [fail2ban]
 enabled = true
 filter = fail2ban
 action = iptables-allports
 # sendmail-whois [name = fail2ban]
 logpath = /var/log/fail2ban.log
 maxretry = 10
 # Find-time: 1 day
 findtime = 86400
 # Ban-time: 1 week
 bantime = 604800

Those. those of intruders who have been blocked by other filters 10 or more times within 24 hours will be blocked by this filter for a week. Please note that the “iptables-allports” action is used here, not “csf-ip-deny”. It's not a mistake. So you need that long-term blocking is not accidentally removed by other filters.

Now, using the example of organizing the protection of nginx against DDoS attacks, consider the creation of Fail2ban filtering rules. Let's start with the nginx configuration, it is necessary to use the capabilities of the ngx_http_limit_conn_module and ngx_http_limit_req_module modules. To do this, add the following lines to the settings:

 limit_conn_zone $ binary_remote_addr zone = perip: 10m;
 limit_conn perip 100;
 limit_conn_zone $ server_name zone = perserver: 10m;
 limit_conn perserver 200;
 limit_req_zone $ binary_remote_addr zone = reqip: 10m rate = 10r / s;
 limit_req zone = reqip burst = 30;

Thus, we set limits for no more than 100 connections from one IP address at a time at a speed of 10-30 new connections per second and no more than 200 to one site from just any number of IP addresses. Excesses will be recorded in the error log, for which we configure the Fail2ban filters.
Create the following configuration files.
"/etc/fail2ban/filter.d/nginx-conn-limit.conf"

 # Fail2Ban configuration file
 #

 [Definition]
 # Option: failregex
 # Notes .: Regexp to catch a generic call from an IP address.
 # Values: TEXT
 #
 failregex = limiting connections by zone. * client: <HOST>

 # Option: ignoreregex
 # Notes .: regex to ignore.  If this regex matches, the line is ignored.
 # Values: TEXT
 #
 ignoreregex =

"/etc/fail2ban/filter.d/nginx-req-limit.conf"

 # Fail2Ban configuration file
 #
 
 [Definition]
 # Option: failregex
 # Notes .: Regexp to catch a generic call from an IP address.
 # Values: TEXT
 #
 failregex = limiting requests. * client: <HOST>
 
 # Option: ignoreregex
 # Notes .: regex to ignore.  If this regex matches, the line is ignored.
 # Values: TEXT
 #
 ignoreregex =

"/etc/fail2ban/filter.d/nginx-dos.conf"

 # Fail2Ban configuration file
 #

 [Definition]
 # Option: failregex
 # Notes .: Regexp to catch a generic call from an IP address.
 # Values: TEXT
 #
 failregex = ^ <HOST> -. * "(GET | POST). * HTTP. *" $

 # Option: ignoreregex
 # Notes .: regex to ignore.  If this regex matches, the line is ignored.
 # Values: TEXT
 #
 ignoreregex =

Add to "/etc/fail2ban/jail.conf":

 [nginx-conn-limit]
 enabled = true
 filter = nginx-conn-limit
 action = csf-ip-deny [name = nginx-conn-limit]
 logpath = /var/log/nginx/error.log
 maxretry = 4
 findtime = 21600
 bantime = 3600
 
 [nginx-req-limit]
 enabled = true
 filter = nginx-req-limit
 action = csf-ip-deny [name = nginx-req-limit]
 logpath = /var/log/nginx/error.log
 maxretry = 4
 findtime = 21600
 bantime = 3600

 [nginx-dos]
 # Based on apache-badbots but a simple IP check (any IP requesting more than
 # 240 pages in 60 seconds, or 4p / s average, is suspicious)
 enabled = true
 filter = nginx-dos
 action = csf-ip-deny [name = nginx-dos]
 logpath = /var/log/nginx/access.log
 maxretry = 240
 findtime = 60
 bantime = 3600

Filters "nginx-conn-limit" and "nginx-req-limit" will block those who exceed the limits on connections to nginx, and "nginx-dos" - those who too often refer to sites: more than 240 pages per minute.

Often the goal of attacks is the widespread CMS WordPress: brute force and a large number of XML-RPC requests. We organize protection from them too.
"/etc/fail2ban/filter.d/nginx-wp-login.conf"

 [Definition]
 failregex = <HOST>. * POST /wp-login.php
 ignoreregex = </ source>

 "/etc/fail2ban/filter.d/nginx-wp-xmlrpc.conf"
 <source lang = "bash"> [Definition]
 failregex = <HOST>. * POST /xmlrpc.php
 ignoreregex =

"/etc/fail2ban/filter.d/nginx-wp-register.conf"

  [Definition]
 failregex = ^ <HOST>. * "GET /wp-login.php\?action=register HTTP /.*". * $
 ignoreregex =

Add to "/etc/fail2ban/jail.conf":

  [nginx-wp-login]
 enabled = true
 filter = nginx-wp-login
 action = csf-ip-deny [name = nginx-wp-login]
 logpath = /var/log/nginx/access.log
 maxretry = 4
 findtime = 600
 bantime = 3600
 
 [nginx-wp-xmlrpc]
 enabled = true
 filter = nginx-wp-xmlrpc
 action = csf-ip-deny [name = nginx-wp-xmlrpc]
 logpath = /var/log/nginx/access.log
 maxretry = 4
 findtime = 600
 bantime = 3600

 [nginx-wp-register]
 enabled = true
 filter = nginx-wp-register
 action = csf-ip-deny [name = nginx-wp-register]
 logpath = /var/log/nginx/access.log
 maxretry = 4
 findtime = 600
 bantime = 3600

Here is what can be observed in the Fail2ban log during a DDoS attack on nginx:

spoiler

 2015-01-04 13: 44: 10,660 fail2ban.actions: WARNING [nginx-req-limit] Ban 188.191.47.46
 2015-01-04 13: 44: 11,668 fail2ban.actions: WARNING [nginx-conn-limit] Ban 109.187.63.199
 2015-01-04 13: 44: 21,061 fail2ban.actions: WARNING [nginx-req-limit] 188.191.47.46 already banned
 2015-01-04 13: 44: 29,382 fail2ban.actions: WARNING [nginx-conn-limit] Ban 178.123.155.115
 2015-01-04 13: 44: 36,584 fail2ban.actions: WARNING [nginx-conn-limit] Ban 109.62.153.190
 2015-01-04 13: 44: 38,246 fail2ban.actions: WARNING [nginx-dos] Ban 5.143.158.88
 2015-01-04 13: 44: 38,826 fail2ban.actions: WARNING [nginx-req-limit] Ban 178.158.206.140
 2015-01-04 13: 44: 41,739 fail2ban.actions: WARNING [nginx-conn-limit] Ban 5.44.168.38
 2015-01-04 13: 44: 49,877 fail2ban.actions: WARNING [nginx-dos] Ban 91.214.131.71
 2015-01-04 13: 44: 52,333 fail2ban.actions: WARNING [nginx-conn-limit] Ban 176.125.48.22
 2015-01-04 13: 44: 53,395 fail2ban.actions: WARNING [nginx-req-limit] Ban 91.207.211.222
 2015-01-04 13: 44: 53,773 fail2ban.actions: WARNING [nginx-dos] Ban 178.158.206.140
 2015-01-04 13: 44: 54,849 fail2ban.actions: WARNING [nginx-conn-limit] Ban 5.143.158.88
 2015-01-04 13: 44: 57,395 fail2ban.actions: WARNING [nginx-req-limit] 91.207.211.222 already banned
 2015-01-04 13: 44: 57,765 fail2ban.actions: WARNING [nginx-dos] Ban 37.232.87.169
 2015-01-04 13: 44: 58,073 fail2ban.actions: WARNING [nginx-conn-limit] Ban 77.34.22.95
 2015-01-04 13: 44: 58,506 fail2ban.actions: WARNING [nginx-req-limit] Ban 93.80.45.244
 2015-01-04 13: 45: 02,733 fail2ban.actions: WARNING [nginx-dos] Ban 176.120.38.238
 2015-01-04 13: 45: 05,615 fail2ban.actions: WARNING [nginx-conn-limit] Ban 178.173.4.162

And this way excessively annoying intruders will be blocked:

spoiler

  2015-01-04 11: 43: 29,618 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 12: 43: 30,160 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 12: 56: 53,543 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 13: 56: 54,279 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 14: 02: 40,932 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 15: 02: 41,040 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 15: 12: 16,906 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 16: 12: 16,937 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 16: 55: 33,362 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 17: 55: 34,142 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 18: 23: 37,665 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 19: 23: 38,136 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 19: 45: 46,850 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 20: 03: 17,247 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-04 20:45: 47.085 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 21: 03: 17,297 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-04 21: 09: 51,996 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-04 21: 16: 22,336 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 22: 09: 52,036 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-04 22: 13: 27,799 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-04 22: 16: 23,295 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-04 22: 24: 56,755 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-04 23: 13: 28,058 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-04 23: 16: 50,235 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-04 23: 24: 56,843 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-05 00: 00: 42,183 fail2ban.actions: WARNING [nginx-wp-login] Ban 95.163.121.129
 2015-01-05 00: 00: 43,851 fail2ban.actions: WARNING [fail2ban] Ban 95.163.121.129
 2015-01-05 00: 16: 50,263 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 00: 23: 22,863 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 01: 00: 42,637 fail2ban.actions: WARNING [nginx-wp-login] Unban 95.163.121.129
 2015-01-05 01: 23: 23,750 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 01: 26: 16,543 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 02: 26: 16,681 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 02: 32: 28,850 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 03: 32: 29,350 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 03: 39: 18,048 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 04: 39: 18,609 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 04: 43: 38,428 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 05: 43: 39,091 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42
 2015-01-05 05: 47: 47,722 fail2ban.actions: WARNING [nginx-wp-login] Ban 92.255.28.42
 2015-01-05 05: 47: 50,212 fail2ban.actions: WARNING [fail2ban] Ban 92.255.28.42
 2015-01-05 06: 47: 48,343 fail2ban.actions: WARNING [nginx-wp-login] Unban 92.255.28.42

Recently, there have been frequent cases when attackers perform distributed brute force search of passwords from different IP addresses on the same class C subnetwork. The Fail2ban algorithm fails to recognize this behavior:

spoiler

 2015-01-05 14: 01: 14,432 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.176 2015-01-05 14: 01: 14,656 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176 .147.249 2015-01-05 14: 01: 35,906 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.198 2015-01-05 14: 02: 39,536 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.154 2015-01-05 14: 02: 53,766 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.132 2015-01-05 14: 02: 53,980 fail2ban.actions: WARNING [nginx-wp- login] Unban 193.176.147.202 2015-01-05 14: 04: 00,782 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.136 2015-01-05 14: 04: 05,007 fail2ban.actions: WARNING [nginx- wp-login] Unban 193.176.147.104 2015-01-05 14: 04: 07,234 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.167 2015-01-05 14: 04: 25,473 fail2ban.actions: WARNING [ nginx-wp-login] Ban 193.176.147.143 2015-01-05 14: 05: 26,993 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.253 2015-01-05 14: 06: 25,719 fail2ban.  actions: WARNING [nginx-wp-login] Unban 193.176.147.230 2015-01-05 14: 06: 27,945 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.162 2015-01-05 14: 07: 23,484 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.153 2015-01-05 14: 07: 56,962 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.140 2015-01-05 14:08 : 11,207 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.142 2015-01-05 14: 09: 37,759 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.137 2015-01-05 14 : 10: 59,757 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.175 2015-01-05 14: 11: 04,030 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.117 2015-01- 05 14: 11: 23,273 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.218 2015-01-05 14: 11: 41,517 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.139 2015- 01-05 14: 13: 22,590 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.204 2015-01-05 14: 13: 24,808 fail2ban.actions: WARNING [nginx-wp-logi  n] Ban 193.176.147.220 2015-01-05 14: 14: 36,124 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.190 2015-01-05 14: 14: 38,356 fail2ban.actions: WARNING [nginx- wp-login] Ban 193.176.147.179 2015-01-05 14: 14: 38,577 fail2ban.actions: WARNING [nginx-wp-login] Ban 193.176.147.137 2015-01-05 14: 14: 49,805 fail2ban.actions: WARNING [ nginx-wp-login] Unban 193.176.147.188 2015-01-05 14: 15: 48,398 fail2ban.actions: WARNING [nginx-wp-login] Unban 193.176.147.131

We will eliminate this drawback with the following script, which will be executed every hour by the cron scheduler.
"/etc/cron.hourly/fail2ban-subnets"

 #! / bin / bash

 log = "/ var / log / fail2ban.log"
 limit = 30
 grep = `which grep`

 $ {grep} "fail2ban.actions. * Ban" $ {log} |  $ {grep} -E -o "(25 [0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?) \. (25 [ 0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?) \. (25 [0-5] | 2 [0-4] [ 0-9] | [01]? [0-9] [0-9]?) \. (25 [0-5] | 2 [0-4] [0-9] | [01]? [0- 9] [0-9]?) "|  awk -F '.'  '{print $ 1 "." $ 2 "." $ 3}' |  sort -u |  while read line
	 do
		 count = $ ($ {grep} -c "fail2ban.actions. * Ban. * $ {line}" $ {log})
			 if [$ {count} -ge $ {limit}]
				 then
					 / usr / sbin / csf -td $ {line} .0 / 24 7d "Subnet $ {line} .0 / 24 is blocked for a week by Fail2ban after $ {count} attempts"
			 fi
	 done

 exit 0

Those. Class C IP networks will be blocked for the entire week if other Fail2ban filters have previously triggered addresses from them 30 or more times.
Here is the result:

 # csf -t

 A / D IP Address Port Dir Time To Live Comment
 DENY 193.176.147.0/24 * in 6d 21h 34m 18s Subnet 193.176.147.0/24 is blocked
 DENY 46.148.30.0/24 * in 6d 21h 34m 19s Subnet 46.148.30.0/24
 DENY 46.148.31.0/24 * in 6d 21h 34m 19s Subnet 46.148.31.0/24 is blocked

When writing this article, we used developments from the following resources:
https://extremeshok.com/5030/ubuntu-lts-fail2ban-with-csf-and-blocking-of-repeat-offenders-scan-log-files-to-ban-malicious-ips-and-prevent-brute- forcing-of-logins-with-configs /
http://tecadmin.net/add-custom-iptables-rules-with-csf/
https://rtcamp.com/tutorials/nginx/block-wp-login-php-bruteforce-attack/
https://rtcamp.com/tutorials/nginx/fail2ban/
https://www.xaker.name/forvb/showthread.php?t=28659
https://beeznest.wordpress.com/2012/06/08/anti-nginx-dos-filter-for-fail2ban-4/
https://debian.pro/1223
http://www.michelem.org/2014/03/02/stopblock-apachenginx-hack-attempts-with-fail2ban/
http://www.krazyworks.com/permanently-ban-ip-with-fail2ban/
http://www.shellhacks.com/ru/RegEx-Nayti-IP-Adresa-v-Fayle-s-Pomoshchyu-Grep

Source: https://habr.com/ru/post/247391/

All Articles

Fail2ban integration with CSF to counter DDoS on nginx

More articles: