Comparison of Security Enhanced NGINX and Hiawatha Web Server in repelling attacks of level 7 (BOTS / SQL / XSS etc.)
Good day.
At the end of the “NGINX week”, we decided to talk about a very interesting build of the “safe” version of NGINX by the Chinese company Neusoft , about which not a single article on Habré was seen, namely SeNginx . In addition to this, we will also tell you about the second web server, which does not receive as much attention as Nginx - Hiawatha Web Server , and try to compare Hiawatha with SeNginx in terms of using them to secure your website or web application.
')
Note that over the 5 years of our hosting company’s existence, we have had the opportunity to see a whole range of attacks of various levels and types on the websites and web applications of our clients, as well as work with the London Metropolitan Police in finding and catching one botnet manager. Today, DDoS attacks have become so frequent that just yesterday Arbor in one of our data centers recorded 49 DDoS attacks with 3/4 levels up to 20-25 Gbit / s ( chart 1 and chart 2 ). And, although Arbor devices can, at a certain setting, deal well with level 3 and 4 attacks , they are almost useless when level 7 attacks are aimed at overloading server resources and also searching for vulnerabilities in your application.
The budget option for repelling attacks of level 7 are various options using DNS RoundRobin + HaProxy, Nginx, Varnish Cache, etc., which either allow you to distribute the attack between several client servers, or give the opportunity to process bots requests "head on" using recoil cached version of the page. This approach, unfortunately, often requires manual adjustment, constant monitoring and is rarely effective in changing types of attacks.
Visual representation of DNS RoundRobin + Nginx ( source ):
After a long search, we found an excellent module for NGINX, written by Russian Perl programmers in 2012 and allowing you to filter bots that come to your site in automatic mode - Roboo ( article on Habré ). The module showed itself from the best side, allowing you to filter hundreds of thousands of requests from bots using NGINX with minimal server load.
We could not figure out how, but the Roboo module was already noticed by Chinese programmers, rewritten to C, and included in the NGINX assembly with security modules, called SeNginx.
So, what is SeNGINX?
SeNginx is an NGINX build with additional security modules, namely:
In addition, modules for Load Balancing (Session Persistence, Fastest Load Balancing Algorithm / Fair Load Balancing Alogorithm), MIME cache module, as well as support for Syslog are present in SeNginx.
As a result, SeNginx is a very powerful tool to counter bots and automated security scanners, which are used to find vulnerabilities in your site.
What is the Hiawatha Web Server?
Hiawatha is a full-fledged web server, written by Hugo Leisink, a student from Amsterdam in 2002, and still supported. Unlike Nginx, Hiawatha was originally conceived as a web server with security features such as protection against SQL, XSS, CSRF and Exploit attacks.
After our communication with Hugo over the past few years, the Roboo module has also been added and tested in Hiawatha to protect against websites from bots, but with one additional feature that we will look at next.
The advantage of Hiawatha is that this web server is very easy to set up and operate. While Nginx is a powerful and sometimes complex tool, Hiawatha can be installed and configured in a short time, including monitoring and protection. Nginx, for example, does not have a convenient way to monitor traffic for each virtual host and requires the configuration of third-party modules to perform advanced monitoring of virtual hosts.
Hiawatha Web Server features:
As a result, Hiawatha is a convenient and fairly easy-to-use web server, which, unlike Nginx, does not have a million settings, but simply installs and works out of the box and has additional security features.
What are the differences between SeNginx and Hiawatha?
Of course, someone will say that comparing Nginx with a little-known web server does not make sense, however, our tests and the rather long use of Hiawatha in the production environment have shown that Hiawatha is an excellent open source tool that does not require much knowledge in configuration and management and is not much inferior Nginx
However, the main difference between SeNginx and Hiawatha in their ways of integrating the Roboo module is the following:
In SeNginx Robot Mitigation, the module works on the principle of On / Off, which means the need to either fine-tune the white list of IP addresses (for example, payment systems) to avoid blocking by this module, or enable the bot protection module directly during a bot attack. This method of protection is not very convenient, because you need to constantly monitor sites that are under attack by bots, or keep the module constantly turned on, which can lead to certain problems with the availability of the site for the "white" bots.
In Hiawatha, the mechanism is implemented to enable the protection module from bots only if the number of connections to your server exceeds a certain barrier. Suppose if during peak times on your web server you handle 10,000 connections, then a bot attack by the network can create a load of 100,000 or more connections to your web server. In this case, setting the Hiawatha settings to a level of 15,000 connections will allow you to automatically connect the bot protection module only when your server is under attack.
As a result, if you do not want to deal with a huge number of SeNginx settings, then it will be easier and faster for you to set up Hiawatha, and forget about botnet attacks for a while. If you are professionally engaged in DDoS protection, then the assembly SeNginx is a great tool that can be precisely customized to the needs of your customers.
At the end of the “NGINX week”, we decided to talk about a very interesting build of the “safe” version of NGINX by the Chinese company Neusoft , about which not a single article on Habré was seen, namely SeNginx . In addition to this, we will also tell you about the second web server, which does not receive as much attention as Nginx - Hiawatha Web Server , and try to compare Hiawatha with SeNginx in terms of using them to secure your website or web application.
')
Note that over the 5 years of our hosting company’s existence, we have had the opportunity to see a whole range of attacks of various levels and types on the websites and web applications of our clients, as well as work with the London Metropolitan Police in finding and catching one botnet manager. Today, DDoS attacks have become so frequent that just yesterday Arbor in one of our data centers recorded 49 DDoS attacks with 3/4 levels up to 20-25 Gbit / s ( chart 1 and chart 2 ). And, although Arbor devices can, at a certain setting, deal well with level 3 and 4 attacks , they are almost useless when level 7 attacks are aimed at overloading server resources and also searching for vulnerabilities in your application.
The budget option for repelling attacks of level 7 are various options using DNS RoundRobin + HaProxy, Nginx, Varnish Cache, etc., which either allow you to distribute the attack between several client servers, or give the opportunity to process bots requests "head on" using recoil cached version of the page. This approach, unfortunately, often requires manual adjustment, constant monitoring and is rarely effective in changing types of attacks.
Visual representation of DNS RoundRobin + Nginx ( source ):
After a long search, we found an excellent module for NGINX, written by Russian Perl programmers in 2012 and allowing you to filter bots that come to your site in automatic mode - Roboo ( article on Habré ). The module showed itself from the best side, allowing you to filter hundreds of thousands of requests from bots using NGINX with minimal server load.
We could not figure out how, but the Roboo module was already noticed by Chinese programmers, rewritten to C, and included in the NGINX assembly with security modules, called SeNginx.
So, what is SeNGINX?
SeNginx is an NGINX build with additional security modules, namely:
- HTTP Robot Mitigation - rewritten in C module Roboo, which protects the site from bots that do not support Javascript or Flash. At present, most of these bots, although there are new bots ( PhantomJS , Zombie JS , etc.), support Java - there are still fewer such botnets.
- Naxsi & Mod Security - protection modules from SQL, XSS and web scanners using Whitelist and rule sets.
- User-Agent Whitelist with DNS Reverse Resolve - the ability to add to the list of permissions bots that are used by search engines (Google, Yandex, etc.) with verification by Reverse DNS, which will prevent access to bots that have UserAgent: googlebot, but those really are not.
- Dynamic IP Blacklist - the module allows you to add the IP addresses of bots or scanners to the black list in automatic mode, write them to a log file and, if desired, unlock them after a certain time
- NetEye Security Layer - a module that allows you to integrate various security modules among themselves and to establish the order in which they perform checks
- Statistics is a convenient statistics module in JSON format that displays both the statistics of the NGINX itself and the security modules - http://demo.senginx.org
In addition, modules for Load Balancing (Session Persistence, Fastest Load Balancing Algorithm / Fair Load Balancing Alogorithm), MIME cache module, as well as support for Syslog are present in SeNginx.
As a result, SeNginx is a very powerful tool to counter bots and automated security scanners, which are used to find vulnerabilities in your site.
What is the Hiawatha Web Server?
Hiawatha is a full-fledged web server, written by Hugo Leisink, a student from Amsterdam in 2002, and still supported. Unlike Nginx, Hiawatha was originally conceived as a web server with security features such as protection against SQL, XSS, CSRF and Exploit attacks.
After our communication with Hugo over the past few years, the Roboo module has also been added and tested in Hiawatha to protect against websites from bots, but with one additional feature that we will look at next.
The advantage of Hiawatha is that this web server is very easy to set up and operate. While Nginx is a powerful and sometimes complex tool, Hiawatha can be installed and configured in a short time, including monitoring and protection. Nginx, for example, does not have a convenient way to monitor traffic for each virtual host and requires the configuration of third-party modules to perform advanced monitoring of virtual hosts.
Hiawatha Web Server features:
- Security - protection against SQL / XSS / CSRF / Exploit attacks, logging in system.log
- Cache support - the ability to set the cache size and convenient setting of file types stored in the cache (CacheRProxyExtensions = css, gif, html, jpg, js, png, txt)
- Reverse Proxy support - as in Nginx, Hiawatha supports reverse proxy support.
- Protection against bots - recently, you can enable protection against bots using Javascript or HTTPHeader (ChallengeClient)
- Hiawatha Monitor - Hiawatha has a web dashboard for collecting statistics from all of its servers into a single database, the data from which can be integrated into your applications.
As a result, Hiawatha is a convenient and fairly easy-to-use web server, which, unlike Nginx, does not have a million settings, but simply installs and works out of the box and has additional security features.
What are the differences between SeNginx and Hiawatha?
Of course, someone will say that comparing Nginx with a little-known web server does not make sense, however, our tests and the rather long use of Hiawatha in the production environment have shown that Hiawatha is an excellent open source tool that does not require much knowledge in configuration and management and is not much inferior Nginx
However, the main difference between SeNginx and Hiawatha in their ways of integrating the Roboo module is the following:
In SeNginx Robot Mitigation, the module works on the principle of On / Off, which means the need to either fine-tune the white list of IP addresses (for example, payment systems) to avoid blocking by this module, or enable the bot protection module directly during a bot attack. This method of protection is not very convenient, because you need to constantly monitor sites that are under attack by bots, or keep the module constantly turned on, which can lead to certain problems with the availability of the site for the "white" bots.
In Hiawatha, the mechanism is implemented to enable the protection module from bots only if the number of connections to your server exceeds a certain barrier. Suppose if during peak times on your web server you handle 10,000 connections, then a bot attack by the network can create a load of 100,000 or more connections to your web server. In this case, setting the Hiawatha settings to a level of 15,000 connections will allow you to automatically connect the bot protection module only when your server is under attack.
As a result, if you do not want to deal with a huge number of SeNginx settings, then it will be easier and faster for you to set up Hiawatha, and forget about botnet attacks for a while. If you are professionally engaged in DDoS protection, then the assembly SeNginx is a great tool that can be precisely customized to the needs of your customers.
Source: https://habr.com/ru/post/260159/
All Articles