Permanent intruder ban with Fail2Ban + MikroTik
A few days ago, I installed Asterisk, downloaded my old configuration with call routing, and intended to connect to a local SIP provider. Just a few minutes after launching, Asterisk found in the logs attempts to authorize on the server, which did not surprise me at all, because Such a picture is observed on any asterisk looking to the Internet. It was a volitional decision to play with your favorite mikrotik and no less favorite python, and figure out what to do with these intruders.
So, we have:
After reading a couple of articles ( one , two ), the following concept was born:
And now to the implementation of the decision.
1. Create a database / table that will contain the following information - IP address, country code, country name, number of bans issued, attack type / service (jail name from Fail2Ban configuration), last attempt, first attempt (with a reserve for the future, perhaps I will somehow use this data).
')
2. Create a script to add entries to the database. The script is written in Python and requires the following additional modules for its work - pygeoip and MySQL-python . Both modules are easily installed using the pip package manager:
The script takes data to connect to the database from the configuration file, which by default tries to find in the same directory, you can also set the path using the "-c" key.
The key point is that the script is executed along with the addition of rules in iptables, therefore I edited the following files:
Thus, after adding the appropriate rules in iptables, our script is executed and adds or updates the data in the database.
3. We create a script for generating blacklist, which later will be imported into our microtic. The script uses the same configuration file to get the settings needed to connect to the database and also searches for it in its root directory, again you can set the path using the "-c" key. At the output, a script / address list is created for importing into a microtic, again in the same directory, you can specify an alternative path using the "-o" key.
This script is executed using the crown, I set the frequency of the launch in 15 minutes.
4. Import the received list into our router.
The resulting script for microtic should be placed in the directory of our web server, I have nginx in the configuration of which I added the following lines:
Instead of a web server, you can use ftp or tftp , it all depends on your taste.
This part is almost completely "stolen" from the second article .
Once an hour, the file is downloaded from the server using the HTTP protocol using the following script (below is the script and the scheduler rule for microtic):
Script to import blacklist:
To use this list, prohibiting rules are created and placed before allowing ones (because the rules are processed in order), in this example 2 rules are created for SSH connections and SIP:
Where ID-Net is the name of my external interface.
This "bicycle" does not claim anything and was assembled "on the knee" for a couple of hours.
I hope for constructive criticism of habrovchan and suggestions for possible improvements.
Archive with all submitted scripts and database schema.
GitHub repository.
So, we have:
- Ubuntu Server 14.04 (I think it doesn't matter, it should work on other distributions)
- Fail2Ban
- Mysql
- Asterisk (or any other service that needs to be protected from brute force attacks)
- MikroTik router
- Arms
- Desire to invent a bicycle
After reading a couple of articles ( one , two ), the following concept was born:
- Ban an attacker for a certain time with the help of Fail2Ban and add an entry with its IP address to the MySQL database
- after a certain number of bans issued, add the IP address to the list of prohibited on the router
And now to the implementation of the decision.
1. Create a database / table that will contain the following information - IP address, country code, country name, number of bans issued, attack type / service (jail name from Fail2Ban configuration), last attempt, first attempt (with a reserve for the future, perhaps I will somehow use this data).
')
Scheme
CREATE DATABASE fail2ban CHARACTER SET utf8; CREATE TABLE `ban_history` ( `id` int(11) unsigned NOT NULL AUTO_INCREMENT, `ip_address` char(15) NOT NULL DEFAULT '', `country_code` varchar(5) DEFAULT NULL, `country_name` varchar(30) DEFAULT NULL, `count` int(11) NOT NULL, `type` varchar(30) DEFAULT NULL, `last_attempt` datetime NOT NULL, `first_attempt` datetime NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; 2. Create a script to add entries to the database. The script is written in Python and requires the following additional modules for its work - pygeoip and MySQL-python . Both modules are easily installed using the pip package manager:
pip install pygeoip MySQL-python Script
#!/usr/bin/env python2 # -*- coding: utf-8 -*- import os import urllib import gzip import StringIO import logging import logging.handlers import MySQLdb import MySQLdb.cursors import ConfigParser import pygeoip from datetime import datetime from sys import exit from optparse import OptionParser def main(config, logger, ip_addr, attack_type, GEOIP_DAT): url = urllib.urlopen('http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz') url_f = StringIO.StringIO(url.read()) handle = gzip.GzipFile(fileobj=url_f) with open(GEOIP_DAT, 'w') as out: for line in handle: out.write(line) if config.has_option('general', 'mysql_ip') and config.has_option('general', 'mysql_user') and config.has_option('general', 'mysql_password') and config.has_option('general', 'mysql_db'): try: logger.info("Connecting to MySQL host: %s" % config.get('general', 'mysql_ip')) db = MySQLdb.connect( host=config.get('general', 'mysql_ip'), user=config.get('general', 'mysql_user'), passwd=config.get('general', 'mysql_password'), db=config.get('general', 'mysql_db'), cursorclass=MySQLdb.cursors.DictCursor ) cursor = db.cursor() logger.debug("Connected") except MySQLdb.Error, e: logger.error("Error %d: %s" % (e.args[0], e.args[1])) exit(2) else: query = """select * from ban_history where ip_address='%s' and type='%s'""" % (ip_addr, attack_type) result = run_query(cursor, query, logger) result = cursor.fetchall() now = datetime.now() gi = pygeoip.GeoIP(GEOIP_DAT, flags=pygeoip.const.MEMORY_CACHE) country_code = gi.country_code_by_addr(ip_addr) country_name = gi.country_name_by_addr(ip_addr) if len(result) > 0: logger.info("Updating blacklist DB record for IP-address %s" % ip_addr) result = result[0] count = result['count'] + 1 query = """update ban_history set count=%s, last_attempt='%s', country_code='%s', country_name='%s' where id=%s""" % (count, now, country_code, country_name, result['id']) result = run_query(cursor, query, logger) db.commit() else: logger.info("Adding IP-address %s into blacklist DB" % ip_addr) count = 1 query = """insert into ban_history (ip_address, country_code, country_name, count, type, last_attempt, first_attempt) values('%s', '%s', '%s', %s, '%s', '%s', '%s')""" % (ip_addr, country_code, country_name, count, attack_type, now, now) result = run_query(cursor, query, logger) db.commit() else: logger.error("Configuration incomplete") exit(3) def run_query(cursor, query, logger): try: logger.debug("Running query \'%s\'" % query) cursor.execute(query) except MySQLdb.Error, e: logger.error("Error %d: %s" % (e.args[0], e.args[1])) exit(2) else: return True if __name__ == '__main__': try: ROOT_PATH = os.path.dirname(os.path.realpath(__file__)) GEOIP_DAT = os.path.join(ROOT_PATH, 'GeoIP.dat') parser = OptionParser(usage="usage: %prog [-c <configuration_file>] [-v] --ip IP-ADDRESS --type TYPE") parser.add_option("-v", "--verbose", action="store_true", default=False, dest="verbose", help="Verbose output") parser.add_option("-c", "--config", action="store", default=False, dest="cfg_file", help="Full path to configuration file") parser.add_option("--ip", action="store", default=False, dest="ip_addr", help="Attacker IP address") parser.add_option("--type", action="store", default=False, dest="attack_type", help="Type of attack (service)") (options, args) = parser.parse_args() verbose = options.verbose ip_addr = options.ip_addr attack_type = options.attack_type # Reading configuration file cfg_file = options.cfg_file if not cfg_file: cfg_file = os.path.join(ROOT_PATH, 'blacklist_db.cfg') config = ConfigParser.RawConfigParser() config.read(cfg_file) # Logging if config.get('general', 'log_file'): LOGFILE = config.get('general', 'log_file') else: LOGFILE = '/tmp/blacklist_db.log' FORMAT = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S') try: rotatetime = logging.handlers.TimedRotatingFileHandler(LOGFILE, when="midnight", interval=1, backupCount=14) except IOError, e: print "ERROR %s: Can not open log file - %s" % (e[0], e[1]) exit(1) except Exception, e: print "Can not configure logger - %s" % e exit(1) formatter = logging.Formatter('%(asctime)s: %(message)s','%y-%m-%d %H:%M:%S') rotatetime.setFormatter(FORMAT) logger = logging.getLogger('BLACKLIST-DB') logger.addHandler(rotatetime) if verbose: lvl = logging.DEBUG console = logging.StreamHandler() formatter = logging.Formatter('%(asctime)s: %(message)s','%y-%m-%d %H:%M:%S') console.setFormatter(formatter) logger.addHandler(console) else: lvl = logging.INFO logger.setLevel(lvl) if ip_addr and attack_type: main(config, logger, ip_addr, attack_type, GEOIP_DAT) else: logger.error("IP address and attack type are needed but not specified") exit(1) except (KeyboardInterrupt): logger.info("CTRL-C... exit") exit(0) except (SystemExit): logger.info("Exit") exit(0) The script takes data to connect to the database from the configuration file, which by default tries to find in the same directory, you can also set the path using the "-c" key.
Sample configuration file
[general]
log_file = /var/log/blacklist_db.log
mysql_ip = localhost
mysql_user = db_user
mysql_password = db_pass
mysql_db = fail2ban
# The number of bans, after which we add the IP address to the blacklist, the default is 10
#ban_count = 10
log_file = /var/log/blacklist_db.log
mysql_ip = localhost
mysql_user = db_user
mysql_password = db_pass
mysql_db = fail2ban
# The number of bans, after which we add the IP address to the blacklist, the default is 10
#ban_count = 10
The key point is that the script is executed along with the addition of rules in iptables, therefore I edited the following files:
/etc/fail2ban/action.d/iptables-allports.conf
# actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> # actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> /// -v --ip <ip> --type <name> /etc/fail2ban/action.d/iptables-multiport.conf
# actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> # actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> /// -v --ip <ip> --type <name> /etc/fail2ban/action.d/iptables-new.conf
(not sure what this action is used for, made changes for fidelity)
# actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> # actionban = iptables -I fail2ban-<name> 1 -s <ip> -j <blocktype> /// -v --ip <ip> --type <name> Thus, after adding the appropriate rules in iptables, our script is executed and adds or updates the data in the database.
3. We create a script for generating blacklist, which later will be imported into our microtic. The script uses the same configuration file to get the settings needed to connect to the database and also searches for it in its root directory, again you can set the path using the "-c" key. At the output, a script / address list is created for importing into a microtic, again in the same directory, you can specify an alternative path using the "-o" key.
Script
#!/usr/bin/env python2 # -*- coding: utf-8 -*- import os import logging import logging.handlers import MySQLdb import MySQLdb.cursors import ConfigParser from sys import exit from optparse import OptionParser def main(config, logger, output): if config.has_option('general', 'ban_count'): ban_count = config.getint('general', 'ban_count') else: ban_count = 10 if config.has_option('general', 'mysql_ip') and config.has_option('general', 'mysql_user') and config.has_option('general', 'mysql_password') and config.has_option('general', 'mysql_db'): try: logger.info("Connecting to MySQL host: %s" % config.get('general', 'mysql_ip')) db = MySQLdb.connect( host=config.get('general', 'mysql_ip'), user=config.get('general', 'mysql_user'), passwd=config.get('general', 'mysql_password'), db=config.get('general', 'mysql_db'), cursorclass=MySQLdb.cursors.DictCursor ) cursor = db.cursor() logger.debug("Connected") except MySQLdb.Error, e: logger.error("Error %d: %s" % (e.args[0], e.args[1])) exit(2) else: contents = ['/ip firewall address-list'] logger.info('Fetching adresses from the blacklist DB') query = """select * from ban_history""" result = run_query(cursor, query, logger) result = cursor.fetchall() for ip in result: if ip['count'] >= ban_count: list_name = '%s_BLC' % ip['type'].upper() logger.info('Adding IP %s into \'%s\' list' % (ip['ip_address'], list_name)) list_line = 'add address=%s list=%s comment=BLACKLIST' % (ip['ip_address'], list_name) contents.append(list_line) if len(contents) > 1: logger.info('Generating mikrotik rsc script...') script_file = open(output, 'w') for item in contents: script_file.write("%s\r\n" % item) script_file.close() logger.info('Done') else: logger.error("Configuration incomplete") exit(3) def run_query(cursor, query, logger): try: logger.debug("Running query \'%s\'" % query) cursor.execute(query) except MySQLdb.Error, e: logger.error("Error %d: %s" % (e.args[0], e.args[1])) exit(2) else: return True if __name__ == '__main__': try: ROOT_PATH = os.path.dirname(os.path.realpath(__file__)) parser = OptionParser(usage="usage: %prog [-c <configuration_file>] [-v] [-o <output_file_path>]") parser.add_option("-v", "--verbose", action="store_true", default=False, dest="verbose", help="Verbose output") parser.add_option("-c", "--config", action="store", default=False, dest="cfg_file", help="Full path to configuration file") parser.add_option("-o", action="store", default=False, dest="output", help="Full path for the generated script file") (options, args) = parser.parse_args() verbose = options.verbose output = options.output if not output: output = os.path.join(ROOT_PATH, 'blacklists.rsc') # Reading configuration file cfg_file = options.cfg_file if not cfg_file: cfg_file = os.path.join(ROOT_PATH, 'blacklist_db.cfg') config = ConfigParser.RawConfigParser() config.read(cfg_file) # Logging if config.get('general', 'log_file'): LOGFILE = config.get('general', 'log_file') else: LOGFILE = '/tmp/blacklist_db.log' FORMAT = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S') try: rotatetime = logging.handlers.TimedRotatingFileHandler(LOGFILE, when="midnight", interval=1, backupCount=14) except IOError, e: print "ERROR %s: Can not open log file - %s" % (e[0], e[1]) exit(1) except Exception, e: print "Can not configure logger - %s" % e exit(1) formatter = logging.Formatter('%(asctime)s: %(message)s','%y-%m-%d %H:%M:%S') rotatetime.setFormatter(FORMAT) logger = logging.getLogger('BLACKLIST-DB') logger.addHandler(rotatetime) if verbose: lvl = logging.DEBUG console = logging.StreamHandler() formatter = logging.Formatter('%(asctime)s: %(message)s','%y-%m-%d %H:%M:%S') console.setFormatter(formatter) logger.addHandler(console) else: lvl = logging.INFO logger.setLevel(lvl) main(config, logger, output) except (KeyboardInterrupt): logger.info("CTRL-C... exit") exit(0) except (SystemExit): logger.info("Exit") exit(0) This script is executed using the crown, I set the frequency of the launch in 15 minutes.
*/15 * * * * /// > /dev/null 2>&1 4. Import the received list into our router.
The resulting script for microtic should be placed in the directory of our web server, I have nginx in the configuration of which I added the following lines:
location /blacklists.rsc { root /////; } Instead of a web server, you can use ftp or tftp , it all depends on your taste.
This part is almost completely "stolen" from the second article .
Once an hour, the file is downloaded from the server using the HTTP protocol using the following script (below is the script and the scheduler rule for microtic):
# , example.com , IP /system script add name="Download_blacklists" source={ /tool fetch url="http://example.com/blacklists.rsc" mode=http; :log info "Downloaded blacklists.rsc"; } # /system scheduler add comment="Download blacklists" interval=1h name="DownloadBlackLists" on-event=Download_blacklists start-date=jan/01/1970 start-time=01:05:00 Script to import blacklist:
# /system script add name="Update_blacklists" source={ /ip firewall address-list remove [/ip firewall address-list find comment="BLACKLIST"]; /import file-name=blacklists.rsc; :log info "Removal old blacklists and add new"; } # /system scheduler add comment="Update BlackList" interval=1h name="InstallBlackLists" on-event=Update_blacklists start-date=jan/01/1970 start-time=01:15:00 To use this list, prohibiting rules are created and placed before allowing ones (because the rules are processed in order), in this example 2 rules are created for SSH connections and SIP:
/ip firewall filter add action=reject chain=forward comment="SIP: Reject Blacklisted IP addresses" dst-port=5060-5061 in-interface=ID-Net protocol=udp src-address-list=ASTERISK_BLC add action=reject chain=forward comment="SSH: Reject Blacklisted IP addresses" dst-port=22 in-interface=ID-Net protocol=tcp src-address-list=SSH_BLC Where ID-Net is the name of my external interface.
This "bicycle" does not claim anything and was assembled "on the knee" for a couple of hours.
I hope for constructive criticism of habrovchan and suggestions for possible improvements.
Archive with all submitted scripts and database schema.
GitHub repository.
Source: https://habr.com/ru/post/248033/
All Articles