Simple way to additional security: SSH - ALERT

Administering a large number of “foreign” servers entails the responsibility to protect the data of our clients. In order to reliably control the list of people with ssh access to servers, it was decided to think over the authorization system from a limited set of hosts.


')
What we have in the problem statement:

• Over 400 physical machines;
• All client services are stored in openvz virtual containers (mostly):
• root access to all servers is closed via ssh;
• sudo access to root privileges is available only to admins on physical servers;
• several servers belonging to our company, half of which are geographically distributed. Let's call them: “access servers”.

What is done:

• The first step, through the configuration management system, we distributed keys to all physical ssh servers, to log in to them from access servers without a password. For each admin their own key.
• The second step is to close ssh access to physical servers from all hosts except access servers.

Of course, there will be exceptions in such a scheme, but this is all individually.

Now the question arises about the security of such a solution: anyone who hacks into account on the access server gets unlimited access to all servers, and the key without a password.
Not very good, to put it mildly!
It was proposed to restrict ssh access to access servers through the firewall, allowing access only from the ip of our employees. Good. But here is the problem: we have quite a few admins, many have dynamic ip. And what if the urgent need to "work" while on a trip, etc.?
We decided to stop on the reports. That is, every time a certain login successfully logs in from ip that is not in the list of valid ones, the task of this event with a high status falls to us in redmine and then we find out what happened using interrogation with addiction.

Implementation:
Current ready-made solutions for ssh auditing were reviewed, but nothing suitable was found, or very monstrous, or not about that.
And as always, they decided to write their bike, that is, the script. Which the:

1. analyzes ssh log for successful authorization.
2. takes data for the last 10 minutes.
3. compares the username / ip pair from the corresponding list
4. in case of successful verification, it does not do it
5. if there is no match, it sends an alert, which subsequently becomes a task in redmine.

The script is written in python. I wrote on python for the second time in my life, so that comments on the code are taken in an objective way. Besides, it works!

Listing attached:

#!/usr/bin/env python import sys, os, time from datetime import datetime, timedelta, date, time as dt_time import socket hostname = socket.gethostname() fileList = "./List.ip" date = datetime.now() - timedelta(minutes=10) date = date.strftime('%H:%M:%S') from commands import * import smtplib def mail(message): smtp_server = "localhost" smtp_port = 25 smtp_user= "root@%s" % hostname subject = 'Ahtung!! Security SSH audit alert' to = "mail@example.ru" mail_lib = smtplib.SMTP(smtp_server, smtp_port) msg = 'From: %s\r\nTo: %s\r\nContent-Type: text/html; charset="utf-8"\r\nSubject: %s\r\n Return-Path: <root@%s>\r\n\r\n' % (smtp_user, to, subject, hostname) msg += message mail_lib.sendmail(smtp_user, to, msg) listLog = getoutput('''cat /var/log/secure |grep "Accepted password for" | awk '{if($3>="'''+date+'''"){print $3 " " $9 " " $11 }}' ''') if listLog: for line in listLog.split('\n'): matchIp = 0 matchName = 0 curIp = line.split()[2] for lineLs in open(fileList): if len(lineLs.strip()) == 0 or lineLs[0] == "#" or lineLs[0] == " ": continue if line.split()[1] == lineLs.split()[0]: matchName = 1 listIp = lineLs.split()[1] i = 0 for c in listIp.strip().split("."): if c.strip() == "0": break i = i + 1 if listIp.strip().split(".")[0:i] == curIp.split(".")[0:i]: matchIp = 1 if not matchIp: if not matchName: print("This user %s not found !!!" % line.split()[1]) message = "This user %s not found !!!" % line.split()[1] mail(message) else: print("Ahtung! User {0} logged from unknown IP {1} in time {2}").format(line.split()[1], line.split()[2], line.split()[0]) message = "Ahtung! User "+line.split()[1]+" logged from unknown IP "+line.split()[2]+" in time "+line.split()[0] mail(message) 

File List.ip looks like:

 #Vasya vasya 1.2.3.4 vasya 12.13.14.0 

Related actions:

• place the script in cron to run every 10 minutes;
• set up a daily rotation of ssh logs at midnight;
• disable key authorization on access servers;
• for more paranoidalness, you can store the username / ip list in the git and scatter on the access servers through hooks.

An additional (relative) plus of this scheme: if you have to dismiss an employee (and this still happens :(), then you just need to kill the session on the access servers.