Geekly Articles each Day
📜 ⬆️ ⬇️

Audit of current vulnerabilities without registration and SMS

Introduction


As is known to anyone who has ever signed up for newsletters on IS, the number of vulnerabilities found per day often exceeds a person’s ability to parse. Especially if there are a lot of servers, especially if there is a zoo from OS and versions.

In this topic I will talk about how we solved this problem. And yes, Perl * is alive :)

Targets and goals


When designing the system, we mkhlystun ** solved two parallel tasks:


Both of these activities pursued a common goal: to correctly prioritize the updates and execute these updates in guaranteed downtime, which we achieved.
')

Scheme of work



  1. Once an hour, each host collects information about current packets and sends its queue
  2. Messages are queued from the queue and placed in the database
  3. Once in 3 hours the robot comes to the base and drives the packages through the audit service
  4. The results for the day generated a report that is sent to all interested

Base scheme


pkgs.sql
CREATE TABLE hosts ( hostname character varying(255) NOT NULL PRIMARY KEY, os character varying(255), pkg_id integer[] ); CREATE INDEX hosts_pkg_id_idx ON hosts USING gin (pkg_id); CREATE TABLE pkg ( id SERIAL NOT NULL PRIMARY KEY, name character varying(255) NOT NULL ); CREATE UNIQUE INDEX pkg_name_idx ON pkg USING btree (name); CREATE TABLE vulners ( id character varying(255) NOT NULL PRIMARY KEY, cvss_score double precision DEFAULT 0.0 NOT NULL, cvss_vector character varying(255), description text, cvelist text ); CREATE TABLE v2p ( pkg_id integer NOT NULL REFERENCES pkg(id), vuln_id character varying(255) NOT NULL REFERENCES vulners(id) );


The hosts table (one host = one entry) is entered into the hosts table, the information in pkg is filled in parallel (one packet - one entry), in order to avoid duplication of information. The array was chosen historically, it is quite possible to live with it

At the same time, information about current vulnerabilities for packages is entered into the vulners table, the v2p connection table allows you to bind many-to-many.

Collection of package information


grabber.pl
 #!/usr/bin/perl # (C) Ivan Agarkov 2016 use strict; use warnings; use JSON; # config our %grabs = ( 'centos|oraclelinux|redhat|fedora' => q(rpm -aq), 'debian|ubuntu' => q(dpkg-query -W -f='${Package} ${Version} ${Architecture}\n'), 'osx' => q(pkgutil --pkgs) ); our %unames = ( 'linux' => q(lsb_release -a), 'darwin' => q(echo "Distributor ID: OSX") ); # global vars our $hostname = `hostname -f`; our ($vercmd, $grabcmd, $operatingsystem, $version); # do uname my $uname = `uname`; chomp $uname; foreach (keys %unames) { $vercmd = $unames{$_} if $uname =~ /$_/i; } die "Version CMD not found" unless $vercmd; # do version check foreach (`$vercmd`) { chomp; /^Distributor ID:\s*(\S[\S\s]+)$/ and $operatingsystem = $1; /^Release:\s*(\S[\S\s]+)$/ and $version = $1; } die "Opetating System not found" unless $operatingsystem; foreach (keys %grabs) { $grabcmd = $grabs{$_} if $operatingsystem =~ /$_/i; } # grab pkgs die "Opetating System not found" unless $grabcmd; my @pkgs; foreach (`$grabcmd`) { chomp; push @pkgs, $_; } chomp $hostname; my $result = { hostname => $hostname, os => $version ? qq($operatingsystem $version) : $operatingsystem, pkgs => [ sort @pkgs ] }; # print JSON->new->encode($result); # done 1;


For those who do not know Perl: the hostname -f , lsb_release -a and rpm -aq | dpkg-query -W commands are simply executed in succession, all this is packaged in JSON and displayed for sending to the message queue.

JSON Transformation to Base


transform.pl
 #!/usr/bin/perl # (C) Ivan Agarkov 2016 use strict; use warnings; use JSON; use DBI; use constant DB => 'dbi:Pg:dbname=pkgs'; # 0. create connection my $dbh = DBI->connect( DB, "", "", { RaiseError => 1, AutoCommit => 0 } ); # 1. read from stdin and parse my $data = JSON->new->decode( join( "", <STDIN> ) ); # if data == array parse foeach if ( ref($data) eq "ARRAY" ) { foreach (@$data) { parse_host($_); } } else { parse_host($data); } # Done 1; ### SUBS ### sub parse_host { $_ = shift; # do parse packages my ( $hostname, $os, @pkgs ) = ( $_->{hostname}, $_->{os}, @{ $_->{pkgs} } ); my @pkgids; eval { foreach (@pkgs) { my $sth = $dbh->prepare("SELECT id FROM pkg WHERE name=?"); $sth->execute( ($_) ); if ( my ($id) = $sth->fetchrow_array ) { push @pkgids, int($id); } else { $dbh->do( "INSERT INTO pkg (name) VALUES(?)", undef, $_ ); push @pkgs, $_; } } }; $dbh->rollback and die "$@" if $@; $dbh->commit; # do parse host eval { my $sth = $dbh->prepare("SELECT os FROM hosts WHERE hostname=?"); $sth->execute( ($hostname) ); if ( my ($os2) = $sth->fetchrow_array ) { if ( lc($os2) ne lc($os) ) { $dbh->do( "UPDATE hosts SET os=? WHERE hostname=?", undef, $os, $hostname ); } } else { $dbh->do( "INSERT INTO hosts (hostname, os) VALUES(?, ?)", undef, $hostname, $os ); } }; $dbh->rollback and die "$@" if $@; $dbh->commit; # do set packages eval { $dbh->do( "UPDATE hosts SET pkg_id=? WHERE hostname=?", undef, [@pkgids], $hostname ); }; $dbh->rollback and die "$@" if $@; $dbh->commit; }


This script receives json as input from the queue, and then decomposes it into the database in three stages:

- First puts the packages, checking their uniqueness
- Then puts the hosts, updating the version if necessary
- then links the hosts and packets through an array

Audit


When we were looking for a way to audit our packages, we went over the options for a long time, until at some conference we had a vulners business card. This is the vulnerability aggregator that isox and videns do . I contacted them and asked for help. The result was the Audit API .

Audit API
To get information about vulnerabilities, it is enough to build a packet blob in json and send it to / api / v3 / audit / audit / .

 POST /api/v3/audit/audit/ HTTP/1.0 Host: vulners.com Content-Type: application/json Content-Length: 377 { "os":"CentOS", "version":"7", "package":["kernel-3.10.0-229.el7.x86_64"]}

In response, the server will give json with a list of current vulnerabilities in the following format:

 { "result": "OK", "data": { "packages": { "kernel-3.10.0-229.el7.x86_64": { "CESA-2015:2152": [ { "package": "kernel-3.10.0-229.el7.x86_64", "providedVersion": "0:3.10.0-229.el7", "bulletinVersion": "3.10.0-327.el7", "providedPackage": "kernel-3.10.0-229.el7.x86_64", "bulletinPackage": "kernel-3.10.0-327.el7.x86_64.rpm", "operator": "lt", "bulletinID": "CESA-2015:2152" } ], "CESA-2015:1978": [ { "package": "kernel-3.10.0-229.el7.x86_64", "providedVersion": "0:3.10.0-229.el7", "bulletinVersion": "3.10.0-229.20.1.el7", "providedPackage": "kernel-3.10.0-229.el7.x86_64", "bulletinPackage": "kernel-3.10.0-229.20.1.el7.src.rpm", "operator": "lt", "bulletinID": "CESA-2015:1978" }, // skipped ], "CESA-2016:0064": [ { "package": "kernel-3.10.0-229.el7.x86_64", "providedVersion": "0:3.10.0-229.el7", "bulletinVersion": "3.10.0-327.4.5.el7", "providedPackage": "kernel-3.10.0-229.el7.x86_64", "bulletinPackage": "kernel-3.10.0-327.4.5.el7.src.rpm", "operator": "lt", "bulletinID": "CESA-2016:0064" }, // skipped ], // skipped ], // skipped "cvss": { "score": 10.0, "vector": "AV:NETWORK/AC:LOW/Au:NONE/C:COMPLETE/I:COMPLETE/A:COMPLETE/" }, "cvelist": [ "CVE-2014-9644", "CVE-2016-2384", // skipped ], "id": "F777" } }


After running in the API, a code was written that pushes the list of packages on the OS and in response receives a list of vulnerabilities and puts them into the database.

audit.pl
 #!/usr/bin/perl # (C) Ivan Agarkov 2016 use strict; use warnings; use lib 'perl5'; use HTTP::Tiny; use DBI; use JSON; use constant VULNERS_AUDIT_API => 'http://vulners.com/api/v3/audit/audit/'; use constant VULNERS_ID_API => 'http://vulners.com/api/v3/search/id/'; use constant DB => 'dbi:Pg:dbname=pkgs'; our %VULNS; our $dbh; our %pkgs = (); # 0. connect to DB $dbh = DBI->connect( DB, "", "", { RaiseError => 1, AutoCommit => 0 } ); # get all OS variations my @os = get_os(); # for each OS get all packages and ask vulners for its vulnerabilities foreach my $os (@os) { eval { my ( $o, $ver ) = split( / /, $os ); my $res = HTTP::Tiny->new->request( 'POST', VULNERS_AUDIT_API, { headers => { 'Content-Type' => 'application/json' }, content => JSON->new->encode( { os => $o, version => $ver, package => [ get_packages($os) ] } ) } ); if ( !$res->{success} ) { die "HTTP Error: $res->{content}"; } my $data = JSON->new->decode( $res->{content} ); my $vulns = $data->{data}->{packages}; return undef unless defined $vulns; foreach ( keys %$vulns ) { my $o = $vulns->{$_}; if ( defined( $pkgs{$_} ) ) { $VULNS{ $pkgs{$_} } = [ keys %$o ]; } } }; print $@ if $@; } # Now get info on each vuln ID ( CESA, USN, etc ) ... my @result; my $res = HTTP::Tiny->new->request( 'POST', VULNERS_ID_API, { headers => { 'Content-Type' => 'application/json' }, content => JSON->new->encode( { id => [ map {@$_} values %VULNS ] } ) } ); if ( !$res->{success} ) { die "HTTP Error: $res->{content}"; } my $data = JSON->new->decode( $res->{content} ); foreach ( values %{ $data->{data}->{documents} } ) { push @result, { id => $_->{id}, cvss_score => $_->{cvss}->{score}, cvss_vector => $_->{cvss}->{vector}, description => $_->{description}, cvelist => join( ', ', @{ $_->{cvelist} } ), }; } # Insert the data to DB eval { $dbh->do( "DELETE FROM v2p", undef ); $dbh->do( "DELETE FROM vulners", undef ); # insert prepared data to vulners table foreach (@result) { $dbh->do( "INSERT INTO vulners (id, cvss_score, cvss_vector, description, cvelist) VALUES (?,?,?,?,?)", undef, $_->{id}, $_->{cvss_score}, $_->{cvss_vector}, $_->{description}, $_->{cvelist} ); } # and link pkg and vuls into v2p foreach my $pkg_id ( keys %VULNS ) { foreach my $vuln_id ( @{ $VULNS{$pkg_id} } ) { $dbh->do( "INSERT INTO v2p(pkg_id,vuln_id) VALUES(?,?)", undef, $pkg_id, $vuln_id ); } } }; $dbh->rollback and die "Error $@" if $@; $dbh->commit; # All done 1; ### SUBS #### sub get_os { my @os; my $sth = $dbh->prepare("SELECT DISTINCT os FROM hosts"); $sth->execute(); while ( my ($os) = $sth->fetchrow_array ) { push @os, $os; } return @os; } sub get_packages { my $os = shift; my $sth = $dbh->prepare( "select DISTINCT p.id,p.name FROM pkg p RIGHT JOIN hosts h ON (p.id=ANY(h.pkg_id)) WHERE h.os=?" ); $sth->execute( ($os) ); my @pkgs; while ( my ( $id, $name ) = $sth->fetchrow_array ) { $pkgs{$name} = $id; push @pkgs, $name; } return @pkgs; }


Algorithm:

- Take the OS list from the hosts table
- For each OS we get a list of packages
- We send packages to Audit API, we receive the list (id) of vulnerabilities
- We send vulnerabilities to Api ID, we get metadata for each
- We write vulnerability metadata to the vulners table
- We write in the database of communication packages and vulnerabilities in the table v2p

Reports


Since our main goal was to get a list of hosts for the priority update, the first report I did was 'top10 hosts to update', selected based on the total CVSS Score *** .

report.pl
 #!/usr/bin/perl # (C) Ivan Agarkov 2016 use strict; use warnings; use lib 'perl5'; use HTTP::Tiny; use DBI; use JSON; use constant DB => 'dbi:Pg:dbname=pkgs'; our $dbh; our @hosts; # 0. connect to DB $dbh = DBI->connect( DB, "", "", { RaiseError => 1, AutoCommit => 0 } ); # get top10 hosts my $sth = $dbh->prepare("SELECT h.hostname,SUM(v.cvss_score) as sum FROM hosts h INNER JOIN pkg p ON(p.id=ANY(h.pkg_id)) INNER JOIN v2p vp ON(vp.pkg_id=p.id) INNER JOIN vulners v ON (v.id=vp.vuln_id) GROUP BY h.hostname ORDER BY sum DESC LIMIT 10"); $sth->execute(); while (my ($host, $sum) = $sth->fetchrow_array) { push @hosts, { hostname => $host, score => $sum, pkgs => [] }; } foreach (@hosts) { $sth = $dbh->prepare("SELECT p.name,SUM(v.cvss_score) AS score FROM pkg p RIGHT JOIN hosts h ON (p.id=ANY(h.pkg_id)) INNER JOIN v2p ON (v2p.pkg_id=p.id) INNER JOIN vulners v ON (v.id=v2p.vuln_id) WHERE h.hostname=? GROUP BY p.name ORDER BY score DESC LIMIT 10"); $sth->execute(($_->{hostname})); while(my ($pkg,$sum) = $sth->fetchrow_array) { push @{$_->{pkgs}}, { package => $pkg, score => $sum }; } } print <<EOF TOP 10 SERVERS TO UPDATE EOF ; foreach (@hosts) { print <<EOF -------------------------------------------------- Hostname: $_->{hostname} Score : $_->{score} Packages: EOF ; foreach (@{$_->{pkgs}}) { print <<EOF Name : $_->{package} Score: $_->{score} EOF } }


This code simply takes the top 10 hosts and for each of them takes the top 10 packets with a total soon. Then you can create tasks and update.

Results of the year of use


For more than a year, we have been sending our data to the guys from Vulners. To date, they are constantly auditing more than 30,000 unique packages. What is nice, all the bugs found were promptly corrected, and the processing speed of every thousand increased from 30 seconds to 400 milliseconds. It is thanks to them that this topic is called "... without registration and SMS")

With regards to business goals, only with the introduction of this system, we began to appear a process of constant updates. To update everything is too big a task for the engineer on duty, and to update the first 10 is quite feasible. For the year, we dropped the total cvss score more than double what we want **** )

Footnotes and explanations


* - It happened historically, I started writing automation on Perl and no one had time to stop me.
** - Misha is not an active user of the habr, but as an engineer he is indispensable :)
*** - Digital metric of vulnerability hazard, from 1.0 (can be scored) to 10.0 (critical vulnerability)
**** - All code can be found here: github.com/annmuor/freeaudit

PS And yet - Perl is alive!

Source: https://habr.com/ru/post/326084/

More articles:


All Articles