Kali Linux: monitoring and logging

→ Part 1. Kali Linux: security policy, protecting computers and network services
→ Part 2. Kali Linux: filtering traffic using netfilter
→ Part 3. Kali Linux: monitoring and logging
→ Part 4. Kali Linux: system protection and monitoring exercises

In the previous two articles in this series, we talked about security policy, about protecting computers and network services, about filtering traffic in Kali Linux. We thank our readers for the useful additions to these materials. In particular, the user imbasoft for referring to SANS Best Practices and for recommending him to familiarize himself with the set of standards of the STO BR IBBS to dive into the topic of business safety management processes. This comment is given to the first material. Thanks to the user loginsin , who made valuable remarks to the second material concerning the iptables rules and some other intricacies of filtering traffic in Linux.


')
Today we want to share with you the translation of section 7.5. Chapter 7 of the book “ Kali Linux Revealed ”, which is dedicated to monitoring and logging.

7.5. Monitoring and logging

Confidentiality and data protection are important aspects of information security, but it is equally important to ensure the smooth and correct operation of the system. If you play the role of a system administrator and security professional, you must ensure reliable and predictable infrastructure. Your area of ​​responsibility is the timely detection of abnormal behavior and deterioration in the quality of work of services.

Monitoring and logging programs play a key role here, allowing you to analyze what is happening in the system and in the network.

In this section, we will look at some tools that can be used to monitor Kali Linux.

7.5.1. Log monitoring using logcheck

The logcheck utility logcheck designed to monitor log files, which it, by default, hourly scans and sends unusual log entries to the administrator's email for further analysis.

The list of files watched by the program is stored in the /etc/logcheck/logcheck.logfiles file. The default settings allow logcheck to work normally, provided that the /etc/rsyslog.conf file has not been completely changed.

Logcheck supports three levels of filtering: paranoid , server and workstation . Using the paranoid level results in logcheck sending extremely long reports to the administrator. Probably, it should be used only on separate servers, such as those that play the role of firewalls. The server level is the default; it is recommended to use it on most servers. The workstation level, which is quite obvious from its name, was created for workstations; thanks to its use, the program produces brief reports, filtering much more messages than when using other levels of filtering.

In all three cases, the logcheck should be configured to exclude from the reports some clearly redundant messages (which ones depend on the services installed on the system). Otherwise, be prepared for the fact that every hour you will receive long uninteresting emails. The mechanism for selecting messages for inclusion in reports is rather complicated, so it is recommended to be patient and master /usr/share/doc/logcheck-database/README.logcheck-database.gz .

The applicable rules can be divided into several types:

• Rules that analyze messages to find attempts to hack the system. These rules are stored in a file located in the /etc/logcheck/cracking.d/ directory;
  
• The rules for hacking attempts to be ignored are described in /etc/logcheck/cracking.ignore.d/ ;
  
• The rules for messages rated as security warnings are stored in /etc/logcheck/violations.d/ ;
  
• The rules for security warnings to be ignored are stored in /etc/logcheck/violations.ignore.d/ ;
  
• And finally, the rules that apply to other messages (they are considered as system events).
  

ignore.d files ignore.d used to ignore messages. For example, a message marked as a hacking attempt or a security warning (according to the rule stored in the /etc/logcheck/violations.d/myfile file) can be ignored only when applying the rule that is stored in the /etc/logcheck/violations.ignore.d/myfile file /etc/logcheck/violations.ignore.d/myfile , or in the /etc/logcheck/violations.ignore.d/myfile-extension file.

Logcheck always reports system events, unless the directories whose naming scheme is shown below indicate that these events should be ignored:

 /etc/logcheck/ignore.d.{paranoid,server,workstation}/ 

Naturally, only those directories that correspond to a filtering level equal to or greater than the selected one are taken into account.

7.5.2. Real-time monitoring

The top — program top — an interactive tool that lists the running processes. The standard sorting of the process list is based on the amount of CPU resources consumed. If necessary, it can be activated using the key P Other ways of sorting processes include sorting by the occupied memory (key M ), total processor usage time (key T ), and process identifier (key N ). The key k allows you to complete the process by entering its identifier. Key r allows you to change the priority of the process.

If the system looks overloaded, top — an excellent tool that helps you know which processes consume the most CPU time or take up too much memory. In particular, it is usually useful to check whether the consumption of resources by the processes corresponds to the role played by the computer, to the services it is intended to support. For example, an unknown process running under the user “www-data” is worth paying attention to. It needs to be investigated, as it may be some kind of program installed and running on the system through a vulnerability in the web application.

The top tool is very flexible; from its help you can find out details on how to customize the data it displays and adapt it to your personal needs and preferences.
The gnome-system-monitor graphical tool is similar to top , it has about the same capabilities.

7.5.3. System change detection

Once the system is installed and configured, most system files should not be changed until the system is updated. Thus, it is useful to track changes in system files, since any unexpected change can be cause for concern and to investigate the reason for such a change. This section shows several common tools suitable for monitoring system files, for detecting their changes, and, if necessary, for organizing an administrator alert.

▍7.5.3.1. Package Auditing with dpkg --verify

The dpkg --verify (or dpkg -V ) command is an interesting tool that allows you to output information about system files that have been modified (perhaps by attackers), but the output of this command should be viewed with skepticism. In order to do its job, the dpkg utility relies on the checksums stored in its own database on the hard disk (you can find it on the path /var/lib/dpkg/info/package.md5sums ). A qualified hacker can easily modify these files so that they contain new checksums for modified files. If the attacker goes even further, he will replace the package on your Debian mirror. In order to protect against such attacks, use the APT digital signature confirmation system (for more on this, see Section 8.3.6. “ Package Authentication ”) for reliable packet checking.

▍What is a checksum file?

We want to remind you that a checksum is a value, often a number (albeit in hexadecimal notation) that contains something like a signature for the contents of a file. The signature is calculated using some algorithm (among them are the widely known MD5 and SHA1), which, in general, ensure that even the smallest change in the file will lead to a change in the checksum. This phenomenon is known as the avalanche effect. A simple digital signature then serves as a means to check if the contents of the file have changed. These algorithms are not amenable to inverse transformation. In other words, for most of them, knowledge of the signature does not allow restoring the data on the basis of which this signature was created. Recent studies in mathematics call into question the absoluteness of this principle, but so far in practice this is not used, since the creation of various data sets having the same checksum is still a very difficult task.

Running the dpkg -V command will check all installed packages and display information about those that have failed. Each character in the package information line indicates that some metadata has been checked. Unfortunately, dpkg does not store the metadata needed for most tests, and thus displays question marks for them. At present, only if the package fails the checksum test, the figure 5 appears at the third position.

 # dpkg -V ??5??????   /lib/systemd/system/ssh.service ??5?????? c /etc/libvirt/qemu/networks/default.xml ??5?????? c /etc/lvm/lvm.conf ??5?????? c /etc/salt/roster 

In this example, dpkg reports a change in the ssh.service file that the administrator made, instead of using the rules override for /etc/systemd/system/ssh.service , (which would be stored in the /etc directory, where the files should be stored specifying configuration changes). In addition, the command displays several configuration files (for such files, the letter “c” is displayed in the second column of the report), which were modified quite reasonably.

▍7.5.3.2. File Monitoring: AIDE

The Advanced Intrusion Detection Environment (AIDE) tool allows you to check the integrity of files and detects any changes compared to a previously saved, properly configured system. The image is stored as a database ( /var/lib/aide/aide.db ) and contains the relevant data (checksums, permissions, time /var/lib/aide/aide.db ) about all the system files.

You can install AIDE like this:

 apt update apt install aide 

After installation, you need to initialize the database with the aideinit command. This operation will then be performed daily (using the /etc/cron.daily/aide script) to check that the system files have not changed. If changes are detected, AIDE writes them to log files ( /var/log/aide/*.log ) and sends the report to the administrator by email.

▍ Protection of the AIDE database

Since AIDE uses a local database to compare the state of files, the correctness of such comparisons is completely tied to the state of the database. If an attacker gains superuser rights on a compromised system, he will be able to replace the database and hide the traces of the hack. One way to prevent such an attack is to store the reference data on read-only media.

To configure the aide package, you can use the parameters in /etc/default/aide . Namely, the internal settings of the program are stored in the /etc/aide/aide.conf and /etc/aide/aide.conf.d/ (in fact, these are the only files used by update-aide.conf to form /var/lib/aide/aide.conf.autogenerated ). The configuration indicates which properties of which files should be checked. For example, the state of the log files is constantly changing, such changes can be ignored, as long as the permissions set for these files do not change. However, in the case of executable files, both their content and permissions must remain unchanged. Although these settings are not so complicated, the syntax of AIDE configuration files cannot be called completely intuitive, so for those who want to understand the AIDE configuration, we recommend reading man aide.conf(5) .

A new version of the database is created daily at /var/lib/aide/aide.db.new . If all committed changes are valid, it can replace the reference database.

The Tripwire tool is very similar to AIDE. Even the syntax of the configuration files is almost the same. The main improvement that tripwire gives is the mechanism for signing the configuration file, with the result that the attacker cannot change it so that it points to a different version of the reference database.

The Samhain tool also provides similar capabilities, as well as some functions designed to detect rootkits (this is discussed in the next box). This program, in addition, can be deployed globally, across the network, and record the results of its work on the server (with a digital signature).

▍Packs checksecurity and chkrootkit / rkhunter

The checksecurity package is a collection of small scripts that perform basic system checks (searching for empty passwords, new setuid files, and so on) and issue warnings if you can find something suspicious. The package name, checksecurity , can give the administrator the impression that by checking the system with this package, you can guarantee its security. However, do not fully rely on this package.

The chkrootkit and rkhunter are designed to detect specific rootkits that can be installed in the system. Rootkits are programs that are created in order to not attract the attention of the administrator, to use the computer imperceptibly. The tests that perform the aforementioned packages cannot be called 100% reliable, but usually if they find something, it is worth paying attention to.

Results

Today we talked about monitoring and logging in Kali Linux. The tools discussed here allow you to maintain the level of protection of the system and its performance in good condition, however, you should always remember that attackers have the means to counter many control systems. In the next article, we will summarize the 7th chapter and offer you some exercises that will allow you to test your knowledge and learn something new about Kali Linux.

Dear readers! How do you organize monitoring of your Linux systems?