Geekly Articles each Day
📜 ⬆️ ⬇️

General overview of standard system monitoring tools

Difficulty level: beginner

An introductory overview of the standard GNU / Linux tools should help novice Linux users to control the OS. The applications used are usually included in the distribution of known distributions. I consider it a useful side effect to be able to orient young system and network administrators in the face of [suspicion] compromise.

Local observation

Magazines.

Logging the system is very important. Log-files are the first source of information about the processes occurring in the system, and, of course, the first purpose of the hacker, sweeping traces of penetration. Cybercriminals always try to hide any evidence of their activities as much as possible, so a good approach is to save or duplicate logs on a separate resource or by e-mail. Naturally, the e-mail box where the system operation reports come should be serviced by a separate node. In this case, as a rule, they can be trusted.
The FHS provides the directory in which the log files should be located. Here is a small list of standard log locations:

Logging http, ftp server, MTA / MDA, network services is a narrower and more professional topic. Understandably, I hope the reasons are not considered in this article.
The output of the dmesg command and the entries in the messages , debug, and daemon.log should be examined first of all in case of (system-wide) problems. The last two especially, if problems are connected with services.
I will make a reservation that the specified paths to the registration files may of course differ in different systems. In such situations, refer to the distribution documentation.
I gave only a list, for example. You can find out the location of the log file from the man pages or configuration files. Usually this should not cause difficulties. For example, where to put records on the work of shared network folders, determines the log file directive of the configuration file (/etc/samba/smb.conf, as a rule) of the corresponding Samba daemon. In the settings of most network services, basically, you can also specify the format and, for some, the level of detail of logging entries.

Changes in the files.

A very good tool for detecting the fact of file modification is the standard find program. Reasonable use of the mtime parameter will help to establish changes in the file system, for example, the command
find /etc -mtime -1

will show files that have been changed in the last day. A very good file search for different times is described in the Unix find tutorial .

Processes.

Analysis of running processes should be carried out using top . Example output:
top - 15:21:36 up 4 days, 19:04, 2 users, load average: 0.06, 0.01, 0.00
Tasks: 92 total, 1 running, 91 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.7%sy, 0.0%ni, 97.4%id, 1.7%wa, 0.3%hi, 0.0%si, 0.0%st
Mem: 256088k total, 249192k used, 6896k free, 5320k buffers
Swap: 265064k total, 304k used, 264760k free, 176876k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27362 decoy 15 0 17400 11m 3616 S 0.3 4.7 3:48.28 rtorrent
1 root 18 0 2912 1848 524 S 0.0 0.7 0:01.83 init
...

The first column of the table contains the process number. It can be used, for example, to remove the command kill . Next (from left to right) follows the name of the user who initiated the launch, then the priority of the task, some service information, in particular, the application's use of system resources, the time during which the process is running, and the launch command.
In addition to the real-time monitor top, there is also a powerful generator of ps process snapshots. For example, you can find out whether a given program is running, say, Midnight Commander, at the moment or not, with the command
ps aux | grep mc | grep -v grep

It is strongly recommended to install htop - a more pleasant and descriptive for the user (it’s enough to see once) the analog top .
')
Network status

Nmap scanner

Colossally powerful tool for which almost separate books on use are written. Here is an example:
$ nmap -sT -v -v -v localhost
Starting nmap 3.55 ( www.insecure.org/nmap ) at 2005-02-04 22:01 EET
Machine 127.0.0.1 MIGHT actually be listening on probe port 80
Host localhost.localdomain (127.0.0.1) appears to be up ... good.
Initiating Connect() Scan against localhost.localdomain (127.0.0.1) at 22:17
Adding open port 111/tcp
Adding open port 25/tcp
Adding open port 113/tcp
Adding open port 22/tcp
Adding open port 80/tcp
Adding open port 631/tcp
The Connect() Scan took 1 second to scan 1660 ports.
Interesting ports on localhost.localdomain (127.0.0.1):
(The 1654 ports scanned but not shown below are in state: closed)
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
111/tcp open rpcbind
113/tcp open auth
631/tcp open ipp

Nmap run completed -- 1 IP address (1 host up) scanned in 0.503 seconds

Consequently, a web server is running on my machine. Another example (we check the node for RPC DCOM):
# nmap -sS -p 135 xxx.xxx.xxx.xxx

Starting nmap 3.55 ( www.insecure.org/nmap ) at 2005-02-04 22:06 EET
Interesting ports on *.* (xxx.xxx.xxx.xxx):
PORT STATE SERVICE
135/tcp filtered msrpc

Nmap run completed -- 1 IP address (1 host up) scanned in 45.798 seconds

Note : some scan types require superuser privileges (root).
The nmap scanner can check not only individual nodes, but entire ranges. As a rule, it is not included in a “clean” installation of the system, but is easy to install and highly recommended for use for auditing purposes.
Sometimes, in case of suspicious anomalies or based on the results of nmap , it is very useful to start the tcpdump traffic analyzer (sniffer), filter out the “unsuspecting” network packets and see what remains. In fact, tcpdump is not a sniffer, it takes pictures, but in such situations it will be extremely useful.
Another great tool to use is netstat . It is used to get a list of active network connections.
$ netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:32769 *:* LISTEN
tcp 0 0 *:62978 *:* LISTEN
tcp 0 0 *:sunrpc *:* LISTEN
tcp 0 0 *:auth *:* LISTEN
tcp 0 0 localhost.localdoma:ipp *:* LISTEN
tcp 0 0 localhost.localdom:5335 *:* LISTEN
tcp 0 0 *:29305 *:* LISTEN
tcp 0 0 localhost.localdom:smtp *:* LISTEN

In conjunction with Linux tools such as lsof and fuser , we learn which programs use which ports.

Conclusion

I tried to review the system monitors for the “little ones”. All the described tools can be found in almost any build of Linux. We are not talking about powerful and heavy third-party software systems IDS and shells for FS.
In conclusion, I would like to say that the proper use of the most standard set of tools allows you to conduct a basic audit. Remember: the problem of security is not the invention of paranoids! Many people think about their own protection after breaking. “The best way to ensure that precautions are not needed is to take them in time.” - Murphy.

A short review lastly ...

Process Analysis:

Network Analysis:

File system analysis:

Source: https://habr.com/ru/post/49204/

More articles:


All Articles