Analyzing network traffic on a server using tshark

In the practice of system administration, it is often necessary to deal with difficult situations in which neither the statistics collection tools (for example, netstat) nor the standard utilities based on the ICMP protocol (ping, traceroute and others) help. In such cases, specialized diagnostic utilities are often used, which make it possible to “listen” to network traffic and analyze it at the level of transmission units of individual protocols. They are called traffic analyzers, and in professional jargon they are sniffers. With their help, you can, firstly, localize network problems and more accurately diagnose them, and secondly, detect spurious traffic and detect malicious software on the network.

Particularly useful are traffic analyzers in cases where the network software is poorly documented or uses its own private protocols.
')
One of the most common and popular traffic analyzers today is Wireshark, distributed under the GNU GPL license. There are versions of Wireshark for various operating systems: Linux, Windows, MacOS, FreeBSD, Solaris.

Tshark uses the libpcap library, which implements the pcap (packet capture) API, to capture. This library is also used by the standard tcpdump utility. Files created in tcpdump can be transferred to tshark for further analysis.

The undoubted advantage of tshark compared to tcpdump is a clearer and more comprehensible output format. In addition, tshark supports a huge number of network protocols (over 300, which covers almost all types of networks ever invented).

On the features of working with tshark we will describe in detail in this article.

Beginning of work

Tshark is included in the distributions of most modern Linux systems and is installed using the standard package manager:

 $ sudo apt-get install tshark

After completing the installation, run the program:

 $ sudo tshark

A list of packets captured in real time will be displayed on the console:

 Capturing on eth0
 0.000000 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.120?  Tell 31.186.98.1
 0.322046 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77?  Tell 31.186.98.226
 0.351801 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len ​​= 224
 0.352414 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 225 Win = 331 Len = 0 TSV = 194287231 TSER = 416767897
 0.600054 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.120?  Tell 31.186.98.1
 0.622913 Cisco_0d: 0d: 96 -> PVST + STP Conf.  Root = 32768/398/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016
 0.800377 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107?  Tell 31.186.98.1
 1.320775 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len ​​= 528
 1.321507 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 753 Win = 331 Len = 0 TSV = 194287474 TSER = 416768866
 1.322109 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77?  Tell 31.186.98.226
 1.400654 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107?  Tell 31.186.98.1
 1.589797 Cisco_0d: 0d: 96 -> PVST + STP Conf.  Root = 32768/401/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016
 2.100769 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107?  Tell 31.186.98.1
 2.322163 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77?  Tell 31.186.98.226
 2.322764 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len ​​= 720
 2.323594 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 1473 Win = 331 Len = 0 TSV = 194287724 TSER = 416769868
 2.520048 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.64?  Tell 31.186.98.1
 2.635370 Cisco_0d: 0d: 96 -> PVST + STP Conf.  Root = 32768/398/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016
 2.200299 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.64?  Tell 31.186.98.1
 3.451774 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len ​​= 528

As can be seen from the above output, tshark shows information about all the packages in a row, including those that we absolutely do not need at the moment. With the help of special options, you can make it so that only the information we really need is output to the console.

Filters

Select the interface to capture

The -i option allows you to capture traffic only for a specific interface. You can view the list of available network interfaces using the command:

 $ sudo tshark -D

 1. eth0
 2. any
 3. lo (Loopback)
 4. nflog
 5. nfqueue
 6. usbmon1

After the -i option, any of the available interfaces is indicated:

 $ sudo tshark -i eth0

With the help of additional arguments, you can get more specialized information. For example, using the host argument, you can capture packets for a specific IP address:

 $ sudo tshark -i eth0 host 192.168.1.100

Both incoming and outgoing packets will be included in the output. To view information only about incoming packets or only about outgoing packets, the dst and src arguments are used respectively:

 $ sudo tshark -i eth0 src host 192.168.1.100 

The command will display to the console a list of packets originating from 192.168.1.100.

 $ sudo tshark -i eth0 dst host 192.168.1.100 

A list of packets arriving at 192.168.1.100 will be displayed on the console.

Similarly, you can capture traffic for a whole subnet — the net argument is used for this:

 $ sudo tshark -i eth0 src net 192.168.1.0/24

You can also specify the port on which the packets will be captured:

 $ sudo tshark -i eth0 host 192.168.1.1 and port 80

Tshark allows you to capture packets for a certain period of time:

 $ sudo tshark -i eth0 -a duration: 10 -w traffic.pcap

In the example above, the -w option was also used. After it, the path to the file in which the received data will be written is indicated.

Capture filters

These filters are used when capturing traffic on the fly. They are recompiled into a set of rules for pcap, according to which packet filtering is performed. Only the information that meets the criteria set using filters is output to the console.

In general, the syntax of the tshark command with filters can be represented as follows:

 $ sudo tshark -i <interface> -f <filter>

In tshark, capture filters use the same syntax as tcpdump. Within the framework of this article we will not consider all existing filters in detail and will limit ourselves to referring to the official manual ).

Reading filters

Tshark can save captured packet information in files. To extract the necessary information from these files, read filters are used, also called rules (option -R). They can also be used when capturing packets on the fly. Information processing is carried out not by pcap, but by means of tshark itself.

These filters provide much wider opportunities for the selection and specification of information.

However, it should be borne in mind that when analyzing large amounts of information on the fly, they may not cope with the tasks assigned to them: they do not have time to filter and drop packets.

In general, the syntax of the rules can be represented as follows:

 $ sudo tshark -R "rule" -r "path to file"

Consider the features of the formation of rules with specific examples.
So the team

 $ sudo tshark -R "ip.addr == 192.168.0.1" -r /tmp/capture.cap

indicates that the file /tmp/capture.cap needs to extract information about outgoing and incoming packets for the IP address 192.168.0.1.

The following rule will indicate that from this file you need to extract information about incoming and outgoing packets for all IP addresses except 192.168.0.1:

 $ sudo tshark -R "! (ip.addr == 192.168.0.1)" -r /tmp/capture.cap 

Similarly, you can set the rules for other protocols and ports (filters eth.addr, udp.port, tcp.port):

 $ sudo tshark -R "eth.addr == 00: 08: 15: 00: 08: 15" -r /tmp/capture.cap

 $ sudo tshark -R "udp.port == 80" -r /tmp/capture.cap

 $ sudo tshark -R "tcp.port == 80" -r /tmp/capture.cap

Rules can also be combined using the logical operators and, or and not:

 $ sudo tshark -R "not arp and not (udp.port == 53)" -r /tmp/capture.cap

The above command indicates that you need to extract a list of all captured packets, except for ARP and DNS (port 53).

Additional settings

Statistics collection

With the -z option, you can collect and display various static information about packages on the console.

For example, the command

 $ sudo tshark -z "proto, colinfo, tcp.srcport, tcp.srcport" -r /tmp/capture.cap

indicates that the source port of all packages should be extracted from the /tmp/capture.cap file.

Consider another example:

 $ sudo tshark -R "http.response and http.content_type contains image" \
 -z "proto, colinfo, http.content_length, http.content_length" \
 -z "proto, colinfo, http.content_type, http.content_type" \ -r /tmp/capture.cap

This command will extract information about all the packages containing images from the /tmp/capture.cap file and output the contents of the content_type and content_length fields to the console:

 439 12.717117 66.249.89.127 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 35
 452 12.828186 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 477
 479 13.046184 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 105
 499 13.075361 203.190.124.6 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 35
 506 13.177414 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 4039
 514 13.190000 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (JPEG JFIF image) http.content_type == "image / jpeg" http.content_length == 11997
 519 13.231228 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (JPEG JFIF image) http.content_type == "image / jpeg" http.content_length == 1033
 523 13.273888 72.233.69.4 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (PNG) http.content_type == "image / png" http.content_length == 1974
 561 728 19.096984 60.254.185.58 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 592
 805 19.471444 60.254.185.58 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 259

Auto save to multiple files

Imagine that we need to keep traffic statistics for a long period of time. To save all the output in one file in such a situation is not very convenient: it is then difficult to analyze.

The output can be saved in several files, the number and size of which are specified by the user. As soon as one file is full, tshark will continue writing to the next. For example, the command:

 $ sudo tshark -b filesize: 100 -a files: 20 -w temp.pcap

will save the output in 20 files of 100 KB each.

In the example above, the -b option means that the ring buffer will be enabled, and filesize sets the size; The -a option indicates an automatic stop upon reaching the specified limit, files — indicates the number of files.

Autosave after a specified time

In the example below, tshark will save the captured information into several files. A new file will be created automatically when the size exceeds 10240 KB or after the interval of 1 s:

 $ sudo tshark -b filesize: 10240 -b duration: 1 -w temp.pcap 
 
 Capturing on eth0 
 34 
 # ls -lrt 
 -rw ------- 1 root 1863 Apr 10 16:13 temp_00001_20140410161312.pcap 
 -rw ------- 1 root 1357 Apr 10 16:13 temp_00002_20140410161313.pcap 
 -rw ------- 1 root 1476 Apr 10 16:13 temp_00003_20140410161314.pcap 
 -rw ------- 1 root root 1216 Apr 10 16:13 temp_00004_20140410161315.pcap

Set Buffer Size

This option can be useful in cases where you have to deal with packet dropping. The default buffer size is 1MB; With the -B option, you can set any other size (in megabytes), after which all data will be flushed to disk:

 $ sudo tshark -B 2

Display statistics for the selected protocol

In tshark, it is also possible to capture only packets transmitted via a protocol specified by the user.

This is, for example, the statistics for the HTTP protocol:

 $ sudo tshark -q -r a.pcap -R http -z http, tree 

 ================================================= ================= 
 HTTP / Packet Counter 
 -------------------------------------------------- ----------------- 
 Total HTTP Packets 7 0.000375 
 HTTP Request Packets 4 0.000214 57.14% 
 GET 4 0.000214 100.00% 
 HTTP Response Packets 3 0.000161 42.86% 
 2xx: Success 2 0.000107 66.67% 
 200 OK 2 0.000107 100.00% 
 3xx: Redirection 1 0.000054 33.33% 
 302 Found 1 0.000054 100.00% 
 5xx: Server Error 0 0.000000 0.00% 
 Other HTTP Packets 0 0.000000 0.00%

Practical examples

In this section, we will look at how tshark can be used for everyday administrative tasks.

Monitoring http requests

To display a list of http requests to the server, use the following command:

 $ sudo tshark 'tcp port 80 and (((ip [2: 2] - ((ip [0] & 0xf) <> 2))! = 0)' -R 'http.request.method == "GET" | | http.request.method == "HEAD" '

The output of this command will look like this:

 190.302141 192.168.0.199 -> 74.125.77.104 HTTP GET / HTTP / 1.1
 190.331454 192.168.0.199 -> 74.125.77.104 HTTP GET /intl/en_com/images/srpr/logo1w.png HTTP / 1.1
 190.353211 192.168.0.199 -> 74.125.77.104 HTTP GET /images/srpr/nav_logo13.png HTTP / 1.1
 190.400350 192.168.0.199 -> 74.125.77.100 HTTP GET / generate_204 HTTP / 1.1

The following command will print to the console a list of 10 URLs from which http requests come:

 $ tshark -r sample1.cap -R http.request -T fields -e http.host -e http.request.uri |
 sed -e 's /?.*$//' |  sed -e 's # ^ (. *) t (. *) $ # http: // 12 #' |  sort |  uniq -c |  sort -rn |  head

View a list of HTTP headers

To view the list of http headers for server requests, use the command:

 $ tshark -r sample1.cap -R http.request -T fields -e http.host -e http.request.uri |
 sed -e 's /?.*$//' |  sed -e 's # ^ (. *) t (. *) $ # http: // 12 #' |  sort |  uniq -c |  sort -rn |  head

Accordingly, the list of headers for http responses can be obtained as follows:

 $ sudo tshark tcp port 80 or tcp port 443 -V -R "http.request"

To include the headers of both requests and responses in the list, the following command is used:

 $ sudo tshark "tcp port 80 or tcp port 443" -V -R "http.request || http.response"

View a list of files of a specific type.

With tshark, you can view lists of files of a specific type transmitted via the http protocol. So, for example, you can view a list of GIF images:

 $ sudo tshark -R "http.response and http.content_type contains image" \
 -z "proto, colinfo, http.content_length, http.content_length" \
 -z "proto, colinfo, http.content_type, http.content_type" \
 -r /tmp/capture.tmp |  grep "image / gif" |  wc -l

Monitoring MySQL queries

You can track real-time queries in the MySQL database using the following command:

 $ sudo tshark -i eth0 -a duration: 60 -d tcp.port == 3306, mysql -T fields -e mysql.query 'port 3306 ′

With the help of tshark, you can write to the log information about all requests to MySQL. Start capturing all MySQL traffic using tcpdump:

 $ sudo tcpdump -i eth0 port 3306 -s 1500 -w tcpdump.out

From the received file with the help of tshark we will extract the list of requests:

 $ sudo tshark -r tcpdump.out -d tcp.port == 3306, mysql -T fields -e mysql.query> query_log.out

Delete empty lines from this list and save the edited version in a new file:

 $ sudo cat query_log.out |  grep -v "^ $" |  grep -v "^ commit" |  grep -v "^ SET autocommit" |  grep -v "^ rollback"> query_log_no_blank.out

Conclusion

Tshark is a tool with very broad capabilities that can hardly be described in detail in a single article. We will be happy to answer all questions in the comments. We will be glad if you share your own experience in diagnosing network problems with tshark.

Readers who can not leave comments here are invited to our blog .