Fast transit traffic analysis

Before any system administrator, sooner or later there is a problem of quantitative analysis of the traffic (from / to, through which protocols / ports, in what volumes, etc.) passing through its network. It is especially unpleasant when this task arises spontaneously, as a side effect of DDoS, and, as usual, there is no money for serious solutions from Cisco or Arbor. And it's good if the gateway for the network is a server on which you can run tcpdump or wireshark, but what to do if:

• the gateway is the provider's device, and the network has only a file server;
• traffic data is not always needed, but from time to time;
• the device does not support the ability to run third-party programs on it;
• there is so much traffic that the server after running tcpdump “glues flippers”;
• or, on the contrary, is it so small that its level is comparable to the share (albeit significant) of normal traffic?

An additional spoon of tar in this “barrel of honey” of the task is also added by the absence of both the tcpdump and tshark miraculous keys “group, sum up / average and sort”.

So, with all the reluctance to reinvent the wheel, I had to roll up our sleeves and write a tool that meets the following requirements:
')

• data source is a router or switch that supports sFlow protocol;
• collector (tcpdump, tshark or sflowtool) data in PCAP format either writes to a file or sends to STDOUT;
• accordingly, the source data can be read by the tool either from the file or from STDIN;
• The base unit for analysis is the packet, not the compound;
• the result should include information about the direction of traffic, the number of past packets, the amount of traffic and the average packet size;
• the possibility of basic grouping and sorting of the result should be provided;
• well, and various passing trivia;
• and with all this, this tool should not duplicate the functionality of existing well-known tools.

It was on this basis that PCAParse was written - as simple as possible tool for obtaining generalized information about traffic passing through the network. To use it, in the simplest case, a D-Link DGS-3XXX switch or equivalent from other manufacturers is enough and running on the aforementioned sflowtool or tcpdump file server on the office gateway. As practice shows, these devices have long ago lost their exotic status and can be found even in small offices.

In order to make it clear what is at stake, I will give a small example:


Of course, an awkward working time immediately catches the eye, but in reality it is the result of tshark, and not a script. Its real performance on AMD A8-6600K @ 3.9 GHz / 8 GB RAM is 15-25 kilopackets / s depending on the data source (read from file, sFlow, etc.).

More demanding users may request more detailed information:


If we take into account that the dump for the example is taken for a regular web server, this information allows you to judge whether there is a DoS / DDoS attack by udp: * on the resource. Well, this knowledge allows us to take some adequate measures. For the convenience of further data processing, a parser-friendly output is provided:


The script is written in perl-e using the Net :: PCAP and NetPacket :: * modules, which provides sufficient performance and cross-platform functionality. At least, on fresh linux and FreeBSD, there were no problems with startup and operation.

Of the known cons:

• lack of support for IPv6 (hopefully for now - work is underway on this);
• checking IP ranges using Data :: Validate :: IP (again, hopefully, temporarily);
• the lack of the “do well” option for tcpdump or tshark (but I hope in the future it may appear there if the script is liked by the authors and contributors of the mentioned tools and its functionality will be sorted there).

PS after the fact it turned out that there is a similar tool - Fastnetmon, but, nevertheless, it is focused on "stationary" use, because it implies the use of a data collector, with which the client part interacts.

References:

• PCAParse - github.com/kornix/pcaparse ;
• Fastnetmon - n0where.net/high-performance-dos-analyzer-fastnetmon ;
• DDS - gul-tech.livejournal.com/5959.html ;
• NFDUMP - nfdump.sourceforge.net ;
• NfSen - nfsen.sourceforge.net