Tester confession: how I rummaged in a competitor's IDS
No self-respecting UTM network security gateway can do without an Intrusion Detection System (IDS) intrusion detection system. Another thing is that the option is often specified by the manufacturer for a tick in order to keep up with competitors. I know from my own experience as a tester, as it happens - there seems to be IDS, but in fact it is no good. That is why, when I was offered to test a relatively inexpensive UTM “piece of hardware,” I suggested that we “run” its IDS first.
UTM hardware Traffic Inspector Next Generation S100 and switch Cisco 2960G
If you have experience with IDS, it is interesting to read about it in the comments to the article. It is desirable - not about the expensive "pieces of iron" (of course, that some Cisco NGFW for a million is fine), but about solutions that are more affordable. I think that local administrators and potential users of network gateways will be curious to discuss with whom how IDS works, and whether it is worth buying at a premium when you can, having danced with a tambourine, achieve the same for less money.
')
In this test, we are talking about the S100 model - the most affordable in the Traffic Inspector Next Generation line (the figures mean that it is designed for 100 subscribers). Here is its description on the official website >>> . In short, the “piece of hardware” is, for example, a network filter, VPN, resource blocker and, of course, IDS.
I propose a simple testing method - we check the throughput and build the dependence of the number of dropped packets on the traffic intensity in increments of 50, 100, 150 and 200 Mb / s. Why take such intervals? We make a start from 100 Mb / s - the most popular client request, and we postpone plus and minus from it to see what happens in more and less loaded modes, as well as in extreme mode 200 Mb / s.
Life experience tells me that with all the active rules, the S100 can fail to pull, so I suggest that the procedure described be carried out in three modes: first, when all the rules are active, then with part of the rules off (let's call it Group No. 1) and, finally, only when active just a few rules (let's call it Group number 2). We form such groups of rules:
- Group number 0: well, this is understandable, all the rules without exception.
- Group number 1: a group of rules of Emerging Threads (the rules are known to me since the testing of IDS Snort, except for the rules of emerging-games).
- Group number 2: a group of rules that I consider just mandatory for IDS (see below), including the addition of p2p rules, because from my own experience I know how large companies have a negative attitude to the fact that their employees actively use corporate network to download your favorite TV shows.
Testing is feasible using the tcpreplay utility . This utility allows you to play pre-recorded network traffic at a certain speed. Command: tcpreplay –i <interface> -l 0 testTI.pcap . The testTI.pcap file contains 1 146 313 packages (we choose this size so that, on the one hand, there is good statistics, on the other - the time for running will not be too long, in our case - no more than 15 minutes). In addition, as I said, we launch a torrent tracker.
The scheme of the test stand:
If someone has questions on the methodology of testing - ready to answer in the comments.
So, the results.
Group number 0
Testing on the full set implies loading all the rules, and 30 305 of them.
When testing, use the default IDS settings:
We start testing with 100 Mb / s and we understand that the piece of iron barely pulls: from 114 thousand packages 109 thousand are discarded! So there is no point in testing for 150, and even more so for 200 Mb / s. On the contrary, I suggest giving the device a chance and conducting an additional test at 10 Mb / s. The result in the table:
Note:
kernel_packets — sent packets;
kernel_drops - dropped packets. As you can see, with the default settings and with the full set of rules, large packet loss occurs (> 30% relative to kernel_packets). Let's hope that the optimization of the settings rules will change the situation;
decoder_packets - the number of packets that the system correctly processed;
detect_alerts - the number of detected attacks. The bulk of the attacks are made up of the “Fragmented Packet” type, but the “Torrent Tracker Identification” attacks were also identified.
Group number 1
Obviously, the piece of hardware is not optimized to work as a full-fledged powerful IDS, but there is room for maneuver, namely, the ability to change the route search mechanism, disable loading the package contents into the reports (payload), and also disable rule groups and certain specific groups of packages. Few experiments with the settings - and we come to the option that I present in the next figure.
The list of active rules that leave:
Test result:
As we see, the picture has significantly improved with this group of rules (the percentage of discarded packets has decreased several times). An attack of the Torrent Tracker Identification type was successfully detected (the Fragmented Packet attacks no longer appeared).
Group number 2
The settings are the same. List of active rules:
The result in the table:
Here the packet processing picture is even better than expected.
Summary
The final results table, expressed as a percentage of kernel_drops relative to kernel_packets, is shown below:
Graphically, the result is as follows:
As you can see, the number of active rules and settings directly affect efficiency. Turning on the settings to the maximum and all the rules at the same time does not make any sense - the loss immediately goes even 10 MB / s. In optimized modes, the “hardware” normally feels up to 100 Mb / s, but on more intensive traffic, the losses increase dramatically. However, for the “office” use of 100 Mb / s is enough. If you drive a device at this speed and select rules, you can achieve quite satisfactory IDS work.
It is possible that in order to use the full set of rules, improvements are needed on using the pf_ring mechanism (https://www.ntop.org/products/packet-capture/pf_ring/) as a mechanism for transferring a package to userspace from the network interface buffer. To do this, you will have to use several copies of Suricata, which, naturally, will select resources from other processes, but perhaps the game will be worth the candle.
I repeat that, in my opinion, the main purpose of the device under test is the firewall, and the IDS option is auxiliary. Honestly, I was ready for the fact that the "piece of iron" test fails. It turned out that with a certain system administrator's gutta perception, the S100 intrusion detection system is fully operational.
PS As mentioned above, I urge readers to write about their experience in using IDS in relatively inexpensive solutions.
PPS The test is laid out in the manufacturer's account, because I do not want to “shine” myself. Nevertheless, I am ready to answer all the questions and discuss them in the comments, but, again, not on my own behalf :)
UTM hardware Traffic Inspector Next Generation S100 and switch Cisco 2960G
If you have experience with IDS, it is interesting to read about it in the comments to the article. It is desirable - not about the expensive "pieces of iron" (of course, that some Cisco NGFW for a million is fine), but about solutions that are more affordable. I think that local administrators and potential users of network gateways will be curious to discuss with whom how IDS works, and whether it is worth buying at a premium when you can, having danced with a tambourine, achieve the same for less money.
')
In this test, we are talking about the S100 model - the most affordable in the Traffic Inspector Next Generation line (the figures mean that it is designed for 100 subscribers). Here is its description on the official website >>> . In short, the “piece of hardware” is, for example, a network filter, VPN, resource blocker and, of course, IDS.
I propose a simple testing method - we check the throughput and build the dependence of the number of dropped packets on the traffic intensity in increments of 50, 100, 150 and 200 Mb / s. Why take such intervals? We make a start from 100 Mb / s - the most popular client request, and we postpone plus and minus from it to see what happens in more and less loaded modes, as well as in extreme mode 200 Mb / s.
Life experience tells me that with all the active rules, the S100 can fail to pull, so I suggest that the procedure described be carried out in three modes: first, when all the rules are active, then with part of the rules off (let's call it Group No. 1) and, finally, only when active just a few rules (let's call it Group number 2). We form such groups of rules:
- Group number 0: well, this is understandable, all the rules without exception.
- Group number 1: a group of rules of Emerging Threads (the rules are known to me since the testing of IDS Snort, except for the rules of emerging-games).
- Group number 2: a group of rules that I consider just mandatory for IDS (see below), including the addition of p2p rules, because from my own experience I know how large companies have a negative attitude to the fact that their employees actively use corporate network to download your favorite TV shows.
Testing is feasible using the tcpreplay utility . This utility allows you to play pre-recorded network traffic at a certain speed. Command: tcpreplay –i <interface> -l 0 testTI.pcap . The testTI.pcap file contains 1 146 313 packages (we choose this size so that, on the one hand, there is good statistics, on the other - the time for running will not be too long, in our case - no more than 15 minutes). In addition, as I said, we launch a torrent tracker.
The scheme of the test stand:
If someone has questions on the methodology of testing - ready to answer in the comments.
So, the results.
Group number 0
Testing on the full set implies loading all the rules, and 30 305 of them.
When testing, use the default IDS settings:
We start testing with 100 Mb / s and we understand that the piece of iron barely pulls: from 114 thousand packages 109 thousand are discarded! So there is no point in testing for 150, and even more so for 200 Mb / s. On the contrary, I suggest giving the device a chance and conducting an additional test at 10 Mb / s. The result in the table:
Note:
kernel_packets — sent packets;
kernel_drops - dropped packets. As you can see, with the default settings and with the full set of rules, large packet loss occurs (> 30% relative to kernel_packets). Let's hope that the optimization of the settings rules will change the situation;
decoder_packets - the number of packets that the system correctly processed;
detect_alerts - the number of detected attacks. The bulk of the attacks are made up of the “Fragmented Packet” type, but the “Torrent Tracker Identification” attacks were also identified.
Group number 1
Obviously, the piece of hardware is not optimized to work as a full-fledged powerful IDS, but there is room for maneuver, namely, the ability to change the route search mechanism, disable loading the package contents into the reports (payload), and also disable rule groups and certain specific groups of packages. Few experiments with the settings - and we come to the option that I present in the next figure.
The list of active rules that leave:
Test result:
As we see, the picture has significantly improved with this group of rules (the percentage of discarded packets has decreased several times). An attack of the Torrent Tracker Identification type was successfully detected (the Fragmented Packet attacks no longer appeared).
Group number 2
The settings are the same. List of active rules:
The result in the table:
Here the packet processing picture is even better than expected.
Summary
The final results table, expressed as a percentage of kernel_drops relative to kernel_packets, is shown below:
Graphically, the result is as follows:
As you can see, the number of active rules and settings directly affect efficiency. Turning on the settings to the maximum and all the rules at the same time does not make any sense - the loss immediately goes even 10 MB / s. In optimized modes, the “hardware” normally feels up to 100 Mb / s, but on more intensive traffic, the losses increase dramatically. However, for the “office” use of 100 Mb / s is enough. If you drive a device at this speed and select rules, you can achieve quite satisfactory IDS work.
It is possible that in order to use the full set of rules, improvements are needed on using the pf_ring mechanism (https://www.ntop.org/products/packet-capture/pf_ring/) as a mechanism for transferring a package to userspace from the network interface buffer. To do this, you will have to use several copies of Suricata, which, naturally, will select resources from other processes, but perhaps the game will be worth the candle.
I repeat that, in my opinion, the main purpose of the device under test is the firewall, and the IDS option is auxiliary. Honestly, I was ready for the fact that the "piece of iron" test fails. It turned out that with a certain system administrator's gutta perception, the S100 intrusion detection system is fully operational.
PS As mentioned above, I urge readers to write about their experience in using IDS in relatively inexpensive solutions.
PPS The test is laid out in the manufacturer's account, because I do not want to “shine” myself. Nevertheless, I am ready to answer all the questions and discuss them in the comments, but, again, not on my own behalf :)
Source: https://habr.com/ru/post/421893/
All Articles