Deep Packet Inspection: Equipment and Application
We in VAS Experts design and implement services in the field of monitoring and analyzing Internet traffic. In our blog, we decided to start talking about the device technologies related to our field of activity.
In the first two materials we talked about the composition of the system and the wiring diagram, and in the second about usage scenarios . Today we will look at what requirements are imposed on the hardware for the full operation of the in-depth traffic analysis systems, and where such systems are applied.
/ Flickr / Sean MacEntee / CC
Deep packet inspection (DPI) systems are most often used to monitor and filter traffic, sometimes to block protocols. Using DPI, you can track data generated by applications and choose the appropriate action strategy.
')
DPI requires a software system installed on a suitable hardware platform. Only properly selected equipment and optimized software in conjunction will give a high level of performance.
The servers for DPI are usually similar to 1U servers, but they focus on network components, not RAM and hard drives. DPI servers can have 4 to 8 ports of 1 GbE RJ45, 4 ports of 10 GbE SFP + or 2 ports 40 GE QSFP +.
Network cards in the DPI-system must support Bypass mode - if the server turns off, the connection between the ports will continue to work using power from the built-in battery and will pass traffic without filtering. Also on the DPI-server is usually installed a system for monitoring the status of work (Advanced Lights Out Management), with which you can manage all parameters remotely or through a graphical interface on the display. The server's motherboard BIOS must be hardware protected from damage and support remote updates.
One or two Intel Xeon E5-2600 V4 (Broadwell-EP) processors and two hard disks combined in RAID 1 are enough for the server to function. Usually the main device has two SSD disks to install the operating system, for example, SmartOS, they are added to 24 HDD or SSD drives, as well as network ports and ports for expansion shelves. To increase the data storage, in addition to the device, JBOD disk arrays are connected, each of which supports up to 70 HDD or SSD disks.
This model allows you to quickly and at low cost to expand the volume of data storage, which is especially important when processing traffic statistics and content caching - images, video and other similar files.
It is best to use ZFS file system and RAID-Z technology to control disks, maintain their integrity and high speed. To ensure fault tolerance, it is desirable to install at least two interchangeable power supplies.
Often, manufacturers of traffic analysis systems offer ready-made kits of hardware and software — such servers are different from standard systems and can be additionally equipped with data storage systems or statistics collection in accordance with legal requirements.
On the other hand, it can be much more convenient if the proposed server is a device on a standard platform that can be easily upgraded. Such systems are more common in Russian companies - Protey, Vas Experts, Peter-Service, Napa Labs.
The company Heavy Reading conducted a survey among telecom companies and collected data on the main areas of application of DPI. The most popular direction has been quality of service (QoS) - this includes monitoring network status and solving equipment problems.
Previously, home subscriber traffic (HSI - High Speed Internet) was hardly controlled - BitTorrent could take the entire free connection bandwidth, but now DPI allows operators to distribute the channel between different applications.
The second most important category for operators was the network subscriber policy management (PCEF). According to research by analysts, this is the largest category of application of DPI in terms of volume and value, and it will only continue to grow. The third most important category providers called network gateways - today DPI is often used on routers in 4G (P-GW) and 3G networks (GGSN).
The fourth category in the survey is the use of DPI to analyze user information. Initially, DPI was used to analyze network traffic and trends, but increasingly, companies are connecting this technology to analyzing the behavior of subscribers in real time, in order to develop more suitable sets of services and at the same time select the right load on the network.
According to experts from the University of California at Berkeley, for a fast and reliable DPI operation, the equipment must have certain properties. DPI is used to check network packets by thousands of identifiers, so for high speed it is necessary to divide this process into parallel streams, that is, the equipment must support parallelism in data processing.
The hardware must process network packets at high speed to match the bandwidth of a gigabit network channel. In addition, the equipment should not consume a lot of energy - overheating means slowing down the work, breakdown or a large load on the cooling system.
Also, the hardware must be flexibly customizable and provide the possibility of future expansion of functions and quick updates, for example, to protect against new viruses.
At Vas Experts, we compared the primary and middle class devices of both domestic and foreign manufacturers, which almost all operators can afford. Based on the data we collected, we can conclude that the most expensive low-end systems were Cisco SCE1000 and Huawei SIG9800-X3, the latter having the highest performance.
As for the middle segment, here we have observed almost complete equality in performance and equipment devices. The only difference is that Russian systems are based on standard components, and this allows you to increase performance with much lower costs, because additional extensions for devices of foreign companies Sandvine, Allot or Procera are much more expensive and require license extensions (you can find a full analysis in our blog ).
Based on this, we can conclude that Russian manufacturers can compete with international companies, but their solutions should be used in networks with low traffic volumes and a small number of subscribers. Otherwise, you should pay attention to closed platforms of foreign developers, which have a little more reliability, stability and optimization.
Unfortunately, a comparison of the specifications specified in the specification will not give a complete understanding of the practical benefits, so it is important to check the equipment on test specimens from manufacturers. However, it is not always possible to get devices for testing, in this case, you need to focus on the experience of people who are already working with specific systems and technologies, look for information on forums and technical support specialists.
Also, when choosing a system, integration costs are of great importance (in addition to performance and functionality) - for many it becomes a decisive factor for the installation of a DPI system. Let us give an example of calculating the effectiveness of the implementation of such a system (ROI).
Assume that the total volume of traffic consumed by subscribers is 20 Gbit per month, the cost of 1 Gbit is $ 3,000, and the cost of the DPI system is $ 75,000. Then the use of the DPI system at the claimed efficiency of 35% reduction in the uplink will give 7 Gbit / s bandwidth savings per month - that’s $ 21,000 per month. In this case, the DPI-system will pay off in 3.6 months (75000/21000 = about 3.6).
If we take into account that adding additional functions (blocking prohibited sites, protection against viruses and attacks, CGNAT) will not have to be paid, the implementation of DPI will pay off even faster, and customers will receive higher quality services for the same price. It is worth noting that Russian companies sell equipment at prices in rubles - this is a serious advantage for operators who receive their income in the same currency. Buying and maintaining the system in this case will be even more profitable.
In the first two materials we talked about the composition of the system and the wiring diagram, and in the second about usage scenarios . Today we will look at what requirements are imposed on the hardware for the full operation of the in-depth traffic analysis systems, and where such systems are applied.
/ Flickr / Sean MacEntee / CC
Deep packet inspection (DPI) systems are most often used to monitor and filter traffic, sometimes to block protocols. Using DPI, you can track data generated by applications and choose the appropriate action strategy.
')
DPI requires a software system installed on a suitable hardware platform. Only properly selected equipment and optimized software in conjunction will give a high level of performance.
Servers for DPI
The servers for DPI are usually similar to 1U servers, but they focus on network components, not RAM and hard drives. DPI servers can have 4 to 8 ports of 1 GbE RJ45, 4 ports of 10 GbE SFP + or 2 ports 40 GE QSFP +.
Network cards in the DPI-system must support Bypass mode - if the server turns off, the connection between the ports will continue to work using power from the built-in battery and will pass traffic without filtering. Also on the DPI-server is usually installed a system for monitoring the status of work (Advanced Lights Out Management), with which you can manage all parameters remotely or through a graphical interface on the display. The server's motherboard BIOS must be hardware protected from damage and support remote updates.
One or two Intel Xeon E5-2600 V4 (Broadwell-EP) processors and two hard disks combined in RAID 1 are enough for the server to function. Usually the main device has two SSD disks to install the operating system, for example, SmartOS, they are added to 24 HDD or SSD drives, as well as network ports and ports for expansion shelves. To increase the data storage, in addition to the device, JBOD disk arrays are connected, each of which supports up to 70 HDD or SSD disks.
This model allows you to quickly and at low cost to expand the volume of data storage, which is especially important when processing traffic statistics and content caching - images, video and other similar files.
It is best to use ZFS file system and RAID-Z technology to control disks, maintain their integrity and high speed. To ensure fault tolerance, it is desirable to install at least two interchangeable power supplies.
Often, manufacturers of traffic analysis systems offer ready-made kits of hardware and software — such servers are different from standard systems and can be additionally equipped with data storage systems or statistics collection in accordance with legal requirements.
On the other hand, it can be much more convenient if the proposed server is a device on a standard platform that can be easily upgraded. Such systems are more common in Russian companies - Protey, Vas Experts, Peter-Service, Napa Labs.
Categories of use
The company Heavy Reading conducted a survey among telecom companies and collected data on the main areas of application of DPI. The most popular direction has been quality of service (QoS) - this includes monitoring network status and solving equipment problems.
Previously, home subscriber traffic (HSI - High Speed Internet) was hardly controlled - BitTorrent could take the entire free connection bandwidth, but now DPI allows operators to distribute the channel between different applications.
The second most important category for operators was the network subscriber policy management (PCEF). According to research by analysts, this is the largest category of application of DPI in terms of volume and value, and it will only continue to grow. The third most important category providers called network gateways - today DPI is often used on routers in 4G (P-GW) and 3G networks (GGSN).
The fourth category in the survey is the use of DPI to analyze user information. Initially, DPI was used to analyze network traffic and trends, but increasingly, companies are connecting this technology to analyzing the behavior of subscribers in real time, in order to develop more suitable sets of services and at the same time select the right load on the network.
Properties and comparison of "iron"
According to experts from the University of California at Berkeley, for a fast and reliable DPI operation, the equipment must have certain properties. DPI is used to check network packets by thousands of identifiers, so for high speed it is necessary to divide this process into parallel streams, that is, the equipment must support parallelism in data processing.
The hardware must process network packets at high speed to match the bandwidth of a gigabit network channel. In addition, the equipment should not consume a lot of energy - overheating means slowing down the work, breakdown or a large load on the cooling system.
Also, the hardware must be flexibly customizable and provide the possibility of future expansion of functions and quick updates, for example, to protect against new viruses.
At Vas Experts, we compared the primary and middle class devices of both domestic and foreign manufacturers, which almost all operators can afford. Based on the data we collected, we can conclude that the most expensive low-end systems were Cisco SCE1000 and Huawei SIG9800-X3, the latter having the highest performance.
As for the middle segment, here we have observed almost complete equality in performance and equipment devices. The only difference is that Russian systems are based on standard components, and this allows you to increase performance with much lower costs, because additional extensions for devices of foreign companies Sandvine, Allot or Procera are much more expensive and require license extensions (you can find a full analysis in our blog ).
Based on this, we can conclude that Russian manufacturers can compete with international companies, but their solutions should be used in networks with low traffic volumes and a small number of subscribers. Otherwise, you should pay attention to closed platforms of foreign developers, which have a little more reliability, stability and optimization.
Practical benefits
Unfortunately, a comparison of the specifications specified in the specification will not give a complete understanding of the practical benefits, so it is important to check the equipment on test specimens from manufacturers. However, it is not always possible to get devices for testing, in this case, you need to focus on the experience of people who are already working with specific systems and technologies, look for information on forums and technical support specialists.
Also, when choosing a system, integration costs are of great importance (in addition to performance and functionality) - for many it becomes a decisive factor for the installation of a DPI system. Let us give an example of calculating the effectiveness of the implementation of such a system (ROI).
Assume that the total volume of traffic consumed by subscribers is 20 Gbit per month, the cost of 1 Gbit is $ 3,000, and the cost of the DPI system is $ 75,000. Then the use of the DPI system at the claimed efficiency of 35% reduction in the uplink will give 7 Gbit / s bandwidth savings per month - that’s $ 21,000 per month. In this case, the DPI-system will pay off in 3.6 months (75000/21000 = about 3.6).
If we take into account that adding additional functions (blocking prohibited sites, protection against viruses and attacks, CGNAT) will not have to be paid, the implementation of DPI will pay off even faster, and customers will receive higher quality services for the same price. It is worth noting that Russian companies sell equipment at prices in rubles - this is a serious advantage for operators who receive their income in the same currency. Buying and maintaining the system in this case will be even more profitable.
Source: https://habr.com/ru/post/313558/
All Articles