Shaper for Linux in user space (NFQUEUE-based)
The times of narrow Internet channels are gradually becoming a thing of the past, but sometimes it is also necessary to shape network traffic. In Linux, there are corresponding kernel mechanisms and utilities for controlling mechanisms for this. All this economy is quite complicated, usually the comprehension of shaping takes more than one day. Although, in simple cases, you can accumulate tc spells from articles or find the script that generates these spells.
As an inquisitive person, it was always interesting, is it possible to make the process of setting up shaping for smaller networks easier? Is it possible to at least roughly detect the important traffic and separate it from the unimportant without DPI and signature analysis? Is it possible to shape traffic in any direction without creating pseudo-interfaces or adding modules to the kernel? And so, after some thinking and googling, I decided to write a simple shaper in userspace. To try to answer questions by experiment.
As a result of the experiment, such a thing turned out to be github.com/vmxdev/damper
It works like this:
')
At startup, two program threads are created. The first NFQUEUE captures packets, they are analyzed, each packet is assigned a “weight” (or priority) and it is stored in a priority queue. When the queue is full, low priority packets are overwritten by high priority ones. Another thread selects the packets with the highest weight and marks them for shipment. Sending takes place at a limited speed, because of this, the actual shaping is obtained.
This mechanism allows you to copy network packets into user space for processing. An application that listens to the appropriate queue must render a verdict regarding the packet (skip or discard). In addition, it is possible to change the package. This mechanism is used by IDS / IPS type Snort or Suricata. Packages are marked for processing in iptables, the goal of NFQUEUE. That is, we can choose any direction (incoming traffic, outgoing, transit, or, say, UDP from port 666 to port 13) and send it to the shaper. There the packets will be analyzed, it is possible to change their order, and when the limit is exceeded, the lowest priority ones will be discarded.
The first working versions of the shaper captured packets, placed them in a queue, and then re-injected them into a raw socket. On StackOverflow and some other articles they write that this is the only way to delay packets before re-sending. The construction worked, but Comrade Vel explained where the package could be retained right in NFQUEUE, the shaper's setup was simplified.
Since the shaper is experimental, I made it not monolithic, but “modular”. Each module calculates the weight of the package according to different criteria. There are 4 modules out of the box, but you can easily write another one. Modules can be used together, you can separately.
Modules:
Packet weights, measured in each module, are multiplied by a factor (you can specify your own for each module) and add. It turns out the resulting weight.
Shaper can keep statistics and draw graphs. Live it looks like this: damper.xenoeye.com . Green — how many bytes / packets are missing; red — how many are dropped. Schedule you can reason / scroll.
The second graph (included by the “wchart yes” directive in the config) is the average packet weights per second, normalized, broken down by module.
The demo works on not-so-fast ARMs (Scaleway bare metal, 32-bit ARMv7), sometimes it may stick slightly.
The modules inhibit_big_flows and entropy have a debug mode that is included in the config. In this mode, modules every N seconds do a dump of current flows with weights.
In addition, there is a mode without speed limit ("limit no" in the config). In this mode, all packets are skipped (without analysis in the modules), but you can keep statistics of past packets / bytes, meditate on the channel load schedule, for example.
Shaper turned out quite simple (well, in my, zamylenny, look). To use, you need to choose the direction in which to shape, add an iptables rule, set the desired speed in the config and run.
Heavy and high-entropy sessions are defined and reduced in priority, but there are no miracles: if the channel is very narrow, comfortable surfing will not work.
At high speeds and a large number of users, I did not test it, but on tens of megabits and with several users it is subjectively better than without a shaper. Although it loads the CPU, but not only from the use of NFQUEUE, but also from the general code clumsiness (and a little from the features of clock_nanosleep ()), it is possible to optimize and optimize.
This, of course, is only proof of concepts, the code is muddled in places and almost never combed.
If someone has ideas, wishes and suggestions about it, it would be interesting to read.
As an inquisitive person, it was always interesting, is it possible to make the process of setting up shaping for smaller networks easier? Is it possible to at least roughly detect the important traffic and separate it from the unimportant without DPI and signature analysis? Is it possible to shape traffic in any direction without creating pseudo-interfaces or adding modules to the kernel? And so, after some thinking and googling, I decided to write a simple shaper in userspace. To try to answer questions by experiment.
As a result of the experiment, such a thing turned out to be github.com/vmxdev/damper
It works like this:
')
At startup, two program threads are created. The first NFQUEUE captures packets, they are analyzed, each packet is assigned a “weight” (or priority) and it is stored in a priority queue. When the queue is full, low priority packets are overwritten by high priority ones. Another thread selects the packets with the highest weight and marks them for shipment. Sending takes place at a limited speed, because of this, the actual shaping is obtained.
A small digression about the mechanism NFQUEUE
This mechanism allows you to copy network packets into user space for processing. An application that listens to the appropriate queue must render a verdict regarding the packet (skip or discard). In addition, it is possible to change the package. This mechanism is used by IDS / IPS type Snort or Suricata. Packages are marked for processing in iptables, the goal of NFQUEUE. That is, we can choose any direction (incoming traffic, outgoing, transit, or, say, UDP from port 666 to port 13) and send it to the shaper. There the packets will be analyzed, it is possible to change their order, and when the limit is exceeded, the lowest priority ones will be discarded.
The first working versions of the shaper captured packets, placed them in a queue, and then re-injected them into a raw socket. On StackOverflow and some other articles they write that this is the only way to delay packets before re-sending. The construction worked, but Comrade Vel explained where the package could be retained right in NFQUEUE, the shaper's setup was simplified.
Modules
Since the shaper is experimental, I made it not monolithic, but “modular”. Each module calculates the weight of the package according to different criteria. There are 4 modules out of the box, but you can easily write another one. Modules can be used together, you can separately.
Modules:
- inhibit_big_flows - suppress large flows. The more bytes transferred between two IP-addresses, the smaller the “weight” of the packet becomes. That is, it increases the likelihood that a packet from a large, heavy session will be dropped. Information is stored in a circular buffer, so that an amnesty comes from time to time (the number of sessions monitored is set in the config file), the flows are replaced with more recent ones.
- bymark - the weight of packages is set by brand. An iptables brand is set according to some criteria, for these packages the coefficient specified in the config will be applied. You can manually raise or lower the priority of some marked traffic
- entropy - the weight is considered depending on the entropy (more precisely, the measure of entropy according to Shannon) flow. The stream is identified by the protocol number and the participant addresses. For TCP / UDP, the source and destination port is also taken into account. The higher the entropy, the less weight. Those. multimedia, encrypted, compressed traffic is more likely to be dropped than the rest.
- And a very primitive random module - it just adds randomness to the process of dropping and reordering packets (if you use only this module, you get the classic RED).
Packet weights, measured in each module, are multiplied by a factor (you can specify your own for each module) and add. It turns out the resulting weight.
Statistics and graphs
Shaper can keep statistics and draw graphs. Live it looks like this: damper.xenoeye.com . Green — how many bytes / packets are missing; red — how many are dropped. Schedule you can reason / scroll.
The second graph (included by the “wchart yes” directive in the config) is the average packet weights per second, normalized, broken down by module.
The demo works on not-so-fast ARMs (Scaleway bare metal, 32-bit ARMv7), sometimes it may stick slightly.
Debugging
The modules inhibit_big_flows and entropy have a debug mode that is included in the config. In this mode, modules every N seconds do a dump of current flows with weights.
In addition, there is a mode without speed limit ("limit no" in the config). In this mode, all packets are skipped (without analysis in the modules), but you can keep statistics of past packets / bytes, meditate on the channel load schedule, for example.
results
Shaper turned out quite simple (well, in my, zamylenny, look). To use, you need to choose the direction in which to shape, add an iptables rule, set the desired speed in the config and run.
Heavy and high-entropy sessions are defined and reduced in priority, but there are no miracles: if the channel is very narrow, comfortable surfing will not work.
At high speeds and a large number of users, I did not test it, but on tens of megabits and with several users it is subjectively better than without a shaper. Although it loads the CPU, but not only from the use of NFQUEUE, but also from the general code clumsiness (and a little from the features of clock_nanosleep ()), it is possible to optimize and optimize.
This, of course, is only proof of concepts, the code is muddled in places and almost never combed.
If someone has ideas, wishes and suggestions about it, it would be interesting to read.
Source: https://habr.com/ru/post/310242/
All Articles