Increase netfilter performance using ipset
iptables - interface to the Linux firewall ( netfilter ). With a large number of iptables rules, the load can be quite high and create problems. In this post I will try to describe what affects the performance of iptables and how to improve it.
For a start: because of what there is a load when a large number of rules? Due to the fact that each network packet processed by the kernel has to be compared across the entire list of rules for each chain. Let me give you a scheme that reflects netfilter chains and the order in which packets pass through them:
The diagram shows through which chains a particular package passes and you can estimate the load. Let's take one simple rule:
')
This rule tells the firewall to drop all packets from the IP address 10.1.1.1 to port 25 of the local machine. I do not specify the table (-t), so the default (-t filter) will be used, which is what we need. It is clear that the DROP action is not always performed. However, a comparison of the source IP address will pass any packet incoming to the local machine. And if there are a thousand such rules?
Now we will think how to speed up the firewall. The first answer is obvious - to reduce the number of rules. But it is unlikely to seriously optimize the records, especially if they were originally compiled correctly. Somewhere instead of the rules for several IP addresses, you can write one rule per subnet. Somewhere instead of the rules for several ports you can use
And there is another option - the use of branching rules. Its essence is that the rules of the same type are grouped into a separate chain, and in the main chain there is one rule that redirects the packet to a separate chain depending on some common attribute of the packets. I will give an example. There are three rules:
What unites these rules? The same protocol is icmp. It is by this criterion and article to group the rules. In general, in this case it is more efficient to use ipset, but more on that later. To group, you need to create a new chain, and then create a rule that will send packets to this chain.
Create PROT_ICMP chain:
Define the rules in this thread:
As you can see, we do not check the protocol (-p) in this chain, because send only ICMP packets to this chain. Finally, send all ICMP packets to this chain:
Now the incoming packet, if it is not ICMP, will pass through only one rule instead of three.
Now suppose the following situation:
Several rules of the same type, but it is not possible to group them or make an additional chain. In such cases, when it is necessary to check on a large number of IP addresses and / or ports, ipset comes to the rescue . ipset is the ip_set kernel module , a number of auxiliary libraries and the ipset utility for setting parameters. Surely there is in the repositories of your distribution, so that the installation should not be a problem.
Used as follows:
* A list is created:
dropips - the name of the list, iphash - the type of the list. The types of lists can be viewed in the man ipset, they are for every taste - to work with ip-addresses, subnets, ports, mac-addresses. iphash is used to store IP addresses, the use of hashing prevents adding to the list of duplicate IP addresses.
* Add an IP address to the list:
* Creates a rule to use the list:
In the article, I did not describe the tuning of network parameters of the kernel, but focused on the rational organization of firewall rules. A lot of information about tuning can be found on the network, these are various net. * Mem *, in the case of a large number of connections and using conntrack, the parameters net.netfilter.nf_conntrack_max, net.netfilter.nf_conntrack * timeout, etc.
Thanks for attention.
Thanks ivlis for pointing out typos.
Main reason for the load
For a start: because of what there is a load when a large number of rules? Due to the fact that each network packet processed by the kernel has to be compared across the entire list of rules for each chain. Let me give you a scheme that reflects netfilter chains and the order in which packets pass through them:
The diagram shows through which chains a particular package passes and you can estimate the load. Let's take one simple rule:
')
-A INPUT -s 10.1.1.1/32 -p tcp --dport 25 -j DROPThis rule tells the firewall to drop all packets from the IP address 10.1.1.1 to port 25 of the local machine. I do not specify the table (-t), so the default (-t filter) will be used, which is what we need. It is clear that the DROP action is not always performed. However, a comparison of the source IP address will pass any packet incoming to the local machine. And if there are a thousand such rules?
What to do?
Now we will think how to speed up the firewall. The first answer is obvious - to reduce the number of rules. But it is unlikely to seriously optimize the records, especially if they were originally compiled correctly. Somewhere instead of the rules for several IP addresses, you can write one rule per subnet. Somewhere instead of the rules for several ports you can use
-m multiport and --dports / --sports a,b,c (a, b, c - port numbers). And so on.Branching rules, additional chains
And there is another option - the use of branching rules. Its essence is that the rules of the same type are grouped into a separate chain, and in the main chain there is one rule that redirects the packet to a separate chain depending on some common attribute of the packets. I will give an example. There are three rules:
-A INPUT -s 10.1.1.1/32 -p icmp -j ACCEPT
-A INPUT -s 10.1.1.2/32 -p icmp -j ACCEPT
-A INPUT -s 10.1.1.3/32 -p icmp -j ACCEPTWhat unites these rules? The same protocol is icmp. It is by this criterion and article to group the rules. In general, in this case it is more efficient to use ipset, but more on that later. To group, you need to create a new chain, and then create a rule that will send packets to this chain.
Create PROT_ICMP chain:
iptables -N PROT_ICMPDefine the rules in this thread:
-A PROT_ICMP -s 10.1.1.1/32 -j ACCEPT
-A PROT_ICMP -s 10.1.1.2/32 -j ACCEPT
-A PROT_ICMP -s 10.1.1.3/32 -j ACCEPTAs you can see, we do not check the protocol (-p) in this chain, because send only ICMP packets to this chain. Finally, send all ICMP packets to this chain:
-A INPUT -p icmp -g PROT_ICMPNow the incoming packet, if it is not ICMP, will pass through only one rule instead of three.
ipset
Now suppose the following situation:
-A INPUT -s 10.1.1.1/32 -j DROP
-A INPUT -s 10.1.1.2/32 -j DROP
-A INPUT -s 10.1.1.3/32 -j DROPSeveral rules of the same type, but it is not possible to group them or make an additional chain. In such cases, when it is necessary to check on a large number of IP addresses and / or ports, ipset comes to the rescue . ipset is the ip_set kernel module , a number of auxiliary libraries and the ipset utility for setting parameters. Surely there is in the repositories of your distribution, so that the installation should not be a problem.
Used as follows:
* A list is created:
ipset -N dropips iphashdropips - the name of the list, iphash - the type of the list. The types of lists can be viewed in the man ipset, they are for every taste - to work with ip-addresses, subnets, ports, mac-addresses. iphash is used to store IP addresses, the use of hashing prevents adding to the list of duplicate IP addresses.
* Add an IP address to the list:
ipset -A dropips 10.1.1.1* Creates a rule to use the list:
iptables -A INPUT -m set --set dropips src -j DROP-m set indicates the use of the ipset module, --set dropips specifies a list of IP addresses, src indicates that you only need to --set dropips IP source. Thus, all packets from the IP addresses specified in the dropips list will be dropped.In the article, I did not describe the tuning of network parameters of the kernel, but focused on the rational organization of firewall rules. A lot of information about tuning can be found on the network, these are various net. * Mem *, in the case of a large number of connections and using conntrack, the parameters net.netfilter.nf_conntrack_max, net.netfilter.nf_conntrack * timeout, etc.
Thanks for attention.
Thanks ivlis for pointing out typos.
Source: https://habr.com/ru/post/108691/
All Articles