Setting up server operation on CentOS with 2 gateways and balancing between them

Instead of intro

The earlier article I read on Habré was taken as a basis, which I personally found enough to understand the mechanism of policy routing as a whole - and it’s catastrophically not enough to implement this type of routing on the company's server. There were 2 serious pitfalls that I had to work on independently, and which cannot be ignored:

• Saving settings in general
• Breaking the settings with the Network Manager utility

I will write my article in the form of the instruction that I wrote for future generations of IT people in my company - so I will give some points from the main article either unchanged or re-told to myself. They will be in italics . For a full understanding of what is written here, I recommend to read it in full.

A little practice, or what we want

And we want to get a routing that:
')

• will send data to the same gateway from which the request came
• will control load balancing between gateways using so-called "Weight" (weight) gateways

For these purposes, we will use the iproute2 utility, included in the standard distribution of all Linux distributions.

We issue IP addresses:

ip aa 11.11.11.11/22 dev eth6 ip aa 22.22.22.22/28 dev eth5 

Define the tables:
To use more than one route, static routing is no longer enough. For OSPF, our scheme is too simple, so let's focus on dynamic routing using tables.

 ip route add default via 22.22.22.17 table 101 ip route add default via 11.11.8.1 table 102 

Wrap outgoing traffic to the gateways from which the incoming came:

 ip rule add from 22.22.22.17 table 101 ip rule add from 11.11.8.1 table 102 

I think now it is not necessary to explain the meaning of these lines. Similarly, server availability can be made at more than two gateways.

Balancing traffic between uplinks
It is done by one elegant team:

 ip route replace default scope global \ nexthop via 22.22.22.17 dev eth5 weight 3 \ nexthop via 11.11.8.1 dev eth6 weight 7 

This entry will replace the existing default routing in the main table. In this case, the route will be selected depending on the weight of the gateway (weight). For example, when specifying weights 7 and 3, 70% of the connections go through the first gateway, and 30% through the second gateway. There is one thing that must be taken into account: the kernel caches routes, and the route for a host through a certain gateway will hang in the table for some time after the last access to this entry. A route to frequently used hosts may not have time to be reset and will be updated all the time in the cache, remaining on the same gateway. If this is a problem, you can sometimes clear the cache manually with the ip route flush cache command.

Result
After running this command, you can check the availability of the server In principle, it could be said that our work was over. However, there is one problem - after a reboot, all settings will be reset.

Saving settings in general

Now it is necessary to force the system to apply the obtained settings after a reboot.

IP addresses
First, we find out which mac-address belongs to which interface. Execute the command:

 ip a 

Next, create ifconfig configuration configuration files using the received mac-addresses. To do this, create files with the prefix ifcfg and the name of the interface in the folder



Create routing tables
Next, you need to create permanent tables, assign names to them (to make it easier to navigate) and set the rules for using these tables:

 cd /etc/iproute2/ echo "101 int" >> rt_tables echo "102 ext" >> rt_tables 

Now go back to the /etc/sysconfig/network-scripts directory and continue working there.
Create the contents of the tables:

 # cat route-eth5 default via 22.22.22.17 table ext # cat route-eth6 default via 11.11.8.1 table int 

Create rules for processing these tables:

 # cat rule-eth5 from 22.22.22.22 lookup ext # cat rule-eth6 from 11.11.11.11 lookup int 

Balancing
Next, replace the static dynamics. Not without a file, because The startup scripts described above are bound to interfaces, and in our case, the rule is written immediately about 2 interfaces. Therefore, it was decided to create a separate script and set it to autoload using the standard Linux autorun mechanism - /etc/rc.local . Script content:

 # cat /etc/network.sh #!/bin/bash /sbin/ip route replace default scope global nexthop via 11.11.8.1 dev eth6 weight 7 nexthop via 22.22.22.17 dev eth5 weight 3 exit 0 

His path to autoload:

 # cat /etc/rc.local #!/bin/sh touch /var/lock/subsys/local /bin/bash /etc/network.sh 

For those who have not earned the execution of the file rc.local, please run the following commands:

 chmod u+x /etc/rc.d/rc.local systemctl start rc-local 

As a result, after a reboot, we get a working system using 2 independent uplinks, the traffic between which is balanced by the weight. And now the sad news: none of this will work.

Breaking the settings with the Network Manager utility

The smaller problem of those that were on my way, but which nevertheless, pretty much spoiled my nerves. The problem with this utility is that it saves and uses settings not from the main configuration files of the system, but from its internal ones. Since it starts along with X11, that is, at the very last moment, when the network is already running, it overwrites the network settings, and it will not be possible to build some complicated configurations on it.

To disable it, you need to run in the console:

 sudo /etc/init.d/NetworkManager stop chkconfig NetworkManager off 

Also, you can go the other way and use the systemctl:

 systemctl disable NetworkManager systemctl stop NetworkManager 

Result
And after performing this action and rebooting, we get a working system using 2 independent uplinks, the traffic between which is balanced with the help of weight.