Dynamic network traffic redirection

The article will discuss how to organize the ability to dynamically switch between network interfaces.

The roots of the question began to grow from the previous project socmetr.ru , where it took to collect a large amount of information from social networks, and thus clogging the only channel with the Internet. The analysis showed that even with compression, the amount of incoming information is so large that it is blocked, while the CPU and Memory capacities are not used by 20%, and the disk subsystem is idle almost all the time, that is, we rested on the width of the channel Provides us provider.

The first thought was to go an extensive way and simply increase its capabilities, cooling down a bit and thinking about it, realized that we were shifting the problem to the future. Needless to say, the question arose: "How can we go comrades?". As a result, implemented the following idea:

Server Tuning

Since the configuration looks monotonous, it makes sense to introduce symbolic symbols, let:

A ₁ is the first network interface, A ₂ is the second ... A _N is the last
IP ₁ is the IP address associated with the network interface A ₁ , IP ₂ is the address associated with A ₂ ... IP _N is the address associated with A _N
GW ₁ is the IP address of the gateway provider P ₁ (Provider 1), GW ₂ is the address of the gateway provider P ₂ ... GW _N is the address of the gateway P _N
GW ₁ _NET is the IP address of the provider's network P ₁ , GW ₂ _NET IP address of the provider's network P ₂ ... GW _N _NET IP address of the provider's network P _N

Then the setting can be done as follows:

 N-  ( N -      ) ip route add {GW1_NET} dev {A1} src {IP1} table T1 ip route add default via {GW1} table T1 ... ip route add {GWn_NET} dev {An} src {IPn} table Tn ip route add default via {GWn} table Tn      : ip route add {GW1_NET} dev {A1} src {IP1} ... ip route add {GWn_NET} dev {An} src {IPn} ip route add default via {GW1}  : ip rule add from {IP1} table T1 ... ip rule add from {IPn} table Tn 

And it's in the bag, we get a configuration that ensures that all requests to a specific interface will receive a response from it.

Using

In order to turn to different network adapters, you must use the design of a specific programming language, or use third-party libraries. In Java, this can be done through sockets, for example:

 socket.bind(new InetSocketAddress(InetAddress.getByName("network-adapter"), port)); 

By implementing this scheme, it was possible to manage the bandwidth of the Internet channel, depending on the business processes and logic occurring on the servers, at different stages of the life cycle of information systems.

Pros:

• the possibility of flexible scaling of the Internet channel (we have the opportunity at any time to increase or decrease the overall Internet channel, both by connecting new providers and by increasing the current bandwidth, it is possible to plan and predict its capacity)
• ability to manage the load, each of them
• the ability to use any channel or their sequence at any time by any subsystem

Minuses:

• initial expenses for the purchase of appropriate equipment
• obligatory presence of many Internet service providers

Perhaps someone has already encountered a similar problem, and we have built a bicycle, so criticism of the decision is only welcome.