One of the customers asked that the connection between his office and the warehouse be permanent and secure. How we did it and what happened and what did not.
Conditions
The customer has two premises. Office and warehouse. There are Microtik-i and two providers in the office and in the warehouse. Warehouse staff use IP-telephony and office resources through a secure tunnel.
Task
It is necessary to organize communication in such a way that the failure of one provider (from either side) does not lead to the fact that Office users will remain without Internet access, and remote Warehouse users without internal Office resources.
Decision
First, let's draw a diagram and try to determine what needs to be done. We assume that the main channels go through the providers OfficeISP1 and SkladISP1, and the backup channels go through OfficeISP2 and SkladISP2. In the picture they are shown in a thick line.
')

Let's make the list of the parameters describing our configuration:
- Office Network Addressing: 192.168.1.0/24
- The address of the internal interface of the office network router: 192.168.1.254
- The external interface address of the office network router in the primary provider’s network (OfficeISP1): OFF.XXX.XXX.LOC
- Gateway main provider in the office (OfficeISP1): OFF.XXX.XXX.ISP
- The external interface address of the office network router in the backup provider's network (OfficeISP2): OFF.YYY.YYY.LOC
- Gateway backup provider in the office (OfficeISP1): OFF.YYY.YYY.ISP
- Warehouse network addressing: 192.168.2.0/24
- Warehouse network router internal interface address: 192.168.2.254
- The external interface address of the warehouse network router in the network of the main provider (SkladISP1): SKL.NNN.NNN.LOC
- Gateway main provider in the office (SkladISP1): SKL.NNN.NNN.ISP
- The external interface address of the office network router in the backup provider's network (SkladISP2): SKL.HHH.HHH.LOC
- Gateway backup provider in the office (SkladISP1): SKL.HHH.HHH.ISP
- Bridge name for organization of local networks in the office and warehouse: bridge-local
- The name of the bridge for the organization of networks of the main providers in the office and the office: bridge-isp1
- The name of the bridge for the organization of networks of backup providers in the Office and Slada: bridge-isp2
Subsequently, you can make such a list for yourself and then, using a replacement, get the configuration you need.
We divide the problem into two
- Failsafe Internet access through two providers.
- Organization of VPN-channels for communication between the Office and the Warehouse.
Failover of Internet access
The first task, at first, I tried to solve with a script that runs every 3 minutes and checks with the help of ping any of the hosts on the Internet (the survivability of which is beyond doubt). However, when testing this technology all the time failed. Why - it is not clear. Then I used a simple and, as it turned out, more efficient technique:
organization of a backup communication channel without scripts . It turned out even better. When the main channel is unavailable, the transition to the backup channel almost instantly occurs.
VPN connections
For the organization of VPN connections, it was decided to use the IPSec protocol and IPIP tunnels. We make the plates for the tunnels. On each side, you will have to describe 4 IPIP tunnels in order to describe all possible combinations of connections.
For Office Router- Office-Warehouse, communication through the main providers: ipip-office-isp1-sklad-isp1
- Office (backup provider) - Warehouse (main provider): ipip-office-isp2-sklad-isp1
- Office (main provider) - Warehouse (backup provider): ipip-office-isp1-sklad-isp2
- Office-Warehouse, communication through backup providers: ipip-office-isp2-sklad-isp2
- ipip-office-isp1-sklad-isp1 local address: OFF.XXX.XXX.LOC
- ipip-office-isp1-sklad-isp1 remote address: SKL.NNN.NNN.LOC
- ipip-office-isp2-sklad-isp1 local address: OFF.YYY.YYY.LOC
- ipip-office-isp2-sklad-isp1 remote address: SKL.NNN.NNN.LOC
- ipip-office-isp1-sklad-isp2 local address: OFF.XXX.XXX.LOC
- ipip-office-isp1-sklad-isp2 remote address: SKL.HHH.HHH.LOC
- ipip-office-isp2-sklad-isp2 local address: OFF.YYY.YYY.LOC
- ipip-office-isp2-sklad-isp2 remote address: SKL.HHH.HHH.LOC
For Warehouse Router- Warehouse-Office, communication through the main providers: ipip-sklad-isp1-office-isp1
- Warehouse (backup provider) - Office (main provider): ipip-sklad-isp2-office-isp1
- Warehouse (main provider) - Office (backup provider): ipip-sklad-isp1-office-isp2
- Warehouse-Office, communication through backup providers: ipip-sklad-isp2-office-isp2
- ipip-sklad-isp1-office-isp1 local address: SKL.NNN.NNN.LOC
- ipip-sklad-isp1-office-isp1 remote address: OFF.XXX.XXX.LOC
- ipip-sklad-isp2-office-isp1 local address: SKL.HHH.HHH.LOC
- ipip-sklad-isp2-office-isp1 remote address: OFF.XXX.XXX.LOC
- ipip-sklad-isp1-office-isp2 local address: SKL.NNN.NNN.LOC
- ipip-sklad-isp1-office-isp2 remote address: OFF.YYY.YYY.LOC
- ipip-sklad-isp2-office-isp2 local address: SKL.HHH.HHH.LOC
- ipip-sklad-isp2-office-isp2 remote address: OFF.YYY.YYY.LOC
For both routers for all IPIP tunnels- IPSec secret (the same for all tunnels) - check the firmware version for Mikrotik !: bgduikneb789o3krjhgt98728550 (or other password)
- Keepalive - by default (the same for all tunnels): 00:05:10 10
- DSCP: inherit
- Don't Fragment: inherit
- Clamp TCP MSS: tick
Create addresses for each of the tunnels in order to organize the routing between the Office and the Warehouse:
- 172.16.1.1/30 - ipip-office-isp1-sklad-isp1
- 172.16.1.2/30 - ipip-sklad-isp1-office-isp1
- 172.16.1.5/30 - ipip-office-isp2-sklad-isp1
- 172.16.1.6/30 - ipip-sklad-isp1-office-isp2
- 172.16.1.9/30 - ipip-office-isp1-sklad-isp2
- 172.16.1.10/30 - ipip-sklad-isp2-office-isp1
- 172.16.1.13/30 - ipip-office-isp2-sklad-isp2
- 172.16.1.14/30 - ipip-sklad-isp2-office-isp2
Create routes. Note how the distance values ​​should be distributed among the routes:
For office- add distance = 1 dst-address = 192.168.2.0 / 24 gateway = 172.16.1.2
- add distance = 5 dst-address = 192.168.2.0 / 24 gateway = 172.16.1.6
- add distance = 10 dst-address = 192.168.2.0 / 24 gateway = 172.16.1.10
- add distance = 15 dst-address = 192.168.2.0 / 24 gateway = 172.16.1.14
For warehouse- add distance = 1 dst-address = 192.168.1.0 / 24 gateway = 172.16.1.1
- add distance = 5 dst-address = 192.168.1.0 / 24 gateway = 172.16.1.5
- add distance = 10 dst-address = 192.168.1.0 / 24 gateway = 172.16.1.9
- add distance = 15 dst-address = 192.168.1.0 / 24 gateway = 172.16.1.13
We give the highest priority to routes through the main providers, the lowest - through backup ones. Intermediate options (in principle) are the same, but in this case we give greater priority to the channel of the main provider from the Office. I pointed out the latest octets for tunnel IP addresses for easier understanding:

What happened
During normal operation of all providers, the main channels for IPSec VPN connections are used. If one of the channels fails, the routing for the users-Internet connection is rebuilt and the corresponding IPSec VPN channel is automatically used. Thus, even with the simultaneous shutdown of one provider on each side, the system will work.
What did not work
Do not reconfigure the phone connection with the provider of IP-telephony. As a result, when switching to the backup channel, the external address for the telephone exchange does not automatically re-register, but this has been left for the near future. Restrictions are imposed by the telephone exchange itself.
"Tips and Tricks"
To work with two providers at once, do not forget to make the same rules for the NAT (for the Office and the Warehouse - the same):
/ ip firewall nat
add action = masquerade chain = srcnat out-interface = bridge-isp1
add action = masquerade chain = srcnat out-interface = bridge-isp2In order for the packets to go between the Office and the Warehouse without any problems, cancel the NAT between them by adding to the top of the table the rule for the Office:
/ ip firewall nat
add chain = srcnat dst-address = 192.168.1.0 / 24 src-address = 192.168.2.0 / 24And for the warehouse:
/ ip firewall nat
add chain = srcnat dst-address = 192.168.2.0 / 24 src-address = 192.168.1.0 / 24And in order that remote desktop users who work on the road or from home can always (with any active provider) use office resources, you can use DynDNS technology. This will allow when switching to a backup provider not to remember the external IP address of the backup channel.