Creating a resilient gateway based on Mikrotik RouterOS
The task was to ensure the fault tolerance of the network router, which was supposed to support several local area networks, three channels to the Internet from different providers, a DMZ and a dozen VPN connections for remote users.
Fail-over was meant to instantly replace equipment in the event of a failure. I opted for the Mikrotik RouterOS, as I had a positive operating experience for this OS. Also, the choice was influenced by the ease of configuration and administration thanks to the Winbox utility.
This example of resiliency works in combat conditions for several years and justified itself. During operation, the configuration was reworked several times, but the option that was lower in my conditions turned out to be optimal. Now turning off one of the routers does not affect the work, they are completely interchangeable.
As an iron, it was decided to use conventional PCs with Core 2 Duo processors, 1 GB of memory and HDD Flash from Transend as a drive. Both routers are located in cute MiniTower cases on a shelf in a server cabinet. The Mikrotik version at that time was installed 4.16, but now it works on 5.22
')
I will not describe the configuration of the entire router, but will only focus on fault tolerance. For better information perception, we will limit ourselves to one provider and 3 internal local networks.
VRRP was chosen as a protocol for fault tolerance. Its principle is that routers have priority: Master and Slave and after a certain time interval they check each other’s availability. If the master fails, Slave will replace it.
Since there are only 3 PCI-based network interfaces on the router (1 integrated was not used), and there are many subnets, VLAN was also used. With that on the physical interface hung VRRP, and already on it VLAN-s. All settings were made on the same router. The second is configured automatically.
Physical Ethernet Interfaces
Virtual VRRP interface
Of the VRRP settings, 3 parameters are interesting.
1) interface = lan interface on which VRRP is hung
2) priority = 101 prioritizes the router. Master or Slave. The main one - who has more.
3) preemption-mode = yes if this mode is turned off: after the slave becomes a master, it will remain with it, even when the master returns to the system.
LAN VLAN
VRRP is used only on the local interface. Its task is to monitor the health of the router as a whole and the connection to the local network. In case of problems, everything else was switched by the script. This decision was due to the fact that IpSec did not work well with VRRP + VLAN on the WAN interface.
VRRP on Mikrotik allows you to use routers in load balancing or fault tolerance modes. In balancing mode, fault tolerance is also observed, but for our task, fault tolerance is sufficient, otherwise we would have to hang VRRP on all interfaces.
Configuring Addressing
Virtual interfaces vrrp-lan on both routers will have the same address 10.1.1.1/32
But the physical lan interfaces will have different addresses (10.1.1.2 and 10.1.1.3) and will also be on the same subnet between themselves and the vrrp interface.
Now it is enough to add the default gateway to the routing table and the initial configuration can be considered complete.
Next, you need to configure the backup scripts, check the status of the Master / Slave and transfer the configuration.
Check the status of the router
If vrrp running = false, then the router is in Slave mode - disable the WAN. If Master - then turn on. We also check the current status of the WAN interface - whether it is on or off, so as not to disturb it again. This script is set to run in the scheduler every 3-10 seconds.
Backup
In addition to sending to the mail (you need to configure the sending parameters first), the script will create a local file named lastconfig.backup, which will be useful to us.
This script is set to run in the scheduler 1 time per day, it costs at night. If you make a lot of configuration changes in a day, put it at your convenience.
Then follow 2 scripts: transferring settings and applying settings on the backup server. Since they should be run only on the backup server, the routers need to be somehow distinguished. I distinguish between the MAC addresses of the integrated network interfaces. Reserve has address FF: FF: 40: 40: 40: 41
Copy and apply the latest actual configuration
That is, we take the lastconfig.backup via FTP and restore it from it. FTP user must be configured, preferably with limited access by IP. Please note that we connect to FTP by the IP address of the local physical interface, which is accessible only between routers.
We put this script in the scheduler a few minutes after the execution of the backup script.
And the last script is applying settings on the backup server. It also uses MAC to identify the router.
Here we change the name of the router, ip-address of the LAN-interface and VRRP priority to a smaller one in order to make the router a slave. Run this script you need to put in startup. Changes will occur on the backup server after copying and applying the latest actual configuration.
Actually, that's all. Let me remind you, we did all the manipulations on the router, which we will have a master. Now save the configuration, transfer it to the slave and use it on it. This can be done by any method described here http://wiki.mikrotik.com/wiki/Manual:Configuration_Management#System_Backup
After applying the configuration and rebooting, the backup router will be commissioned with the settings we need.
It was written from memory, but in the process of setting different sources were used. Basically wiki.mikrotik.com/wiki/Main_Page
Fail-over was meant to instantly replace equipment in the event of a failure. I opted for the Mikrotik RouterOS, as I had a positive operating experience for this OS. Also, the choice was influenced by the ease of configuration and administration thanks to the Winbox utility.
This example of resiliency works in combat conditions for several years and justified itself. During operation, the configuration was reworked several times, but the option that was lower in my conditions turned out to be optimal. Now turning off one of the routers does not affect the work, they are completely interchangeable.
As an iron, it was decided to use conventional PCs with Core 2 Duo processors, 1 GB of memory and HDD Flash from Transend as a drive. Both routers are located in cute MiniTower cases on a shelf in a server cabinet. The Mikrotik version at that time was installed 4.16, but now it works on 5.22
')
I will not describe the configuration of the entire router, but will only focus on fault tolerance. For better information perception, we will limit ourselves to one provider and 3 internal local networks.
VRRP was chosen as a protocol for fault tolerance. Its principle is that routers have priority: Master and Slave and after a certain time interval they check each other’s availability. If the master fails, Slave will replace it.
Since there are only 3 PCI-based network interfaces on the router (1 integrated was not used), and there are many subnets, VLAN was also used. With that on the physical interface hung VRRP, and already on it VLAN-s. All settings were made on the same router. The second is configured automatically.
Interface Setup
Physical Ethernet Interfaces
/ interface ethernet
set 0 arp = enabled auto-negotiation = yes cable-settings = default \
disable-running-check = yes disabled = no full-duplex = yes l2mtu = 1600 \
mtu = 1500 name = lan speed = 1Gbps
set 1 arp = enabled auto-negotiation = yes cable-settings = default \
disable-running-check = yes disabled = no full-duplex = yes l2mtu = 1600 \
mtu = 1500 name = wan speed = 1Gbps
Virtual VRRP interface
/ interface vrrp
add arp = enabled authentication = none disabled = no interface = lan \
interval = 2s mtu = 1500 name = vrrp-lan \
preemption-mode = yes priority = 101 v3-protocol = ipv4 version = 3 vrid = 2
Of the VRRP settings, 3 parameters are interesting.
1) interface = lan interface on which VRRP is hung
2) priority = 101 prioritizes the router. Master or Slave. The main one - who has more.
3) preemption-mode = yes if this mode is turned off: after the slave becomes a master, it will remain with it, even when the master returns to the system.
LAN VLAN
/ interface vlan
add arp = enabled disabled = no interface = vrrp-lan mtu = 1500 name = vlan101 \
use-service-tag = no vlan-id = 101
add arp = enabled disabled = no interface = vrrp-lan mtu = 1500 name = vlan102 \
use-service-tag = no vlan-id = 102
add arp = enabled disabled = no interface = vrrp-lan mtu = 1500 name = vlan103 \
use-service-tag = no vlan-id = 103
VRRP is used only on the local interface. Its task is to monitor the health of the router as a whole and the connection to the local network. In case of problems, everything else was switched by the script. This decision was due to the fact that IpSec did not work well with VRRP + VLAN on the WAN interface.
VRRP on Mikrotik allows you to use routers in load balancing or fault tolerance modes. In balancing mode, fault tolerance is also observed, but for our task, fault tolerance is sufficient, otherwise we would have to hang VRRP on all interfaces.
Configuring Addressing
/ ip address
add address = 192.168.101.1 / 24 disabled = no interface = vlan101 network = 192.168.101.0
add address = 192.168.102.1 / 24 disabled = no interface = vlan102 network = 192.168.102.0
add address = 192.168.103.1 / 24 disabled = no interface = vlan103 network = 192.168.103.0
add address = 10.1.1.2 / 29 disabled = no interface = lan network = 10.1.1.0
add address = 10.1.1.1 / 32 disabled = no interface = vrrp-lan network = 10.1.1.1
add address = 77.77.77.70 / 30 disabled = no interface = wan network = 77.77.77.68
Virtual interfaces vrrp-lan on both routers will have the same address 10.1.1.1/32
But the physical lan interfaces will have different addresses (10.1.1.2 and 10.1.1.3) and will also be on the same subnet between themselves and the vrrp interface.
Now it is enough to add the default gateway to the routing table and the initial configuration can be considered complete.
/ ip route
add disabled = no distance = 1 dst-address = 0.0.0.0 / 0 gateway = 77.77.77.69 scope = 30 target-scope = 10
Next, you need to configure the backup scripts, check the status of the Master / Slave and transfer the configuration.
Check the status of the router
: if ([/ interface get vrrp running] = false) do = {
: if ([/ interface get wan disabled] = false) do = {
/ interface disable wan
}
}
: if ([/ interface get vrrp running] = true) do = {
: if ([/ interface get wan disabled] = true) do = {
/ interface enable wan
}
}
If vrrp running = false, then the router is in Slave mode - disable the WAN. If Master - then turn on. We also check the current status of the WAN interface - whether it is on or off, so as not to disturb it again. This script is set to run in the scheduler every 3-10 seconds.
Backup
: local mserver
: local mkomu
: local msubject
: local bjmeno
: set mkomu "root@server.ru"
: set msubject ("Backup". [/ system identity get name])
: set bjmeno ([/ system identity get name]. "-". [: pick [/ system clock get date] 7 11]. [: pick [/ system clock get date] 0 3]. [: pick [/ system clock get date] 4 6]. ".backup")
/ system backup save name = $ bjmeno
/ system backup save name = lastconfig
: delay 5
: put ($ mserver. "\ n")
/ tool e-mail send subject = $ msubject file = $ bjmeno to = $ mkomu body = ("Backup from". [/ system clock get date]. "mikrotik". [/ system identity get name]. ".")
: put (“Backup”. $ bjmeno. "\ n")
In addition to sending to the mail (you need to configure the sending parameters first), the script will create a local file named lastconfig.backup, which will be useful to us.
This script is set to run in the scheduler 1 time per day, it costs at night. If you make a lot of configuration changes in a day, put it at your convenience.
Then follow 2 scripts: transferring settings and applying settings on the backup server. Since they should be run only on the backup server, the routers need to be somehow distinguished. I distinguish between the MAC addresses of the integrated network interfaces. Reserve has address FF: FF: 40: 40: 40: 41
Copy and apply the latest actual configuration
: local interA [/ interface ethernet find mac-address = "FF: FF: 40: 40: 40: 41"]
: if ($ interA! = "") do = {
/ tool fetch address = 10.1.1.2 src-path = lastconfig.backup mode = ftp user = ftp password = VeryHightPassword !!! 11
: delay 10
system backup load name = lastconfig.backup
}
That is, we take the lastconfig.backup via FTP and restore it from it. FTP user must be configured, preferably with limited access by IP. Please note that we connect to FTP by the IP address of the local physical interface, which is accessible only between routers.
We put this script in the scheduler a few minutes after the execution of the backup script.
And the last script is applying settings on the backup server. It also uses MAC to identify the router.
: local intA [/ interface ethernet find mac-address = "FF: FF: 40: 40: 40: 41"]
: if ($ intA! = "") do = {
/ system identity set name = router-slave
/ ip address remove [/ ip address find address = "10.1.1.2/29"]
/ ip address add address = 10.1.1.3 / 29 interface = lan
/ interface vrrp set priority = 100 preemption-mode = yes numbers = vrrp-lan
}
Here we change the name of the router, ip-address of the LAN-interface and VRRP priority to a smaller one in order to make the router a slave. Run this script you need to put in startup. Changes will occur on the backup server after copying and applying the latest actual configuration.
Actually, that's all. Let me remind you, we did all the manipulations on the router, which we will have a master. Now save the configuration, transfer it to the slave and use it on it. This can be done by any method described here http://wiki.mikrotik.com/wiki/Manual:Configuration_Management#System_Backup
After applying the configuration and rebooting, the backup router will be commissioned with the settings we need.
It was written from memory, but in the process of setting different sources were used. Basically wiki.mikrotik.com/wiki/Main_Page
Source: https://habr.com/ru/post/164873/
All Articles