Mikrotik and OSPF. What had to face and how we overcame it

Good day. Today I would like to talk about what prevented us from living, our brains exploded for a long time - Mikrotik and OSPF.
Mikrotik is in itself a good piece of hardware, with low cost, a lot of opportunity, but unfortunately, not without flaws.
We have the following network diagram:


ROU1 - Cisco c3845
ROU2 - Cisco c3845
MT1 - Mikrotik 1100Ahx2
MT2 - Mikrotik 1100Ahx2
c3550 - Cisco c3550
GW68 - MikroTik RB751U-2HnD
GW69 - MikroTik RB751U-2HnD

On ROU1 and ROU2, we have tunnels to MT1 and to MT2 (IPSec will be skipped), the OSPF process for the backbone is started. The following subnet is 10.0.0.0/19:


On MT1 and MT2, the tunnel addresses are 172.20.64.2 and 172.20.64.6 respectively. The area backbone is already there by default.


But we must also have a connection between MT1 and MT2 directly.
For this we use a new area IRK. Cisco c3550 is not only a switch, it is our root device, all the vlans are set on it, providers are thrown through it, vrf is created on it, etc. (if it will be interesting, then I will describe the whole network organization)
Create an IRK on the c3550 area and two vlan to connect with MT1 and MT2. Area IRK is the main area of ​​the RU - the regional node in which all routers connected to the router work.

Now let's take these vlans on MT1 and MT2, and get a new ospf instance and area, add a link between MT1 and MT2 and add the address.

It seems that I did not forget what we got:

1. We have two tunnels to the central routers, with each microtic one by one.
2. Cost 10 and 40 respectively. That is, with the live main router (provider) MT1, the route from backbone 10.0.0.0/19 we get through the tunnel to ROU1, while dead through the tunnel to ROU2
3. We have a direct link between MT1 and MT2, in order not to drive traffic always through the c3550.

Set, set. Everything is working:

# MT1
ADo dst-address = 10.0.0.0 / 19 gateway = 172.20.64.1 gateway-status = 172.20.64.1 reachable via ipip_yyy_vl03_rou1 distance = 110 scope = 20 target-scope = 10 ospf-metric = 40 ospf-type = external-type-1

# MT2
ADo dst-address = 10.0.0.0 / 19 gateway = 172.20.64.105 gateway-status = 172.20.64.105 reachable via ether2 distance = 110 scope = 20 target-scope = 10 ospf-metric = 50 ospf-type = external-type-1

On MT2 route up to 10ki via ether2, as it should be in principle.

Problem number 1:

Once, on MT1 the provider dies. Subnet 10.0.0.0/19 becomes available through the tunnel to ROU2, everything should be as follows:

# MT2
ADo dst-address = 10.0.0.0 / 19 gateway = 172.20.64.5 gateway-status = 172.20.64.5 reachable via ipip_yyy_vl03_rou2 distance = 110 scope = 20 target-scope = 10 ospf-metric = 70 ospf-type = external-type-1

BUT! As soon as the provider on MT1 comes to life, we see the following:

# MT1
ADo dst-address = 10.0.0.0 / 19 gateway = 172.20.64.1 gateway-status = 172.20.64.1 reachable via ipip_yyy_vl03_rou1 distance = 110 scope = 20 target-scope = 10 ospf-metric = 40 ospf-type = external-type-1

# MT2
ADo dst-address = 10.0.0.0 / 19 gateway = 172.20.64.5 gateway-status = 172.20.64.5 reachable via ipip_yyy_vl03_rou2 distance = 110 scope = 20 target-scope = 10 ospf-metric = 70 ospf-type = external-type-1

We see that each router is looking into its own tunnel. That does not suit us at all. Reboot MT2. It begins to receive a route through ether2 from MT1. Disorder.

As a temporary solution, we got all of the regional nodes on ROU1 and ROU2. Everything worked very nicely until one day our tsisk said: too many routing processes

And we again returned to the problem with the backbone. Long support battles do not lead to anything, but one fine moment we get the answer that the microtic during the redistribution of the routes between the arias (in this case the route from the backbone is redistributed via IRK)
starts incorrectly install routes. And we get the picture that every router looks into its tunnel.
Then the solution comes by itself:
We add IP on ether2, we bring a new subnet in backbone.

# MT1
/ ip address add address = 172.20.64.121 / 30 comment = "MT-MT BB" interface = ether2
/ routing ospf network add area = backbone network = 172.20.64.120 / 30

# Do not redistribute the route through IRK
/ routing filter add action = discard chain = ospf-in prefix = 10.0.0.0 / 19

# MT2
/ ip address add address = 172.20.64.122 / 30 comment = "MT-MT BB" interface = ether2
/ routing ospf network add area = backbone network = 172.20.64.120 / 30

# Do not redistribute the route through IRK
/ routing filter add action = discard chain = ospf-in prefix = 10.0.0.0 / 19

And we get that the 10.0.0.0/19 route is not redistributed through someone else's erie, but is received within one. Fuf, one problem won.

Now for consideration of the second problem, we give the GW config, I will omit the config on MT1 and MT2, there is nothing complicated there.

In the figure we have GW68 and GW69, I will give the settings for only one and in abbreviation:

On Mikrotik 751:
ether2 - backup provider
ether3 - primary provider
ether5 - LAN
The question arises why not ether2 primary provider. We first made ether1 - the main provider, but it turned out that some kind of polusoftovy port and IPSec performance was completely nonexistent, just transferred to ether3, in order not to edit the config much.


How it all works:

GW68 establishes two tunnels to MT1 and to MT2, each through its own provider. There is no default on microtics, addresses MT1 and MT2 are prescribed by statics.
On the tunnel to MT1 there is a cost 10, on the tunnel to MT2 cost 40. As soon as the tunnel comes to life, we get the default and all routes with MT1 through OSPF. All traffic is wrapped on MT1. As soon as the main provider falls off,
after the expiration of the OSPF timer, routs through the tunnel to MT2 become active.

Problem 2:

With a live main channel, we see in the GW68 log

20:12:26 route, packet, ospf, info database, packet has different master status flag
20:12:26 route, ospf, info new master flag = false

At this moment, our traffic itself begins to run through the tunnel to MT2. The tunnel to MT1 is live, and the traffic runs through the tunnel to MT2. And since stores usually have a reserve of megabytes,
then we get sucking money for no reason.
To return everything to the tunnel before the MT1, it was necessary to turn off on the MT1 and again turn on the tunnel to GW68. Moreover, this problem occurred only in some cities, and not on all routers.
On the forum Mikrotik, in those. support no one helped us. One day, my colleague, trying to raise the tunnel MT1-Juniper SRX-650, armed with a bunch of alcohol, sniffer and debugger :)
And debugging IPSec, I came across authorization errors OSPF microtic. We disabled authorization on the OSPF interfaces and voila, the problem disappeared by itself.
How does the authorization go there, where it fails, neither he nor I now remember. But the issue is resolved. Why somewhere worked fine and with authorization, I can not say.

With the microtic, you can only guess at the coffee grounds, since each answer of the support is a new question, each new firmware gives a new problem :)