⬆️ ⬇️

Do you know static routing well?

A static route is the first thing that anyone encounters when learning the concept of IP packet routing. It is believed that this is the simplest topic of all, everything is simple and obvious. I will try to show that even such a primitive technology can contain many nuances.



Reservation. When writing a topic, I assume that the reader is familiar with the concept of routing, knows how to do static routes and does not consider the word “ARP” abusive. However, even experienced telecom operators will surely find something new here.

All examples were tested on the IOS 15.2M line. The behavior of other operating systems may vary.

And there will be no dynamic routing here.



We work with the following topology:





How does a static route appear?



To begin with, we will execute the command that everyone knows, and see in debigs what will happen:

R00(config)#ip route 3.1.1.0 255.255.255.0 10.0.0.3 


 IP-ST(default): updating same distance on 3.1.1.0/24 IP-ST(default): 3.1.1.0/24 [1], 10.0.0.3 Path = 8, no change, not active state IP-ST(default): 3.1.1.0/24 [1], 10.0.0.3 Path = 2 3 7 RT: updating static 3.1.1.0/24 (0x0): via 10.0.0.3 RT: add 3.1.1.0/24 via 10.0.0.3, static metric [1/0], add succeed, active state IP ARP: creating incomplete entry for IP address: 10.0.0.3 interface GigabitEthernet0/1 IP ARP: sent req src 10.0.0.1 30e4.db16.7791, dst 10.0.0.3 0000.0000.0000 GigabitEthernet0/1 IP ARP: rcvd rep src 10.0.0.3 0019.aad6.ae10, dst 10.0.0.1 GigabitEthernet0/1 


IOS created the route, and immediately sent an arp request in search of next hop, which we have is 10.0.0.3. And immediately the question: how did the router know that the request should be sent to the interface Gi0 / 1? Surely someone will say "from the list of local interfaces," and cruelly mistaken. Routing does not work that way. In fact, IOS made a recursive query to the routing table to find out how to get to next hop:

 R00#show ip route 10.0.0.3 Routing entry for 10.0.0.0/24 Known via "connected", distance 0, metric 0 (connected, via interface) Routing Descriptor Blocks: * directly connected, via GigabitEthernet0/1 Route metric is 0, traffic share count is 1 


And here it is, our Gi0 / 1. IOS will find out that with recursive queries to the RIB it is necessary to end as soon as it finds a route with the "directly connected" flag. But what if, in response to the initial request for 10.0.0.3, it does not return a connected route at all, but an intermediate one that refers to another next hop? Let's come back to this a bit later, but for now let's remember what CEF is.

')

Approximately in all documentation focused on beginners, it is said that each packet moves according to the routing table. In fact, on all more or less modern platforms this is no longer the case, because the routing table (hereinafter referred to as RIB) is not optimized for fast data transfer at all. This table allows us to estimate the scale of the disaster (although the process switching has many drawbacks besides non-optimal queries — for example, constant switching of the CPU scheduler between contexts, which is very expensive). CEF is a serious optimization. In a modern implementation, he builds two tables — FIB (Forwarding Information Base, a packet transfer table, based on a coherent graph with the terrible name 256-way mtrie) and an adjacency table. The first one is built on the basis of the routing table and in one pass allows you to get all the necessary information. It is built in advance, even before the first corresponding package appears.



Let's return to our static route. Here is the entry in the routing table:

 R00#show ip route 3.1.1.0 Routing entry for 3.1.1.0/24 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * 10.0.0.3 Route metric is 0, traffic share count is 1 


Where to send the package? Where to look for 10.0.0.3? Unclear. It is necessary to once again request the routing table, this time about 10.0.0.3, and, if necessary, perform a few more iterations until we find out the connected interface. And about this way, we are actually several times reducing the performance of the router.



This is what CEF says:

 R00#show ip cef 3.1.1.0 detail 3.1.1.0/24, epoch 0 recursive via 10.0.0.3 attached to GigabitEthernet0/1 


Simple and concise. There is an interface, there is next hop, to which you need to send a packet. What was said about the adjacency table?

 R00#show adjacency 10.0.0.3 detail Protocol Interface Address IP GigabitEthernet0/1 10.0.0.3(10) 0 packets, 0 bytes epoch 0 sourced in sev-epoch 2 Encap length 14 0019AAD6AE1030E4DB1677910800 ARP 


Pay attention to some long sequence in the last but one line. Something it reminds ... We look mac 10.0.0.3:

 R00#show arp | in 10.0.0.3 Internet 10.0.0.3 1 0019.aad6.ae10 ARPA GigabitEthernet0/1 


We look at your mac address on gi0 / 1:

 R00#show int gi0/1 GigabitEthernet0/1 is up, line protocol is up Hardware is CN Gigabit Ethernet, address is 30e4.db16.7791 (bia 30e4.db16.7791) 


Yeah. That terrible line is just two poppies that need to be substituted into the Ethernet header at the encapsulation stage, and ethertype 0x0800, i.e. banal IPv4. And in two CEF tables there is absolutely all the information that is needed to successfully send a packet further down the chain.



If someone has a question, why should a piece of iron hold two tables at once instead of one, then I will give an obvious answer: usually, the router has few interfaces (and neighbors as well) and many routes. What is the point of duplicating the same Macs thousands of times in the FIB? Memory does not happen much, especially on hardware platforms, be it the new-fledged ASRs or even the L3 switches of the Catalyst line. All of them involve CEF in the transmission of packets.



And by the way, let's return for a moment to the initial debug. Disable CEF with no ip cef (never do this) and compare the result:

 IP-ST(default): updating same distance on 3.1.1.0/24 IP-ST(default): 3.1.1.0/24 [1], 10.0.0.100 Path = 8, no change, not active state IP-ST(default): 3.1.1.0/24 [1], 10.0.0.100 Path = 2 3 7 RT: updating static 3.1.1.0/24 (0x0): via 10.0.0.100 RT: add 3.1.1.0/24 via 10.0.0.100, static metric [1/0], add succeed, active state 


Route added. Arp request was not. And rightly so - why did RIB give up mac address? If you let ping up, for example, 3.1.1.1, then most likely it will be like this:

 R00#ping 3.1.1.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 3.1.1.1, timeout is 2 seconds: .!!!! Success rate is 80 percent (4/5), round-trip min/avg/max = 1/1/4 ms 


The first packet is discarded, and the router sends an arp request in order to find out the mac address 10.0.0.3 if it was not previously known. CEF, however, always knows in advance the mac address of the next hop.



With this sorted out. Now back to the question of what will happen if the next hop of a static route is not at all on the directly connected interface. We proceed simply:

 R00(config)#ip route 10.0.0.3 255.255.255.255 100.100.100.101 


where Gi0 / 2 has the address 100.100.100.100/24.

 R00#show ip cef 3.1.1.0 detail 3.1.1.0/24, epoch 0 recursive via 10.0.0.3 recursive via 100.100.100.101 recursive via 100.100.100.0/24 attached to GigabitEthernet0/2 


How bad everything is ... And what if we have a route for a whole supernetwork?

 R00(config)#no ip route 10.0.0.3 255.255.255.255 100.100.100.101 R00(config)#ip route 10.0.0.0 255.0.0.0 100.100.100.101 R00#show ip cef 3.1.1.0 detail 3.1.1.0/24, epoch 0 recursive via 10.0.0.3 attached to GigabitEthernet0/1 


Now our routing table looks like this:

 R00#show ip route 10.0.0.0/8 is variably subnetted, 2 subnets, 3 masks S 10.0.0.0/8 [1/0] via 100.100.100.101 C 10.0.0.0/24 is directly connected, GigabitEthernet0/1 L 10.0.0.1/32 is directly connected, GigabitEthernet0/1 100.0.0.0/8 is variably subnetted, 2 subnets, 2 masks C 100.100.100.0/24 is directly connected, GigabitEthernet0/2 L 100.100.100.100/32 is directly connected, GigabitEthernet0/2 


Seems alright. The new route at 100.100.100.101 does not apply to 10.0.0.3, since its mask / 8 is much shorter than / 24 at the connected interface. But suddenly Gi0 / 1, which contained next hop for 3.1.1.0/24, for some unknown reason went down, and its connected route disappeared from the RIB.

 %LINK-5-CHANGED: Interface GigabitEthernet0/1, changed state to administratively down %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down RT: interface GigabitEthernet0/1 removed from routing table RT: del 10.0.0.0 via 0.0.0.0, connected metric [0/0] RT: delete subnet route to 10.0.0.0/24 RT: del 10.0.0.1 via 0.0.0.0, connected metric [0/0] RT: delete subnet route to 10.0.0.1/32 IP-ST(default): updating GigabitEthernet0/1 


Here is what became:

 R00#show ip cef 3.1.1.0 detail 3.1.1.0/24, epoch 0 recursive via 10.0.0.3 recursive via 10.0.0.0/8 recursive via 100.100.100.101 recursive via 100.100.100.0/24 attached to GigabitEthernet0/2 


Oh. Now packets on the 3.1.1.0/24 network go somewhere not there. I can not imagine a scenario when the expected behavior of a static route is switching to another interface. If there is a backup path behind that interface, you still need to create another static route ...



What to do? Specify the interface immediately in the route. Recreate the route:

 R00(config)#no ip route 3.1.1.0 255.255.255.0 10.0.0.3 R00(config)#ip route 3.1.1.0 255.255.255.0 Gi0/1 10.0.0.3 


Raise Gi0 / 1. We look where the route now leads to 3.1.1.0/24:

 R00#show ip route 3.1.1.0 Routing entry for 3.1.1.0/24 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * 10.0.0.3, via GigabitEthernet0/1 Route metric is 0, traffic share count is 1 


There is already an interface. Therefore, there will be no recursive queries to the routing table. Check FIB:

 R00#show ip cef 3.1.1.0 3.1.1.0/24 nexthop 10.0.0.3 GigabitEthernet0/1 


Yes, no "recursive". And if you again pay gi0 / 1? The route has disappeared.

 R00#show ip route 3.1.1.0 % Network not in table R00#show ip cef 3.1.1.0 0.0.0.0/0 no route 


And despite the fact that the route to 10.0.0.3 was still:

 R00#show ip cef 10.0.0.3 10.0.0.0/8 nexthop 100.100.100.101 GigabitEthernet0/2 


And what will happen if the path to next hop gives the default route and the route to 3.1.1.0/24 does not refer to the interface?

 R00(config)#no ip route 10.0.0.0 255.0.0.0 100.100.100.101 R00(config)#ip route 0.0.0.0 0.0.0.0 100.100.100.101 R00(config)#no ip route 3.1.1.0 255.255.255.0 Gi0/1 10.0.0.3 R00(config)#ip route 3.1.1.0 255.255.255.0 10.0.0.3 R00#show ip route 3.1.1.0 % Network not in table R00#show ip cef 3.1.1.0 detail 0.0.0.0/0, epoch 0, flags default route recursive via 100.100.100.101 recursive via 100.100.100.0/24 attached to GigabitEthernet0/2 


Note that the first line after “show ip cef” is “0.0.0.0/0”, and not “3.1.1.0/24”. Despite the fact that next hop is formally, in fact, all iterations of polling the routing table (except the first) ignore the default route, which is logical, otherwise any query to the routing table would almost always be resolved (“resolving” means finding an interface in which need to send a package). Therefore, our static route is missing, but packages still fly to Gi0 / 2. It seems to be the same as without explicitly specifying the interface? Not really. Let's say the routing protocol was told to "redistribute static". If the static route is gone, then the announcement also responds. And if not, the router will continue to tell everyone to “go through me there,” and this will almost certainly turn L3 into a ring for the prefix 3.1.1.0/24, which could be accessed from somewhere else. But wait, we agreed not to touch the dynamic routing ...



And what if you specify an interface in a static route, but do not specify the IP address of the next hop? Answer: in the case of Ethernet, if proxy arp is not disabled at next hop, the connectivity will not be broken, but the router can be VERY bad. More details . If you say “ip route 3.1.1.0 255.255.255.0 gi0 / 1”, then nothing particularly terrible will happen, even a couple of hundred entries in the arp table any router will digest (and there are workaround scripts where the optimal solution is just such a crutch ), but the “ip route 0.0.0.0 0.0.0.0 gi0 / 1” on the border router will surely kill it. Therefore, remember the general rule: if you create a static route with next hop on the Ethernet interface, its IP address should always be indicated. Exceptions - only when you are very well aware of what you are doing, why you are doing it and why you cannot do it otherwise.



And finally, we will make one very bad thing.

 R00(config)# ip route 3.1.1.0 255.255.255.0 10.0.0.3 R00(config)#ip route 10.0.0.3 255.255.255.255 3.1.1.1 


The first route is in order, tested a hundred times. But the second is strange - it leads through the first. And the first now refers to the second, and we have endless recursion. Here is what happened:

 IP-ST(default): updating same distance on 3.1.1.0/24 IP-ST(default): 3.1.1.0/24 [1], 10.0.0.3 Path = 8, no change, not active state IP-ST(default): 3.1.1.0/24 [1], 10.0.0.3 Path = 2 3 7 RT: updating static 3.1.1.0/24 (0x0): via 10.0.0.3 RT: add 3.1.1.0/24 via 10.0.0.3, static metric [1/0], add succeed, active state IP-ST(default): updating same distance on 10.0.0.3/32 IP-ST(default): 10.0.0.3/32 [1], 3.1.1.1 Path = 8, no change, not active state IP-ST(default): 10.0.0.3/32 [1], 3.1.1.1 Path = 2 3 7 RT: updating static 10.0.0.3/32 (0x0): via 3.1.1.1 RT: add 10.0.0.3/32 via 3.1.1.1, static metric [1/0], add succeed, active state 


Added successfully. But then it appeared in debags:

 RT: recursion error routing 3.1.1.1 - probable routing loop RT: recursion error routing 10.0.0.3 - probable routing loop 


And there was an entry in the log with severity 3:

 %IPRT-3-RIB_LOOP: Resolution loop formed by routes in RIB 


 R00#show ip cef 10.0.0.3 detail 10.0.0.3/32, epoch 0 Adj source: IP adj out of GigabitEthernet0/1, addr 10.0.0.3 359503C0 Dependent covered prefix type adjfib cover 10.0.0.0/24 1 RR source [no flags] recursive via 3.1.1.1, unresolved recursive-looped R00#show ip cef 3.1.1.0 detail 3.1.1.0/24, epoch 0, flags cover dependents Covered dependent prefixes: 1 notify cover updated: 1 recursive via 10.0.0.3, unresolved recursive-looped 


However, the RIB does not see any crime:

 Routing entry for 3.1.1.0/24 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * 10.0.0.3 Route metric is 0, traffic share count is 1 R00#show ip route 10.0.0.3 Routing entry for 10.0.0.3/32 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * 3.1.1.1 Route metric is 0, traffic share count is 1 


Conclusion - never do that.



Why a static route can not get into the routing table?



Any networker should immediately give one of the explanations regarding any source of routes in IOS: there is another route to the same prefix, but with a smaller AD (do everyone remember Administrative Distance?). The route, the source of which is “connected”, always has AD = 0, and no other source of routes can bring anything lower than “1”, not even a static route with an explicit indication of the interface. Example connected:

 R00# show ip route C 10.0.0.0/24 is directly connected, GigabitEthernet0/1 


Those. While the Gi0 / 1 interface is in the up state and has an address from the subnet 10.0.0.0/24, not a single static route to this prefix appears in the routing table.



Another option is “different route sources add routes to the same prefix with the same AD”. The behavior of iOS in this case is not documented, the general recommendation is “never do that.”



But let's look at other, less obvious examples. For example, static routes can be created with the word "permanent", which translates as "permanent", and then they will always hang in the routing table. Right? Not.



Add it and see:

 R00(config)#ip route 3.1.1.0 255.255.255.0 10.0.0.3 permanent R00#show ip cef 3.1.1.0 3.1.1.0/24 nexthop 10.0.0.3 GigabitEthernet0/1 


Put Gi0 / 1, and see:

 R00#show ip cef 3.1.1.0 3.1.1.0/24 unresolved via 10.0.0.3 


It is in the RIB, and other routing protocols can use it:

 R00#show ip route 3.1.1.0 Routing entry for 3.1.1.0/24 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * 10.0.0.3, permanent Route metric is 0, traffic share count is 1 


And now, without raising Gi0 / 1:

 R00(config)#no ip route 3.1.1.0 255.255.255.0 10.0.0.3 permanent R00(config)#ip route 3.1.1.0 255.255.255.0 10.0.0.3 permanent 


Just recreated it, without changing anything. And that's what happened:

 IP-ST(default): updating same distance on 3.1.1.0/24 IP-ST(default): 3.1.1.0/24 [1], 10.0.0.3 Path = 8, no change, not active state IP-ST(default): cannot delete, PERMANENT R00#show ip route 3.1.1.0 % Network not in table R00#show ip cef 3.1.1.0 0.0.0.0/0 no route 


Standing, speak? Not. There is one small nuance: in order for the permanent route to fit forever into the routing table, it is necessary that it should be resolved for at least a split second. Although what else is "forever"? When it was left hanging in the air without a resolving interface, it suffices to say “clear ip route *” or, all the more, “reload” so that it disappears from the RIB.



But we will continue. Let's do this:

 R00(config)#ip route 3.1.1.0 255.255.255.0 Gi0/1 10.0.0.3 R00(config)#ip route 3.1.1.10 255.255.255.255 3.1.1.1 


Like normal routes. What will happen? With the second - absolutely nothing.

 IP-ST(default): 3.1.1.10/32 [1], 3.1.1.1 Path = 8, no change, not active state IP-ST(default): 3.1.1.10/32 [1], 3.1.1.1 Path = 2 3 6 8, no change, not active state 


The bottom line is this. Suppose there is a route on XXXX through YYYY. We add a route on X1.X1.X1.X1 (this prefix is ​​completely covered by XXXX) via X2.X2.X2.X2 (and it is also covered by XXXX). IOS makes a logical conclusion: the second route does not carry any new information and is completely useless, so you can not install it in the RIB.



And now feint ears.

 R00(config)#no ip route 3.1.1.10 255.255.255.255 3.1.1.1 R00(config)#ip route 3.1.1.10 255.255.255.255 Gi0/1 3.1.1.1 IP-ST(default): updating same distance on 3.1.1.10/32 IP-ST(default): 3.1.1.10/32 [1], GigabitEthernet0/1 Path = 1 RT: updating static 3.1.1.10/32 (0x0): via 3.1.1.1 Gi0/1 RT: network 3.0.0.0 is now variably masked RT: add 3.1.1.10/32 via 3.1.1.1, static metric [1/0], add succeed, active state IP ARP: creating incomplete entry for IP address: 3.1.1.1 interface GigabitEthernet0/1 IP ARP: sent req src 10.0.0.1 30e4.db16.7791, dst 3.1.1.1 0000.0000.0000 GigabitEthernet0/1 R00#show ip cef 3.1.1.10 3.1.1.10/32 nexthop 3.1.1.1 GigabitEthernet0/1 


And this brings us to another important point. Specifying an interface in a static route allows you to bypass many checks, since the static route no longer needs to perform recursive queries to the RIB in search of a path to next hop, and when it is added, it will not trigger triggers on other routes. But this does not cancel the main requirement: next hop must be resolved to a specific interface, and that interface must be up. The fact that there will no longer be recursive requests to the RIB means that the specified IP address of the next hop is located directly behind the interface, and will most likely respond to the arp request (from the router's point of view). If proxy arp is enabled on the neighbor Gi0 / 1 router, then in response to an arp request, it will most likely return its mac address, and everything will be fine. Is that an extra entry in the arp table ...



But still, you shouldn't do that.



It is necessary to mention another important point. The static route should, in theory, disappear from the routing table as soon as it stops resolving. But in practice there are many situations where next hop disappears, but at the same time a static route remains for some time. For example, when next hop is resolved via a route received from a dynamic routing protocol. The thing is that the process that monitors the presence of next hop in the RIB cannot always receive a notification about the disappearance of the route, and it has to periodically (every 60 seconds by default) recheck whether everything is fine. This will cause a noticeable delay in network convergence.



To change the check interval, for example, for 10 seconds, you can use the command:

 ip route static adjust-time 10 

Source: https://habr.com/ru/post/174167/



All Articles