The global consequences of a single mistake in Quagga

^{On the left, you see the avatar savannah.gnu.org , where the Quagga repository lies. It seemed to us that he was coming to the event.}

About two weeks ago, the Qrator Radar team encountered an interesting network incident, the clarification of the circumstances of which resulted in an internal investigation-investigation, with the search for victims and perpetrators, as well as attempts to rectify the situation. 09/30/2017 our team drew attention to an unusually large number of "flashing" BGP sessions.

A quick analysis showed that all problem sessions had a number of similar symptoms: they broke off due to some kind of “broken” announcements, it also gave the impression that not only we are having sessions break - as a result, significant rerouting of routing occurs around the world.

Having spent some time figuring out the causes of this problem, we found that the broken announcement comes for the prefix 186.177.184.0/23 from AS262197 from Costa Rica, and in other areas this announcement comes without any errors, but with a prepend ... 563 . On the one hand, this policy is useless, since the effect of the policy of the prepend is fading away at values ​​in the region of 5, on the other hand, such an announcement remains legitimate.
')
We asked questions to the users with whom we had blinked sessions, and in most cases the BGP session on the user's side was configured using Quagga or Brocade routers (which is also based on Quagga). Having assembled a stand, we were convinced that the problem was indeed localized in Quagga implementation - after processing this route, Quagga created an announcement with several anomalies at once: the route turned out to be the wrong length of the AS_PATH attribute, as well as the very value of the AS_PATH attribute turned out to be incorrect.

As a result, in accordance with RFC4271 , a similar announcement led to the rupture of BGP sessions, which began to rupture BGP sessions around the world. For operators whose upstream providers use Quagga, this anomaly, due to the constant breaks in the BGP session, was able to result in partial or complete network unavailability. We found several hundred networks affected by this incident.

This anomaly was repeated last weekend, but for another prefix - 186.176.186.0/23 . It ended only yesterday, October 16, 2017, thus the total duration of the incident was more than two days.

At the end of last week’s implementation of Quagga, as part of release 1.2.2, an error was fixed with the length of the attribute AS_PATH . Our team also sent an additional patch that fixes the second problem with the contents of this attribute itself. As of last week, the error in the code was partially corrected in the framework of release 1.2.2 . The Qrator Radar team provided an additional patch for Quagga, correcting the formation of the incorrect value of the AS_PATH attribute, and yesterday it was adopted , but is not yet related to the current release.

We strongly recommend all Quagga users to upgrade their software to version 1.2.2 and keep track of subsequent updates. Also, in the appendix to this publication, we attach a patch that fixes both the above problems with Quagga announcements.

Quagga team thanks for your quick response.

Link to patch.
Check if your AS is offended.