Parse BGP NOTIFICATION by RFC

Is it important to know the header formats of the transmitted data? In many training courses on networks, the analysis of protocol headers takes more or less time, but usually this does not work without it. Being descriptive in nature, their study often causes boredom, and the availability of automatic analysis tools does not improve the picture. Sometimes the headline does contain an interesting approach, but in most cases it all fits into one principle and the next new format is usually not surprising. The most interesting thing is, of course, all sorts of combinations of those options that affect the functionality, but is it worth remembering which options fit in the header in what order?

I can not say that I remember it, I still can not say that it bothers me in my work. I can't even say that I often have to see the headers in some kind of unprocessed form, because, as a rule, the messages in the syslog or on the console are already reformatted into coherent English sentences. But, sometimes, you have to look deeper, for example, this year was fruitful for such messages:

Ericsson SmartEdge

notification msg sent (nbr 192.0.2.1, context 0x40030044 32 bytes, repeated 89 times, code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 0708 0003 02ed 5bdc 3f01 

Cisco, the same on the other hand

 NOTIFICATION received from 192.0.2.2 (External AS 64496): code 3 (Update Message Error) subcode 4 (attribute flags error), Data: e0 07 08 00 03 02 ed 5b dc 3f 01 

Let's try to sort it out with our hands as written in RFC4271 . We will not look for the cause - just a header analysis. There will be many quotes and, most likely, nothing new for those who already know how to do it. For the rest, read on.

To make a variety, we will not limit ourselves to one message and analyze this issue from GitHub FRRouting:

 %NOTIFICATION: received from neighbor 192.168.0.1 3/5 (UPDATE Message Error/Attribute Length Error) 3298 bytes 50 02 0c de 02 ff 00 00 0c b9 00 00 00 ae 00 04 00 3e 00 04 00 3e 00 04 00 35 00 04 00 35 ...    00 04 00 35 

What can we read? We see the message type, its value and sub-value, the IP address of the neighborhood (which is already known to us) and the hexadecimal string, which we do not understand.

Let's start with the main one from the IANA website, where there are all the necessary codes with reference to RFC4271 . If you gain strength and read the RFC from cover to cover, then everything should become clear, but we will try to understand only the format of our two messages without affecting the behavioral aspects.

Parse NOTIFICATION

From the content, by name, we are satisfied with paragraph 4.5 of the NOTIFICATION Message Format . Having opened that, we really find the format of the header we need and a list of all codes with reference to the relevant sections (we’ll hide all the quotes below under spoilers):

  0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Error code | Error subcode | Data (variable) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Error Code: This 1-octet unsigned integer indicates the type of NOTIFICATION. The following Error Codes have been defined: Error Code Symbolic Name Reference 1 Message Header Error Section 6.1 2 OPEN Message Error Section 6.2 3 UPDATE Message Error Section 6.3 4 Hold Timer Expired Section 6.5 5 Finite State Machine Error Section 6.6 6 Cease Section 6.7 

Our section 6.3 UPDATE Message Error with error code 3. We see it in all messages, along with the qualifying code:

code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 0708 0003 02ed 5bdc 3f01

code 3 (Update Message Error) subcode 4 (attribute flags error), Data: e0 07 08 00 03 02 ed 5b dc 3f 01

3/5 (UPDATE Message Error/Attribute Length Error) 3298 bytes 50 02 0c de 02 ff 00 00 0c b9 00 00 00 ae 00 04 00 3e 00 04 00 3e 00 04 00 35 00 04 00 35 ... 00 04 00 35

There is no tablet here, so we read several pages of text. Each paragraph describes the behavior in different situations, which is characterized by its error code.

In our case, the flags used cannot be used with this attribute:

 If any recognized attribute has Attribute Flags that conflict with the Attribute Type Code, then the Error Subcode MUST be set to Attribute Flags Error. The Data field MUST contain the erroneous attribute (type, length, and value). 

and - the actual length of the attribute does not match the expected one:

 If any recognized attribute has an Attribute Length that conflicts with the expected length (based on the attribute type code), then the Error Subcode MUST be set to Attribute Length Error. The Data field MUST contain the erroneous attribute (type, length, and value). 

The most important thing that is written here and that will help us in the future is that besides the error codes, the message MUST contain an erroneous attribute from the UPDATE message in the data field in the TLV format (type, length, value). That same hexadecimal string that we cannot yet interpret. However, we still have a problem in identifying this field, now associated with various agreements adopted by manufacturers of devices for displaying the log of logs.

In the example with Cisco, the beginning of the word "Data" is clearly indicated:
code 3 (Update Message Error) subcode 4 (attribute flags error), Data : e0 07 08 00 03 02 ed 5b dc 3f 01

In other cases, there is no such binding. In this case, the line with Ericsson contains much more data than that of Cisco, although we know that these messages are identical. Therefore, we go back a step to the NOTIFICATION description and see that we have completely disassembled it; the variable length data field is the last one in the message.

Let's go higher in the hierarchy and start reading from the beginning of section 4.1 :

  0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + + | Marker | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 

Each message starts with a field:

 This 16-octet field is included for compatibility; it MUST be set to all ones. 

16 bytes long and consisting of all units ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff . We see the same pattern in the Ericsson log:
code 3/4 (update: attribute flags error) - 0000 0000 ( ffff ffff ffff ffff ffff ffff ffff ffff ) 0020 0303 04e0 0708 0003 02ed 5bdc 3f01

Next, the length field is two bytes:
code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff ( 0020 ) 0303 04e0 0708 0003 02ed 5bdc 3f01 , with a value of 32 decimal, which is the same as decoding from the notification msg sent (nbr 192.0.2.1, context 0x40030044 32 bytes , and even further:

 This 1-octet unsigned integer indicates the type code of the message. This document defines the following type codes: 1 - OPEN 2 - UPDATE 3 - NOTIFICATION 4 - KEEPALIVE 

and we know that it is number 3 - NOTIFICATION: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 ( 03 ) 03 04e0 0708 0003 02ed 5bdc 3f01

And then the message itself and what we recognized ( section 4.5 ) 03 04 - the type and subtype of the error: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 03 ( 03 04 ) e0 0708 0003 02ed 5bdc 3f01

Ericsson repeated to us not only the data field that starts with e0 , but the entire generated message. In the Cisco log, the byte sequence begins with it. The FRRouting log also contains the completely decrypted BGP header of the NOTIFICATION message followed by the data field, to which we return to decoding.

We are directed to the description of the UPDATE format, since we expect there to find a description of attribute formats in order to understand what we are getting. UPDATE contains many fields, the attributes are set in the Path Attributes variable length field:

 A variable-length sequence of path attributes is present in every UPDATE message, except for an UPDATE message that carries only the withdrawn routes. Each path attribute is a triple <attribute type, attribute length, attribute value> of variable length. 

Let's start with the Type field, which consists of two single-byte parts:

  Attribute Type is a two-octet field that consists of the Attribute Flags octet, followed by the Attribute Type Code octet. 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Attr. Flags |Attr. Type Code| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 

Here again, you need to read everything, because everything is decorated in solid text. It follows that the flag field uses the first 4 bits from 0 to 3, and the second byte defines the attribute itself. Let's put it all in the table:

Bits 4-7 should be 0 when transmitting and ignored when receiving, we do not pay attention to them. Bit 3 defines the size of the length field in the TLV:

 The fourth high-order bit (bit 3) of the Attribute Flags octet is the Extended Length bit. It defines whether the Attribute Length is one octet (if set to 0) or two octets (if set to 1). 

Of particular interest is the partial flag, which can be set for transitive optional attributes. The flag description is scattered in several places throughout the RFC. There are the following situations of its use:

1. If the attribute is not recognized, this flag is set, and the attribute itself is passed on - section 9 :
   
   Update Message handling

    If an optional transitive attribute is unrecognized, the Partial bit (the third high-order bit) in the attribute flags octet is set to 1, and the attribute is retained for propagation to other BGP speakers. 

2. If the attribute is recognized by the next speaker, the flag is still not reset - section 5 :
   
   Path Attributes

    Paths with unrecognized transitive optional attributes SHOULD be accepted. If a path with an unrecognized transitive optional attribute is accepted and passed to other BGP peers, then the unrecognized transitive optional attribute of that path MUST be passed, along with the path, to other BGP peers with the Partial bit in the Attribute Flags octet set to 1. If a path with a recognized, transitive optional attribute is accepted and passed along to other BGP peers and the Partial bit in the Attribute Flags octet is set to 1 by some previous AS, it MUST NOT be set back to 0 by the current AS. 

3. And if not the original speaker wants to add a transitive attribute, then the flag is also set:
   
   (ibid)

    New, transitive optional attributes MAY be attached to the path by the originator or by any other BGP speaker in the path. If they are not attached by the originator, the Partial bit in the Attribute Flags octet is set to 1. 

Let's start parsing UPDATE

The flags field in the first case e0: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04 ( e0 ) 0708 0003 02ed 5bdc 3f01 - in binary 1110 0000 - optional, transitive set by the partial flag, the length field is specified in one byte.

code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 ( 07 ) 08 0003 02ed 5bdc 3f01 - attribute code 7 - AGGREGATOR

Data size is 8 bytes: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 07 ( 08 ) 0003 02ed 5bdc 3f01

In the second case, the well-known attribute (50 - 0101 0000), the size field is two bytes, AS_PATH (02), length 3294 bytes (0c de): 3/5 (UPDATE Message Error/Attribute Length Error) 3298 bytes ( 50 02 0c de ) 02 ff 00 00 0c b9 00 00 00 ae 00 04 00 3e 00 04 00 3e 00 04 00 35 00 04 00 35 ... 00 04 00 35

The rest of the string is the data field of the attribute itself. To do this, move on in section 4.3 :

 AGGREGATOR is an optional transitive attribute of length 6. The attribute contains the last AS number that formed the aggregate route (encoded as 2 octets), followed by the IP address of the BGP speaker that formed the aggregate route (encoded as 4 octets). This SHOULD be the same address as the one used for the BGP Identifier of the speaker. 

You should consider working with 32-bit ASn RFC6793 to get the following result:

4 bytes is AS197357: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 0708 ( 0003 02ed ) 5bdc 3f01 ,

and 4 bytes IP address 91.220.63.1: code 3/4 (update: attribute flags error) - 0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303 04e0 0708 0003 02ed ( 5bdc 3f01 ) - in the amount of 8 bytes, as indicated in the length field.

In the second case, AS_PATH, TLV itself:

 The path segment type is a 1-octet length field with the following values defined: Value Segment Type 1 AS_SET: unordered set of ASes a route in the UPDATE message has traversed 2 AS_SEQUENCE: ordered set of ASes a route in the UPDATE message has traversed The path segment length is a 1-octet length field, containing the number of ASes (not the number of octets) in the path segment value field. 

Type and length fields are 1 byte long.

Type 2, data size (ff - 255 ASn): 3/5 (UPDATE Message Error/Attribute Length Error) 3298 bytes 50 02 0c de ( 02 ff ) 00 00 0c b9 00 00 00 ae 00 04 00 3e 00 04 00 3e 00 04 00 35 00 04 00 35 ... 00 04 00 35

The data itself is AS numbers of 4 bytes each: 3/5 (UPDATE Message Error/Attribute Length Error) 3298 bytes 50 02 0c de 02 ff ( 00 00 0c b9 ) ( 00 00 00 ae ) ( 00 04 00 3e ) ( 00 04 00 3e ) ( 00 04 00 35 ) ( 00 04 00 35 ) ... 00 04 00 35 - AS3257, AS174, AS262206 AS262206, AS262197, AS262197, AS262197, AS262197 ...

Got to the end. It seemed not difficult? .. but boring. Of course, this did not give us much in understanding the problem, since most of the document, which we didn’t even touch upon, describes not the formats, but the required behavior. But at least we know the direction to the source.

Unfortunately, or maybe not - the RFC is not an easy walk to read, in some places the data are tabulated, in others exactly in the same cases they are written directly in the text and you have to read a lot to understand the structure. But I am glad that they are there and the manufacturers were able to read them and bring them to realization.

PS In the case of FRRouting there is a patch , hinting that the problem is not in format.

In the first case, apparently, a patch is also needed, since formally I didn’t see any problems in sharing flags for the AGGREGATOR attribute, maybe someone saw it.