The myth of the futility of QoS without network overload
At work, I have several times come across the opinion that it is not necessary to configure QoS in a non-overloaded ethernet network for the successful functioning of services such as IPTV and VoIP. This opinion has cost me and my colleagues many nerve cells and hours to diagnose phantom problems, so I will try to tell as simply as possible why this opinion is wrong.
My name is Vladimir and I work as a network engineer in one of the smaller ISPs in St. Petersburg.
One of the services we provide is L2VPN for transport of IPTV streams. On the example of this service I will lead the story.
It all starts with contacting technical support from a client-operator with a complaint about the quality of IPTV - the picture is frayed (“artifacts”), the sound disappears, in general a standard set. IPTV in our network is classified into an assured forwarding queue, so the diagnosis is to run through the glands on the route and check that there are no losses in the AF queue for egress, and there are no physical errors on the ingress. After that, we cheerfully report to the client that no losses were found in our area of ​​responsibility, we recommend the client to look for a problem with himself or the IPTV provider, and go to drink tea with cookies.
')
But the client presses and continues to insist that we are to blame, but he is fine. We check everything again, we look at the correctness of the classifiers and labeling of packets from the client, a dialogue is started and at some stage we ask the question “how is your QoS configured on the network?”, To which we get the answer “no, we have interfaces even for 50 % not loaded so we do not need QoS ". Sigh and go.
Usually the download schedule for which everyone is watching has an interval of 5 minutes. If “real time” is a few seconds, starting from 1. Unfortunately and fortunately, modern network equipment operates not for 5 minutes and not for 1 second even, but for picoseconds. The fact that the interface was not 100% loaded in a second does not mean that it was not loaded in just a few milliseconds either.
Here we come to the conceptual concept - microburst . This is such a very short period of time when the amount of data received by the device becomes more than the interface is able to send.
Usually the first reaction - how so? We live in an era of high-speed interfaces! 10Gb / s is already commonplace, 40 and 100Gb / s are being introduced everywhere, and we are already waiting for 1Tb / s interfaces.
In fact, the higher the speed of the interfaces, the harder the microbursts and their effect on the network become.
The mechanism of occurrence is very simple, I will consider it on the example of three 1Gb / s interfaces, where the traffic from two of them goes through the third.
This is the only prerequisite for the emergence of microburst - that the speed of incoming (ingress) interfaces exceeds the speed of the outgoing (egress) interface. Nothing like? This is the traditional scheme of the aggregation level in the ethernet network - many ports (ingress) merge traffic into one uplink (egress). So everything is being built by networks - from telecom operators to data centers.
Each egress interface has a tx-ring send queue, which is a ring buffer. Packages are added there for sending to the network and of course this buffer has a finite size. But ingress interfaces on the sending side also have the same ring buffers that provide the same line-rate. What happens if they start sending traffic at the same time? Our egress interface does not have enough space in its tx-ring, since it will fill up twice as fast as it is able to send packets. The remaining packages need to be stored somewhere. In general, this is a different buffer, which we call the queue. While there is no space in the tx-ring, the package is stored in the queue and waits for free space in the tx-ring. But the trouble is - the queue memory is also finite. What happens if ingress interfaces work on line-rate long enough? The memory in the queue will also end. In this case, the new package is nowhere to be stored, and it will be dropped - this situation is called tail drop.
How long does it take for such a scenario to become a reality? Let's count.
The most difficult thing is to find the buffer capacity of the interface. Vendors are not very actively publish such information. But let's take, for example, a period of 200ms - it usually does not make sense to keep a packet in the queue longer, and this is already very much.
For the 1Gb / s interface, we need (1,000,000,000 * 0.2) / 8 = 25MB of memory. How long does it take to work on line-rate to two 1Gb / s interfaces in order to completely buffer the buffer? 200ms. This is the time during which 25MB is transmitted at a speed of 1Gb / s. Yes, ingress interfaces, we have two, but the egress interface does not sit idle, and sends data at the same speed, so 200ms.
This is relatively much. How about 10Gb / s ingress interface? How long will it take to reload 200ms buffer 1Gb / s interface? ~ 22ms. This is significantly less.
And how much memory does it take to store 200ms for a 10Gb / s interface? Already 250MB. This is not so much by modern standards, but the wind is blowing in this direction - speeds are growing, and in order to maintain the depth of the buffer, more and more memory is required, which results in engineering and economic problems, and the smaller the buffer, the faster microbursts will fill it.
It turns out the eternal question for vendor engineers - how much memory should the interface give to the hardware? A lot is expensive and every next millisecond becomes meaningless and meaningless. Little - microbursts will lead to large packet losses and complaints from customers.
For other scenarios, you can calculate it yourself, but the result is always the same - a fully clogged queue and tail drops, and on the graph the interface shelf doesn't even smell like that, and in any period - which is 5 minutes, which is 1 second.
This situation in packet networks is inevitable - the interface will work on the line-rate for less than a second, and there will already be losses. The only way to avoid it is to build a network so that the ingress speed never exceeds the egress speed, and this is impractical and unrealistic.
Further logic is already traceable and fairly obvious. There are packet losses, but QoS is not configured - priority traffic is not classified in any way and does not differ from other traffic, and falls into one general queue, where it has equal chances to be dropped.
What to do? Configure QoS. It is necessary to classify the priority traffic and place it in a separate queue of which allocate a larger amount of memory. Configure packet sending algorithms so that priority packets get into the tx-ring earlier than others - thus, their queue will be cleared faster.
For example, in our practice we use the following approach to queues:
Assured forwarding (AF) - "hold but deliver." In the AF queue, traffic is classified that requires guaranteed delivery, but is not sensitive to delays. This queue is allocated a large amount of memory, but relatively little space is given in the tx-ring, and packets get there later than others. A striking example of such traffic is IPTV - it is buffered on the client (VLC or STB), so it can be delayed, but the loss will turn into an image artifact.
Expedited forwarding (EF) - “deliver instantly or throw away.” This queue will be allocated a minimum (or no memory at all) for the queue, but the highest priority is set for getting into the tx-ring so that the package is sent as soon as possible. An example of traffic is VoIP. The voice cannot be delivered late, otherwise the telephony codec will not be able to correctly collect it - the subscriber will hear a croak. At the same time, the loss of individual packets on the overall quality of the voice does not greatly affect - it is so not perfect for people.
There is also network control (NC) and best effort (BE), for network management and everything else, respectively, and traffic can also be, for example, newsgroups, which is a hybrid between VoIP and IPTV, but this is a completely separate topic, and configure QoS for them should be separately in each network, depending on the topology and other factors. All together, in general, it looks like this (picture from the Cisco site):
I hope now you will configure QoS in your network?
My name is Vladimir and I work as a network engineer in one of the smaller ISPs in St. Petersburg.
One of the services we provide is L2VPN for transport of IPTV streams. On the example of this service I will lead the story.
It all starts with contacting technical support from a client-operator with a complaint about the quality of IPTV - the picture is frayed (“artifacts”), the sound disappears, in general a standard set. IPTV in our network is classified into an assured forwarding queue, so the diagnosis is to run through the glands on the route and check that there are no losses in the AF queue for egress, and there are no physical errors on the ingress. After that, we cheerfully report to the client that no losses were found in our area of ​​responsibility, we recommend the client to look for a problem with himself or the IPTV provider, and go to drink tea with cookies.
')
But the client presses and continues to insist that we are to blame, but he is fine. We check everything again, we look at the correctness of the classifiers and labeling of packets from the client, a dialogue is started and at some stage we ask the question “how is your QoS configured on the network?”, To which we get the answer “no, we have interfaces even for 50 % not loaded so we do not need QoS ". Sigh and go.
Usually the download schedule for which everyone is watching has an interval of 5 minutes. If “real time” is a few seconds, starting from 1. Unfortunately and fortunately, modern network equipment operates not for 5 minutes and not for 1 second even, but for picoseconds. The fact that the interface was not 100% loaded in a second does not mean that it was not loaded in just a few milliseconds either.
Here we come to the conceptual concept - microburst . This is such a very short period of time when the amount of data received by the device becomes more than the interface is able to send.
Usually the first reaction - how so? We live in an era of high-speed interfaces! 10Gb / s is already commonplace, 40 and 100Gb / s are being introduced everywhere, and we are already waiting for 1Tb / s interfaces.
In fact, the higher the speed of the interfaces, the harder the microbursts and their effect on the network become.
The mechanism of occurrence is very simple, I will consider it on the example of three 1Gb / s interfaces, where the traffic from two of them goes through the third.
This is the only prerequisite for the emergence of microburst - that the speed of incoming (ingress) interfaces exceeds the speed of the outgoing (egress) interface. Nothing like? This is the traditional scheme of the aggregation level in the ethernet network - many ports (ingress) merge traffic into one uplink (egress). So everything is being built by networks - from telecom operators to data centers.
Each egress interface has a tx-ring send queue, which is a ring buffer. Packages are added there for sending to the network and of course this buffer has a finite size. But ingress interfaces on the sending side also have the same ring buffers that provide the same line-rate. What happens if they start sending traffic at the same time? Our egress interface does not have enough space in its tx-ring, since it will fill up twice as fast as it is able to send packets. The remaining packages need to be stored somewhere. In general, this is a different buffer, which we call the queue. While there is no space in the tx-ring, the package is stored in the queue and waits for free space in the tx-ring. But the trouble is - the queue memory is also finite. What happens if ingress interfaces work on line-rate long enough? The memory in the queue will also end. In this case, the new package is nowhere to be stored, and it will be dropped - this situation is called tail drop.
How long does it take for such a scenario to become a reality? Let's count.
The most difficult thing is to find the buffer capacity of the interface. Vendors are not very actively publish such information. But let's take, for example, a period of 200ms - it usually does not make sense to keep a packet in the queue longer, and this is already very much.
For the 1Gb / s interface, we need (1,000,000,000 * 0.2) / 8 = 25MB of memory. How long does it take to work on line-rate to two 1Gb / s interfaces in order to completely buffer the buffer? 200ms. This is the time during which 25MB is transmitted at a speed of 1Gb / s. Yes, ingress interfaces, we have two, but the egress interface does not sit idle, and sends data at the same speed, so 200ms.
This is relatively much. How about 10Gb / s ingress interface? How long will it take to reload 200ms buffer 1Gb / s interface? ~ 22ms. This is significantly less.
And how much memory does it take to store 200ms for a 10Gb / s interface? Already 250MB. This is not so much by modern standards, but the wind is blowing in this direction - speeds are growing, and in order to maintain the depth of the buffer, more and more memory is required, which results in engineering and economic problems, and the smaller the buffer, the faster microbursts will fill it.
It turns out the eternal question for vendor engineers - how much memory should the interface give to the hardware? A lot is expensive and every next millisecond becomes meaningless and meaningless. Little - microbursts will lead to large packet losses and complaints from customers.
For other scenarios, you can calculate it yourself, but the result is always the same - a fully clogged queue and tail drops, and on the graph the interface shelf doesn't even smell like that, and in any period - which is 5 minutes, which is 1 second.
This situation in packet networks is inevitable - the interface will work on the line-rate for less than a second, and there will already be losses. The only way to avoid it is to build a network so that the ingress speed never exceeds the egress speed, and this is impractical and unrealistic.
Further logic is already traceable and fairly obvious. There are packet losses, but QoS is not configured - priority traffic is not classified in any way and does not differ from other traffic, and falls into one general queue, where it has equal chances to be dropped.
What to do? Configure QoS. It is necessary to classify the priority traffic and place it in a separate queue of which allocate a larger amount of memory. Configure packet sending algorithms so that priority packets get into the tx-ring earlier than others - thus, their queue will be cleared faster.
For example, in our practice we use the following approach to queues:
Assured forwarding (AF) - "hold but deliver." In the AF queue, traffic is classified that requires guaranteed delivery, but is not sensitive to delays. This queue is allocated a large amount of memory, but relatively little space is given in the tx-ring, and packets get there later than others. A striking example of such traffic is IPTV - it is buffered on the client (VLC or STB), so it can be delayed, but the loss will turn into an image artifact.
Expedited forwarding (EF) - “deliver instantly or throw away.” This queue will be allocated a minimum (or no memory at all) for the queue, but the highest priority is set for getting into the tx-ring so that the package is sent as soon as possible. An example of traffic is VoIP. The voice cannot be delivered late, otherwise the telephony codec will not be able to correctly collect it - the subscriber will hear a croak. At the same time, the loss of individual packets on the overall quality of the voice does not greatly affect - it is so not perfect for people.
There is also network control (NC) and best effort (BE), for network management and everything else, respectively, and traffic can also be, for example, newsgroups, which is a hybrid between VoIP and IPTV, but this is a completely separate topic, and configure QoS for them should be separately in each network, depending on the topology and other factors. All together, in general, it looks like this (picture from the Cisco site):
I hope now you will configure QoS in your network?
Source: https://habr.com/ru/post/246791/
All Articles