SDN: revolution or evolution? Yandex Workshop
Hi, my name is Daniel Ginsburg, I work in Yandex as a network architect.
In my report, I tried to give my definition of the concept Software-defined Network, which today is understood in the industry both too broadly and too narrowly. My story also touched upon the historical roots of the SDN, its future, and practical deployment issues.
')
Video of the report
To begin, let us try to answer the question, do we all have a good network? Maybe we do not need to redo anything? I am sure that this is not the case, and now I will try to substantiate my point of view.
What is wrong now in the networked world? The first thing that kills us is complexity. Our network solutions, our networks are complex. The difficulty prevents us from scaling, managing networks, making them fragile. Difficulty is our enemy. The transfer of useful traffic (data-plane is complicated. The data-plane of our glands knows a lot of things: to forward millions of encapsulations, to make a million different tunnels (and we continue to invent new ones). There are people who build absolutely horrible constructions from this.
For example, they drive a multicast through IRB , which sticks out in VPLS , under which link aggregation groups lie. Such a thing is impossible to scale, impossible to debug. And it is expensive, expensive in terms of iron and in terms of working time.
All this wealth of data-plane capabilities must somehow be managed. And since this whole economy is so complicated, it is difficult to manage it.
We build wild sandwiches from protocols, and our control protocols evolve momentarily.
My favorite (though not the most glaring) example of the momentary is the Multiprotocol BGP . In order to announce a route on it, next hop in this announcement must have the same address family (v4, v6 or something else). When we want a v4 route with v6 next hop, we begin to invent new RFCs. Accordingly, we have n NLRI , then n 2 RFC potentially appears. This is a terrible, flawed and inexpressive abstraction.
When I create a network, I first of all imagine how the traffic should flow. From here I immediately imagine how my forwarding tables should be filled. And then I begin to think, how can I, with the help of existing protocols, fill these tables in the way I want. And this is not always the case. It is because of inexpressive abstractions.
All this is very expensive. A network element that has tremendous complexity in data-plane and control-plane , in a particular case, uses, say, 5% of opportunities, and is paid for all 100. And it is paid not only with the price of a piece of hardware, but also with the complexity of the software that is used on it, bugs in it, network fragility.
When I need a new feature in the hardware, I come to the vendor. If I'm lucky, I will get this feature in a year, and refracted through the mind of a vendor who knows "how best."
Among other things, because of this, we have extremely difficult automation. Each vendor invents its own automatic configuration method. If I have a new vendor, I need to modify the entire control and automation system. Not from scratch, of course, but significantly.
Obviously, you need to do something with it. For example, remake everything and call it SDN.
I cannot give an exact definition of this term, no one can. Some associate it with the OpenFlow initiative, some use it simply as an adjective in the synonymous series “good”, “fashionable”, “modern”.
So what is SDN? This is something that will give us comfortable abstractions. They will be simple, understandable natural, expressive and complete. This will allow you to implement the necessary functions of the network yourself, and not wait for the vendor to do this. Naturally, automation will be simplified.
First, to separate the passage of traffic (data-plane) and signaling / control (control plane). Secondly, to make the data-plane elements as simple as possible and, thirdly, to centralize the control-plane. This all allows you to quickly and easily implement convenient abstractions in the control-plane independently of the data-plane.
Now we will sort all these ideas in order.
The first question that arises in connection with the separation of data-plane and control-plane is how much should they be separated?
This is not a new idea, they have done it or tried to do it many times already. For example, any modern router is designed. There are data-plane elements: forwarding engines that take packets and decide where to forward them, there is a switching matrix connecting them. The control module exists separately. User traffic flows through, control traffic enters the control engine, which programs the forwarding engine.
This approach allows you to scale a separate network element. On the other hand, such an approach extremely complicates the failure model. Such composite construction breaks in parts. Moreover, even the data-plane can break partially. For example, if you have a broken switching matrix, and the control engine did not notice this, all user traffic will simply be dropped. Management traffic will continue to go, and everyone will think that our network element is alive.
On the diagram, this is not the case, but the blue traffic goes through a separate internal out-of-band control network. And this network may also refuse. If we take and divide the data-plane and the control-plane not within the same network element, but across the entire network, the out-of-band-management network will become as complex as our data-plane. Accordingly, it will break as often and badly as the main network.
The question also arises: how is it to manage the control network if it is as complex as the network transmitting useful traffic?
Let us turn to the idea of ​​simplifying the data-plane elements. We have to come up with a fairly simple and general abstraction for managing the network element. Simple, understandable, quite flexible and expressive.
How simple should this abstraction be? If you don’t go deep into the details, the data-plane looks like this: a packet comes to us, we take fields from the header, lookup in the forwarding table, modify the packet headers and send to the next interface.
The reality is somewhat more complicated. Here is a very simplified diagram of what is happening in a single chip of a network element:
What abstraction of data-plane elements does OpenFlow offer us? We have fields, there is a table of lukap, we do a search, determine what needs to be done and implement it. Everything is extremely simple.
One of the problems with abstraction in OpenFlow 1.0 is a combinatorial explosion. For example, take a simple artificial task. We need to pass traffic to N hosts on the same M TCP ports, and reset the rest of the traffic. In OpenFlow 1.0, for this we need NxM records, we have to list all the host-port combinations. Scale this will not work.
In OpenFlow 1.1 and all subsequent versions, the idea of ​​multiple tables appears. This eliminates the combinatorial explosion. The idea is relatively elegant, but it has a problem, it reflects badly on the gland. The fact is that iron, which costs reasonable money, does not know how to make Lukapas in any order.
In modern iron there is an opportunity to do multi-stage processing. We find the corresponding entry in the table, and then we carry out several processing steps: add a label, overwrite the title, then something else, etc. This is called indirect next hops: one next hop in this case refers to the other by the index of the table. Those. finding the next is easy. However, in the construction with multiple tables, we have no direct links, and we will be forced to do a multifield lookup several times, which is quite expensive. However, we get rid of the combinatorial explosion and the same problem with N hosts and M ports is solved much easier: we will not have two tables: N records in one and M records in another.
The Forwarding Abstractions working group (FAWG) operates within the framework of the Open Networking Foundation, which develops and standardizes the OpenFlow. The idea being developed by this group is to interrogate network elements about the sequence of lukap they can do. Those. when the controller makes a request, the network element provides it with information about the entire monstrous pipeline that is presented in the previous picture.
But where did the abstraction go?
If we have introduced an abstraction, and in order to effectively use and implement it, we have to tell the controller about all the implementation details, which means that we either do something that is not abstracted or we are not abstracted there.
Let's try to determine how centralized the control-plane should be. There is room for maneuver. Centralization and decentralization is an oscillatory process.
Those who have been working for a long time in the industry will most likely agree with me. One example, in which we have already gone through several cycles. We buy BRAS and put it in the middle. We understand that we do not scale anything, spreading it around the edges. Now control is lost, we return to centralization again, buy a BRAS more and put it in the middle. And so on.
This is a natural process that cannot be avoided. It is dictated not only by fashion and marketing, but also by completely objective factors, technical progress. With centralization and decentralization in the control-plane, everything will be exactly the same.
What do the radicals from the OpenFlow approach suggest to us in this regard? It is assumed that the network element itself is extremely stupid and incapable of any independent actions, what the controller says to it is what it does.
Accordingly, the controller will have to respond to all events on the network. If an element is unable to transfer traffic from one path to another by itself, then the controller should see this failure and react to it. This design is very dangerous in terms of scaling. We'll have to take a big controller again, divide it into several, distribute it so that it can cope with the load. Those. we are again in a round of centralization / decentralization.
The main thing is to temper the revolutionary fervor and approach the SDN from an evolutionary point of view.
All that SDN promises us, all that we talked about in the beginning is really good. This is what we need to strive for. We must make every effort to make these promises come true.
So what do we need to do for this?
First, standardize one data-plane mechanism. There is no reasonable way to control the whole variety of protocols, constructions and mechanisms that exist in today's data-plane.
Secondly, you need to choose the right element abstraction, which would be simple and at the same time flexible. In addition, it is necessary to provide the ability to implement network functions both centrally and decentralized. You also need to find a way to combine differently implemented functions.
Both management styles — centralized and decentralized — are important. There are naturally centralized functions, for example, high-level policies.
And at the same time, there are functions decentralized in nature, for example, rerouting in case of failures. In addition, there are functions that can be implemented as it is profitable here and now. These functions include the calculation of the path. For example, inside the data center I have no need to do traffic-engineering, there is simply no need to save bandwidth. But in the WAN traffic-engineering may already be required, and it can and sometimes needs to be done centrally.
A network can have a domain structure: for example, DC and WAN, access and core, RAN and blcackhaul. Different parts of the network perform different functions, they are arranged differently, and different types of control are suitable for them. Accordingly, in order for the network to be unified, different parts of the network need to be “stitched” and “superimposed” on each other. On top of the domains we can make transports, and on top of these transports - “services”.
Proper abstraction of an element really should hide its complexity. The wild scheme, which we considered at the beginning, in fact displays only what is happening inside a single chip. When we talk about a multi-slot distributed box, it gets even more complicated. And it needs to be hidden, we do not want to know what is happening there.
This abstraction should allow to combine different management styles. If some function is implemented in the classical way in the form of distributed signaling, and the other in the form of centralized control, they should be able to interact with each other.
One promising approach is the I2F i2rs . This is a slightly higher level model than the proposed OpenFlow. For example, i2rs operates not with entries in the forwarding table, but with the concept of a route.
What technical progress allows us to do today and in the near future?
Today, there are two of the most current applications of SDN: network virtualization and service linking. We need to understand that we have a virtualization level and a transport level, the task of which is to drag packages.
We have IP / MPLS as a transport. We are able to make a simple MPLS network, to make transport.
95% of the MPLS control-plane complexity is concentrated in the organization of services. It is here that efforts must be focused.
Virtualization must start from the host. Modern processors allow us to do it quite cheaply. We make a virtual switch on the host, and virtualization begins with it.
Such a network should be managed hybrid: centralized controls should be distributed. This will give us the opportunity to take network separation policies and compile them into certain rules of hybrid controllers that manage the imposed network.
At the same time, whatever our policies, we do not touch the transport. It does not change anything, he does not know about our policies or about the imposed network.
Summing up, I would like to repeat the idea that SDN, no matter what is meant by it, really promises us good things, they need to be implemented. However, this needs to be done with the right tools, rather than equating SDN and OpenFlow.
In my report, I tried to give my definition of the concept Software-defined Network, which today is understood in the industry both too broadly and too narrowly. My story also touched upon the historical roots of the SDN, its future, and practical deployment issues.
')
Video of the report
To begin, let us try to answer the question, do we all have a good network? Maybe we do not need to redo anything? I am sure that this is not the case, and now I will try to substantiate my point of view.
Complexity
What is wrong now in the networked world? The first thing that kills us is complexity. Our network solutions, our networks are complex. The difficulty prevents us from scaling, managing networks, making them fragile. Difficulty is our enemy. The transfer of useful traffic (data-plane is complicated. The data-plane of our glands knows a lot of things: to forward millions of encapsulations, to make a million different tunnels (and we continue to invent new ones). There are people who build absolutely horrible constructions from this.
For example, they drive a multicast through IRB , which sticks out in VPLS , under which link aggregation groups lie. Such a thing is impossible to scale, impossible to debug. And it is expensive, expensive in terms of iron and in terms of working time.
All this wealth of data-plane capabilities must somehow be managed. And since this whole economy is so complicated, it is difficult to manage it.
We build wild sandwiches from protocols, and our control protocols evolve momentarily.
My favorite (though not the most glaring) example of the momentary is the Multiprotocol BGP . In order to announce a route on it, next hop in this announcement must have the same address family (v4, v6 or something else). When we want a v4 route with v6 next hop, we begin to invent new RFCs. Accordingly, we have n NLRI , then n 2 RFC potentially appears. This is a terrible, flawed and inexpressive abstraction.
When I create a network, I first of all imagine how the traffic should flow. From here I immediately imagine how my forwarding tables should be filled. And then I begin to think, how can I, with the help of existing protocols, fill these tables in the way I want. And this is not always the case. It is because of inexpressive abstractions.
All this is very expensive. A network element that has tremendous complexity in data-plane and control-plane , in a particular case, uses, say, 5% of opportunities, and is paid for all 100. And it is paid not only with the price of a piece of hardware, but also with the complexity of the software that is used on it, bugs in it, network fragility.
Feature velocity
When I need a new feature in the hardware, I come to the vendor. If I'm lucky, I will get this feature in a year, and refracted through the mind of a vendor who knows "how best."
Among other things, because of this, we have extremely difficult automation. Each vendor invents its own automatic configuration method. If I have a new vendor, I need to modify the entire control and automation system. Not from scratch, of course, but significantly.
What to do with it
Obviously, you need to do something with it. For example, remake everything and call it SDN.
I cannot give an exact definition of this term, no one can. Some associate it with the OpenFlow initiative, some use it simply as an adjective in the synonymous series “good”, “fashionable”, “modern”.
So what is SDN? This is something that will give us comfortable abstractions. They will be simple, understandable natural, expressive and complete. This will allow you to implement the necessary functions of the network yourself, and not wait for the vendor to do this. Naturally, automation will be simplified.
First, to separate the passage of traffic (data-plane) and signaling / control (control plane). Secondly, to make the data-plane elements as simple as possible and, thirdly, to centralize the control-plane. This all allows you to quickly and easily implement convenient abstractions in the control-plane independently of the data-plane.
Now we will sort all these ideas in order.
Separation of data-plane and control-plane
The first question that arises in connection with the separation of data-plane and control-plane is how much should they be separated?
This is not a new idea, they have done it or tried to do it many times already. For example, any modern router is designed. There are data-plane elements: forwarding engines that take packets and decide where to forward them, there is a switching matrix connecting them. The control module exists separately. User traffic flows through, control traffic enters the control engine, which programs the forwarding engine.
This approach allows you to scale a separate network element. On the other hand, such an approach extremely complicates the failure model. Such composite construction breaks in parts. Moreover, even the data-plane can break partially. For example, if you have a broken switching matrix, and the control engine did not notice this, all user traffic will simply be dropped. Management traffic will continue to go, and everyone will think that our network element is alive.
On the diagram, this is not the case, but the blue traffic goes through a separate internal out-of-band control network. And this network may also refuse. If we take and divide the data-plane and the control-plane not within the same network element, but across the entire network, the out-of-band-management network will become as complex as our data-plane. Accordingly, it will break as often and badly as the main network.
The question also arises: how is it to manage the control network if it is as complex as the network transmitting useful traffic?
Simplify network elements
Let us turn to the idea of ​​simplifying the data-plane elements. We have to come up with a fairly simple and general abstraction for managing the network element. Simple, understandable, quite flexible and expressive.
How simple should this abstraction be? If you don’t go deep into the details, the data-plane looks like this: a packet comes to us, we take fields from the header, lookup in the forwarding table, modify the packet headers and send to the next interface.
The reality is somewhat more complicated. Here is a very simplified diagram of what is happening in a single chip of a network element:
What abstraction of data-plane elements does OpenFlow offer us? We have fields, there is a table of lukap, we do a search, determine what needs to be done and implement it. Everything is extremely simple.
One of the problems with abstraction in OpenFlow 1.0 is a combinatorial explosion. For example, take a simple artificial task. We need to pass traffic to N hosts on the same M TCP ports, and reset the rest of the traffic. In OpenFlow 1.0, for this we need NxM records, we have to list all the host-port combinations. Scale this will not work.
In OpenFlow 1.1 and all subsequent versions, the idea of ​​multiple tables appears. This eliminates the combinatorial explosion. The idea is relatively elegant, but it has a problem, it reflects badly on the gland. The fact is that iron, which costs reasonable money, does not know how to make Lukapas in any order.
In modern iron there is an opportunity to do multi-stage processing. We find the corresponding entry in the table, and then we carry out several processing steps: add a label, overwrite the title, then something else, etc. This is called indirect next hops: one next hop in this case refers to the other by the index of the table. Those. finding the next is easy. However, in the construction with multiple tables, we have no direct links, and we will be forced to do a multifield lookup several times, which is quite expensive. However, we get rid of the combinatorial explosion and the same problem with N hosts and M ports is solved much easier: we will not have two tables: N records in one and M records in another.
The Forwarding Abstractions working group (FAWG) operates within the framework of the Open Networking Foundation, which develops and standardizes the OpenFlow. The idea being developed by this group is to interrogate network elements about the sequence of lukap they can do. Those. when the controller makes a request, the network element provides it with information about the entire monstrous pipeline that is presented in the previous picture.
But where did the abstraction go?
If we have introduced an abstraction, and in order to effectively use and implement it, we have to tell the controller about all the implementation details, which means that we either do something that is not abstracted or we are not abstracted there.
Centralization and decentralization
Let's try to determine how centralized the control-plane should be. There is room for maneuver. Centralization and decentralization is an oscillatory process.
Those who have been working for a long time in the industry will most likely agree with me. One example, in which we have already gone through several cycles. We buy BRAS and put it in the middle. We understand that we do not scale anything, spreading it around the edges. Now control is lost, we return to centralization again, buy a BRAS more and put it in the middle. And so on.
This is a natural process that cannot be avoided. It is dictated not only by fashion and marketing, but also by completely objective factors, technical progress. With centralization and decentralization in the control-plane, everything will be exactly the same.
What do the radicals from the OpenFlow approach suggest to us in this regard? It is assumed that the network element itself is extremely stupid and incapable of any independent actions, what the controller says to it is what it does.
Accordingly, the controller will have to respond to all events on the network. If an element is unable to transfer traffic from one path to another by itself, then the controller should see this failure and react to it. This design is very dangerous in terms of scaling. We'll have to take a big controller again, divide it into several, distribute it so that it can cope with the load. Those. we are again in a round of centralization / decentralization.
So what to do in reality?
The main thing is to temper the revolutionary fervor and approach the SDN from an evolutionary point of view.
All that SDN promises us, all that we talked about in the beginning is really good. This is what we need to strive for. We must make every effort to make these promises come true.
So what do we need to do for this?
First, standardize one data-plane mechanism. There is no reasonable way to control the whole variety of protocols, constructions and mechanisms that exist in today's data-plane.
Secondly, you need to choose the right element abstraction, which would be simple and at the same time flexible. In addition, it is necessary to provide the ability to implement network functions both centrally and decentralized. You also need to find a way to combine differently implemented functions.
Both management styles — centralized and decentralized — are important. There are naturally centralized functions, for example, high-level policies.
And at the same time, there are functions decentralized in nature, for example, rerouting in case of failures. In addition, there are functions that can be implemented as it is profitable here and now. These functions include the calculation of the path. For example, inside the data center I have no need to do traffic-engineering, there is simply no need to save bandwidth. But in the WAN traffic-engineering may already be required, and it can and sometimes needs to be done centrally.
A network can have a domain structure: for example, DC and WAN, access and core, RAN and blcackhaul. Different parts of the network perform different functions, they are arranged differently, and different types of control are suitable for them. Accordingly, in order for the network to be unified, different parts of the network need to be “stitched” and “superimposed” on each other. On top of the domains we can make transports, and on top of these transports - “services”.
Proper abstraction of an element really should hide its complexity. The wild scheme, which we considered at the beginning, in fact displays only what is happening inside a single chip. When we talk about a multi-slot distributed box, it gets even more complicated. And it needs to be hidden, we do not want to know what is happening there.
This abstraction should allow to combine different management styles. If some function is implemented in the classical way in the form of distributed signaling, and the other in the form of centralized control, they should be able to interact with each other.
One promising approach is the I2F i2rs . This is a slightly higher level model than the proposed OpenFlow. For example, i2rs operates not with entries in the forwarding table, but with the concept of a route.
Where are we now
What technical progress allows us to do today and in the near future?
Today, there are two of the most current applications of SDN: network virtualization and service linking. We need to understand that we have a virtualization level and a transport level, the task of which is to drag packages.
We have IP / MPLS as a transport. We are able to make a simple MPLS network, to make transport.
95% of the MPLS control-plane complexity is concentrated in the organization of services. It is here that efforts must be focused.
Virtualization must start from the host. Modern processors allow us to do it quite cheaply. We make a virtual switch on the host, and virtualization begins with it.
Such a network should be managed hybrid: centralized controls should be distributed. This will give us the opportunity to take network separation policies and compile them into certain rules of hybrid controllers that manage the imposed network.
At the same time, whatever our policies, we do not touch the transport. It does not change anything, he does not know about our policies or about the imposed network.
Conclusion
Summing up, I would like to repeat the idea that SDN, no matter what is meant by it, really promises us good things, they need to be implemented. However, this needs to be done with the right tools, rather than equating SDN and OpenFlow.
Source: https://habr.com/ru/post/227061/
All Articles