Scalable networks in Openstack. Part 2: VlanManager
Posted by: Piotr Siwczak
In the first part of the article, I described the main network operation mode in OpenStack, in particular, the FlatManager network manager and its addition, FlatDHCPManager. In this article I will talk about VlanManager. While managers working in flat mode are designed for simple and small deployments, VlanManager is suitable for large internal clouds and public clouds. As the name suggests, VlanManager relies on the use of virtual local area networks (“virtual LANs”). The purpose of VLANs is to divide the physical network into separate broadcast domains (in such a way that groups of nodes in different virtual networks do not see each other). VlanManager is trying to fix two major drawbacks of network managers, namely:
-Lack of scalability (managers operating in flat mode rely on a single L2 broadcast domain for all OpenStack installations)
- Lack of user isolation (a single pool of IP addresses that is used by all users)
')
In this article, I concentrate on VlanManager, which uses multi-node network mode in OpenStack. Outside the sandbox, this is safer than single-node mode, since multi-node mode does not suffer from a single failure resulting from running a separate instance of the nova-network service on the entire openstack cluster. However, it is possible to use VlanManager in single-node mode. (You can read more about the ratio of “multi-node” and “single-node” mode here ).
When working with managers in the flat mode during the network configuration process, the administrator usually performs the following actions:
-Creates one large network with fixed ip-addresses (usually with a network mask of 16 or less bits), which is shared by all users:
-Creates users
-After the users create their copies, assigns each a free IP address from the common pool of IP addresses.
Typically, in this mode, IP addresses are allocated instances as follows:
tenant_1:
tenant_2:
We see that the tenant_1 and tenant_2 instances are in the same IP network, 10.0.0.0.
If VlanManager is used, the administrator acts as follows:
-Creates a new user, writes tenantID
-Creates a dedicated fixed ip-network for a new user:
-After creating an instance of the user will automatically be assigned an IP address from the private pool of IP addresses.
Thus, compared to FlatDHCPManager, we additionally define two things for the network:
-Connection network with a specific user (--project_id =). Thus, no one else can take IP addresses from the network.
- Allocation of a separate virtual network for this network (--vlan = 102).
From now on, as soon as the user creates a new virtual machine, it automatically obtains an IP address from the allocated pool. It is also placed on a dedicated virtual network vlan, which is automatically created and maintained by the OpenStack platform. Thus, if we have created two different networks for two users, the situation is as follows:
tenant_1:
tenant2:
It is clear that the instances of users are located in different pools of IP addresses. But how are virtual networks supported?
VlanManager here does three things:
-Creates a dedicated bridge for the user's network on the compute node.
-Creates the vlan interface on top of the eth0 interface of the compute node.
- Starts and configures the dnsmasq process corresponding to the bridge so that the user instance can be loaded from it.
Let's assume that a user called “t1” creates its own instance of t1_vm_1 . It falls on one of the compute nodes. Here is the network diagram:
We see that a dedicated bridge “br102” was created with a vlan interface called “vlan102”. In addition, the dnsmasq process was created, which listens at 10.0.2.1. When an instance t1_vm_1 is loaded, it gets its address from the dnsmasq process based on a static lease (for details on how dnsmasq is managed by the OpenStack platform, see the previous article ).
Now, let's assume that the user “t1” creates another instance called t1_vm_2 , and he accidentally ends up on the same compute node as the previous instance created:
Both instances are connected to the same bridge, since they belong to the same user, and thus they are on the same dedicated network of users. In addition, they receive their DHCP configuration from a single dnsmasq server.
Now, imagine that the user “t2” creates its first instance . It also falls on a single computing node with the user “t1”. In addition, a dedicated bridge, the vlan interface, and the dnsmasq process are configured for its network:
Thus, it turns out that, depending on the number of users, the situation is quite normal when you have a sufficiently large number of network bridges and dnsmasq processes, and all work on a single computing node.
There is nothing strange in this situation - OpenStack will automatically manage all of them. In contrast to the use of managers in the flat mode, here both instances of users are located on different bridges that are not connected to each other. This will ensure traffic separation at the L2 level. In the case of the “t1” user, ARP broadcasts transmitted via br102, and then via vlan102, are not visible on the br103 and vlan103 ports, and vice versa.
Above, we talked about how it works on a single computing node. Most likely you will use multiple compute nodes. Usually we aim for the largest number of compute nodes. Then, most likely, the user “t1 ″ will be distributed among several compute nodes. This means that the dedicated network must also be distributed among several compute nodes. However, it must meet two requirements:
- Must be provided to communicate instances of t1, which are located on different physical computer nodes
- The t1 network distributed across several computing nodes should be isolated from other user networks
Typically, compute nodes are connected to the network with one cable. We would like several users to share this channel without seeing each other’s traffic.
There is a technology that satisfies this requirement - Vlan tagging (802.1q). Technically, each Ethernet frame is supplemented with a 12-bit field called VID (Vlan ID), which has the vlan number. Frames with the same Vlan tag belong to the same L2 translation domain; thus, devices with traffic tagged with the same Vlan ID may be reported.
Therefore, it is clear that it is possible to isolate networks of users by marking them with different Vlan IDs.
How does it work in practice? Take a look at the diagrams above.
Traffic for user “t1” leaves the compute node via the interface “vlan102 ″. Vlan102 is a virtual interface connected to eth0. Its sole purpose is to tag frames with the number “102 ″ using the 802.1q protocol.
The traffic for the user “t2” leaves the computing node via the interface “vlan103 ″, which is tagged with tag 103. Since the stream has different vlan tags, the traffic“ t1′s ”does not intersect with the traffic“ t2 '.
They have no idea about each other, although they both use the same physical interface eth0, and then the ports and trunk of the switch.
Then we need to tell the switch that the tagged traffic needs to be sent through its ports. This is accomplished by putting this switch port in trunk mode (as opposed to the default “access” mode). In short, a trunk allows the switch to send frames tagged by VLANs; More information on vlan trunks can be found in this article (802.1q). At this time, the configuration of the switch is performed by the system administrator . Openstack does not perform this configuration on its own. Not all switches support vlan trunking. This must be checked in advance before purchasing a switch.
In addition, if you are using devstack + virtualbox to experiment with VlanManager in a virtual environment, make sure that you select “PCNET - Fast III” as an adapter to connect to your VLAN.
Having done this, we come to the following communication model:
The thick black line from the compute nodes to the switch is a physical channel (cable). Vlan traffic with tags 102 and 103 (red and green dotted lines) is carried over the same cable. Traffic does not mix (two lines never intersect).
What does the traffic look like when the user “t1” wants to send a packet from 10.0.2.2 to 10.0.2.5?
-Package runs from 10.0.2.2 to the bridge br102 and up to vlan102, where it is assigned tag 102.
-The traffic goes beyond the switch that processes the vlan tags. Once it reaches the second compute node, its vlan tag is learned.
-Based on the results of the study, the computational node decides to send it to the vlan102 interface.
-Vlan102 removes the Vlan ID field from the packet so that the packet can reach the instances (instances do not have a tagged interface).
-Then he goes through br102 and reaches the address 10.0.2.5.
VlanManager setup
To configure the VlanManager network settings in OpenStack, put the following lines in the nova.conf file:
#We point OpenStack to use VlanManager here:
network_manager = nova.network.manager.VlanManager
#Interface on which virtual interfaces will be created:
vlan_interface = eth0
#The first tag number for private vlans
# (in this case, vlan numbers lower than 100 can serve our
#internal purposes and will not be consumed by tenants):
vlan_start = 100
In any case, VlanManager is the most sophisticated network interaction model currently offered in OpenStack. L2 scalability and isolation of traffic of different users is provided.
However, this manager has his limitations. For example, for each user network, it assigns ip pools (L3 level) to vlan networks (L2 level) (remember? Each user's network is defined by a pair of ip + vlan pool). Thus, it is not possible to make two different users use the same IP addressing scheme independently in different L2 domains.
In addition, the field length of the vlan tag is only 12 bits, which allows you to create a maximum of 4096 vlan. This means that you can have no more than 4096 potential users, not so much on the scale of the cloud.
These restrictions will be circumvented by emerging solutions, such as Quantum , the new network manager for the OpenStack platform and software-defined networking .
In the next article in this series, I will explain how floating ip addresses work.
Original article in English
In the first part of the article, I described the main network operation mode in OpenStack, in particular, the FlatManager network manager and its addition, FlatDHCPManager. In this article I will talk about VlanManager. While managers working in flat mode are designed for simple and small deployments, VlanManager is suitable for large internal clouds and public clouds. As the name suggests, VlanManager relies on the use of virtual local area networks (“virtual LANs”). The purpose of VLANs is to divide the physical network into separate broadcast domains (in such a way that groups of nodes in different virtual networks do not see each other). VlanManager is trying to fix two major drawbacks of network managers, namely:
-Lack of scalability (managers operating in flat mode rely on a single L2 broadcast domain for all OpenStack installations)
- Lack of user isolation (a single pool of IP addresses that is used by all users)
')
In this article, I concentrate on VlanManager, which uses multi-node network mode in OpenStack. Outside the sandbox, this is safer than single-node mode, since multi-node mode does not suffer from a single failure resulting from running a separate instance of the nova-network service on the entire openstack cluster. However, it is possible to use VlanManager in single-node mode. (You can read more about the ratio of “multi-node” and “single-node” mode here ).
Differences between managers working in the “flat” mode and VlanManager
When working with managers in the flat mode during the network configuration process, the administrator usually performs the following actions:
-Creates one large network with fixed ip-addresses (usually with a network mask of 16 or less bits), which is shared by all users:
nova-manage network create --fixed_range_v4=10.0.0.0/16 --label=public-Creates users
-After the users create their copies, assigns each a free IP address from the common pool of IP addresses.
Typically, in this mode, IP addresses are allocated instances as follows:
tenant_1:
tenant_2:
We see that the tenant_1 and tenant_2 instances are in the same IP network, 10.0.0.0.
If VlanManager is used, the administrator acts as follows:
-Creates a new user, writes tenantID
-Creates a dedicated fixed ip-network for a new user:
nova-manage network create --fixed_range_v4=10.0.1.0/24 --vlan=102 \
--project_id="tenantID"-After creating an instance of the user will automatically be assigned an IP address from the private pool of IP addresses.
Thus, compared to FlatDHCPManager, we additionally define two things for the network:
-Connection network with a specific user (--project_id =). Thus, no one else can take IP addresses from the network.
- Allocation of a separate virtual network for this network (--vlan = 102).
From now on, as soon as the user creates a new virtual machine, it automatically obtains an IP address from the allocated pool. It is also placed on a dedicated virtual network vlan, which is automatically created and maintained by the OpenStack platform. Thus, if we have created two different networks for two users, the situation is as follows:
tenant_1:
tenant2:
It is clear that the instances of users are located in different pools of IP addresses. But how are virtual networks supported?
How VlanManager configures network settings
VlanManager here does three things:
-Creates a dedicated bridge for the user's network on the compute node.
-Creates the vlan interface on top of the eth0 interface of the compute node.
- Starts and configures the dnsmasq process corresponding to the bridge so that the user instance can be loaded from it.
Let's assume that a user called “t1” creates its own instance of t1_vm_1 . It falls on one of the compute nodes. Here is the network diagram:
We see that a dedicated bridge “br102” was created with a vlan interface called “vlan102”. In addition, the dnsmasq process was created, which listens at 10.0.2.1. When an instance t1_vm_1 is loaded, it gets its address from the dnsmasq process based on a static lease (for details on how dnsmasq is managed by the OpenStack platform, see the previous article ).
Now, let's assume that the user “t1” creates another instance called t1_vm_2 , and he accidentally ends up on the same compute node as the previous instance created:
Both instances are connected to the same bridge, since they belong to the same user, and thus they are on the same dedicated network of users. In addition, they receive their DHCP configuration from a single dnsmasq server.
Now, imagine that the user “t2” creates its first instance . It also falls on a single computing node with the user “t1”. In addition, a dedicated bridge, the vlan interface, and the dnsmasq process are configured for its network:
Thus, it turns out that, depending on the number of users, the situation is quite normal when you have a sufficiently large number of network bridges and dnsmasq processes, and all work on a single computing node.
There is nothing strange in this situation - OpenStack will automatically manage all of them. In contrast to the use of managers in the flat mode, here both instances of users are located on different bridges that are not connected to each other. This will ensure traffic separation at the L2 level. In the case of the “t1” user, ARP broadcasts transmitted via br102, and then via vlan102, are not visible on the br103 and vlan103 ports, and vice versa.
Support for user networks on multiple computing sites
Above, we talked about how it works on a single computing node. Most likely you will use multiple compute nodes. Usually we aim for the largest number of compute nodes. Then, most likely, the user “t1 ″ will be distributed among several compute nodes. This means that the dedicated network must also be distributed among several compute nodes. However, it must meet two requirements:
- Must be provided to communicate instances of t1, which are located on different physical computer nodes
- The t1 network distributed across several computing nodes should be isolated from other user networks
Typically, compute nodes are connected to the network with one cable. We would like several users to share this channel without seeing each other’s traffic.
There is a technology that satisfies this requirement - Vlan tagging (802.1q). Technically, each Ethernet frame is supplemented with a 12-bit field called VID (Vlan ID), which has the vlan number. Frames with the same Vlan tag belong to the same L2 translation domain; thus, devices with traffic tagged with the same Vlan ID may be reported.
Therefore, it is clear that it is possible to isolate networks of users by marking them with different Vlan IDs.
How does it work in practice? Take a look at the diagrams above.
Traffic for user “t1” leaves the compute node via the interface “vlan102 ″. Vlan102 is a virtual interface connected to eth0. Its sole purpose is to tag frames with the number “102 ″ using the 802.1q protocol.
The traffic for the user “t2” leaves the computing node via the interface “vlan103 ″, which is tagged with tag 103. Since the stream has different vlan tags, the traffic“ t1′s ”does not intersect with the traffic“ t2 '.
They have no idea about each other, although they both use the same physical interface eth0, and then the ports and trunk of the switch.
Then we need to tell the switch that the tagged traffic needs to be sent through its ports. This is accomplished by putting this switch port in trunk mode (as opposed to the default “access” mode). In short, a trunk allows the switch to send frames tagged by VLANs; More information on vlan trunks can be found in this article (802.1q). At this time, the configuration of the switch is performed by the system administrator . Openstack does not perform this configuration on its own. Not all switches support vlan trunking. This must be checked in advance before purchasing a switch.
In addition, if you are using devstack + virtualbox to experiment with VlanManager in a virtual environment, make sure that you select “PCNET - Fast III” as an adapter to connect to your VLAN.
Having done this, we come to the following communication model:
The thick black line from the compute nodes to the switch is a physical channel (cable). Vlan traffic with tags 102 and 103 (red and green dotted lines) is carried over the same cable. Traffic does not mix (two lines never intersect).
What does the traffic look like when the user “t1” wants to send a packet from 10.0.2.2 to 10.0.2.5?
-Package runs from 10.0.2.2 to the bridge br102 and up to vlan102, where it is assigned tag 102.
-The traffic goes beyond the switch that processes the vlan tags. Once it reaches the second compute node, its vlan tag is learned.
-Based on the results of the study, the computational node decides to send it to the vlan102 interface.
-Vlan102 removes the Vlan ID field from the packet so that the packet can reach the instances (instances do not have a tagged interface).
-Then he goes through br102 and reaches the address 10.0.2.5.
VlanManager setup
To configure the VlanManager network settings in OpenStack, put the following lines in the nova.conf file:
#We point OpenStack to use VlanManager here:
network_manager = nova.network.manager.VlanManager
#Interface on which virtual interfaces will be created:
vlan_interface = eth0
#The first tag number for private vlans
# (in this case, vlan numbers lower than 100 can serve our
#internal purposes and will not be consumed by tenants):
vlan_start = 100
Conclusion
In any case, VlanManager is the most sophisticated network interaction model currently offered in OpenStack. L2 scalability and isolation of traffic of different users is provided.
However, this manager has his limitations. For example, for each user network, it assigns ip pools (L3 level) to vlan networks (L2 level) (remember? Each user's network is defined by a pair of ip + vlan pool). Thus, it is not possible to make two different users use the same IP addressing scheme independently in different L2 domains.
In addition, the field length of the vlan tag is only 12 bits, which allows you to create a maximum of 4096 vlan. This means that you can have no more than 4096 potential users, not so much on the scale of the cloud.
These restrictions will be circumvented by emerging solutions, such as Quantum , the new network manager for the OpenStack platform and software-defined networking .
In the next article in this series, I will explain how floating ip addresses work.
Original article in English
Source: https://habr.com/ru/post/181354/
All Articles