Scalable networks in OpenStack. Part 1: Flat Topology
Posted by: Piotr Siwczak
Recently, networks in OpenStack have evolved from a simple, hardly usable model to a more advanced one with the ability to completely isolate the owners. Support for various user requirements in OpenStack is implemented using the so-called. “Network managers”. The network manager defines the network topology of a particular OpenStack deployment. Starting with the Essex version, the user can select one of three options for network managers: FlatManager, FlatDHCPManager, VlanManager. In this part of the article we will look at the first two, the second part will be set aside for the last.
The FlatManager and FlatDHCPManager managers have a lot in common. Both are based on the concept of bridged networking using a single bridge. As an example, consider a network consisting of several nodes.
')
For each computational node, 1 virtual bridge is created, whose name is specified in the Nova configuration using the option:
All virtual machines started by OpenStack join this dedicated bridge.
Network bridge on the OpenStack compute node
The main disadvantage of using a single bridge per compute node is the known limitation of network bridges in Linux — the bridge can be connected to only one physical interface (the restriction can be circumvented using VLAN, but they are not supported by FlatDHCPManager and FlatManager). For this reason, isolation at the second level is impossible, and all nodes operate in the same ARP domain.
At the core of FlatManager and FlatDHCPManager is the idea of ​​having 1 “flat” pool of IP addresses defined for the entire cluster. This address space is shared between all user machines, regardless of which owner they belong to. Each owner can pick up any address available in the pool.
Network Manager FlatManager provides the most primitive set of operations. Its role is only to connect the user machine to the bridge on the compute node. By default, the manager does not configure the IP on the user machine. This task falls on the shoulders of the system administrator and can be performed using an external DHCP server.
Network topology using FlatManager
Network manager FlatDHCPManager connects the specified user machine to the bridge and provides a DHCP server.
Thus, on each computing node:
• the network bridge receives the address from the “flat” pool of IP addresses
• the dnsmasq process is started, which listens to the IP address of the bridge interface
• the bridge acts as the default gateway for all user machines running on the compute node
Network topology using FlatDHCPManager
For the dnsmaq service, the FlatDHCPManager network manager on each compute node creates static entries to ensure that the same IP address is obtained for a given user machine. The record file is created based on the Nova database and consists of the MAC address, IP and machine name. It is envisaged that the dnsmasq server distributes addresses only to machines running locally on the compute node. This is achieved by filtering entries from the 'instances' table across the 'host' field. Also, the default gateway is set to the IP address of the bridge interface. In the following figure, you can see that the default gateway depends on which compute node the virtual machine is located on.
Network gateway for machines running on different compute nodes
The following is the routing table for the vm_1 and vm_3 machines, each of which has a different default gateway:
root @ vm_1: ~ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 eth0
root @ vm_3: ~ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.0.4 0.0.0.0 UG 0 0 0 eth0
By default, all virtual machines in a “flat” network can see each other, regardless of who owns them. Isolation can be achieved with the following flag from nova.conf:
This sets up iptables policies that block any traffic between machines (even within the same owner) until it is allowed through the security group.
From a practical point of view, “flat” network managers look suitable for homogeneous, rather small internally corporate clouds in which there are no owners as such, or their number is severely limited. Usually, the use case is a dynamically scalable web server or HPC cluster. For such tasks, it is usually sufficient to have a single IP address space, managed either by an external DCHP server or by the dnsmasq process provided by OpenStack. On the other hand, this solution is limited in terms of scalability, since all virtual machines must be in the same L2 domain.
Restrictions in scalability and division of ownership are, to varying degrees, removed in the network manager VlanManager, which we will discuss in our next post.
Original article in English
Recently, networks in OpenStack have evolved from a simple, hardly usable model to a more advanced one with the ability to completely isolate the owners. Support for various user requirements in OpenStack is implemented using the so-called. “Network managers”. The network manager defines the network topology of a particular OpenStack deployment. Starting with the Essex version, the user can select one of three options for network managers: FlatManager, FlatDHCPManager, VlanManager. In this part of the article we will look at the first two, the second part will be set aside for the last.
The FlatManager and FlatDHCPManager managers have a lot in common. Both are based on the concept of bridged networking using a single bridge. As an example, consider a network consisting of several nodes.
')
For each computational node, 1 virtual bridge is created, whose name is specified in the Nova configuration using the option:
flat_network_bridge=br100All virtual machines started by OpenStack join this dedicated bridge.
Network bridge on the OpenStack compute node
The main disadvantage of using a single bridge per compute node is the known limitation of network bridges in Linux — the bridge can be connected to only one physical interface (the restriction can be circumvented using VLAN, but they are not supported by FlatDHCPManager and FlatManager). For this reason, isolation at the second level is impossible, and all nodes operate in the same ARP domain.
At the core of FlatManager and FlatDHCPManager is the idea of ​​having 1 “flat” pool of IP addresses defined for the entire cluster. This address space is shared between all user machines, regardless of which owner they belong to. Each owner can pick up any address available in the pool.
Flatmanager
Network Manager FlatManager provides the most primitive set of operations. Its role is only to connect the user machine to the bridge on the compute node. By default, the manager does not configure the IP on the user machine. This task falls on the shoulders of the system administrator and can be performed using an external DHCP server.
Network topology using FlatManager
FlatDHCPManager
Network manager FlatDHCPManager connects the specified user machine to the bridge and provides a DHCP server.
Thus, on each computing node:
• the network bridge receives the address from the “flat” pool of IP addresses
• the dnsmasq process is started, which listens to the IP address of the bridge interface
• the bridge acts as the default gateway for all user machines running on the compute node
Network topology using FlatDHCPManager
For the dnsmaq service, the FlatDHCPManager network manager on each compute node creates static entries to ensure that the same IP address is obtained for a given user machine. The record file is created based on the Nova database and consists of the MAC address, IP and machine name. It is envisaged that the dnsmasq server distributes addresses only to machines running locally on the compute node. This is achieved by filtering entries from the 'instances' table across the 'host' field. Also, the default gateway is set to the IP address of the bridge interface. In the following figure, you can see that the default gateway depends on which compute node the virtual machine is located on.
Network gateway for machines running on different compute nodes
The following is the routing table for the vm_1 and vm_3 machines, each of which has a different default gateway:
root @ vm_1: ~ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 eth0
root @ vm_3: ~ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.0.4 0.0.0.0 UG 0 0 0 eth0
By default, all virtual machines in a “flat” network can see each other, regardless of who owns them. Isolation can be achieved with the following flag from nova.conf:
allow_same_net_traffic=FalseThis sets up iptables policies that block any traffic between machines (even within the same owner) until it is allowed through the security group.
From a practical point of view, “flat” network managers look suitable for homogeneous, rather small internally corporate clouds in which there are no owners as such, or their number is severely limited. Usually, the use case is a dynamically scalable web server or HPC cluster. For such tasks, it is usually sufficient to have a single IP address space, managed either by an external DCHP server or by the dnsmasq process provided by OpenStack. On the other hand, this solution is limited in terms of scalability, since all virtual machines must be in the same L2 domain.
Restrictions in scalability and division of ownership are, to varying degrees, removed in the network manager VlanManager, which we will discuss in our next post.
Original article in English
Source: https://habr.com/ru/post/180585/
All Articles