Your own AWS. Part 1
Hello again!
No it's not fez
Last time, I briefly explained what the open-source market has to create a pocket cloud.
Now it is time to tell you about OpenStack itself and its implementation in the form of Mirantis OpenStack 5.1.
')
Instead of a thousand words
For a start, I will briefly tell you what OpenStack consists of. So, Horizon is the web interface itself, most of the work with the cloud takes place through it. In rare cases, you will need to do something from the console. Nova - is engaged in provisioning your virtualok. Everything related to libvirt setup, quota distribution, the choice of physical hardware for placement is located in nova, also this component is responsible for one of the network options - nova-network. Neutron (in the past Quantum) - is responsible for everything connected with the network - interfaces, virtual routers, dhcp, routing. At the core lies the Open vSwitch . If something cannot be done via the web interface, then most likely it is implemented in the console. Cinder - is responsible for working with block devices, all questions on issuing additional disks to your virtual machine are sent here. Glance - here are stored the original images of virtualoks. Snapshots of working systems are also stored here. Swift - object storage. It has an extensive API for working with files, compatible with Amazon S3. Cellometr - monitoring everything and everyone. Collects data from all components of the cloud, warehouses, analyzes. Works through REST API. Keystone is the last and most important component. Performs authorization and authentication of all other components, as well as users, the speed of its work directly depends on how quickly you will get access to other components. Can integrate with LDAP. Heat - with its help you can automate the installation of virtual machines of any configuration. It uses HOT (Heat orchestration template) - in fact, a yaml-file with a description of the main parameters and a separate userdata section in which you can write both sh-scripts and bat.
Also in OpenStack Juno will be added a couple of new components. Ironic - work with "bare" iron. In essence, what Fuel is doing right now is loading on PXE, working with IPMI and other amenities. Trove - database as a service (DBaaS). More details here .
What is it you can look at the link at the beginning of the article. Now I will tell you what is configured here and how. The first thing you see after the installation is the login window, followed by the environment creation button. Here, the basic parameters of your cloud are described, such as whether we build a wake-up cloud or not, what to use as a backend - LVM or Ceph, which network we want, we build on real hardware (KVM), virtual (QEMU) or even just want to manage the already working vSphere ESXi. For the latter, you must have vCenter installed. As the OS, you can choose either CentOS 6.5 or Ubuntu 12.04.4. My configuration looks like non-HA / Ceph / KVM / Neutron GRE, respectively, some of the things described in the article may not be suitable for another scheme. Many questions can cause the choice of the network, some settings depend on this parameter in the future.
Disclaimer: openstack divides the network into segments. Management network - for the work of the OpenStack service commands, from the same network your servers will receive ip-addresses. Storage - a network that drives all traffic from your storage. Public is your real network, it is routed “to the world” and from it you issue addresses for virtual computers in the cloud. You can share networks both physically and using VLAN. Fuel requires another, untagged network. It will be live DHCP + PXE and it will be carried out checking the availability of servers.
Clickable
Nova-network - transferred to the status of deprecated. Only suitable for all home use when you do not need to share a cloud between different customers. Able to work only with layer 3 of the OSI model.
Neutron with VLAN - requires 802.1Q support on the router. The melanox driver with the SR-IOV plugin and iSER also works with this option. It is assumed that you know the number of clients in advance, because it requires you to specify the final number of VLAN IDs used. There may be problems with CentOS, there are “VLAN splinters” in Fuel to eliminate them.
Neutron with GRE is an alternative network option. Connectivity between nodes is performed using GRE tunnels, this creates an additional load on the processor, but eliminates the need to count in advance the number of clients and configure VLANs on routers. Also, unlike the previous version, there is no need for a separate network interface.
Disclaimer 2: Currently there is a limitation in the Fuel UI when setting up the neutron network - Public and Floating IP must be from the same network segment (CIDR). Theoretically, this can be done after installing all components of the cloud through the Open vSwitch CLI. More information about the difference between networks and examples of setting up some models of switches can be found here . I also recommend that you read the wonderful article about VxLAN.
I would describe it as “network RAID”. This is a distributed file system that provides both block devices and object storage. The system is critical to timing, so the time on all servers must be synchronized. It has very good documentation , because I will not be engaged in its translation here. Fuel is quite good * copes with the setting, so that in the basic configuration, you practically do not need to finish anything with a file.
Disk configuration window in Fuel
The disk can be divided into 4 logical parts. And if everything seems to be clear with the first three, then Virtual Storage has been asking me questions for a long time. As it turned out, it can not be made less than 5GB, but this is still half the trouble. The main problem came out when I tried to use qcow2 images to create virtual machines - this section was necessary for OpenStack to convert the image to raw and put it into cinder. It was also required to create snapshots with qcow2. As a result, I had to spend a few days catching non-obvious errors from the logs and reading the documentation in order to go to work only with raw images of the system.
Focus on columns (total)
And finally, a little about the allocation of resources. Initially, Fuel configures cpu_allocation_ratio = 8.0, which means that you can select 8 "virtual" for 1 real cpu of your server. You can change this parameter in /etc/nova/nova.conf and after restarting openstack-nova-api should see it in the webmord, but because of a bug in the API this is not yet possible. The same problem will arise if you change ram_allocation_ratio (1 is by default). How applications behave inside virtuallok when all available memory is exhausted twice (ram_allocation_ratio = 2) I have yet to figure out. But remembering how it worked in OpenVZ, I do not advise you to change this parameter greatly.
In the next, and most likely final, article, I will try to tell how to build several hypervisors (KVM + Docker) within one cloud, how to automate the configuration of virtual machines using Heat and Murano, as well as a little about monitoring and searching for a narrow neck in your cloud.
No it's not fez
Last time, I briefly explained what the open-source market has to create a pocket cloud.
Now it is time to tell you about OpenStack itself and its implementation in the form of Mirantis OpenStack 5.1.
')
Openstack
Instead of a thousand words
For a start, I will briefly tell you what OpenStack consists of. So, Horizon is the web interface itself, most of the work with the cloud takes place through it. In rare cases, you will need to do something from the console. Nova - is engaged in provisioning your virtualok. Everything related to libvirt setup, quota distribution, the choice of physical hardware for placement is located in nova, also this component is responsible for one of the network options - nova-network. Neutron (in the past Quantum) - is responsible for everything connected with the network - interfaces, virtual routers, dhcp, routing. At the core lies the Open vSwitch . If something cannot be done via the web interface, then most likely it is implemented in the console. Cinder - is responsible for working with block devices, all questions on issuing additional disks to your virtual machine are sent here. Glance - here are stored the original images of virtualoks. Snapshots of working systems are also stored here. Swift - object storage. It has an extensive API for working with files, compatible with Amazon S3. Cellometr - monitoring everything and everyone. Collects data from all components of the cloud, warehouses, analyzes. Works through REST API. Keystone is the last and most important component. Performs authorization and authentication of all other components, as well as users, the speed of its work directly depends on how quickly you will get access to other components. Can integrate with LDAP. Heat - with its help you can automate the installation of virtual machines of any configuration. It uses HOT (Heat orchestration template) - in fact, a yaml-file with a description of the main parameters and a separate userdata section in which you can write both sh-scripts and bat.
Also in OpenStack Juno will be added a couple of new components. Ironic - work with "bare" iron. In essence, what Fuel is doing right now is loading on PXE, working with IPMI and other amenities. Trove - database as a service (DBaaS). More details here .
Fuel
What is it you can look at the link at the beginning of the article. Now I will tell you what is configured here and how. The first thing you see after the installation is the login window, followed by the environment creation button. Here, the basic parameters of your cloud are described, such as whether we build a wake-up cloud or not, what to use as a backend - LVM or Ceph, which network we want, we build on real hardware (KVM), virtual (QEMU) or even just want to manage the already working vSphere ESXi. For the latter, you must have vCenter installed. As the OS, you can choose either CentOS 6.5 or Ubuntu 12.04.4. My configuration looks like non-HA / Ceph / KVM / Neutron GRE, respectively, some of the things described in the article may not be suitable for another scheme. Many questions can cause the choice of the network, some settings depend on this parameter in the future.
Disclaimer: openstack divides the network into segments. Management network - for the work of the OpenStack service commands, from the same network your servers will receive ip-addresses. Storage - a network that drives all traffic from your storage. Public is your real network, it is routed “to the world” and from it you issue addresses for virtual computers in the cloud. You can share networks both physically and using VLAN. Fuel requires another, untagged network. It will be live DHCP + PXE and it will be carried out checking the availability of servers.
Clickable
Nova-network - transferred to the status of deprecated. Only suitable for all home use when you do not need to share a cloud between different customers. Able to work only with layer 3 of the OSI model.
Neutron with VLAN - requires 802.1Q support on the router. The melanox driver with the SR-IOV plugin and iSER also works with this option. It is assumed that you know the number of clients in advance, because it requires you to specify the final number of VLAN IDs used. There may be problems with CentOS, there are “VLAN splinters” in Fuel to eliminate them.
Neutron with GRE is an alternative network option. Connectivity between nodes is performed using GRE tunnels, this creates an additional load on the processor, but eliminates the need to count in advance the number of clients and configure VLANs on routers. Also, unlike the previous version, there is no need for a separate network interface.
Disclaimer 2: Currently there is a limitation in the Fuel UI when setting up the neutron network - Public and Floating IP must be from the same network segment (CIDR). Theoretically, this can be done after installing all components of the cloud through the Open vSwitch CLI. More information about the difference between networks and examples of setting up some models of switches can be found here . I also recommend that you read the wonderful article about VxLAN.
Ceph
I would describe it as “network RAID”. This is a distributed file system that provides both block devices and object storage. The system is critical to timing, so the time on all servers must be synchronized. It has very good documentation , because I will not be engaged in its translation here. Fuel is quite good * copes with the setting, so that in the basic configuration, you practically do not need to finish anything with a file.
Disk configuration window in Fuel
The disk can be divided into 4 logical parts. And if everything seems to be clear with the first three, then Virtual Storage has been asking me questions for a long time. As it turned out, it can not be made less than 5GB, but this is still half the trouble. The main problem came out when I tried to use qcow2 images to create virtual machines - this section was necessary for OpenStack to convert the image to raw and put it into cinder. It was also required to create snapshots with qcow2. As a result, I had to spend a few days catching non-obvious errors from the logs and reading the documentation in order to go to work only with raw images of the system.
Hidden text
* - There are some problems with Ceph as a backend for glance / cinder. In part, they are caused by errors in the puppet scripts of Fuel itself, in part these are features of the OpenStack logic. For example, ceph can work normally only with RAW images. If you use Qcow2 (which, incidentally, out of the box implies copy-on-write, unlike raw), then OpenStack to work with this image will cause 'qemu-img convert', which gives a noticeable load on the disk system, so on the processor. Also, when working with raw, there are currently problems with creating snapshots — instead of making the same CoW, Ceph has to copy the entire image. And this in turn dramatically increases the consumption of space. True to the release of Mirantis OpenStack 6 they promised to fix this problem. You should also carefully monitor the free space on the controllers, where Fuel has a ceph monitor. As soon as you have less than 5% of free space on the root partition, ceph-mon is frayed and problems begin to arise with access to ceph-osd.
There is one more not very pleasant feature in Horizon itself - it incorrectly considers the place available on storages. So, for example, if you have 5 servers on which total 1Tb Ceph is stored, in the dashboard you will see 5Tb of available space. The fact is that ceph reports the total amount of storage from each node, and horizon simply summarizes this data.
There is one more not very pleasant feature in Horizon itself - it incorrectly considers the place available on storages. So, for example, if you have 5 servers on which total 1Tb Ceph is stored, in the dashboard you will see 5Tb of available space. The fact is that ceph reports the total amount of storage from each node, and horizon simply summarizes this data.
Focus on columns (total)
And finally, a little about the allocation of resources. Initially, Fuel configures cpu_allocation_ratio = 8.0, which means that you can select 8 "virtual" for 1 real cpu of your server. You can change this parameter in /etc/nova/nova.conf and after restarting openstack-nova-api should see it in the webmord, but because of a bug in the API this is not yet possible. The same problem will arise if you change ram_allocation_ratio (1 is by default). How applications behave inside virtuallok when all available memory is exhausted twice (ram_allocation_ratio = 2) I have yet to figure out. But remembering how it worked in OpenVZ, I do not advise you to change this parameter greatly.
In the next, and most likely final, article, I will try to tell how to build several hypervisors (KVM + Docker) within one cloud, how to automate the configuration of virtual machines using Heat and Murano, as well as a little about monitoring and searching for a narrow neck in your cloud.
Source: https://habr.com/ru/post/239535/
All Articles