The previous part defined the terminology. We now turn to a detailed explanation of "how it works." (to those who can not wait to "take and run" can do with the help of the manuals on the site xen'a).

In this part: more about the pools and an overview of the device hosts and a little bit about Xen.

Pula

The entire XCP is a single pool (support for multiple pools is in very distant plans). In principle, a company can have multiple pools. If you use different hardware for the hosts, you will have to form pools within the same hardware; in the degenerate case, this means “one host - one pool”. In this configuration, there is no live migration capability, and the only option is to export / import a virtual machine (slowly and automatically). Accordingly, you need to have at least a pair of identical machines in order to get the possibility of live migration (identical means “exactly the same”, up to processor stepping). A little earlier, questions arose whether it was possible to collect different iron into the pool. The official answer is no. If you really want to, then you can, but all the glitches that might have arisen will be your own personal rake.
')
Each pool host (its device is discussed below) has complete information on the status of all machines in the pool (stores the entire pool base). Only one host, referred to as the "pool master", can perform operations. The wizard can easily change on the go (without rebooting and stopping the virtual machines), in the case of the wizard's “death” its role can be transferred to another host by force. If the former master then loads, then there will be a “pool split” - the division of the pool into two unequal parts, when the former master believes that he is the master, and all the others obey the new master. This problem is easily solved by changing the settings of the "former master".

You should play with the masters carefully, precisely because of the games with who the master is, I got the situation of " Ghost in the Xen " (when the virtual machine was started that was not in the list of cloud machines). However, since XCP 0.1.1, the situation has changed a bit, and the behavior of hosts in the absence of a live master has become more reasonable.


The master accepts commands and sends them to the slaves (the hosts in the pool that are not masters are called slaves). However, the pool can be managed from the console of any host - in this case, the entered commands are simply sent to the wizard and the user is shown the result. More information about the management below.

A pool stores a configuration that (with some exceptions) is completely identical for all hosts. Thanks to this, the virtual machine can run on any host, the hosts are indistinguishable from the point of view of the virtual machine operation (this is precisely why the name “XCP” includes the cloud - the platform allows you to forget about specific machines and work simply with many equivalent bricks).

Hosts may still have features at the lower level, but they are carefully disguised for the higher level. This is mainly related to network connectivity and storage — PBD and PIF on some hosts may be unplugged, that is, not connected. This can be either an administrative decision or an external reason (the port on the switch broke, the iSCSI target refused to connect to one of the hosts, etc.). In this case, the following situation occurs:

If the command to start / migrate the machine is given without specifying “where / where” (just xe vm-start , xe host-evacuate ), then “unsuitable” hosts are ignored, and the command is processed using the available hosts. If the administrator explicitly indicates where to start the virtual machine, they can tell him that the machine cannot be started on this host, because part of the host resources are not ready. In normal administrative use, it is not necessary to specify exactly “where to start the machine” (all machines are equivalent), so that “half-working” hosts do not cause problems.

Another interesting feature is Load Balancing. It is of two types - native to XCP and external. Native load balancing is performed only when the machine is turned on (there are two behavioral strategies: greedy and modest, greedy starts the machine where there are most resources, and modest where it has enough resources, that is, tries to occupy as few hosts as possible under the task). External balancing (usually with migration) is given to the user.

In addition, XCP provides a DMC (dynamic memory control) mechanism, a mechanism in which the boundaries of RAM are set for each virtual machine, and, if necessary, XCP can “pull” other machines to provide memory to a new neighbor. However, we will talk about memory management much later.

Host device

A full description of how Xen's virtualizer works somewhat beyond what is planned in the articles (if I ever feel like a guru enough in this area, I will write); For now, I will describe very superficially how the heart of XCP works - the Xen hypervisor. Xen is loaded before the OS boots and takes control of the processor, memory, and I / O operations. It is in this wording. Unlike VMWare, Xen knows nothing about disk operations or about the network. All he manages are interrupts, DMA, memory and processor. To ensure work with devices, the first operating system is loaded: (in XCP it is CentOS), which gets access to all physical devices, the right to command the hypervisor (to command, it does not have direct access to other domains or hypervisor memory) and the duty to serve virtual devices for virtual machines. In fact, from the point of view of the hypervisor, a commanding OS is just another virtual machine, no more.

In Xen terminology, the running machine is called a 'domain'. Important: domain destruction is just a “shutdown” of the virtual machine. Sometimes domains are destroyed even without shutting down the machine (for example, if the machine migrates from host to host, the domain is created on the receiving host and destroyed on the sending host).

The OS Commander is called dom0 (this is a generally accepted term and we will only use it later). Guest domains are called domU. In Xen, there is the concept of stubdomain (domains that are delegated to work with specific pieces of hardware and provide virtual pieces of hardware based on them), but in XCP it is not used. Just as the opportunity to “forward” the glands into the guest domains is not used. All hardware is in dom0. Point.

Dom0, in addition to a specially adapted kernel, contains several more components. It:

• xapi is the brain of XCP. Primary service, commanding the hypervisor, migrating and handling API calls. xapi is in some sense an analogue of xend in “ordinary” zen (this is the main architectural difference between xen and XCP).
• binding xen maintenance devices: blktap, xenstore, consoled (we will discuss them a little later)
• open vswitch (similarly, a little later, in short - this is a virtual switch for virtual networks).
• squeezed is a daemon providing dynamic memory control.
• stunnel, providing communication between the hosts in the pool (I forgot to mention that all the interaction between the hosts and the network is encrypted? Well, that is implied).
• xsconsole is a simple ncurses menu for managing a host using a menu. It starts up after the host boots.
• xe is a pretty cool console utility with a powerful autocomplete (when I say powerful, I mean really powerful), allowing you to do everything that the pool can do with the pool. In fact, with a little syntactic sugar for easy operation, it completely covers everything XenAPI.

Of course, besides this, there is a lot of auxiliary - from yum to openoffice Linux components for working with the network, iscsi, nfs, FC, etc. Of course, among them ssh-server and ntp-server.

By the way, about NTP ... This is the most important component for XCP, since the synchronization of time on hosts is vital for the normal migration of machines. As far as I know, operating XCP without configured ntp is extremely fraught. The source of time is not necessary to have a "global" - it is enough to synchronize with a common source of time.

When the host boots up, it loads Xen, which starts the dom0 kernel. Further downloads are described in detail in the RHEL / CentOS manuals. All other applications run as normal daemons. The entire infrastructure is divided into 4 parts (this is my personal division, and not generally accepted):

1. host as a computer (OS and application software)
2. Xen and his harness
3. XCP's cloud part
4. administrative part (console utilities, xsconsole, vncterm)

Now about each part (except the first, I hope, the centos will not cause confusion) more ...

Xen'ovskaya part, although hidden from the user, but still present. There is xenstored, there is a set of utilities for working with XenStore (xenstore-ls, -read, -write, -rm), there is a xc python module hidden in the giblets, there is xentop and xentrace. Especially you don’t need to think about it during normal operation, you need to remember about it if you are sawing something of your own, or if something incomprehensible happened.

The XCP part of it is actually a topic of conversation.

xapi provides the API, provides the http (s) server for it, controls the work of the virtual machines (in particular, it is xapi that performs the migration), sends the information to the master and receives commands from it. The master, accordingly, collects information, accepts and gives commands, sends changes to all hosts. The XCP database is stored entirely in memory, its dump is saved to disk. (No DBMS is used for this). xapi is written in OCaml, that lovers poking around in the source text can plunge into great bewilderment and confusion.

In addition, each xapi on the host receives commands from the master and executes them. These are not only commands regarding virtual machines, but also regarding the configuration of the host itself — network settings, restart / shutdown commands, management (highly mediated) of mounting block devices and NFS-ball, user management. Xapi represents the level of abstraction, which allows you to switch from terminology 'eth0 to host-123' to xe pif-param-list uuid=(someuuid) , and most importantly, to use it from any host and not from the specific one where eth0 is located .

xapi uses stunnel to organize secure connections between the master and the slaves and the slaves with each other ... Since encryption is not a free operation when transferring large amounts of data (I have met with this only when machines are being migrated, but the amount of data transferred is less than Guest system RAM) it can eat up a relatively decent percentage of the processor. In principle, if the pool is 10-15 hosts in size, then we can expect that the xapi + stunnel on the master will eat almost the entire core. (By the way, all dom0 in XCP is strictly single core). The load of slaves is significantly lower, I could not create more than 25% of the xapi load.

The administrative part is represented by the server of the consoles (talking about the console of a freshly installed viral machine), a primitive menu on the console of the host itself (you can even call it through ssh) and the most powerful control system, the command line xe. In addition to xe, XCP also contains a couple of dozen utilities, each of which is used in very rare cases. All staffing goes through 'xe'.

The peculiarity of xe is that it can work and not on the host pool (in particular, there is a windows version of xe). Xe is a separate package, it can be installed on any suitable machine (at least on the admin working station). Xe connects to the pool wizard using the API and, after authorization, allows you to manage the pool in much the same way as locally.

In addition to console utilities, there are many alternative options. This XenCenter (paid graphics utility from tsitriksa for Windows), and OpenXenManager (cross-platform XenCenter clone in python), and a few dozen graphical shells of varying degrees of convenience. One of the sponsors of XCP is actively promoting its version of the Xen Cloud Control System, which supports automatic balancing with the migrations of machines (my hands do not reach to see). However, full-fledged work (not causing WTF syndrome) is possible only from the command line or through the API, since any graphics system is forced to slightly simplify the relationship between objects.

We will talk more about administration and API much later.

Lastly, I’ll repeat in more detail how the pools and hosts relate.

A host can refer to one and only one pool. If the host is one, it is itself a pool. If a host is accepted into a “foreign” pool, then it “forgets” about its pool and accepts a foreign pool. The master, accordingly, does not forget anyone, but he accepts other people's hosts, just by adding them to the data of his “own” pool. If the host goes out of the pool, it removes the old data of its “own” pool from the backup (note: virtual machines are not stored in such a backup, so consider that joining and leaving the pool is destructive).

The pool cannot be destroyed, if you take the machines out of the pool, then as a result only one host, who is his own master, remains in the pool. He can not say "kill the pool", you can either take it to another pool, or demolish the system.

In the next part: VBD VDI PBD SR VIF PIF Open vSwitch