The article deals with the principle of operation of xvc0 both from the point of view of the domU and from the point of view of the dom0, and also describes what is done with this console.
Immediately I warn you, the topic is interesting only to those who work closely with Xen.

The console in terms of Xen'a

The console and XenStore are two devices in a virtual machine that are not advertised through the XenStore, but are written to the holy of holies - start info page domain. The standard mechanism for all other devices is their announcement in the XenStore. It is clear that announcing XenStore in XenStore is somewhat inconvenient, so this device "must be from the very beginning."

The console could be placed in the list of "normal" devices. But he was taken to a special class and made “equal to XenStore” for the sake of ease of debugging and diagnostics. The sooner the kernel can write to the console (and we can read it), the more likely it is that the information on the console will be enough to determine the cause of the failure.
')
At the very lowest level, the console uses a ring buffer, and the size of one page of memory (4kb for i386 / x86_64). This ring buffer is divided in a ratio of 1024/2048, the first half is used for input (buffer for “pressed” buttons), the second for output (characters and esc-codes, which are displayed on the screen).

What is a ring buffer ? This is a data structure in which the address space is closed (after 4095 there is 0, 1, 2 ..., 4095, 0, 1 ... and so on to infinity). There are two markers in the buffer - the beginning of the queue and the end. Obviously, new data is written to the end, and read from the beginning. Or vice versa (this is a question of terms). Only one condition is checked - the end has not reached the beginning. Or the beginning has not reached the end.

I will leave the algorithmic part aside. It is much more important that the memory page, which is used for the ring buffer, is available at the same time as dom0 and domU (in general, it is available for two domains, but in our particular case one of the domains is dom0).

This moment is difficult to understand (and its explanation is far beyond the description of the “console”), but in short it looks like this: one and the same physical memory page (roughly speaking, the bits on the memory bar) have two addresses. One is in dom0, the second is in domU. Xen when accessing the machine to the memory translates the virtual address to the physical. And it does this so that both domains can write to the same physical page. At the same time the address of the page is different

“Everyone has his own,” by the way, not just a fad. If the address of the memory page for domU can be fixed, then it will not work for dom0 - there is a bunch of domU around it, and it is clear that each machine has its own memory page for the console.

So, we have a data transfer mechanism from domU to dom0 and back (this mechanism is actually used almost everywhere, but in our particular case it is used also for the operation of the console).

The second interaction mechanism is the event channel, a kind of interrupt in PV mode. In fact, these two mechanisms, plus hypercall, is all that Xen has for guest machines, so the console is little different from the network driver, for example.

Where does the domU find out about its console?

Console from domU point of view

When a virtual machine starts, a special program in the dom0 creates its domain. This program is called domain_builder. In addition to downloading the kernel code, it also fills in the start info page , information on which the domain can find everything else. The address of this page is transferred (in the case of x86) in the ESI register. On other architectures, of course, the methods are different.

It is very important that this information (about the position of the console and the number of the event channel) is not constant - the domain can migrate at any time, which, in fact, means that the contents of start_info_page suddenly change at the domain.

In this place there should have been approximately 50-60kb explanations of exactly how ring buffer and binding to the event channel work, but with the permission of the reader, I’ll skip this part, because the story is about the console, and not about how xen works A more detailed analysis up to the source text of each section is in the remarkable book “The Definitive Guide to the Xen Hypervisor”.

What happens to the console driver?

Like most zen devices, the driver is divided into two halves: the backend and the frontend.

Frontend works in domU, and that is what we will look at. This driver, after finding the corresponding data in start_info_page, creates a character device (to be exact, it is created by udev). The name of this device is the source for a decent amount of headaches for novice Xen administrators, since during the existence of the hypervisor, it has changed several times, and management can quite bear such nonsense in this matter.

The name can be / dev / xvc0, / dev / hvc0, / dev / ttyS0. The first is “xen virtual console”, the second is “h? virtual console ”is the common name for virtualisers in Linux (as h stands for I will not say), ttyS0 is a legacy of the early Xen era, when the virtual console was disguised as a serial port.

So, the device is created. What else? When loading the kernel, it “knows” where it is necessary to write messages of the loading time - to the virtual console.

The download ends ... And the novice administrator panics, because "it is stuck." Crones start up - that's all, the virtual machine stops responding to the buttons and does not ask for a login.

To explain what is happening (and what should be happening) let's think about what happens on a regular console when the download is complete. The init process reads the inittab and starts the processes specified in it (getty and the like). getty configures the console (or serial port) and runs / bin / login, which displays the long-awaited “login:”. If getty does not start, it means that there is no one to write login. "It hung."

The solution is simple: 0:12345:respawn:/sbin/getty xvc0 9600 in / etc / inittab.

Note that the name xvc0 must match what the driver creates. And the driver creates what was passed to it in the form of extra via the kernel command line arguments (console = xvc0). If not passed, then it is used by default, and it is rather difficult to guess it from the bay-flounder.

So, we register, we load, we see cherished login, we try to log in the root, and ... we are told that the name is wrong.

The fact is that login reads the / etc / securetty file to determine from which devices you can log in. In some distributions, xvc0 is not registered, and we need to register it with our hands. We register, hurray. Log in.

Sorry, lost the style, I do not write a guide "how to see the console", but a description of what is happening.

So what happens in a properly configured domU ?

1. Driver reads data from start_info_page
2. Driver creates character device
3. init runs getty for this device (it may not be init, by the way, you can run getty with the necessary parameters even from rc.local, even via ssh).
4. getty starts login
5. login starts -bash, which becomes responsible for the control terminal for all other processes (roughly speaking, it processes Ctrl-C, Ctrl-Z, etc., as appropriate).
6. Bash continues to live a normal life, runs the program. Programs write to stdout that is normal for them, read from stdin, etc., it all gets to the driver, from the driver to the block buffer, from where it gets to dom0 to ...

xenconsoled

(console in terms of dom0)
Here we come to the second, much more interesting part of the xen'a console.

The dom0 listens to a special xenconsoled daemon (by the way, IBM authorship, not XenSource). Perhaps you can write an alternative to him, but I have not yet seen such brave ones.

Xenconsoled deals with all the rubbish associated with receiving ring buffer content and sending events to the event channel.

In order to give the domain administrator access to the console, xenconsoled uses the unix98 mechanism, which allows you to create a pseudo-terminal. More precisely, TWO devices are created that are connected to each other by an invisible loopback. One of them does not even appear as a device, but remains an abstraction with a file handler, and the second appears in / dev / pty /. In fact, everything that will be written to / dev / pty / X will exit this handler while reading. The reverse is also true - what is written to this handler will come from / dev / pty / X. I think it's quite clear what xenconsoled does with the data from the handler. Writes them to the ring buffer console. And reads from there, writing in the same handler.

When a new domain appears, xenconsoled writes the path to / dev / pty / X to it in xenstore (for example, / dev / pty / 0, / dev / pty / 1). As soon as the domain disappears (reboot, shutdown, migration), the pseudo-terminal disappears.

What to do with it / dev / pty? Well, actually, the same as with any com-port, at the back end of which there is a computer, a router, etc. Connect terminalkoy. Dudes can use putty or another graphical terminal, other people can do with minicom. For Xena, a special terminalka xenconsole (/ usr / (local) / lib / xen / bin / xenconsole) is written, which allows you to disable using the Ctrl-] combination.

It, by the way, launches it xm console (xl console too).

Is magic revealed? Yes. Further begins 'value added services'. If we provide virtualization services and we want to give people the opportunity to look at the console of their virtual machine without issuing them administrator rights to dom0 ...

Then everything is very simple. An alternative to xenconsole is vncterm (a special program that “takes” a character device and makes a VNC connection from it), allowing you to look at the console through VNC. VNCterm clings to the pseudo-terminal, receives bytes from there, draws them (console_codes) and, if a user is connected to the VNC, sends him a video stream. From it, he receives buttons for sending them to the pseudo-terminal (and further along the chain).

This method of access to the console for clients has several drawbacks: firstly, VNC is usually not encrypted. Secondly, it requires a separate password to connect. Thirdly, the connection is broken when you migrate or reboot the computer (because the domain is new, it is “listened” by a completely different VNCterm).

Graphic console?

For the graphical console in Xen, a completely different mechanism is used - framebuffer. A scribbled area into which the virtual machine can write, and from where the console driver reads this data. Console data (resolution and color depth) are advertised through the xenstore. Unlike the sleek and compact text console, the framebuffer is very large. For example, in 1920x1050 resolution it occupies as much as 8 megabytes. Plus, of all acceleration, only scrolling up is available.

In fact, framebuffer was added to Xen specifically for the HVM mode (much beyond the scope of this article), and the HVM mode itself was largely due to the existence of Microsoft Windows, which even in Core Server has not yet learned how to boot into the text console. However, HVM is successfully used for linux, but the capabilities and performance of the HVM are inferior to the PV mode.

Another headbuffer headache is a mouse. Obviously, without a mouse, Windows Guy is not nice, so you have to create ring buffer separately for the keyboard, for the mouse, and for framebuffer updates. At the same time, in contrast to the relatively quiet keyboard, the mouse requires a much higher speed of work (both in the sense of sending messages about mouse movement, and in the sense of delay in drawing the reaction).

I can look at the framebuffer from the outside by only one method - VNC, since the entire layer of text-based terminals is out of business. This, by the way, explains why in the Xen Cloud Platform the default mode for all virtual machines is VNC - it turns out to be the same for both HVM and PV modes.

Third way

Just for completeness, I have to mention it - in the domU you can "forward" physical devices. For example, a video card. In this case, the domain is one on one with a conventional video card and can do what it wants with it. Including opengl and other directxs. However, the video card is available only to one domain (two video cards - two domains, etc.), plus, you should forget about the migration, since the probability that the driver after the crash will be able to correctly initialize the video card is somewhat controversial. In real life (in server virtualization), this method is not used.

Future text console

Currently, Xen4 is actively completing support for multiple consoles. Unlike the "debug", the main console, they will be announced through XenStor, and according to the principles of their work they will not differ much from the existing "only" one.