Paravirtualization in Xen: no bootloader anywhere

PV-GRUB (not to be confused with the pygrub crutch) is a GRUB 0.9x-based boot loader for Xen para-virtual machines, which allows you to load the OS kernel directly from the DomU (guest) environment, which adds significant independence from the host system to the guest launch process. One of the largest implementations can be considered Amazon EC2 cloud hosting, which also uses the Xen hypervisor and allows customers to use PV-GRUB to download any custom kernel ( Use your own kernel with Amazon EC2 ) for paravirtual systems.

In this matter, everything would be extremely positive if the problem is not quite common for free software: the development of this branch of GRUB has been completely stopped for the benefit of GRUB2 for several years, and the Xen developer community seems to be based on the current version of GRUB in the coming years PV replacement loader is not ready to provide. PV-GRUB itself, despite being, in general, part of the official Xen distribution, is currently excluded at least from the corresponding packages in the Debian and Ubuntu repositories, and users who are used to the conveniences, meanwhile, demand (Debian bug # 588839: Include pv-grub to securely boot guest kernels ) and require (Ubuntu bug # 798583: xen-utils-4.1 does not have pvgrub ) ... Although for me personally the motivation of Xen package maintainers referring to some vague problems is not very clear to me with dependencies, while packages with GRUB 0.97 are still available and workable, and to the existing functionality no complaints.

Here I will try to describe, perhaps, the most acceptable solution to the immediate problem at the moment - the self-assembly of PV-GRUB and the necessary configuration of DomU for its use , since no supernatural efforts are required for this.

For those interested in understanding the implementation of the almost traditional bootloader for the Xen para-virtual environment, it would be appropriate to immediately throw up some materials about the functional stub domains that appeared in Xen 3.3. A brief introduction is on the Xen.org blog ( Xen 3.3 Feature: Stub Domains , Xen 3.3 Feature: HVM Device Model Domain ). The rest is to look through the presentation ( PDF ) and listen to the report "Stub domains: A step towards Dom0 disaggregation", presented at Xen Summit North America.
')
To build the bootloader, you need the Xen source code. It is best to take the version that is already used on your host system. Since I use Debian GNU / Linux "Wheezy", for me it will be Xen 4.1.4. You can build on any system with a minimum developer set (gcc + binutils + make). From the official Xen website ( Xen 4.1 series , Xen 4.2 series ), download and unpack the tarball with the source code:

  $ wget http://bits.xensource.com/oss-xen/release/4.1.4/xen-4.1.4.tar.gz
 $ tar xzf xen-4.1.4.tar.gz 

Go to the stubdom directory. Run the bootloader assembly:

  $ make grub 

If you need to build a loader for an architecture other than the system architecture, you can set it via the XEN_TARGET_ARCH variable:

  $ XEN_TARGET_ARCH = x86_32 make grub 

In the process does not do without some pitfalls. The assembly of this component stops at the banal errors of including some header files along nonexistent paths.

  In file included from xc_core.c: 69: 0:
 xc_core.h: 26: 35: fatal error: xen / libelf / elfstructs.h: No such file or directory
 compilation terminated 

We are looking for the culprit:

  $ find .. -type f -name xc_core.h
 ../tools/libxc/xc_core.h 

Replace the string in it

 #include "xen/libelf/elfstructs.h" 

on

 #include "xen/elfstructs.h" 

When you run make grub again, there are several more like this:

  xc_dom.h: 17: 31: fatal error: xen / libelf / libelf.h: No such file or directory
 xc_hvm_build.c: 34: 31: fatal error: xen / libelf / libelf.h: No such file or directory
 ../../xen/common/libelf/libelf-private.h:70:31: fatal error: xen / libelf / libelf.h: No such file or directory 

All are solved by replacing

 #include <xen/libelf/libelf.h> 

on

 #include <xen/libelf.h> 

The build should now complete successfully. The last lines of output that you are likely to see will be:

  gzip -f -9 -c /home/user/xen-4.1.4/stubdom/mini-os-x86_64-grub/mini-os> /home/user/xen-4.1.4/stubdom/mini-os-x86_64 -grub / mini-os.gz
 make [1]: Leaving directory `/home/user/xen-4.1.4/extras/mini-os' 

The resulting compressed mini-os-x86_64-grub/mini-os.gz can be renamed to a more understandable pv-grub-x86_64.gz (or pv-grub-x86_32.gz if compiled under x86_32), and then put into the /usr/lib/xen-4.1/boot directory /usr/lib/xen-4.1/boot your your Dom0, where it hvmloader same is true for Debian with the xen-utils-4.1 package installed, there is a hvmloader in the same hvmloader ).

Before directly switching to the use of the bootloader, it is worth doing some preparatory procedures for your guest system.
It is worth noting that the old GRUB is not directly friends with modern file systems like Ext4, not to mention less common ones. Therefore, you should first make sure that the partition where you will have the kernel image, the initrd and the GRUB configuration files (by default, everything is stored in the /boot directory) will be available to the bootloader for reading. I have already stepped on a rake in the process of completing the quest described here: despite the theoretically existing backward compatibility of Ext4 with Ext2, there are quite a few different extensions (features) that completely eliminate this compatibility, especially if you didn’t think about it when creating your Ext4 section. Before creating a separate partition under /boot , I didn’t have a special need. Now, the solution could be just to remove the above-mentioned starting trio to the micro-partition (I think, double the volume from the current contents of /boot will be quite enough), formatted in Ext2. Do not forget to edit /etc/fstab your DomU, adding there /boot section and changing the names of the block devices of the other partitions, if necessary. The same applies to the disk configuration of the virtual machine itself.
You can check how much the existing partition is suitable for mounting in backward compatibility mode with Ext2 using mount -t ext2 elementary.

On the virtual machine itself, you need to install the old version of GRUB ( grub-legacy package in Debian or the most neutered grub-legacy-ec2 in Ubuntu). Requests to install GRUB in the boot sector of a partition should be answered negatively - it does not make sense.
After installation, make sure that the directory for storing the GRUB configuration exists and is available for writing. Running update-grub will create a bootloader menu file, adding kernels to the boot list.

  $ mkdir / boot / grub
 $ update-grub
 Searching for GRUB installation directory ... found: / boot / grub
 Probing devices to guess BIOS drives.  This may take a long time.
 Boot for default file ...
 Searching for GRUB installation directory ... found: / boot / grub
 Testing for an existing GRUB menu.lst file ...

 Generating /boot/grub/menu.lst
 Searching for splash image ... none found, skipping ...
 Found kernel: /vmlinuz-3.2.0-4-amd64
 Updating /boot/grub/menu.lst ... done 

Let's look at the created / /boot/grub/menu.lst / /boot/grub/menu.lst / /boot/grub/menu.lst :

  title Debian GNU / Linux, kernel 3.2.0-4-amd64
 root (/ dev / xvda1)
 kernel /vmlinuz-3.2.0-4-amd64 root = UUID = f5731cf7-420a-4094-acf7-bec5976a0b62 ro
 initrd /initrd.img-3.2.0-4-amd64

 title Debian GNU / Linux, kernel 3.2.0-4-amd64 (single-user mode)
 root (/ dev / xvda1)
 kernel /vmlinuz-3.2.0-4-amd64 root = UUID = f5731cf7-420a-4094-acf7-bec5976a0b62 ro single
 initrd /initrd.img-3.2.0-4-amd64 

Here it is striking that the configurator incorrectly filled in the root option, substituting the disk and partition names of the block device instead of the understandable GRUB . When selecting these menu items, GRUB will simply swear at the wrong option value.
To remedy the situation, we slightly modify the generated /boot /boot/grub/menu.lst adding the groot option with an explicit indication of the sequence number of the boot "disk", which is subsequently mounted as /boot (since there are no full disks with the partition table in the paravirtual environment):

  groot = (hd0) 

You can also reduce the timeout before starting the download, so as not to waste time in vain:

  ## timeout sec
 # Set a timeout, in SEC seconds, before automatically booting the default entry
 # (normally the first entry defined).
 timeout 1 

In order for the groot value to be correctly perceived by the update-grub configuration generator, we will additionally create the /boot/grub/device.map file, where we will enter a single line relating to the mentioned boot partition:

  (hd0) / dev / xvda1 

Now update-grub should generate the correct menu file, which will also not be broken if you want to update the version of the main kernel or add additional ones.

  title Debian GNU / Linux, kernel 3.2.0-4-amd64
 root (hd0)
 kernel /vmlinuz-3.2.0-4-amd64 root = UUID = f5731cf7-420a-4094-acf7-bec5976a0b62 ro
 initrd /initrd.img-3.2.0-4-amd64

 title Debian GNU / Linux, kernel 3.2.0-4-amd64 (single-user mode)
 root (hd0)
 kernel /vmlinuz-3.2.0-4-amd64 root = UUID = f5731cf7-420a-4094-acf7-bec5976a0b62 ro single
 initrd /initrd.img-3.2.0-4-amd64 

If it is necessary to transfer any additional parameters to the kernel that were previously specified directly in the virtual machine configuration via the extra option, this can be done by modifying kopt in the same / kopt .

As a final chord, it remains to reconfigure the DomU startup process: the bootloader file name must be specified in the kernel option. If the boot loader image is in the default directory (in the same place as the hvmloader ), then the full path is not required. The extra option specifies the partition and path to the GRUB menu file.

  kernel = 'pv-grub-x86_64.gz'
 extra = '(hd0) /grub/menu.lst' 

It is necessary to get rid of the previous ramdisk and root options.
To make sure that the new configuration is fully operational, the first launch of DomU should be done in the auto-connect mode to the console ( xm start -c ). If everything is done correctly, you will see the GRUB menu and the subsequent kernel boot process with mounting the root partition.

Note : The most likely reason for the absence of PV-GRUB in Xen packages of various distributions is voiced on the xen-users mailing list .

Do you need to support the construction of a jail box?