Instantly launch almost any OS under Linux using libvirt + qemu

In the course of working on CoreOS and Kubernetes, I often had to reproduce the environments of users to help them solve the problem. Agree that when you start the OS with the help of the container, it is not always possible to achieve the full functionality of the OS, because often had to deal with issues related to systemd.

So the idea was born to write crutches, which can easily help me to raise a cluster from almost any operating system, such OpenStack in miniature. Currently, the following OS is fully supported as a guest:

Ubuntu
Debian
CentOS 6/7 + atomic
Fedora + atomic
CoreOS

Partially:

FreeBSD ( manual setup of network and ssh keys is required)
openSUSE (manual setup of network and ssh keys is required)
Windows (you won’t get to the console, but you can use the graphical interface with virt-manager)

The publication was contributed by a colleague who accidentally saw me testing the code. And now about the advantages over Vagrant . The first advantage is time. For example, three virtual machines are created in just 20-30 seconds .

At the time of starting work on Vagrant scripts under Linux, it could run images exclusively using VirtualBox, support for libvirt / qemu was still raw. For VirtualBox, you had to look for specially prepared images and it took a long time to restart the cluster from virtual machines.

Currently, most operating system developers provide images prepared for work in an OpenStack environment ("cloud" images). These images are very convenient to configure due to the presence of cloud-init . With the help of cloud-init at the first boot stage, you can easily embed SSH public keys into the image and configure the network.

QEMU can work with virtual machine images as with layers ( spanshots , almost like Docker). We can download the official "cloud" image and use it as a base for all virtual machines from the cluster. In this case, all changes on the disk will be recorded as the difference between the base image and the current state (like LVM snapshots). This method also allows you to easily remove all machines and recreate them within a few seconds.

Redeploy example of 3 debian machines

Delete the already created cluster from Debian and re-create it from scratch using the downloaded official image debian-8.5.0-openstack-amd64.qcow2

 $ virsh list Id Name State ---------------------------------------------------- 12 debian1 running 13 debian2 running 14 debian3 running ./remove_cluster.sh debian Are you sure to remove 'debian1 debian2 debian3 '? (Type 'y' when agree) y Domain debian1 destroyed Domain debian1 has been undefined Vol debian1.qcow2 deleted # Host debian1 found: line 8 /home/user/.ssh/known_hosts.debian updated. Original contents retained as /home/user/.ssh/known_hosts.debian.old Domain debian2 destroyed Domain debian2 has been undefined Vol debian2.qcow2 deleted # Host debian2 found: line 9 /home/user/.ssh/known_hosts.debian updated. Original contents retained as /home/user/.ssh/known_hosts.debian.old Domain debian3 destroyed Domain debian3 has been undefined Vol debian3.qcow2 deleted Host debian3 not found in /home/user/.ssh/known_hosts.debian Pool debian destroyed Pool debian has been undefined $ date && time ./deploy_vms_cluster.sh -o debian -s 3 && date Thu Aug 25 10:26:15 CEST 2016 # <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Using default 8.5.0 channel for debian Using default current release for debian Will use following path to SSH public key: /home/user/.ssh/id_rsa.pub Pool debian created Pool debian defined from /dev/stdin Formatting '/home/user/libvirt_images/debian/debian1.qcow2', fmt=qcow2 size=2147483648 backing_file=/home/user/libvirt_images/debian/debian-8.5.0-openstack-amd64.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 Pool debian refreshed Total translation table size: 0 Total rockridge attributes bytes: 763 Total directory bytes: 4458 Path table size(bytes): 40 Max brk space used 22000 187 extents written (0 MB) Pool debian refreshed Starting install... Creating domain... | 0 B 00:00:00 Domain creation completed. Pool debian defined from /dev/stdin Formatting '/home/user/libvirt_images/debian/debian2.qcow2', fmt=qcow2 size=2147483648 backing_file=/home/user/libvirt_images/debian/debian-8.5.0-openstack-amd64.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 Pool debian refreshed Total translation table size: 0 Total rockridge attributes bytes: 763 Total directory bytes: 4458 Path table size(bytes): 40 Max brk space used 22000 187 extents written (0 MB) Pool debian refreshed Starting install... Creating domain... | 0 B 00:00:00 Domain creation completed. Pool debian defined from /dev/stdin Formatting '/home/user/libvirt_images/debian/debian3.qcow2', fmt=qcow2 size=2147483648 backing_file=/home/user/libvirt_images/debian/debian-8.5.0-openstack-amd64.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 Pool debian refreshed Total translation table size: 0 Total rockridge attributes bytes: 763 Total directory bytes: 4458 Path table size(bytes): 40 Max brk space used 22000 187 extents written (0 MB) Pool debian refreshed Starting install... Creating domain... | 0 B 00:00:00 Domain creation completed. Use following command to connect to your cluster: 'ssh -i "/home/user/.ssh/id_rsa" debian@debian1' real 0m4.916s user 0m2.112s sys 0m0.268s Thu Aug 25 10:26:20 CEST 2016 # <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< $ ssh debian1 Warning: Permanently added 'debian1,192.168.122.30' (ECDSA) to the list of known hosts. The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. debian@debian1:~$ date Thu Aug 25 08:26:38 UTC 2016 # <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Virtual machines are ready for use in 20 seconds (laptop with SSD), on the HDD about 10 seconds longer.

This is what the contents of the ~/libvirt_images/debian directory look like

 $ find ~/libvirt_images/debian -ls 4456467 4 drwxrwxr-x 5 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian 4853836 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian2 4853838 4 -rwxrwxr-x 1 user user 157 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/rebuild_iso.sh 4853837 376 -rw-rw-r-- 1 libvirt-qemu kvm 382976 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/cidata.iso 4985097 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/openstack 4985103 4 drwxrwxr-x 2 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/openstack/latest 4985104 4 -rw-rw-r-- 1 user user 574 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/openstack/latest/user_data 4985105 4 -rw-rw-r-- 1 user user 152 Aug 25 11:13 /home/user/libvirt_images/debian/debian2/openstack/latest/meta_data.json 4456468 7752 -rw-r--r-- 1 libvirt-qemu kvm 7995392 Aug 25 11:13 /home/user/libvirt_images/debian/debian1.qcow2 4456470 7752 -rw-r--r-- 1 libvirt-qemu kvm 7995392 Aug 25 11:13 /home/user/libvirt_images/debian/debian3.qcow2 4722884 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian1 4722886 4 -rwxrwxr-x 1 user user 157 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/rebuild_iso.sh 4722885 376 -rw-rw-r-- 1 libvirt-qemu kvm 382976 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/cidata.iso 4853832 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/openstack 4853833 4 drwxrwxr-x 2 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/openstack/latest 4853834 4 -rw-rw-r-- 1 user user 574 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/openstack/latest/user_data 4853835 4 -rw-rw-r-- 1 user user 152 Aug 25 11:13 /home/user/libvirt_images/debian/debian1/openstack/latest/meta_data.json 4985106 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian3 4985112 4 -rwxrwxr-x 1 user user 157 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/rebuild_iso.sh 4985111 376 -rw-rw-r-- 1 libvirt-qemu kvm 382976 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/cidata.iso 4985107 4 drwxrwxr-x 3 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/openstack 4985108 4 drwxrwxr-x 2 user user 4096 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/openstack/latest 4985109 4 -rw-rw-r-- 1 user user 574 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/openstack/latest/user_data 4985110 4 -rw-rw-r-- 1 user user 152 Aug 25 11:13 /home/user/libvirt_images/debian/debian3/openstack/latest/meta_data.json 4456469 7752 -rw-r--r-- 1 libvirt-qemu kvm 7995392 Aug 25 11:13 /home/user/libvirt_images/debian/debian2.qcow2 4456471 474092 -rw-rw-r-- 1 libvirt-qemu kvm 485465600 Aug 25 10:18 /home/user/libvirt_images/debian/debian-8.5.0-openstack-amd64.qcow2

The network and DNS are configured using dnsmasq and send host-name "<hostname>" . At the network configuration stage, we assign the hostname to the virtual machine, then it requests the IP address via DHCP and sends its hostname to dnsmasq, dnsmasq writes the hostname to the internal DNS. Using dnsmasq as a resolver we get the IP of a new virtual machine.

I also ran virtual machines inside virtual machines. Nested KVM virtualization works fine, but additional configuration may be required.

Disadvantages:

Scripts written in bash
You must configure the libvirt environment in advance
There is no flexibility that Vagrantfile or Dockerfile provides, after deploying you have to run your favorite puppet / chef / ansible / etc.
Currently there is no support for PXE boot images to emulate / test deploy from scratch with providers such as Hetzner / OVH / etc.
Need help to improve.
It is required to come up with a beautiful name.
Plans to add support for FreeBSD and bhyve.

Well, the repository itself is here: https://github.com/kayrus/scripts . There is also information on setting up the libvirt environment.

As a bonus, the repository already has a Kubernetes cluster deployment script under CoreOS.

PS I guess I'm not the only one who came up with something like that. I would appreciate links to no more mature solutions.

Source: https://habr.com/ru/post/308480/

All Articles

Instantly launch almost any OS under Linux using libvirt + qemu

More articles: