Running FreeBSD on Linux KVM
The task: to run FreeBSD from under Linux, preferably with a minimum of changes to the Linux system during initial configuration and updates, the ability to run on the workstation and on the server, and with minimal performance loss.
As a VPS-farm, you can use any common distribution of Linux. In our case, it will be Ubuntu 12.10 with a kernel 3.5.0 for amd64.
The guest system will be FreeBSD 9.1 for i386. The i386 architecture is chosen because of the significantly lower consumption of RAM by 32-bit applications as compared to 64-bit ones.
')
Linux-KVM (“Kernel-based Virtual Machine”) will be used as a virtualization system.
Pros of KVM:
Cons KVM:
Next, we assume that all disk images and configuration files are stored in the home directory in the virt subdirectory:
Install the necessary software in Linux:
The virtualization itself is performed by the module as part of the kernel, but in the packages that apt-get installs by dependencies, there are control and auxiliary utilities, KVM-specific settings for basic Linux services (for example, for udev), etc.
Check for hardware support:
Hardware virtualization (a) must be supported by the processor and (b) must be enabled in the BIOS. Otherwise, KVM will not work and the guest launch command will either complete with an error (if the" -enable-kvm " key is specified), or switch to a substantially less efficient software virtualization mode based on QEMU.
The standard kvm-ok shell script performs sample checks and advises a repair method if it fails.
KVM supports many options for organizing a guest network (see an example for a brief official review ). Currently "man kvm" contains9! 8 variants of the " -net " key with several dozens of possible subkeys, and often the "-net" needs to be indicated twice in the container launch command with different sets of subkeys - to create a guest interface, and to create an interface in the base system to communicate with the guest. Network settings may be the most unobvious part of KVM during initial development.
For more or less serious use, two options make sense:
Both require superuser privileges, in our case they have the same set of parameters for"-net ..." , but differ in the set of actions in the "-net ..., script = ... " script that KVM calls when the container starts to configure the network interface created in the base system. The bridge version is somewhat simpler, so our script ~ / virt / kvm-ifup-bridge.sh will do the following:
In various manuals, it is recommended to configure the bridge in advance by editing / etc / network / interfaces , but for test purposes it is easier for the workstation to create it at the moment when it becomes really needed, i.e. at the time of the first launch of the first container.
If in the external network it is unacceptable to light up additional MAC addresses, then instead of a bridge, you can use routing and ProxyARP . If the external network allows exactly one MAC and one IP, then in the base system, to exit the guest systems to the outside world, you will have to use routing, the IP address on the internal interfaces, and NAT. In both cases, you will need to either configure static IP in the guest systems, or configure a DHCP server in the basic system to configure the guests.
MAC addresses for guest network interfaces KVM can automatically generate at start, but if you plan to release guests to the outside world via a network bridge, it is better to assign them permanent MAC addresses. In particular, if a DHCP server is running on the external network, this will help the guest system to get the same IP from it every time it starts. First, let's “compile” the base MAC address:
For containers we will replace the last number with their sequence number. We will use the same number for their names and for VNC consoles. For example, the container with number 25 will be called “kvm_25”, have MAC 00: 16: 3e: xx: xx: 25 and listen to VNC connections on port 59 25 . In ordernot to stump hemorrhoids with different number systems, not to have unnecessary problems, it is recommended to choose numbers from 10 to 99. Of course, this approach is not used in VDS hosting, but it is suitable for personal needs.
1. Boot from the CD image, install the OS on an empty hdd image, turn off the VM.
2. Edit the startup script (disable the CD), boot from hdd, configure virtio support in the guest OS, turn off the VM.
3. We edit the startup script (we change the disk and network types from IDE and Realtek to virtio), boot.
Download the ISO image of the FreeBSD installation disk:
Create a hard disk image:
The image format is selected with the "-f" key: raw (default), qcow2, vdi, vmdk, cloop, etc. Raw is understandable to any software, but provides a minimum of opportunities and immediately takes the maximum possible place. Qcow2 is more compact (supports dynamic increase in size) and more functional (supports snapshots, compression, encryption, etc.), but is recognized only by systems based on QEMU.
Script to run ~ / virt / freebsd9.start
In the window that opens, the CD Loader and the FreeBSD installer should start. We perform the installation in the usual way. Almost all parameters can be left as default.
Sudo is needed because KVM loader requires superuser rights to create a TAP interface.
Two " -net " keys create two interconnected network interfaces: TAP in the base system and virtual Realtek-8139 in the guest system.
The " -enable-kvm " key ensures that QEMU will not automatically select software emulation mode if KVM was unable to start.
The " -name " key defines the title of the console window, can be used to search the process list, etc.
The CD is selected as the boot disk (" -boot order = d"). The option is valid only when the container is turned on, i.e. When rebooting, the system search will start from the first disk.
The " -m " key sets the size of the guest RAM. The default is 128 megabytes. This may be enough for the installer to work, but after successful installation, the very first attempt to assemble a large project from the ports with "-m 256M" and the swap section for 512 megabytes (the size is automatically selected by the installer) caused a kernel trap.
The KVM loader works as a normal user process, so to turn off a virtual machine, simply press Ctrl + C in the console (naturally, when the guest OS is running, it is better not to do this and use poweroff in the guest console). The bootloader communicates with the virtualization system in the kernel through the symbolic pseudo-device / dev / kvm , so any user who has the right to write to it can start virtual machines. As a rule, the group " kvm " is created for such users in the system.
To run in the background, the bootloader has the " -daemonize " key.
Before running in the freebsd9.start script, you need to comment out the " boot " and " cdrom " lines. Then we start it and after the FreeBSD boot is complete, we enter its command line with superuser privileges.
The guest drivers for virtio support for FreeBSD are not yet included in the base kernel, but are distributed as a port, so we will need to install the ports tree:
Drivers require the source code for the current kernel to build:
After that we assemble and install the drivers themselves:
The following lines must be added to /boot/loader.conf :
They can be copied from / var / db / pkg / virtio-kmod * / + DISPLAY. If you forget, the FreeBSD kernel will crash when booting into the “mountroot>” prompt , because it will not be able to see the disk device from the root file system. You will need to reboot, enter the boot manager command line, and manually load these modules in front of the kernel with the “load” command.
In / etc / rc.conf you need to insert one of two lines:
If a large number of settings are already tied to the old network interface, the second option allows you to avoid their ubiquitous changes. But he makes the general scheme a little more confusing.
In / etc / fstab, you must replace all " / dev / ada " with " / dev / vtbd ". If the disk is set up by the installer automatically, fstab will become like this:
If you forget or incorrectly edit fstab, on the next boot, you will be taken to the “mountroot” prompt and you will have to manually type in it “ufs: / dev / vtbd0p2”.
If a virtual copy of a real-life device (such as a Realtek network card or a SCSI disk) is provided to the container, it is first accessed through the device driver in the guest system. The driver converts high-level data read / write calls to low-level interrupt operations, registers, I / O ports, and so on. They are intercepted by the virtualization system and perform the reverse work - it translates into high-level calls for an external system (for example, reading and writing a disk image file).
If a virtio device is provided in the container, the guest system driver immediately transfers data to the external system and back. The driver is simplified, low-level virtualization of physical resources is not required.
They write that the transition to virtio speeds up the drive in the guest system by half , and the network is almost an order of magnitude .
Another interesting feature of virtio is associated with dynamic allocation of memory for the guest system (" ballooning ") and merging blocks of memory with the same content ( KSM , "Kernel Samepage Merging").
VirtualBox and KVM use the virtio compatible mechanism, so the set of guest drivers for them is the same. In Linux, guest drivers are already included in the standard kernel, for FreeBSD they are distributed as a port (see above), for Windows they are written by the KVM developers (see here ).
Change the ~ / virt / freebsd9.start line with the network interface and drive:
... for the following:
If the FreeBSD boot succeeds, you can verify with the following commands that the virtual devices are now being used:
By default, KVM renders the guest console in the graphics window using the SDL library. This option is poorly suited to run the container in the background, to run on a server without graphics, and to access the console over the network.
To solve this problem, the KVM container can provide access to the guest console via the VNC network protocol. In ~ / virt / freebsd9.start, insert in the startup parameters:
Now when you start the KVM container, it will not open a graphical window, but a network connection. You can see it, for example, with the command " sudo netstat -ntlp | grep -w kvm ".
Install the client application (for example, tightvncviewer ) and connect to the console:
Note: If there is no keyboard response in the VNC window, click on it.
A VNC connection can be password protected, but, unfortunately, it is impossible to assign a password directly from the command line. You will need to either connect to the container control console via a separate control socket (a brief description of how to configure it and how to connect to it), or open it in the main VNC window by pressing Ctrl + Alt + Shift + 2 .
In addition to SDL and VNC, a curses text-based interface is supported (the "-curses" or "-display curses" key). Theoretically, it could be useful for background running on screen . In practice, KVM sends its own diagnostic garbage to the console being created and makes its use inconvenient.
As a VPS-farm, you can use any common distribution of Linux. In our case, it will be Ubuntu 12.10 with a kernel 3.5.0 for amd64.
The guest system will be FreeBSD 9.1 for i386. The i386 architecture is chosen because of the significantly lower consumption of RAM by 32-bit applications as compared to 64-bit ones.
')
Linux-KVM (“Kernel-based Virtual Machine”) will be used as a virtualization system.
Brief comparison of KVM with alternatives
Pros of KVM:
- does not require prior installation of the hypervisor and resource scheduling, unlike Xen / ESXi / Hyper-V, and can be run on any Linux distribution, including desktop, for study and testing,
- unlike all other virtualization systems (except for LXC and with OpenVZ reservations), it is included in the base Linux kernel and developed by key Linux developers (first of all - RedHat ),
- unlike LXC and OpenVZ, it is capable of launching an arbitrary OS, incl. Linux with its own copy of the kernel and a set of drivers.
Cons KVM:
- the processor must have virtualization hardware support,
- there are no convenient graphical shells for running and editing virtual machines,
- There is no transparent access to files, processes and console consoles from the base system (it is in LXC and OpenVZ).
Setting up the environment
Next, we assume that all disk images and configuration files are stored in the home directory in the virt subdirectory:
mkdir -p ~/virt && cd ~/virt Install the necessary software in Linux:
apt-get update && apt-get -y install kvm The virtualization itself is performed by the module as part of the kernel, but in the packages that apt-get installs by dependencies, there are control and auxiliary utilities, KVM-specific settings for basic Linux services (for example, for udev), etc.
Check for hardware support:
kvm-ok Hardware virtualization (a) must be supported by the processor and (b) must be enabled in the BIOS. Otherwise, KVM will not work and the guest launch command will either complete with an error (if the
The standard kvm-ok shell script performs sample checks and advises a repair method if it fails.
Container network
KVM supports many options for organizing a guest network (see an example for a brief official review ). Currently "man kvm" contains
For more or less serious use, two options make sense:
- The basic system provides guest transparent access to the external network through the so-called. network bridge
- The base system acts as a router between the external and guest network (“router”).
Both require superuser privileges, in our case they have the same set of parameters for
- if the bridge is missing, it creates it and adds an external physical interface to it,
- assigns the same IP to the bridge as the physical interface
- moves all routes from the physical interface to the bridge,
- connects to the bridge a virtual interface for communication with the guest system.
#!/bin/sh # Constants BRIDGE_IFACE="br0" # Variables iface="$1" gwdev="$(ip route get 8.8.8.8 | grep ' via ' | sed -e 's,.* dev ,,' -e 's, .*,,' | head -1)" my_ip="$(ip addr list dev $gwdev | grep ' inet ' | sed -e 's,.* inet ,,' -e 's, .*,,' | head -1)" # Create and configure bridge if ! ip link list "$BRIDGE_IFACE" >/dev/null 2>&1 then echo "Create bridge $BRIDGE_IFACE..." brctl addbr "$BRIDGE_IFACE" brctl addif "$BRIDGE_IFACE" "$gwdev" ip link set "$BRIDGE_IFACE" up ip addr add "$my_ip" dev "$BRIDGE_IFACE" fi # Move routes from physical iface to bridge if test "$gwdev" != "$BRIDGE_IFACE" then ip route list dev "$gwdev" | grep -v 'scope link' \ | while read line; do ip route delete $line dev "$gwdev" ip route add $line dev "$BRIDGE_IFACE" done fi # Add virtual iface to bridge ip link set "$iface" up brctl addif "$BRIDGE_IFACE" "$iface" In various manuals, it is recommended to configure the bridge in advance by editing / etc / network / interfaces , but for test purposes it is easier for the workstation to create it at the moment when it becomes really needed, i.e. at the time of the first launch of the first container.
If in the external network it is unacceptable to light up additional MAC addresses, then instead of a bridge, you can use routing and ProxyARP . If the external network allows exactly one MAC and one IP, then in the base system, to exit the guest systems to the outside world, you will have to use routing, the IP address on the internal interfaces, and NAT. In both cases, you will need to either configure static IP in the guest systems, or configure a DHCP server in the basic system to configure the guests.
MAC addresses for guest network interfaces KVM can automatically generate at start, but if you plan to release guests to the outside world via a network bridge, it is better to assign them permanent MAC addresses. In particular, if a DHCP server is running on the external network, this will help the guest system to get the same IP from it every time it starts. First, let's “compile” the base MAC address:
perl -e '$XEN_RESERVED = "00:16:3e"; printf "%s:%02x:%02x:%02x\n", $XEN_RESERVED, int(rand(0x7f)), int(rand(0xff)), int(rand(0xff));' For containers we will replace the last number with their sequence number. We will use the same number for their names and for VNC consoles. For example, the container with number 25 will be called “kvm_25”, have MAC 00: 16: 3e: xx: xx: 25 and listen to VNC connections on port 59 25 . In order
Action plan
1. Boot from the CD image, install the OS on an empty hdd image, turn off the VM.
2. Edit the startup script (disable the CD), boot from hdd, configure virtio support in the guest OS, turn off the VM.
3. We edit the startup script (we change the disk and network types from IDE and Realtek to virtio), boot.
Preparing for the first boot
Download the ISO image of the FreeBSD installation disk:
wget http://mirror.yandex.ru/freebsd/releases/ISO-IMAGES/9.1/FreeBSD-9.1-RELEASE-i386-disc1.iso Create a hard disk image:
kvm-img create -f qcow2 freebsd9.img 8G kvm-img info freebsd9.img The image format is selected with the "-f" key: raw (default), qcow2, vdi, vmdk, cloop, etc. Raw is understandable to any software, but provides a minimum of opportunities and immediately takes the maximum possible place. Qcow2 is more compact (supports dynamic increase in size) and more functional (supports snapshots, compression, encryption, etc.), but is recognized only by systems based on QEMU.
First launch and installation
Script to run ~ / virt / freebsd9.start
#!/bin/sh MACBASE="00:16:3e:33:28" VM_ID=10 DIR=$HOME/virt sudo kvm \ -net "nic,model=rtl8139,macaddr=$MACBASE:$VM_ID" \ -net "tap,ifname=tap$VM_ID,script=$DIR/kvm-ifup-bridge.sh,downscript=/bin/true" \ -name "kvm_$VM_ID" \ -enable-kvm \ -m 512M \ -hda $DIR/freebsd9.img \ -cdrom "$DIR/FreeBSD-9.1-RELEASE-i386-disc1.iso" \ -boot order=d \ ## END ## In the window that opens, the CD Loader and the FreeBSD installer should start. We perform the installation in the usual way. Almost all parameters can be left as default.
Explanation of the launch team
Sudo is needed because KVM loader requires superuser rights to create a TAP interface.
Two " -net " keys create two interconnected network interfaces: TAP in the base system and virtual Realtek-8139 in the guest system.
The " -enable-kvm " key ensures that QEMU will not automatically select software emulation mode if KVM was unable to start.
The " -name " key defines the title of the console window, can be used to search the process list, etc.
The CD is selected as the boot disk (" -boot order = d"). The option is valid only when the container is turned on, i.e. When rebooting, the system search will start from the first disk.
The " -m " key sets the size of the guest RAM. The default is 128 megabytes. This may be enough for the installer to work, but after successful installation, the very first attempt to assemble a large project from the ports with "-m 256M" and the swap section for 512 megabytes (the size is automatically selected by the installer) caused a kernel trap.
The KVM loader works as a normal user process, so to turn off a virtual machine, simply press Ctrl + C in the console (naturally, when the guest OS is running, it is better not to do this and use poweroff in the guest console). The bootloader communicates with the virtualization system in the kernel through the symbolic pseudo-device / dev / kvm , so any user who has the right to write to it can start virtual machines. As a rule, the group " kvm " is created for such users in the system.
To run in the background, the bootloader has the " -daemonize " key.
Second launch and setup virtio
Before running in the freebsd9.start script, you need to comment out the " boot " and " cdrom " lines. Then we start it and after the FreeBSD boot is complete, we enter its command line with superuser privileges.
The guest drivers for virtio support for FreeBSD are not yet included in the base kernel, but are distributed as a port, so we will need to install the ports tree:
portsnap fetch extract Drivers require the source code for the current kernel to build:
csup -h cvsup2.ru.FreeBSD.org /usr/share/examples/cvsup/standard-supfile After that we assemble and install the drivers themselves:
make -C /usr/ports/emulators/virtio-kmod install clean The following lines must be added to /boot/loader.conf :
virtio_load="YES" virtio_blk_load="YES" virtio_pci_load="YES" virtio_balloon_load="YES" if_vtnet_load="YES" They can be copied from / var / db / pkg / virtio-kmod * / + DISPLAY. If you forget, the FreeBSD kernel will crash when booting into the “mountroot>” prompt , because it will not be able to see the disk device from the root file system. You will need to reboot, enter the boot manager command line, and manually load these modules in front of the kernel with the “load” command.
In / etc / rc.conf you need to insert one of two lines:
ifconfig_vtnet0="DHCP" # ..ifconfig_re0 ifconfig_vtnet0_name="re0" # ..ifconfig_re0 ! If a large number of settings are already tied to the old network interface, the second option allows you to avoid their ubiquitous changes. But he makes the general scheme a little more confusing.
In / etc / fstab, you must replace all " / dev / ada " with " / dev / vtbd ". If the disk is set up by the installer automatically, fstab will become like this:
# Device Mountpoint FStype Options Dump Pass# /dev/vtbd0p2 / ufs rw 1 1 /dev/vtbd0p3 none swap sw 0 0 If you forget or incorrectly edit fstab, on the next boot, you will be taken to the “mountroot” prompt and you will have to manually type in it “ufs: / dev / vtbd0p2”.
What is virtio and why is it needed at all?
If a virtual copy of a real-life device (such as a Realtek network card or a SCSI disk) is provided to the container, it is first accessed through the device driver in the guest system. The driver converts high-level data read / write calls to low-level interrupt operations, registers, I / O ports, and so on. They are intercepted by the virtualization system and perform the reverse work - it translates into high-level calls for an external system (for example, reading and writing a disk image file).
If a virtio device is provided in the container, the guest system driver immediately transfers data to the external system and back. The driver is simplified, low-level virtualization of physical resources is not required.
They write that the transition to virtio speeds up the drive in the guest system by half , and the network is almost an order of magnitude .
Another interesting feature of virtio is associated with dynamic allocation of memory for the guest system (" ballooning ") and merging blocks of memory with the same content ( KSM , "Kernel Samepage Merging").
VirtualBox and KVM use the virtio compatible mechanism, so the set of guest drivers for them is the same. In Linux, guest drivers are already included in the standard kernel, for FreeBSD they are distributed as a port (see above), for Windows they are written by the KVM developers (see here ).
Third launch
Change the ~ / virt / freebsd9.start line with the network interface and drive:
-net "nic,model=rtl8139,macaddr=$MACBASE:$VM_ID" \ -hda $DIR/freebsd9.img \ ... for the following:
-net "nic,model=virtio,macaddr=$MACBASE:$VM_ID" \ -drive "file=$DIR/freebsd9.img,if=virtio" \ If the FreeBSD boot succeeds, you can verify with the following commands that the virtual devices are now being used:
ifconfig df; swapinfo kldstat dmesg | grep vt Guest console
By default, KVM renders the guest console in the graphics window using the SDL library. This option is poorly suited to run the container in the background, to run on a server without graphics, and to access the console over the network.
To solve this problem, the KVM container can provide access to the guest console via the VNC network protocol. In ~ / virt / freebsd9.start, insert in the startup parameters:
-vnc localhost:$VM_ID \ Now when you start the KVM container, it will not open a graphical window, but a network connection. You can see it, for example, with the command " sudo netstat -ntlp | grep -w kvm ".
Install the client application (for example, tightvncviewer ) and connect to the console:
apt-get install vncviewer vncviewer :10 Note: If there is no keyboard response in the VNC window, click on it.
A VNC connection can be password protected, but, unfortunately, it is impossible to assign a password directly from the command line. You will need to either connect to the container control console via a separate control socket (a brief description of how to configure it and how to connect to it), or open it in the main VNC window by pressing Ctrl + Alt + Shift + 2 .
In addition to SDL and VNC, a curses text-based interface is supported (the "-curses" or "-display curses" key). Theoretically, it could be useful for background running on screen . In practice, KVM sends its own diagnostic garbage to the console being created and makes its use inconvenient.
Source: https://habr.com/ru/post/167099/
All Articles