Geekly Articles each Day
📜 ⬆️ ⬇️

Forwarding a video card to a guest OS from a KVM hypervisor using VFIO technology

Introduction


The article describes a method for forwarding physical devices from a KVM hypervisor to a guest OS using VFIO technology implemented in linux kernel 3.9. *.
The material is not a guide to action (approx. Tutorial). All the instructions described are at your own risk.

The experiment is performed under Ubuntu OS 13.10:


')
Hardware part of the stand:



The video card for the guest system and the host, perhaps, presumably, any (NVIDIA, ATI RADEON).
The motherboard must have an I / O memory management unit (IOMMU) - AMD-Vi or VT-d technology.
If an Intel processor is used, it must also support VT-d.
Regardless of the chosen platform, IVRS / DMAR tables should be implemented in the BIOS of the motherboard.

Training


It is strongly recommended to update the BIOS. boards from the manufacturer's site before the start of the experiment.

We check the support for forwarding technology.
$ dmesg | grep AMD-Vi [ 1.279788] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40 [ 1.279790] AMD-Vi: Interrupt remapping enabled [ 1.292879] AMD-Vi: Lazy IO/TLB flushing enabled


We include modules in the core.
 $ cat /etc/modules # /etc/modules: kernel modules to load at boot time. lp rtc pci_stub vfio vfio_iommu_type1 vfio_pci #vfio_pci_vga (For now, this component is in the stock Ubuntu kernel.) kvm kvm_amd 

 $ sudo update-initramfs -u


We disable the devices being forwarded (for convenience, the controller usb ports are also forwarded).
 $ lspci -nn | grep NVIDIA 03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF114 [GeForce GTX 560 Ti] [10de:1200] (rev a1) 03:00.1 Audio device [0403]: NVIDIA Corporation GF114 HDMI Audio Controller [10de:0e0c] (rev a1) 

 $ cat /etc/default/grub | grep GRUB_CMDLINE_LINUX_DEFAULT GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pci-stub.ids=10de:1200,10de:0e0c,1002:4397,1002:4396" 

 $ sudo update-grub


After a reboot, it should work:
 $ dmesg | grep pci-stub [ 3.163062] pci-stub: add 10DE:1200 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 3.163088] pci-stub 0000:03:00.0: claimed by stub [ 3.163100] pci-stub: add 10DE:0E0C sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 3.163116] pci-stub 0000:03:00.1: claimed by stub [ 3.163124] pci-stub: add 1002:4397 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 3.163136] pci-stub: add 1002:4396 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000


Create a script to connect the device to the VFIO-PCI.
 $ cat /usr/bin/vfio-bind #!/bin/bash modprobe vfio-pci for dev in "$@"; do vendor=$(cat /sys/bus/pci/devices/$dev/vendor) device=$(cat /sys/bus/pci/devices/$dev/device) if [ -e /sys/bus/pci/devices/$dev/driver ]; then echo $dev > /sys/bus/pci/devices/$dev/driver/unbind fi echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id done 

 $ sudo chmod 755 /usr/bin/vfio-bind


And we add it to the autorun (we take the keys from the lspci call - discussed above).
 $ cat /etc/rc.local #!/bin/sh -e /usr/bin/vfio-bind 0000:03:00.0 0000:03:00.1 0000:00:12.0 0000:00:12.2 exit 0


Small process automation
Attention: set your parameters!

 $ cat pass.sh #!/bin/sh #configs modules="pci_stub;vfio;vfio_iommu_type1;vfio_pci;kvm;kvm_amd" grub="quiet splash pci_stub.ids=10de:1200,10de:0e0c,1002:4397,1002:4396" autorun="/usr/bin/vfio-bind 0000:03:00.0 0000:03:00.1 0000:00:12.0 0000:00:12.2" #functions split() { #$1 - file, $2 - param arr=$(echo $2 | tr ";" "\n") for x in $arr do echo $x >> $1 done } replaceParam() { #$1 - file, $2 - param, $3 - value cur_val=`cat $1 | grep $2` sed "s/$cur_val/$2=\"$3\"/g" -i $1 } autorunAdd() { #$1 - param str='$i \'"$1"'\n' sed -i -e "$str" /etc/rc.local } pause() { read -p "$1" -n1 -s echo } #info virtualization dmesg | grep AMD-Vi pause 'Press any key to continue or CTRL-C to exit...' #modules load split '/etc/modules' "$modules" update-initramfs -u #grub config replaceParam '/etc/default/grub' 'GRUB_CMDLINE_LINUX_DEFAULT' "$grub" update-grub autorunAdd "$autorun" 

 $ cat vfio-bind #!/bin/bash modprobe vfio-pci for dev in "$@"; do vendor=$(cat /sys/bus/pci/devices/$dev/vendor) device=$(cat /sys/bus/pci/devices/$dev/device) if [ -e /sys/bus/pci/devices/$dev/driver ]; then echo $dev > /sys/bus/pci/devices/$dev/driver/unbind fi echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id done 

 $ sudo cp vfio-bind /usr/bin/ $ sudo chmod 755 /usr/bin/vfio-bind $ sudo chmod 755 pass.sh $ sudo bash pass.sh



Launch


WINDOWS 7 with paravirtual VIRTIO drivers is used as a guest OS. In accordance with this selected virtual machine equipment.

 $ cat win7_vfio_test.sh #! /bin/sh IS_TAP0_EXISTS=`ifconfig | grep -c tap0` if [ $IS_TAP0_EXISTS = 0 ] ; then sudo tunctl -t tap0 && sudo ifconfig tap0 0.0.0.0 promisc up && sudo brctl addif br0 tap0 && echo 'Success: tap0 interface created.' || echo 'Error: tap0 interface is not created.' else echo 'Success: tap0 interface already exists.' fi #init kvm-qemu sudo qemu-system-x86_64 \ -boot menu=on \ -enable-kvm \ -M q35 \ -m 1024 \ -cpu host \ -rtc base=localtime \ -smp 1,sockets=1,cores=1,threads=1 \ -bios /usr/share/qemu/bios.bin \ -acpitable file=/usr/share/seabios/q35-acpi-dsdt.aml \ -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root \ -device ahci,bus=pcie.0,id=ahci \ -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x6 \ -drive file='/dev/sdc6',if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 \ -drive file='windows7.iso',if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 \ -drive file='virtio-win-0.1-74.iso',if=none,id=drive-ide0-2-0,readonly=on,format=raw -device ide-cd,bus=ide.2,unit=0,drive=drive-ide0-2-0,id=ide0-2-0 \ -netdev tap,ifname=tap0,id=hostnet0,script=no,downscript=no -device virtio-net-pci,netdev=hostnet0,mac=52:54:00:26:7F:96,id=net0 \ -device vfio-pci,host=00:12.0,bus=pcie.0 \ -device vfio-pci,host=00:12.2,bus=pcie.0 \ -device vfio-pci,host=03:00.0,bus=root,addr=00.0,multifunction=on,x-vga=on \ -device vfio-pci,host=03:00.1,bus=root,addr=00.1 \ -vga none \ -nographic #-vnc 127.0.0.1:0


In the VM, the hard disk partition is forwarded -drive file = / dev / sdc6 .
The initial installation was done without forwarding physical devices (you can use -vnc 127.0.0.1 stray or a standard console).
Arguments -vga none -nographic should be added when forwarding video card components (-device vfio-pci).

Source: https://habr.com/ru/post/211909/

More articles:


All Articles