Forwarding a video card to a virtual machine
It is said that modern hardware virtualization support technologies (VT-d from Intel, IOMMU from AMD) allow the physical device on the PCI bus to be directly managed by the virtual machine. Including a video card.
Imagination draws this configuration: a desktop server with a hypervisor, it runs a guest user operating system that has access to the necessary I / O devices, one or two unpretentious servers as needed, and how many virtual machines are needed for inhuman experiments. We manage the hypervisor through the console in the guest OS either remotely, from a laptop, for example.
Inspired by this picture, I decided to try it, but it turned out that passthrough the video adapter was not a trivial task. Only a month after three butting with iron and reading the forums managed to get a positive result. I tried VMware and Xen as a hypervisor . It turned out only with Xen.
Short.
Iron:
Soft:
At first, I experimented with VMware vSphere 5.0 for a long time. Actually, the hardware configuration was chosen just for her. On the way, a number of interesting details opened up: for example, the VT-d should be supported by the processor (they write that processors with the K index are not suitable), and by the chipset and the motherboard. With video cards in general, the trouble is: it’s known that with the majority this trick doesn’t work, with some (rather short list), some succeed, others don’t. A long and informative (although not too joyful) discussion was here:
VMware Communities: VMDirectPath and ATI Radeon . Radeon 3450 went, perhaps, in favorites as one of the most forwarded cards.
I went through a decent amount of various iron combinations. The competition involved two motherboards, three video cards, plus an integrated SandyBridge video (IGD), three network adapters and one processor. Several times I threw these fruitless attempts for a week or two, then I came up with some options. On the way, there was one moment when it almost turned out: the virtual machine correctly identified the monitor, but it did not go further. I rested on the fact that the card seemed to be normally forwarded to a virtual machine, and the device manager appeared to be exactly, but Catalyst stubbornly refused to deal with it. The map is as lively but not working.
You could try a lot more: Windows XP and Linux as guest systems (set Windows 7 in 32 and 64-bit versions), get another video card ... In the end, spat and decided to go in from the other end, trying another hypervisor. Without further thinking, he took what was in plain sight: Xen as part of the Xen Cloud Platform (XCP).
XCP delivered without a hitch.
')
For a while I stumped the question: how to steer this system? In the sense, there must be some kind of management console, preferably under Windows? Having picked up half a day with conditionally- staffed OpenXenManager, I came to the conclusion that either the skis do not go, or this cross-platform tool does not live on Windows. Once or twice she connected to the server, but she died somewhere in the process, the rest of the times she hung dumb when connected, merging an irresistible stream of exceptions into the Python console.
Fortunately, a broader view of the surrounding Internet has revealed to me that Citrix XenCenter can perfectly control opensen-Xen, and is completely free. However, when connecting, it shouts that the Evaluation period will expire on your server in N days, but people in the know write that it’s just that they don’t know about the opensource of the server’s edition, but in fact everything will work.
XenCenter allows you to create-enable-suppress virtuals, and device forwarding must be configured from the sysadmin-friendly command line interface.
Against expectations, problems did not happen here. I did everything according to the manual , and it was enough for him alone. Here the people complain that the documentation for Xen is not enough. So another time, and well, that is not enough, if this is enough. How much I read about vSphere, and all to no avail ... However, I don’t want to say bad words about vSphere. Under it, but the iron is so tuned that Xen flew straight with a whistle.
So, with the help of XenCenter, I organized a virtual machine about two cores and 4 GB of memory, rolled the seventh 64-bit Windows and went to forward it.
Corresponding to the manual, edit
We do xe vm-list and find the uuid of our
We execute the
Write a single-line view script
In addition to the video adapter, I still have a sound on the card - so, remembering the rake I was attacking in the vSphere, I add both devices: 0/0000: 02: 00.0, 0/0000: 02: 00.1.
Execute the script. To control
We stop and restart the virtual machine (they write that it is this way, and not a reboot - I will not check it once more).
At the first attempt, the card is in my first PCIe slot (01: 00.0, 01: 00.1) and is used by default by the hypervisor. After restarting the virtual machine, the monitor goes out.
In XenCenter (from a laptop) we go into the virtual console and after logging in to Windows we see that it asks for a reboot. A sign that she has found a new device. We will not refuse her. Reboot. Indeed, the Device Manager has a new Radeon 3450 video adapter with the Microsoft WDDM 1.1 driver. From previous experience it is known that the driver is needed native. I download and install fresh ATI Catalist 12.1, the one after installation, as usual, asks for a reboot. Reboot ... Opanki. The first attempt is covered with a copper basin: the hypervisor falls. Yes ... vSphere in such a situation won a convincing victory over the virtual machine, arranging for it BSOD.
We re-issue the host and, on the recommendation of the best dog breeders, see what the team writes to us
It seems that the transfer of the card to the hot does not shine. Okay. Let's give the hypervisor your VGA adapter, the benefit of the video cards is enough for me now. We rearrange the Radeon 3450 in the second slot, in the first we place the 3470 lying next to it. We attach a monitor to each card. We include a host, we start a virtualka. Windows asks to restart after changing the configuration. Reboot. Login ...
In the XenCenter console, the screen saver freezes, and I do not immediately realize that the right monitor is lit in blue
It still happened.
Total, on Xen has grown together for 3 days (after 3 months practiced on VMware).
I logged in. The picture on the monitor is the most ordinary, without features. The resolution of 1920x1200 holds. Not tupit (although not drove tests). YouTube video is playing normally.
On this joyful note, I will complete my truthful report, and I will do the forwarding of the keyboard, mouse, and other peripherals.
Update:
I threw the keyboard and mouse, I write from virtual under Win7. Nothing here, you can live.
Performance Index 3.5
I haven’t even turned off Aero and other visual effects that areharmful to our cause , but it works smartly.
USB forwarding is rude and cynical:
That is, donated a virtual USB controller. On the other hand, Xen is still without them.
What else? Put XenCenter, Xen normally administered (who would doubt). An external USB drive, of course, also hooks normally. Now we need to understand how to forward a CD-ROM.
Unfortunately, nothing came out of my USB Passthrough , as well as the Xen VGA Passthrough ( Scraelos ) rightly mentioned in the comments, because there is no "/ etc / xen / cfgfile" file under the XCP. How to register the necessary settings using
Update: ToDo:
Update 06.02.2012:
dom0 at IGD refused to change over in a categorical form. In addition, I tried again to forward the primary VGA adapter - to no avail. Returned to the previous configuration of the adapters.
Rested in the problem with probrosom CD (DVD) -Writer. The CD-ROM is being forwarded normally, but only RO, and I need RW. On this occasion, I found 2 recommendations: plug in and forward a separate SATA controller and use a USB CD / DVD-Writer (the benefit of USB is being forwarded). Unfortunately, even these (imkho, crutch) solutions have not worked for me at the moment. The controller refused to forward. Attempting to connect a regular SATA CD / DVD drive via a USB-SATA adapter did not lead to anything good. I continue the experiments.
Imagination draws this configuration: a desktop server with a hypervisor, it runs a guest user operating system that has access to the necessary I / O devices, one or two unpretentious servers as needed, and how many virtual machines are needed for inhuman experiments. We manage the hypervisor through the console in the guest OS either remotely, from a laptop, for example.
Inspired by this picture, I decided to try it, but it turned out that passthrough the video adapter was not a trivial task. Only a month after three butting with iron and reading the forums managed to get a positive result. I tried VMware and Xen as a hypervisor . It turned out only with Xen.
Short.
Iron:
- Gigabyte GA-Q67M-D2H-B3 motherboard
BIOS Firmware - F5
Setup: CPU-> Intel Virtualization Technology = ON
Setup: Chipset-> North Bridge-> VT-d = ON - Intel Core i5 2500 3.3 GHz processor
- Memory 16 GB
- ATI Radeon HD 3470 in the first PCIe slot, used by the hypervisor
- ATI Radeon HD 3450 in the second PCIe slot, guest OS is given
- Intel network adapter in a PCI slot
Soft:
- XCP (Xen Cloud Platform) 1.0 (XenServer build 42052c)
- Citrix XenCenter to manage
- Windows 7 64 bit as guest OS, ATI Catalyst 12.1
At first, I experimented with VMware vSphere 5.0 for a long time. Actually, the hardware configuration was chosen just for her. On the way, a number of interesting details opened up: for example, the VT-d should be supported by the processor (they write that processors with the K index are not suitable), and by the chipset and the motherboard. With video cards in general, the trouble is: it’s known that with the majority this trick doesn’t work, with some (rather short list), some succeed, others don’t. A long and informative (although not too joyful) discussion was here:
VMware Communities: VMDirectPath and ATI Radeon . Radeon 3450 went, perhaps, in favorites as one of the most forwarded cards.
I went through a decent amount of various iron combinations. The competition involved two motherboards, three video cards, plus an integrated SandyBridge video (IGD), three network adapters and one processor. Several times I threw these fruitless attempts for a week or two, then I came up with some options. On the way, there was one moment when it almost turned out: the virtual machine correctly identified the monitor, but it did not go further. I rested on the fact that the card seemed to be normally forwarded to a virtual machine, and the device manager appeared to be exactly, but Catalyst stubbornly refused to deal with it. The map is as lively but not working.
You could try a lot more: Windows XP and Linux as guest systems (set Windows 7 in 32 and 64-bit versions), get another video card ... In the end, spat and decided to go in from the other end, trying another hypervisor. Without further thinking, he took what was in plain sight: Xen as part of the Xen Cloud Platform (XCP).
XCP delivered without a hitch.
')
For a while I stumped the question: how to steer this system? In the sense, there must be some kind of management console, preferably under Windows? Having picked up half a day with conditionally- staffed OpenXenManager, I came to the conclusion that either the skis do not go, or this cross-platform tool does not live on Windows. Once or twice she connected to the server, but she died somewhere in the process, the rest of the times she hung dumb when connected, merging an irresistible stream of exceptions into the Python console.
Fortunately, a broader view of the surrounding Internet has revealed to me that Citrix XenCenter can perfectly control opensen-Xen, and is completely free. However, when connecting, it shouts that the Evaluation period will expire on your server in N days, but people in the know write that it’s just that they don’t know about the opensource of the server’s edition, but in fact everything will work.
XenCenter allows you to create-enable-suppress virtuals, and device forwarding must be configured from the sysadmin-friendly command line interface.
Against expectations, problems did not happen here. I did everything according to the manual , and it was enough for him alone. Here the people complain that the documentation for Xen is not enough. So another time, and well, that is not enough, if this is enough. How much I read about vSphere, and all to no avail ... However, I don’t want to say bad words about vSphere. Under it, but the iron is so tuned that Xen flew straight with a whistle.
So, with the help of XenCenter, I organized a virtual machine about two cores and 4 GB of memory, rolled the seventh 64-bit Windows and went to forward it.
Corresponding to the manual, edit
/boot/extlinux.conf by inserting the line "iommu=1 iommu_inclusive_mapping=1" after each occurrence of "/boot/xen.gz"
extlinux /boot .
pciback skip the steps associated with the pciback module - they write that in the six it is already compiled into the kernel.
We do xe vm-list and find the uuid of our
uuid=d103a91d-5c38-844f-14d5-64b3c495eb08 , I have uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
We execute the
lspci command and find our map in the output, for example, 02: 00.0 VGA compatible ..., 02: 00.1 Audio ... (the two surprisingly correspond to the slot number where the card is inserted).
Write a single-line view script
xe vm-param-set other-config:pci=0/0000:02:00.0,0/0000:02:00.1 uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
In addition to the video adapter, I still have a sound on the card - so, remembering the rake I was attacking in the vSphere, I add both devices: 0/0000: 02: 00.0, 0/0000: 02: 00.1.
Execute the script. To control
xe vm-param-list uuid=d103a91d-5c38-844f-14d5-64b3c495eb08 | more xe vm-param-list uuid=d103a91d-5c38-844f-14d5-64b3c495eb08 | more - really
We stop and restart the virtual machine (they write that it is this way, and not a reboot - I will not check it once more).
At the first attempt, the card is in my first PCIe slot (01: 00.0, 01: 00.1) and is used by default by the hypervisor. After restarting the virtual machine, the monitor goes out.
In XenCenter (from a laptop) we go into the virtual console and after logging in to Windows we see that it asks for a reboot. A sign that she has found a new device. We will not refuse her. Reboot. Indeed, the Device Manager has a new Radeon 3450 video adapter with the Microsoft WDDM 1.1 driver. From previous experience it is known that the driver is needed native. I download and install fresh ATI Catalist 12.1, the one after installation, as usual, asks for a reboot. Reboot ... Opanki. The first attempt is covered with a copper basin: the hypervisor falls. Yes ... vSphere in such a situation won a convincing victory over the virtual machine, arranging for it BSOD.
We re-issue the host and, on the recommendation of the best dog breeders, see what the team writes to us
dmesg . She writes, among other things, this:
pciback 0000:01:00.1: secondary bus reset failed for device — all functions need to be co-assigned — err: 6
pciback 0000:01:00.1: FLR functionality not supported; attempts to use secondary bus reset unsuccessful;
pciback 0000:01:00.1: FLR not performed for device
It seems that the transfer of the card to the hot does not shine. Okay. Let's give the hypervisor your VGA adapter, the benefit of the video cards is enough for me now. We rearrange the Radeon 3450 in the second slot, in the first we place the 3470 lying next to it. We attach a monitor to each card. We include a host, we start a virtualka. Windows asks to restart after changing the configuration. Reboot. Login ...
In the XenCenter console, the screen saver freezes, and I do not immediately realize that the right monitor is lit in blue
It still happened.
Total, on Xen has grown together for 3 days (after 3 months practiced on VMware).
I logged in. The picture on the monitor is the most ordinary, without features. The resolution of 1920x1200 holds. Not tupit (although not drove tests). YouTube video is playing normally.
On this joyful note, I will complete my truthful report, and I will do the forwarding of the keyboard, mouse, and other peripherals.
Update:
I threw the keyboard and mouse, I write from virtual under Win7. Nothing here, you can live.
Performance Index 3.5
I haven’t even turned off Aero and other visual effects that are
USB forwarding is rude and cynical:
xe vm-param-set other-config:pci=0/0000:02:00.0,0/0000:02:00.1,0/0000:00:1a.0,0/0000:00:1d.0 uuid=d103a91d-5c38-844f-14d5-64b3c495eb08
That is, donated a virtual USB controller. On the other hand, Xen is still without them.
What else? Put XenCenter, Xen normally administered (who would doubt). An external USB drive, of course, also hooks normally. Now we need to understand how to forward a CD-ROM.
Unfortunately, nothing came out of my USB Passthrough , as well as the Xen VGA Passthrough ( Scraelos ) rightly mentioned in the comments, because there is no "/ etc / xen / cfgfile" file under the XCP. How to register the necessary settings using
xe - I did not understand. If experts Xen help, I will be very grateful.
Update: ToDo:
- Forward CD-ROM (-)
- Configure static IP (+)
- Test performance (partially done)
- Try to transplant dom0 to IGD, one PCIe slot will be freed (-)
- Try to transplant dom0 on the onboard NIC, free PCI slot (+)
- Try Vpom dom0 keyboard on PS / 2
- Organize a file system for data exchange between guest OS
- Arrange switching between guest OS (scripts, probably)
Update 06.02.2012:
dom0 at IGD refused to change over in a categorical form. In addition, I tried again to forward the primary VGA adapter - to no avail. Returned to the previous configuration of the adapters.
Rested in the problem with probrosom CD (DVD) -Writer. The CD-ROM is being forwarded normally, but only RO, and I need RW. On this occasion, I found 2 recommendations: plug in and forward a separate SATA controller and use a USB CD / DVD-Writer (the benefit of USB is being forwarded). Unfortunately, even these (imkho, crutch) solutions have not worked for me at the moment. The controller refused to forward. Attempting to connect a regular SATA CD / DVD drive via a USB-SATA adapter did not lead to anything good. I continue the experiments.
Source: https://habr.com/ru/post/137327/
All Articles