Raising the Thunderbolt Ethernet Adapter on XenServer

Good afternoon, dear Habro community, I got a hand to write a post about how I made the Thunderbolt Ethernet Adapter from Apple work under XenServer 6.2, under the cut a manual that will help you save time if you decide to do the same.

As it turned out in the course of the play, installing something on Citrix XenServer is quite troublesome, and if you really need to assemble a driver from the list, which is exactly what we need, then this is not even an achievable task. My assumption that this was done not by chance, but for reasons of security and performance of the host system, because each extra package in the system requires overhead and reduces the security of the system as a whole.

Background (Who is not interested, go to the next section )

In 2010, already far away, I had the task of raising a server with several virtual machines at home that would carry out their non-complex task around the clock, and most importantly would not interfere with the peace of people near this piece of iron. After a long selection ( thanks to the successfully found ad on Avito ), the choice fell on a Mac Mini Server with C2D 2.5Ghz, 8GB Ram, 2x500Gb HDD characteristics, looking ahead to say that this car is still on duty on duty, in fact, it is said - done, and now he is adorning on the table, all such aluminum with a white top, in general, candy. The search for the hypervisor began, first of all due to my own stupidity and oversupply of money, Parallels Server for Mac 4.0 Mac mini edition was chosen, which cost me about 9k wood, and was able to simultaneously run as many as two (!) Virtual machines, for the initial purposes of this enough and everything was fine.
')
But over time, as usual, this began to be missed, it took more VM, it was said - done, some more Mac Mini with an already more impressive config, i7 2.0 (4 cores 8 streams) were added, packed in 16Gb Ram and everything would be fine However, it became sad to launch only 2 VMs on one machine, which you need - then 1 core and 1GB Ram it was decided to study the Internet to search for another more interesting hypervisor, which allowed it to run ... well, at least not two VMs.

During the survey, HyperV and ProxMox were tested, but the choice was on Xen, or rather on the Xen Cloud Platform, which, to my great regret, did not last long and somehow I saw a message one day that XCP was closed and XenServer now free, nothing to do, go to XenServer. Further - more, a pack of servers, I want High Available, for these cases a separate Xeon e3-1240v2 server with 16GB Ram was purchased on board, in the Hot Swap baskets of which a WD Red 3TB HDD bundle, Nas4Free on a USB flash drive was mounted, iSCSI target was raised moon, assembled Xen cluster, which is able to HA migration on the move and other nice buns, but over time I began to notice that for n VMs a 1Gb / s throughput channel is not enough, and it ran like the VM images themselves NAS'a server and data stored on the same NAS'e. It was decided to put a reasonable switch, which is able to 802.3ad, and then on NAS'e to grow the number of ethernet channels up to 6x1GB / s, and on servers up to 2x1GB / s, but as soon as the Apple Thunderbolt Ethernet Adapter fell into the hands, it began in the farm morning, which resulted in this article on Habré ...

Let's get started

And so we have:

• Apple Thunderbolt Ethernet Adapter (MD463ZM), which is a 10/100 / 1000BASE-T PCIe network card from Broadcom, model NetXtreme BCM57762
• Motherboard with a Thunderbolt connector (the interface is nothing more than PCIe, only as a wire)
• Windows-based computer to use XenCenter

In our case, the experimental was Mini Mac Server mid 2011 with XenServer 6.2 installed on it and updated to SP1.

1. We connect the adapter to the connector, insert the network cable and reboot the server (it cannot be determined without rebooting), check whether the adapter is determined:

[root@xNode1 ~]# lspci ... 0a:00.0 PCI bridge: Intel Corporation DSL3510 Thunderbolt Controller [Cactus Ridge] 0b:00.0 PCI bridge: Intel Corporation DSL3510 Thunderbolt Controller [Cactus Ridge] 0c:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM57762 Gigabit Ethernet PCIe 

It remains a mystery to me why Broadcom did not include support for this adapter in the standard driver, but the fact remains that they did not do this, most likely due to some unknown agreement with Apple.


2. As mentioned earlier, installing something on the host system is quite problematic, for the reason that all unnecessary packages and even “make” are cut out from there, and the default repo is set to Citrix, there is no way you can block the Citrix repo and go through baserepo, but this is not our case, we will assume that we support the policy of the manufacturer and do not want to litter / harm our system.

In Citrix, they are not fools, and at the level that they have cleaned the system of "extras", they created an image of the system with the same kernel and the entire set of packages we need to build driver images, which can then be easily installed on the host system. This beast is called Driver Development Kit (DDK), you can download it here: http://support.citrix.com/article/CTX139817 (as of this writing, the current version is 6.2 SP1). This iso-Schnick is nothing more than an image of a virtual machine. However, in order to import it to our Xen server, you first need to mount it in your operating system, then in the Xen Center right click on our server and select import. To import, you must select the ova.xml file located in the ddk folder on the mounted image. (Do not forget to add an ethernet interface when importing)


3. After the image was imported, we assigned the root password, you need to get the driver source itself, it is located here: http://www.broadcom.com/support/ethernet_nic/netxtreme_desktop.php at the very bottom, called Linux (tg3) , for the convenience of habro-residents, I posted the already edited source: http://my-files.ru/Download/wqet0i/tg3-drv.zip .

Fill the file on the DDK server, and then unpack it:

 [root@localhost ~]# wget http://my-files.ru/Download/wqet0i/tg3-drv.zip ... 2014-01-14 01:48:13 (776 KB/s) - `tg3-drv.zip` saved [349509/349509] [root@localhost ~]# unzip tg3-drv.zip 

Before the assembly, we need a script that will build the driver image for us. The version of the script I have corrected can be taken http://my-files.ru/Download/ntqvvd/tg3-proj.zip , the original http://discussions.citrix.com/topic/301614-bcm5754-tg3-and-vlan-tagging -do-not-work-in-xs6 / , post 16 (registration is required).

Download the file, and then unpack it:

 [root@localhost ~]# wget http://my-files.ru/Download/ntqvvd/tg3-proj.zip ... 2014-01-14 02:29:38 (72.6 MB/s) - `tg3-proj.zip` saved [1738/1738] [root@localhost ~]# unzip tg3-proj.zip 

4. We proceed to build the driver image:

 [root@localhost ~]# cd tg3-drv/ [root@localhost tg3-drv]# make tg3_flags.h [root@localhost tg3-drv]# cd .. [root@localhost ~]# mv tg3-drv tg3-3.133d [root@localhost ~]# tar czvpf tg3-3.133d.tar.gz tg3-3.133d/ [root@localhost ~]# cp tg3-3.133d.tar.gz /usr/src/redhat/SOURCES/ 

Translated by the above steps, we prepared the source for the build script, now it remains only to run the script, but before running the script set the current time in the image, otherwise the script will end with a warning:

 [root@localhost ~]# cd tg3-proj/ [root@localhost tg3-proj]# date 011311282014.30 //(,  , , , 4  , ) Mon Jan 13 11:28:30 EST 2014 [root@localhost tg3-proj]# make ... Total translation table size: 0 Total rockridge attributes bytes: 780 Total directory bytes: 0 Path table size(bytes): 10 Max brk space used 21000 283 extents written (0 MB) 

The tg3.iso, tg3.iso.md5, tg3.metadata.md5 files appeared in the folder, etc., we are only interested in tg3.iso, it should be sent to the host server, this is done with the help of the scp command, which uses ssh to download files:

 [root@localhost tg3-proj]# scp tg3.iso root@192.168.0.2:/root/tg3.iso 

5. The most difficult thing is already behind, only the driver installation on the host server is left; for this we log in to the server, you can through the console in the Xen Center and perform the following actions:

 mkdir -p /mnt/tmp mount /root/tg3.iso /mnt/tmp -o loop,ro cd /mnt/tmp/ ./install.sh cd /root/ umount /mnt/tmp shutdown -r now 

And so, the final touch after a reboot, the adapter will not appear in the available devices, in order for it to appear, you need to go to the server console, call the manager through the “xsconsole” command and there in the item “Network and Management Interface” select the item “Emergency” Network Reset ”, after resetting your adapter and rebooting the Thunderbolt Ethernet Adapter will be available for use.

PS For the lazy, the image that I have already collected and verified: here .