Set the virtual machine IP over MAC without using DHCP

The article describes the use of scripts for CentOS and Windows XP, which set the IP in accordance with the MAC of the VM network interface, as well as the complexity of managing the network interface in Windows

The system we are developing is very actively working with virtual machines. When the next machine is required by the system kernel, a template image is copied and this copy is launched. Thus, a lot of copies of essentially the same machine can work at the same time.

Of course, at startup, each virtual machine is identical to the template one. Including inherited and the network settings. All virtual machines work on the same subnet, which means that they should not use the static IP that they got from the patterned machine - otherwise conflicts will arise. That is, each machine must receive its own IP. It would seem that the solution is very simple - use a DHCP server and dynamic IP.
')
However, there is another option, which I will discuss in this article.

1. Why do we need virtual machines
2. The problem of the same static addresses when running multiple VM clones
3. We code IP in MAC
4. IP Allocation by MAC: DHCP
5. IP Allocation by MAC: mac2ip script
a) Linux
b) Windows
6. Remarks

Update
Update 2

Spoiler: Actually, the most interesting is in points 3) and 5), the rest is for those who want to see the whole picture.

1. Why do we need virtual machines

Our project, Nerrvana , performs functional tests of sites in different browsers. These tests work with a special functional testing framework - Selenium, which allows you to emulate user actions in the browser (element clicks, mouse movements, character input, text reading), take screenshots of pages and some other things.
A site test is a sequence of actions that a user could do on a site and checks that the result of these actions is exactly as expected. For example, the simplest test - login to the site. It is necessary to open the login page, enter the login, password, press "Enter", and make sure that we are logged in - say, after seeing the standard greeting. Everyone knows that browsers can display and work on the same page in completely different ways, and therefore it makes sense to run the same tests in the most popular browsers. As already mentioned, it is this work that our system does.

Tests in the selected browsers are performed simultaneously and completely independently of each other. We called the test execution in one of the browsers a speck. That is, for example, if I want to perform a login test on IE 8 and FF 3.6 browsers, our system will make two independent specs — run the test code using the selected browsers. Not going into much detail, I’ll say that we create at least two virtual machines for the work of each spec. One machine, the “hub”, will actually perform the tests — there are Java and PHP on which tests should be written. On the second machine, the “tester,” Selenium RC and the required browser work. Via Selenium RC there is an interaction between the tests and the browser. After performing each spec, the virtual machines on which it worked are destroyed.

Since there can be a lot of simultaneously running tests, and everyone can use several specs, virtual machines can also work relatively well - 50, for example.

2. The problem of the same static addresses when running multiple VM clones

For each type of virtual machine there is a template image. When the next system is required by the system kernel, this image is copied and a copy of the image is launched - the image itself remains unchanged. Thus, a lot of copies of essentially the same machine can work at the same time.

And then there is some problem.
Naturally, at the time of launch, each virtual machine is absolutely identical to the template one - because all data is stored in an image that cannot be changed. Including inherited and the network settings. Since all virtual machines work on the same subnet, it is clear that they should not use the static IP that they got from the patterned machine - otherwise address conflicts will arise. That is, each machine must dynamically get its own IP.
It would seem that the solution is very simple - to use a DHCP server, which will give each machine a unique address.

However, not all so simple. The fact is that the core of the system actively interacts with virtual machines. It should just do a lot of work with them: for example, make sure that the virtual machines started successfully, run Selenium RC on the tester, upload to the hub and run the tests themselves, monitor their performance, and then get the results back (screenshots, logs, etc.). d.) Work is done through ssh.
Those. the core of the system, whatever one may say, must know the IP addresses of the machines that have just been launched at its request.

We have seen two approaches to solving the problem:
1) After the virtual machine has risen, it receives a random address from DHCP, and then somehow registers itself in the database - i.e. indicates that I am a machine of this kind, received such an address from DHCP.
2) Somehow the kernel makes it clear to the virtual machine what address it should use, i.e. IP matching - there is a virtual machine before the machine starts.

The first option seemed to us quite difficult for several reasons. This option made the VM too independent of the kernel, there was an extra connection - from the virtual machine to the database, there were difficulties with matching the requested and registered VMs, and some problems related to the kernel architecture.
We liked the second option much more, because the kernel immediately gained complete control over the VM, and did not end up in suspense, waiting for the VM to register itself.

3. We code IP in MAC

How to influence the not yet running machine to tell it the future ip-address? We found this way: when you start the virtual machine, you can set the MAC address of the virtual network card. And the virtual machine itself can recognize it at any time. That is, MAC can be used as a carrier of information about IP (or about something else).

How exactly will we encode the IP in the MAC address? For virtualization, we use xen, and virtual network cards xen must have a MAC that looks like this: 00: 16: 3E: XX: XX: XX, where 00: 16: 3E is the code of the manufacturer of the network card. We can use the last three bytes at our discretion (of course, remembering that the MAC must be unique).
Suppose that our system will work in the subnet 10.0.0.0/8, and therefore the logical solution is to use the values ​​for the last three bytes of the MAC corresponding to the last three bytes of the IP address we want to encode.
That is, for the address 10.1.1.3 we will use MAC 00: 16: 3E: 01: 01: 03.

It remains to force the machine to get the desired IP corresponding to its MAC. This again can be done in two ways.

4. IP Allocation by MAC: DHCP

The first way is pretty obvious. We will launch a DHCP server that will be configured to issue an IP by MAC address in accordance with the described encoding method.

The script will look like this:
1) The kernel requires a VM.
2) The kernel scans the pool of IP addresses, selects the first unused IP (for example, 10.4.0.15), and converts it to MAC (00: 16: 3E: 04: 00: 0F)
4) The kernel copies the template VM image of the desired type, prepares the VM configuration file (in which it also indicates the received MAC)
5) The kernel starts the VM
6) VM accesses DHCP for an address, which is checked against the MAC / IP lookup table, and issues IP 10.4.0.15.
7) At this time, the kernel periodically pings 10.4.0.15, and, having received the answer, it starts working with the virtual machine (of course, having waited before starting sshd)

- DHCP server is required
- if we move to another subnet or get another pool of IP addresses, we will have to change the configuration of not only the kernel, but also DHCP
+ Standard IP approach is used.

5. IP Allocation by MAC: mac2ip script

The second method is less obvious and requires some additional work.
It lies in the fact that the VM at startup will execute a special script that will receive the MAC, convert it to IP, and assign the network board. The scenario of work, thus, will be almost the same - only point 6 will change. It will look like this:

6) The VM calculates its IP based on its MAC, and sets it before starting the interface.

+ No additional link is required in the form of DHCP and storage of MAC-IP correspondence tables there.
- for each OS you need your own mac2ip script

We implemented this option.
We work with virtual machines with CentOS 5.6 and Windows XP PRO SP3, and therefore we needed two mac2ip scripts - for each of the systems.
Consider both scripts.

a) Linux

#!/bin/bash #      10 IP1=10 IFCFG=/etc/sysconfig/network-scripts/ifcfg-eth0 NETWORK=/etc/sysconfig/network case "$1" in *start) ;; *) exit ;; esac #  MAC  ,      MAC=$(ifconfig eth0|grep HWaddr|awk '{print $NF}'|grep ^00:16:3E) if [[ -z "$MAC" ]] ; then echo "Can't determine MAC address" >&2 exit 1 fi #  MAC  IP set -- $(echo $MAC|awk -F: '{print $4,$5,$6}') IPADDR=${IP1}.$((0x$1)).$((0x$2)).$((0x$3)) #    sed -i -e "s/^IPADDR.*/IPADDR=$IPADDR/" $IFCFG sed -i -e "/^HWADDR/d" $IFCFG sed -i -e "s/^HOSTNAME.*/HOSTNAME=localhost/" $NETWORK 

And we force this script to run before launching the network.

b) Windows

The same work in Windows XP is done in much more sophisticated ways. Perhaps over time, there will be a more efficient way to convert MAC to IP.

The first problem I encountered is the impossibility of changing the interface address BEFORE its inclusion in relatively easy ways. Thus, the template VM should have the “network connection” turned off by default, otherwise two simultaneously running copies of Windows immediately after launch will try to use the same address (it is static, because for VM we do not use DHCP).

Ok, this is not a problem - turn off the interface. However, the getmac utility, which we will use to get the mac address of the network, cannot return the MAC for the interface that is turned off! Therefore, we will first have to assign a random IP to the interface, enable it, learn the MAC, and then set the desired IP.

For device manipulations, the devcon utility is used.
Here's what it looks like:

 @echo off SET MAC= SET IP= SET MASK=255.255.255.0 SET GATEWAY= rem   IP.  IP      , rem           . rem 100 - ,  ,      rem     IP. set /A TEMP_THIRD_BYTE=100+%RANDOM:~0,2% set /A TEMP_FOURTH_BYTE=100+%RANDOM:~0,2% set TEMP_IP="192.168.%TEMP_THIRD_BYTE%.%TEMP_FOURTH_BYTE%" set TEMP_GATEWAY="192.168.%TEMP_THIRD_BYTE%.1" rem    netsh interface ip set address local static %TEMP_IP% %MASK% %TEMP_GATEWAY% 1 rem  .     C:\devcon\i386\devcon enable *VEN_10E* rem     MAC FOR /F "Tokens=4-6 Delims=- " %%d in ('getmac^|find "Device\Tcpip_"') do ( set /a dec_d=0x%%d set /a dec_e=0x%%e set /a dec_f=0x%%f ) rem   IP   SET IP=10.%dec_d%.%dec_e%.%dec_f% SET GATEWAY=10.%dec_d%.0.1 rem      netsh interface ip set address local static %IP% %MASK% %GATEWAY% 1 netsh interface ip set dns local static %GATEWAY% 

We add this script to autoload - for example, like this (a reboot is required):

 reg ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run" /v "mac2ip" /t REG_SZ /d "c:\init\mac2ip.bat" 

Thus, after these scripts are executed, the virtual machine will receive the address specified by the kernel, and will be ready for operation.
The kernel finds out about it by earning a ping to this address.

Perhaps this scheme will add visibility (on click - full size in a new window):

6. Remarks

The method we used — with the mac2ip script at startup — turned out to be rather slow in Win XP. In any case, its current implementation does its job in 10-15 seconds. At the same time, the time that elapses from the start of the virtual machine prior to the execution of the mac2ip script is 15-20 seconds.

However, we are not in a hurry to switch to using the first method (DHCP with IP to MAC binding), because:

- first, VMs are not completed in a regular way for them (i.e. centos / windows do not perform shutdowns), but by performing virsh destroy for a VM (just the same as pulling out power). This saves a lot of time, and the integrity of the used VM still doesn’t interest us - it will be immediately removed after use. So, the rented VM address will not be released immediately in this case, but will be released after the default-lease-time DHCP expires. This means that we can not immediately start a VM with the same MAC (and the same IP). It is unlikely that setting default-lease-time too small (seconds) is a good idea. A more realistic option is to study and use OMAPI / omshell, and with their help, delete unnecessary DHCP records immediately after stopping the VM.

- secondly, for Linux, getting the address from DHCP will be slower than the current option with the assignment of a static address.

- in the third - of course, there will be other pitfalls.

So, the relatively slow work of the current version of the script in Windows is not enough reason to switch to using the option with DHCP.

The ideal solution would be to speed up the mac2ip script for Windows. I will be glad to advice - since this is my first experience in managing windows network interface from scripts, then perhaps there is already a bicycle.


Update: First, The_Kf opened his eyes to the banal way of getting the MAC on the disabled interface using ipconfig.
Secondly, gribozavr conducted an experiment which showed that it is not necessary to take care of the rented IP lease-time at all, because when binding IP to MAC, DHCP will issue an IP in any case to a machine with the same MAC, even if the rent has not expired. He also pointed to the stateless autoconfiguration of IPv6.
Thirdly, when communicating with akshakirov, I suddenly realized why we didn’t immediately look more closely at DHCP.

Update 2: fourthly, amarao suggested using xenstore to transfer IP to VM. In the guest Linux machine, you just need to install xenstore-utils, but for Windows, you may need to write a utility to read from the xenstore.