Server from image: DHCP + TFTP + Initrd + OpenVZ
Greetings. Often, large projects use fairly large sets of identical servers that have the same software configuration (read the root). And often, the administrators of these machines need to maintain them in a symmetrical state - the same sets of packages, configs, etc. etc. One of the solutions to this problem is to download such machines over the network, so that they have a common root and keep it in RAM, and stored data (for example / var / www for web servers) are kept on hard drives mounted after the download. About this and talk.
One of the goals that I set is maximum simplicity. Therefore, this article uses the minimum set of tools, and they are as easy to use as possible. We will pack the image directly into the initrd, and issue it with the same TFTP server, which will give the kernel and the loader.
As an optional continuation, in the last section I will talk about the use of OpenVZ in this task - in my opinion, it’s more convenient to work with a virtual host than with the chroot environment - especially if you make massive server updates (and even for banal testing is convenient).
If you decide to use OpenVZ - think about settling a DHCP and TFTP server inside the VZ-container (especially if you already have several host systems) - this will allow you to deploy a DHCP server in case of failure of the main host system.
So let's go.
Everything is simple here. I used dhcpd. In the default config we enter:
')
Restart dhcpd. Now it will offer machines to boot over the network.
For tftp tftp-hpa was used. To work correctly in /etc/xinetd.d/tftp (in ubuntu, the file needed to be created independently) the following lines fit in:
As can be seen from the config, we will store all the good intended for downloading machines over the network in / var / lib / tftpboot. Restart xinetd. TFTP is now ready too
PXELinux is the twin brother of SYSLinuxʻa (which is used, probably in all Linux-CD installations), but focused on network booting. I myself have taken the bootloader from a Ubuntu netboot image, although I suspect there are other ways to get it. Put the binary (pxelinux.0) in / var / lib / tftpboot /. Its default configs can be located in the pxelinux.cfg subdirectory, then the default will be used as the main one (the full path for it will be: /var/lib/tftpboot/pxelinux.cfg/default). But we indicated in the dhcpd config that the servers of the bla-bla.ru group will retrieve their configs from configs / bla-bla.ru / config (the full path /var/lib/tftpboot/configs/bla-bla.ru/config ). Therefore, we create the necessary directories and write to the config:
As you can see, the kernel will be stored in kernels /, and the image itself is in images /. Of course, you will specify your kernel version later. But all this later, but for now we will be engaged in the manufacture of the image itself.
As I said, our image will be stored directly in the initrd. This way is much easier than using SquashFS + UnionFS (because the latter is not included in the kernel, and therefore requires chronic recompilation of the latter). And simpler / safer nfs_root. does not require additional services that may become additional points of failure.
I compiled my image in Ubuntu, for Debian, the procedure will be the same. So, we need debootstrap. Create a directory somewhere (let it be / root / image) and deploy an image into it. Well, for example, Ubuntu Lucid:
Now chroot` there and put the kernel (in the box). We exit the chroot`a, and drag the resulting kernel into / var / lib / tftpboot / kernels / and enter it into the config from the previous section. By the way, it can be removed from the environment itself so that it does not occupy precious space.
As you probably already understood, this environment is the future image. Here you can put the necessary packages and edit configs. Also, you definitely need to create / etc / fstab (even if you don’t want to fill it, although I would write the / proc;)) and the init file in the root (for some reason, Kernel Panic was obtained without it, although the file was intentionally there was no right to execute, and the content was empty). I also advise you to enter the receipt of ip-addresses via dhcp in / etc / network / interfaces - our DHCP server can issue static addresses.
Now we are ready to assemble an image. But before that you need to know about one particular process: cpio - does not know how to pack hardlinks . This means that if you do not disassemble them, you will lose some utilities - for example, ifup, ifdown, mkfs, and some other files.
To solve this problem, I wrote a small crutch on Perl, which parses them: the code is here . We tell this script as a parameter to the / root / image directory so that it can disassemble the links there. Next, go to the directory with the environment (/ root / image) and say:
This command will pack the image as an initrd, put it in the right place and create a link to it with current (which is specified in the config file).
That's all! When you try to boot on the network, the machine should unwind from the image we created.
With the help of OpenVZ we will make a live model of the image. It (OpenVZ) is convenient for our purposes, first of all, because the images of VZ virtualos are stored on the disk “as is” in the form of a tree, and therefore they are conveniently packaged. In my case, I went a little further and collected DHCP + TFTP also inside an OpenVZ container (separate, of course).
If you also want to use OpenVZ in solving this task - we will assume that you have already configured a server with everything you need for this virtualization system (support in the kernel, vzctl) - good, there is enough documentation on this topic.
Create a virtual machine:
Run it is not necessary. Delete the entire contents of the private directory (by default / var / lib / vz / private / 100, where 100 is the VEID specified during creation) and put the directory tree in it, which we did in the previous paragraph.
If you have already entered the auto-receiving network settings into the system being deployed from the image, do not be too lazy to enter the DHCP server and the address of this virtual machine into the configuration (you can find the MAC address of the container by going to the host system through vzctl enter and executing ifconfig is there), otherwise it risks to remain without a network.
Run:
With a positive outcome virtualka comes down to the DHCP-server for the address and will be available on the network. Now you can work with it in the same way as with a full-fledged machine from which images will be created.
To automatically build images, I wrote a small Perl script that works like this:
As you can see, the script is the simplest and, if desired, it can be rewritten to respond to any other change. You can hang his cron and he will follow the process himself, collecting as needed images.
There are two obvious disadvantages of the proposed option - the consumption of RAM for the use of a RAM disk and the higher boot time compared to hard drives (by obtaining an image and unpacking it). The flip side is the resulting functionality: once you have assembled an image, you will be able to commission a new server in minutes, and each assembled image can serve as a rollback point in case of an unsuccessful update.
Of course, only the general principles and manipulations necessary to obtain a basic system packed in an initrd with the ability to boot over the network are described here. Various options for finishing, such as the mdadmʻa config and small scripts that make life easier (for example, for partitioning disks on a new server) are left to the discretion of the reader.
I hope that it will be useful to someone. Thanks for attention. :)
A couple of words about what we will do
One of the goals that I set is maximum simplicity. Therefore, this article uses the minimum set of tools, and they are as easy to use as possible. We will pack the image directly into the initrd, and issue it with the same TFTP server, which will give the kernel and the loader.
As an optional continuation, in the last section I will talk about the use of OpenVZ in this task - in my opinion, it’s more convenient to work with a virtual host than with the chroot environment - especially if you make massive server updates (and even for banal testing is convenient).
If you decide to use OpenVZ - think about settling a DHCP and TFTP server inside the VZ-container (especially if you already have several host systems) - this will allow you to deploy a DHCP server in case of failure of the main host system.
So let's go.
DHCP
Everything is simple here. I used dhcpd. In the default config we enter:
# dhcpd pxelinux
option space pxelinux;
option pxelinux.magic code 208 = string;
option pxelinux.configfile code 209 = text;
option pxelinux.pathprefix code 210 = text;
option pxelinux.reboottime code 211 = unsigned integer 32;
site-option-space "pxelinux";
option pxelinux.magic f1:00:74:7e;
if exists dhcp-parameter-request-list {
option dhcp-parameter-request-list = concat(option dhcp-parameter-request-list,d0,d1,d2,d3);
}
group
{
option pxelinux.configfile "configs/bla-bla.ru/config"; # pxelinux`a
filename "/var/lib/tftpboot/pxelinux.0"; #
#
host first.bla-bla.ru
{
hardware ethernet 48:5b:39:90:b9:06; #MAC
fixed-address 192.168.0.100;
option host-name "first.bla-bla.ru";
}
#
host second.bla-bla.ru
{
hardware ethernet 48:5b:39:90:b9:07; #MAC
fixed-address 192.168.0.101;
option host-name "second.bla-bla.ru";
}
}
subnet 192.168.0.0 netmask 255.255.255.0 #, dhcp-
{
option routers 192.168.0.1; #
option domain-name-servers 192.168.0.2; #DNS
range 192.168.0.100 192.168.0.150; #
}
deny unknown-clients; #
')
Restart dhcpd. Now it will offer machines to boot over the network.
Tftp
For tftp tftp-hpa was used. To work correctly in /etc/xinetd.d/tftp (in ubuntu, the file needed to be created independently) the following lines fit in:
service tftp
{
port = 69
socket_type = dgram
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = /var/lib/tftpboot
disable = no
}
As can be seen from the config, we will store all the good intended for downloading machines over the network in / var / lib / tftpboot. Restart xinetd. TFTP is now ready too
Boot Loader (PXELinux)
PXELinux is the twin brother of SYSLinuxʻa (which is used, probably in all Linux-CD installations), but focused on network booting. I myself have taken the bootloader from a Ubuntu netboot image, although I suspect there are other ways to get it. Put the binary (pxelinux.0) in / var / lib / tftpboot /. Its default configs can be located in the pxelinux.cfg subdirectory, then the default will be used as the main one (the full path for it will be: /var/lib/tftpboot/pxelinux.cfg/default). But we indicated in the dhcpd config that the servers of the bla-bla.ru group will retrieve their configs from configs / bla-bla.ru / config (the full path /var/lib/tftpboot/configs/bla-bla.ru/config ). Therefore, we create the necessary directories and write to the config:
default linux
timeout 100
label linux
kernel kernels/vmlinuz-2.6.32
append panic=15 initrd=images/bla-bla.ru/current
As you can see, the kernel will be stored in kernels /, and the image itself is in images /. Of course, you will specify your kernel version later. But all this later, but for now we will be engaged in the manufacture of the image itself.
Cooking image
As I said, our image will be stored directly in the initrd. This way is much easier than using SquashFS + UnionFS (because the latter is not included in the kernel, and therefore requires chronic recompilation of the latter). And simpler / safer nfs_root. does not require additional services that may become additional points of failure.
I compiled my image in Ubuntu, for Debian, the procedure will be the same. So, we need debootstrap. Create a directory somewhere (let it be / root / image) and deploy an image into it. Well, for example, Ubuntu Lucid:
debootstrap lucid /root/imageNow chroot` there and put the kernel (in the box). We exit the chroot`a, and drag the resulting kernel into / var / lib / tftpboot / kernels / and enter it into the config from the previous section. By the way, it can be removed from the environment itself so that it does not occupy precious space.
As you probably already understood, this environment is the future image. Here you can put the necessary packages and edit configs. Also, you definitely need to create / etc / fstab (even if you don’t want to fill it, although I would write the / proc;)) and the init file in the root (for some reason, Kernel Panic was obtained without it, although the file was intentionally there was no right to execute, and the content was empty). I also advise you to enter the receipt of ip-addresses via dhcp in / etc / network / interfaces - our DHCP server can issue static addresses.
Now we are ready to assemble an image. But before that you need to know about one particular process: cpio - does not know how to pack hardlinks . This means that if you do not disassemble them, you will lose some utilities - for example, ifup, ifdown, mkfs, and some other files.
To solve this problem, I wrote a small crutch on Perl, which parses them: the code is here . We tell this script as a parameter to the / root / image directory so that it can disassemble the links there. Next, go to the directory with the environment (/ root / image) and say:
find | cpio -H newc -o -p > ../bla-bla.ru; gzip -9 ../bla-bla.ru;cp ../bla-bla.ru.gz /var/lib/tftpboot/images/bla-bla.ru/2011-03-16;ln -s /var/lib/tftpboot/images/bla-bla.ru/2011-03-16 /var/lib/tftpboot/images/bla-bla.ru/currentThis command will pack the image as an initrd, put it in the right place and create a link to it with current (which is specified in the config file).
That's all! When you try to boot on the network, the machine should unwind from the image we created.
Openvz
With the help of OpenVZ we will make a live model of the image. It (OpenVZ) is convenient for our purposes, first of all, because the images of VZ virtualos are stored on the disk “as is” in the form of a tree, and therefore they are conveniently packaged. In my case, I went a little further and collected DHCP + TFTP also inside an OpenVZ container (separate, of course).
If you also want to use OpenVZ in solving this task - we will assume that you have already configured a server with everything you need for this virtualization system (support in the kernel, vzctl) - good, there is enough documentation on this topic.
Create a virtual machine:
vzctl create 100 --hostname model.bla-bla.ruRun it is not necessary. Delete the entire contents of the private directory (by default / var / lib / vz / private / 100, where 100 is the VEID specified during creation) and put the directory tree in it, which we did in the previous paragraph.
If you have already entered the auto-receiving network settings into the system being deployed from the image, do not be too lazy to enter the DHCP server and the address of this virtual machine into the configuration (you can find the MAC address of the container by going to the host system through vzctl enter and executing ifconfig is there), otherwise it risks to remain without a network.
Run:
vzctl start 100With a positive outcome virtualka comes down to the DHCP-server for the address and will be available on the network. Now you can work with it in the same way as with a full-fledged machine from which images will be created.
To automatically build images, I wrote a small Perl script that works like this:
- In the title, we declare the VEIDs of the “models” (virtual machines from which we will shoot images) and the dhcp server. If your dhcp-server does not live on the VZ-container - just correct $ out_path)
- For each virtualka specified in the script, the modification time of / var / lib / apt / extended_states is checked (this file is changed each time the package list is changed - setting / updating something or via apt-get); the resulting time stamp is recorded in / var / cache / netboot / number (directory must exist).
- If the time stamp from the script cache is lower than the change time of the scanned file, the virtual machine stops.
- Next, the / var / cache / apt is cleaned (to save space) and a cpio image is assembled, after which the virtual machine is started back. Previously, the previous script is executed that parses the hard links. It is believed that it lies in the directory described in PATH, and it is called nb_links_breaker.
- In the final stage, the cpio image is pressed by gzip, it moves to the right place (by default - / var / lib / tftpboot / images / virtual_name / date and a link from current is attached to it.
As you can see, the script is the simplest and, if desired, it can be rewritten to respond to any other change. You can hang his cron and he will follow the process himself, collecting as needed images.
Total
There are two obvious disadvantages of the proposed option - the consumption of RAM for the use of a RAM disk and the higher boot time compared to hard drives (by obtaining an image and unpacking it). The flip side is the resulting functionality: once you have assembled an image, you will be able to commission a new server in minutes, and each assembled image can serve as a rollback point in case of an unsuccessful update.
Of course, only the general principles and manipulations necessary to obtain a basic system packed in an initrd with the ability to boot over the network are described here. Various options for finishing, such as the mdadmʻa config and small scripts that make life easier (for example, for partitioning disks on a new server) are left to the discretion of the reader.
I hope that it will be useful to someone. Thanks for attention. :)
Source: https://habr.com/ru/post/115531/
All Articles