Become the master of loading Linux
First, we will introduce udev and learn how to use it to examine the devices installed on the computer right at boot time: as an example, we will automatically select the video card settings for Xorg. Then we will solve the problem of working with one image on dozens of computers at the same time by implementing our own handler in the initramfs, and at the same time we optimize the system for network boot. To further reduce the load time and reduce the load on the network, try replacing NFS with NBD, and help TFTP using HTTP. In the end, we will return to the beginning - to the boot server, which will also be switched to read-only mode.
This article is more a study, not a ready-made guide (all solutions work, they just are not always optimal). In the end, you will have enough knowledge to do everything the way you want it.
')
Start look here:
Initial server setup
Preparing the image for network boot
We stopped at the fact that we downloaded the VirtualBox machine over the network and launched Firefox. If you now try to do the same with a real computer, then the screen will show the cyclical authorization of the user username and unsuccessful attempts to start the graphical environment - Xorg does not find the necessary driver.
For the graphics mode to work in VirtualBox, we have installed everything you need. It was originally planned that our diskless system will function on any hardware, butdue to laziness we will not try to embrace the immense, so we limit ourselves to supporting graphics solutions from the following dominant manufacturers: nVidia, Intel and AMD.
Switch on the client machine to the second terminal by pressing Ctrl + Alt + F2 and install open drivers:
Most likely, this time Xorg will not be able to choose the appropriate settings for each case on its own, and judging from the screen of the downloaded client, it will seem that nothing has changed at all.
The easiest way to find out which video devices are in the system is to enter a command in the console:
But we will not look for easy ways, and as a reward we will receive a new portion of knowledge.
I mentioned earlier that the device manager in Archlinux is called udev . It is included in the systemd package under the name systemd-udevd.
As new devices are discovered in the boot system, the kernel creates their hierarchy in the / devices directory. First, the PCI system itself appears, then it detects the tires on which the end devices “sit”, and their drivers sort the devices into classes. Devices within classes are detected in parallel, just as systemd runs services in parallel to achieve the next goal.
Asynchronous search for devices leads to the fact that if several devices belonging to the same class are present on the computer at the same time, they can be found in a different order from switching to switching on, for example, first one video card and then the other, and their names will be change between themselves. Fortunately, the emergence of a new element in the device hierarchy is a udev event that can be tracked and actions taken.
For the udev manager, rules were designed to streamline chaos and simplify the life of installed programs. The rules are stored in the folders /usr/lib/udev/rules.d/ and /etc/udev/rules.d/ (the latter, as in the case of hooks, has a higher priority and the files from there are checked first). The appearance of a new element in the device hierarchy is accompanied by checking all established udev rules, and automatically performing the actions specified there, in the event of a match. Usually these steps consist in renaming devices and setting up links to them in directories inside / dev and / sys for ease of use in programs.
Video card drivers refer them to the drm subsystem (class), so information about such devices is duplicated in the / sys / class / drm directory. The first video card found in the system, by default, gets the name "card0", if it has several video outputs, then they get names like "card0-CON-n", where "CON" is the connector type (VGA, HDMI, DVI and etc.), and “n” is the serial number of the connector (moreover, some manufacturers number the connectors starting with “0” and others with “1”). The next video card becomes “card1”, etc.
If nothing is done, in view of the parallelism of detection, the next time card1 is turned on, it can become card0 and vice versa. Udev will add such devices to / dev with one or another name. Cases where this udev behavior is undesirable are described in detail on the Internet, and they mainly discuss various USB devices. We also need to run a certain program when detecting video cards, which we will write later, but for now we’ll find out what udev is known for.
To find out the same thing that knows about the udev video card, enter the command on the client:
Notice the tree structure using the paradigm of the parent and child devices. In the lines beginning with “looking at ...” the path to this device is relative to the / sys directory, i.e. by accessing the video card along the path / sys / class / drm / card0, we found that this is actually a link to /sys/devices/pci0000:00/0000:00:02.0/drm/card0.
The parent device /devices/pci0000:00/0000:00:02.0 has a vendor attribute with a manufacturer ID. Udev has access to an extensive database and can translate this code into a digestible form:
Compare with the command output:
Connect to the boot server. And create a file with the rules:
Each rule is written in a new line. The first part is to identify the udev event to which you want to apply the action specified at the end of the line. To identify the event, data is used that can be obtained from the output of the command “udevadm info -a -p / sys ...”.
The rule from the first line works for all devices with the name (core) card0, card1 ... of the drm subsystem. The second rule will work only for active devices from the drm subsystem to which the monitor is currently connected (it will not work for card0, card1, but only for names like card0-HDMI-1, since only such devices have enabled attributes and status). When an event coincides with its description, the same program is executed, in which in the first case one parameter% n is sent (the kernel sequence number, which for card0 will be “0”), and in the second - the additional parameter% k (the kernel name itself “ card0 ").
The / etc / default / xdevice program will change the contents of the file in the /etc/X11/xorg.conf.d/ folder, which contains information about the video adapter settings for xorg. For different manufacturers, we will prepare different templates that take into account the specifics of the implementation. It is enough to specify the minimum necessary information to uniquely identify the device, and xorg will do the rest on its own:
We will indicate the necessary data in the template itself, or we will obtain by examining the output of the udevadm info command.
The program will be triggered for each output of each video card to which the monitor is connected. To simplify the task, let's make the last discovered option work so that at least one monitor works on multimonitor systems. This is not the best way to configure, and it would be better to check the graphics subsystem once before reaching the graphical.target, but our option is working and is suitable for learning the udev rules in action.
Create a program file with the following content:
Make the file executable:
Turning off the automatic loading of the VirtualBox service, since now it will be launched by our program only when necessary:
Add xorg configuration file templates with settings optimized for the main manufacturers:
At the end of the xorg configuration, we will do the “Windows like” switch of the keyboard layout with the Alt + Shift combination:
Logs of work of all components of Archlinux are saved in the log . If everything is left as it is, the journal can be quite inflated, so we will limit its size, say 30 MB (add or uncomment the line):
Each action is logged in the / var / log / journal folder. In our case, the data transfer is carried out over the network, which in practice has a low bandwidth. You can delete the folder with the magazine, it will only be saved in RAM, which is ideal for a diskless client:
With various errors in the operation of applications, automatic kernel dumps are created in the / var / lib / systemd / coredump folder. We will turn them off for the same reason:
Disable SWAP :
Remove unnecessary localization . This simple action will save over 65 MB. At the same time, let's see how programs are installed from AUR (in fact, they are built from source). Log on to the boot server with standard user rights and follow these steps:
Package is ready. Install it from the file, not from the repository, so the S key is replaced with U (correct the file name if the version of the program you have compiled does not match mine):
Now set up. Comment out the “NEEDCONFIGFIRST” line at the beginning of the file and indicate the localizations used at the very end:
Configure and run the program:
If we try to load the existing system on several computers at the same time, then all copies will change the same folders on the server. If one client deletes a file, it will suddenly disappear from the other. The surest way to guard against change is to go into read-only mode.
The problem is that for the normal operation of the system, it is necessary to write data to some folders. The solution on the surface — connecting these folders via fstab like tmpfs — is great for / var / log, for example. But what to do, for example, with the / etc directory, because our udev rule changes files there, and other programs work with it actively? You can save information somewhere before mounting, and then rewrite it back. Or immediately move everything somewhere else, and then return. One thing is clear: you have to test for a long time and monitor the operation of the system in order to understand what other folders to make available for recording, or to set up all the programs so that they leave the products of their livelihoods strictly in the space provided. Too tricky. I propose to deploy the entire system in RAM. It remains only to pre-rewrite into it all the most necessary for work.
There is one folder in which nothing is recorded during the work, if we do not install anything - this is / usr. If you mount it at a later stage in initramfs with read-only access, then Firefox’s work will not be affected in any way. Be sure to compare the size of the / usr directory with the size of everything else, and it turns out that there will be not much to copy, and if you exclude everything that is superfluous ... Did you also think about rsync ?
Install rsync on the client:
Copying is to be done at the stage of intramfs, therefore, we need a new handler, let's call it “live”. First, we will save all the necessary mount parameters for the original root directory by analyzing the / etc / fstab file using the findmnt utility. Then unmount the root directory from the / new_root folder, where it is always located inside the initramfs. In its place, create a ramfs with the ability to write and prepare a mount point / srv / new_root, where we will return the original root directory. All that is needed is to rewrite all the most needed files and directories, with the exception of the / usr folder, which we will read in read-only mode. Copies of files in ramfs will be available for reading and writing.
We refer to the file / etc / fstab twice: the first time we get information on the mount parameters of the root directory, and the second time we check if there is any information on / usr in fstab. For late mounting of / usr in Archlinux there is a special usr handler, which we will not interfere with doing our work. If / usr is mounted in some special way, then our handler skips it.
The text mentions the / etc / default / live_filter file with filtering rules intended for rsync, we need to remember to prepare it. We do this automatically from the handler installer:
Rsync "does not see" more than one directory. Files and folders in the directory are checked by each rule in order to the first match ("+" - the object is copied, "-" - the object is not copied). If there is no match, the file is copied, and the directory is created empty. Next, rsync enters the "surviving" directory and again applies the rules to its contents. This is repeated until there is absolutely nothing left.
In our case, the root directory does not fall under any rule, so its structure is completely transferred (all files are copied and empty directories are created). The directories / boot, / dev, / lost + found, / mnt, / opt, / proc, / root, / run, / srv, / sys, / tmp fall under the effect of the last rule "- / * / *", m. e. none of their contents is copied anywhere, but they themselves are created. The / etc directory immediately falls under the "+ / etc / *" rule, and all its contents are copied, but first only within one directory (later on, its entire structure will be transferred in order, because for the nesting levels / etc / * / and then there are no rules). A similar beginning is waiting for the directory / home - folders of all users fall under the rule "+ / home / *" and will be recreated in a copy (empty for now). The following rule "+ /home/*/.config" copies the .config directories that are attached to the home folders of each user, and "- / home / * / * /" excludes all other directories (the rule comes after the "rescue", so for /home/*/.config does not work). About the files themselves from the home directory says nothing, so they are completely transferred. Files from excluded subdirectories are not copied, because these directories were not created. The "- / var / cache / * / *" rule retains the entire directory structure in / var / cache, but their contents are not migrated. The remaining rules are similar.
Add a handler to the initramfs:
Generate initramfs:
Physically, of course, network overclocking is now impossible without changing equipment, but software optimizations are not prohibited. We need to transfer the contents of the associated / usr folder over the network. We are not able to send this data, but we are able to reduce the amount of space they occupy - to archive. , — , .
squashfs fstab, . — ( ) — :
:
initramfs /usr usr, :
, :
20% , . , live ramfs .
/srv/source_usr.sfs ramfs rsync, fstab , , RAM, .
, : « ?». , , squashfs NFS ( ), Network Block Device , . « », « », . btrfs zlib, squashfs .
initramfs NBD- AUR mkinitcpio-nbd ( ):
$root/boot/grub/grub.cfg :
, :
NBD /dev/nbd0, :
NBD ( ). NBD , , «» . NBD , , . NBD IP netcat , , IP :
. :
initramfs - net_nfs4, nbd:
initramfs:
$root/srv/source_usr.sfs $root — /usr , /usr:
:
NBD :
. habrahabr, , , « », copyonwrite . Copyonwrite , , . . . NBD , man' .
, :
:
, netcat netstat:
NBD:
, , .
3% ( ). , NFS 10 , NBD .
. — TFTP . , iPXE , kvaps .
username.
, :
vmlinuz-linux initramfs HTTP. :
root :
DHCP , :
HTTP :
:
« »:
:
, :
TFTP HTTP iPXE. , , . , .
live . rsync /srv, . systemd:
/srv/new_root/srv /srv , .
, « », , USB . , . , . , VPN , …
( ), VirtualBox ( > USB ). lsblk, . , HABR /mnt.
rsync:
:
— /dev/sdb
/mnt .
PS . . . .
.
This article is more a study, not a ready-made guide (all solutions work, they just are not always optimal). In the end, you will have enough knowledge to do everything the way you want it.
')
Start look here:
Initial server setup
Preparing the image for network boot
We stopped at the fact that we downloaded the VirtualBox machine over the network and launched Firefox. If you now try to do the same with a real computer, then the screen will show the cyclical authorization of the user username and unsuccessful attempts to start the graphical environment - Xorg does not find the necessary driver.
We start video cards
For the graphics mode to work in VirtualBox, we have installed everything you need. It was originally planned that our diskless system will function on any hardware, but
Switch on the client machine to the second terminal by pressing Ctrl + Alt + F2 and install open drivers:
pacman -S xf86-video-ati xf86-video-nouveau xf86-video-intel Most likely, this time Xorg will not be able to choose the appropriate settings for each case on its own, and judging from the screen of the downloaded client, it will seem that nothing has changed at all.
The easiest way to find out which video devices are in the system is to enter a command in the console:
lspci | grep -i vga 00:02.0 VGA compatible controller: InnoTek Systemberatung GmbH VirtualBox Graphics Adapter But we will not look for easy ways, and as a reward we will receive a new portion of knowledge.
Take a closer look at udev
I mentioned earlier that the device manager in Archlinux is called udev . It is included in the systemd package under the name systemd-udevd.
As new devices are discovered in the boot system, the kernel creates their hierarchy in the / devices directory. First, the PCI system itself appears, then it detects the tires on which the end devices “sit”, and their drivers sort the devices into classes. Devices within classes are detected in parallel, just as systemd runs services in parallel to achieve the next goal.
Asynchronous search for devices leads to the fact that if several devices belonging to the same class are present on the computer at the same time, they can be found in a different order from switching to switching on, for example, first one video card and then the other, and their names will be change between themselves. Fortunately, the emergence of a new element in the device hierarchy is a udev event that can be tracked and actions taken.
For the udev manager, rules were designed to streamline chaos and simplify the life of installed programs. The rules are stored in the folders /usr/lib/udev/rules.d/ and /etc/udev/rules.d/ (the latter, as in the case of hooks, has a higher priority and the files from there are checked first). The appearance of a new element in the device hierarchy is accompanied by checking all established udev rules, and automatically performing the actions specified there, in the event of a match. Usually these steps consist in renaming devices and setting up links to them in directories inside / dev and / sys for ease of use in programs.
Video card drivers refer them to the drm subsystem (class), so information about such devices is duplicated in the / sys / class / drm directory. The first video card found in the system, by default, gets the name "card0", if it has several video outputs, then they get names like "card0-CON-n", where "CON" is the connector type (VGA, HDMI, DVI and etc.), and “n” is the serial number of the connector (moreover, some manufacturers number the connectors starting with “0” and others with “1”). The next video card becomes “card1”, etc.
If nothing is done, in view of the parallelism of detection, the next time card1 is turned on, it can become card0 and vice versa. Udev will add such devices to / dev with one or another name. Cases where this udev behavior is undesirable are described in detail on the Internet, and they mainly discuss various USB devices. We also need to run a certain program when detecting video cards, which we will write later, but for now we’ll find out what udev is known for.
To find out the same thing that knows about the udev video card, enter the command on the client:
udevadm info -a -p /sys/class/drm/card0 command output
Udevadm info
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
The rule is to match.
one single parent device.
looking at device '/devices/pci0000:00/0000:00:02.0.0/drm/card0':
KERNEL == "card0"
SUBSYSTEM == "drm"
DRIVER == ""
looking at parent device '/devices/pci0000:00/0000:00:02.0':
KERNELS == "0000: 00: 02.0"
SUBSYSTEMS == "pci"
DRIVERS == ""
ATTRS {irq} == "18"
ATTRS {subsystem_vendor} == "0x0000"
ATTRS {broken_parity_status} == "0"
ATTRS {class} == "0x030000"
ATTRS {driver_override} == "(null)"
ATTRS {consistent_dma_mask_bits} == "32"
ATTRS {dma_mask_bits} == "32"
ATTRS {local_cpus} == "00000000,00000000,00000000,00000001"
ATTRS {device} == "0xbeef"
ATTRS {enable} == "1"
ATTRS {msi_bus} == "1"
ATTRS {local_cpulist} == "0"
ATTRS {vendor} == "0x80ee"
ATTRS {subsystem_device} == "0x0000"
ATTRS {boot_vga} == "1"
ATTRS {numa_node} == "- 1"
ATTRS {d3cold_allowed} == "0"
looking at parent device '/ devices / pci0000: 00':
KERNELS == "pci0000: 00"
SUBSYSTEMS == ""
DRIVERS == ""
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
The rule is to match.
one single parent device.
looking at device '/devices/pci0000:00/0000:00:02.0.0/drm/card0':
KERNEL == "card0"
SUBSYSTEM == "drm"
DRIVER == ""
looking at parent device '/devices/pci0000:00/0000:00:02.0':
KERNELS == "0000: 00: 02.0"
SUBSYSTEMS == "pci"
DRIVERS == ""
ATTRS {irq} == "18"
ATTRS {subsystem_vendor} == "0x0000"
ATTRS {broken_parity_status} == "0"
ATTRS {class} == "0x030000"
ATTRS {driver_override} == "(null)"
ATTRS {consistent_dma_mask_bits} == "32"
ATTRS {dma_mask_bits} == "32"
ATTRS {local_cpus} == "00000000,00000000,00000000,00000001"
ATTRS {device} == "0xbeef"
ATTRS {enable} == "1"
ATTRS {msi_bus} == "1"
ATTRS {local_cpulist} == "0"
ATTRS {vendor} == "0x80ee"
ATTRS {subsystem_device} == "0x0000"
ATTRS {boot_vga} == "1"
ATTRS {numa_node} == "- 1"
ATTRS {d3cold_allowed} == "0"
looking at parent device '/ devices / pci0000: 00':
KERNELS == "pci0000: 00"
SUBSYSTEMS == ""
DRIVERS == ""
Notice the tree structure using the paradigm of the parent and child devices. In the lines beginning with “looking at ...” the path to this device is relative to the / sys directory, i.e. by accessing the video card along the path / sys / class / drm / card0, we found that this is actually a link to /sys/devices/pci0000:00/0000:00:02.0/drm/card0.
The parent device /devices/pci0000:00/0000:00:02.0 has a vendor attribute with a manufacturer ID. Udev has access to an extensive database and can translate this code into a digestible form:
udevadm info -q property -p /sys/devices/pci0000:00/0000:00:02.0 DEVPATH=/devices/pci0000:00/0000:00:02.0 ID_MODEL_FROM_DATABASE=VirtualBox Graphics Adapter ID_PCI_CLASS_FROM_DATABASE=Display controller ID_PCI_INTERFACE_FROM_DATABASE=VGA controller ID_PCI_SUBCLASS_FROM_DATABASE=VGA compatible controller ID_VENDOR_FROM_DATABASE=InnoTek Systemberatung GmbH MODALIAS=pci:v000080EEd0000BEEFsv00000000sd00000000bc03sc00i00 PCI_CLASS=30000 PCI_ID=80EE:BEEF PCI_SLOT_NAME=0000:00:02.0 PCI_SUBSYS_ID=0000:0000 SUBSYSTEM=pci USEC_INITIALIZED=24450 Compare with the command output:
lspci | grep -i vga 00:02.0 VGA compatible controller: InnoTek Systemberatung GmbH VirtualBox Graphics Adapter. Dynamic video card configuration with udev
Connect to the boot server. And create a file with the rules:
export root=/srv/nfs/diskless nano $root/etc/udev/rules.d/10-graphics.rules KERNEL=="card[0-9]*", SUBSYSTEM=="drm", RUN+="/etc/default/xdevice %n" KERNEL=="card*", SUBSYSTEM=="drm", ATTR{enabled}=="enabled", ATTR{status}=="connected", RUN+="/etc/default/xdevice %n %k" Each rule is written in a new line. The first part is to identify the udev event to which you want to apply the action specified at the end of the line. To identify the event, data is used that can be obtained from the output of the command “udevadm info -a -p / sys ...”.
The rule from the first line works for all devices with the name (core) card0, card1 ... of the drm subsystem. The second rule will work only for active devices from the drm subsystem to which the monitor is currently connected (it will not work for card0, card1, but only for names like card0-HDMI-1, since only such devices have enabled attributes and status). When an event coincides with its description, the same program is executed, in which in the first case one parameter% n is sent (the kernel sequence number, which for card0 will be “0”), and in the second - the additional parameter% k (the kernel name itself “ card0 ").
The / etc / default / xdevice program will change the contents of the file in the /etc/X11/xorg.conf.d/ folder, which contains information about the video adapter settings for xorg. For different manufacturers, we will prepare different templates that take into account the specifics of the implementation. It is enough to specify the minimum necessary information to uniquely identify the device, and xorg will do the rest on its own:
Section "Device" Identifier " " Driver " " Option "AccelMethod" " " BusID "PCI: PCI, " EndSection We will indicate the necessary data in the template itself, or we will obtain by examining the output of the udevadm info command.
The program will be triggered for each output of each video card to which the monitor is connected. To simplify the task, let's make the last discovered option work so that at least one monitor works on multimonitor systems. This is not the best way to configure, and it would be better to check the graphics subsystem once before reaching the graphical.target, but our option is working and is suitable for learning the udev rules in action.
Create a program file with the following content:
nano $root/etc/default/xdevice Hidden text
#!/bin/sh # xorg CONF_FILE=/etc/X11/xorg.conf.d/20-device.conf # "" # , get_vendor(){ local card=$(get_path $1) udevadm info -q property -p ${card%\/drm*} | \ awk '/^ID_VENDOR_FROM_DATABASE/{split($1,a,"=");print tolower(a[2])}' } # PCI x:y:z get_bus(){ local bus=$(get_path $1) echo ${bus%\/drm*} | \ sed 's\:\.\g' | \ awk '{n=split($0,a,".");printf "%i:%i:%i",a[n-2],a[n-1],a[n]}' } # get_path(){ udevadm info -q path -p /sys/class/drm/$1 } # . make_conf(){ local filename="xorg-device-$(get_vendor $1).conf" cat /etc/X11/$filename | \ sed 's\%BUS%\'$(get_bus $1)'\g'| \ sed 's\%ID%\'$1'\g' > $CONF_FILE } # virtualbox, check_vbox(){ local vendor=$(get_vendor $1) [ "$vendor" == "innotek" ]] && systemctl start vboxservice } # card_numb=$1 if [ -z "$2" ] # virtualbox then card_name="card$card_numb" check_vbox $card_name && make_conf $card_name else card_name=$2 make_conf $card_name fi Make the file executable:
chmod +x $root/etc/default/xdevice Turning off the automatic loading of the VirtualBox service, since now it will be launched by our program only when necessary:
systemctl disable vboxservice Add xorg configuration file templates with settings optimized for the main manufacturers:
nano $root/etc/X11/xorg-device-intel.conf Section "Device" Identifier "Intel %ID%" Driver "intel" Option "AccelMethod" "uxa" BusID "PCI:%BUS%" EndSection AMD, nVidia, VirtualBox
Add your templates and do not forget to install drivers for these devices. Specify in the comments proven combinations.
nano $root/etc/X11/xorg-device-innotek.conf Section "Device" Identifier "VirtualBox %ID%" Driver "vboxvideo" BusID "PCI:%BUS%" EndSection nano $root/etc/X11/xorg-device-advanced.conf Section "Device" Identifier "AMD %ID%" Driver "radeon" Option "AccelMethod" "exa" BusID "PCI:%BUS%" EndSection nano $root/etc/X11/xorg-device-nvidia.conf Section "Device" Identifier "nVidia %ID%" Driver "nouveau" Option "AccelMethod" "exa" BusID "PCI:%BUS%" EndSection Add your templates and do not forget to install drivers for these devices. Specify in the comments proven combinations.
At the end of the xorg configuration, we will do the “Windows like” switch of the keyboard layout with the Alt + Shift combination:
nano $root/etc/X11/xorg.conf.d/50-keyboard.conf Section "InputClass" Identifier "keyboard-layout" MatchIsKeyboard "on" Option "XkbLayout" "us,ru" Option "XkbVariant" ",winkeys" Option "XkbOptions" "grp:alt_shift_toggle" EndSection We optimize the system
Logs of work of all components of Archlinux are saved in the log . If everything is left as it is, the journal can be quite inflated, so we will limit its size, say 30 MB (add or uncomment the line):
nano $root/etc/systemd/journald.conf ... SystemMaxUse=30M ... Each action is logged in the / var / log / journal folder. In our case, the data transfer is carried out over the network, which in practice has a low bandwidth. You can delete the folder with the magazine, it will only be saved in RAM, which is ideal for a diskless client:
rm -r $root/var/log/journal With various errors in the operation of applications, automatic kernel dumps are created in the / var / lib / systemd / coredump folder. We will turn them off for the same reason:
nano $root/etc/systemd/coredump.conf ... Storage=none ... Disable SWAP :
echo -e 'vm.swappiness=0\nvm.vfs_cache_pressure=50' > $root/etc/sysctl.d/99-sysctl.conf Remove unnecessary localization . This simple action will save over 65 MB. At the same time, let's see how programs are installed from AUR (in fact, they are built from source). Log on to the boot server with standard user rights and follow these steps:
curl -o localepurge.tar.gz https://aur.archlinux.org/packages/lo/localepurge/localepurge.tar.gz tar -xvvzf localepurge.tar.gz cd localepurge makepkg -s Package is ready. Install it from the file, not from the repository, so the S key is replaced with U (correct the file name if the version of the program you have compiled does not match mine):
sudo pacman --root $root --dbpath $root/var/lib/pacman -U localepurge-0.7.3.4-1-any.pkg.tar.xz Now set up. Comment out the “NEEDCONFIGFIRST” line at the beginning of the file and indicate the localizations used at the very end:
nano $root/etc/locale.nopurge ... # NEEDSCONFIGFIRST ... ru ru_RU ru_RU.UTF-8 en en_US en_US.UTF-8 Configure and run the program:
arch-chroot $root /usr/bin/localepurge-config arch-chroot $root localepurge Go to read-only
If we try to load the existing system on several computers at the same time, then all copies will change the same folders on the server. If one client deletes a file, it will suddenly disappear from the other. The surest way to guard against change is to go into read-only mode.
The problem is that for the normal operation of the system, it is necessary to write data to some folders. The solution on the surface — connecting these folders via fstab like tmpfs — is great for / var / log, for example. But what to do, for example, with the / etc directory, because our udev rule changes files there, and other programs work with it actively? You can save information somewhere before mounting, and then rewrite it back. Or immediately move everything somewhere else, and then return. One thing is clear: you have to test for a long time and monitor the operation of the system in order to understand what other folders to make available for recording, or to set up all the programs so that they leave the products of their livelihoods strictly in the space provided. Too tricky. I propose to deploy the entire system in RAM. It remains only to pre-rewrite into it all the most necessary for work.
There is one folder in which nothing is recorded during the work, if we do not install anything - this is / usr. If you mount it at a later stage in initramfs with read-only access, then Firefox’s work will not be affected in any way. Be sure to compare the size of the / usr directory with the size of everything else, and it turns out that there will be not much to copy, and if you exclude everything that is superfluous ... Did you also think about rsync ?
Redoing the file system on the fly
Install rsync on the client:
pacman -S rsync Copying is to be done at the stage of intramfs, therefore, we need a new handler, let's call it “live”. First, we will save all the necessary mount parameters for the original root directory by analyzing the / etc / fstab file using the findmnt utility. Then unmount the root directory from the / new_root folder, where it is always located inside the initramfs. In its place, create a ramfs with the ability to write and prepare a mount point / srv / new_root, where we will return the original root directory. All that is needed is to rewrite all the most needed files and directories, with the exception of the / usr folder, which we will read in read-only mode. Copies of files in ramfs will be available for reading and writing.
nano $root/etc/initcpio/hooks/live cat $ root / etc / initcpio / hooks / live
#!/usr/bin/bash run_latehook() { local source options fstype local target="/" local fstab=/new_root/etc/fstab local place=/new_root/srv/new_root local filter=${place}/etc/default/live_filter if source=$(findmnt -snero source --tab-file=$fstab -T $target); then options=$(findmnt -snero options --tab-file=$fstab -T $target) fstype=$(findmnt -snero fstype --tab-file=$fstab -T $target) umount /new_root mount -t ramfs none /new_root -o rw,defaults [ ! -d "$place" ] && mkdir -p $place mount ${fstype:+-t ${fstype}} ${options:+-o ${options}} $source $place mount -o remount,ro${options:+-,${options}} $source $place rsync -aAX ${place}/* /new_root --filter="merge $filter" ! findmnt -snero source --tab-file=$fstab -T /usr && bind_usr $place # "/" , # fstab cat ${place}/etc/fstab | grep -v $source > $fstab fi } bind_usr(){ local place=$1 mount --bind ${place}/usr /new_root/usr mount -o remount,ro,bind ${place}/usr /new_root/usr } We refer to the file / etc / fstab twice: the first time we get information on the mount parameters of the root directory, and the second time we check if there is any information on / usr in fstab. For late mounting of / usr in Archlinux there is a special usr handler, which we will not interfere with doing our work. If / usr is mounted in some special way, then our handler skips it.
The text mentions the / etc / default / live_filter file with filtering rules intended for rsync, we need to remember to prepare it. We do this automatically from the handler installer:
nano $root/etc/initcpio/install/live #!/usr/bin/bash build() { make_filter > /etc/default/live_filter add_binary "/usr/bin/rsync" "/bin/rsync" add_binary findmnt add_runscript } make_filter() { cat <<EOF + /etc/* + /home/* + /home/*/.config - /home/*/*/ + /var/* - /var/cache/*/* - /var/lib/pacman/*/* - /var/lib/systemd/*/* + /var/log/*/ - /var/log/* - /var/tmp/* - /*/* EOF } Rsync "does not see" more than one directory. Files and folders in the directory are checked by each rule in order to the first match ("+" - the object is copied, "-" - the object is not copied). If there is no match, the file is copied, and the directory is created empty. Next, rsync enters the "surviving" directory and again applies the rules to its contents. This is repeated until there is absolutely nothing left.
In our case, the root directory does not fall under any rule, so its structure is completely transferred (all files are copied and empty directories are created). The directories / boot, / dev, / lost + found, / mnt, / opt, / proc, / root, / run, / srv, / sys, / tmp fall under the effect of the last rule "- / * / *", m. e. none of their contents is copied anywhere, but they themselves are created. The / etc directory immediately falls under the "+ / etc / *" rule, and all its contents are copied, but first only within one directory (later on, its entire structure will be transferred in order, because for the nesting levels / etc / * / and then there are no rules). A similar beginning is waiting for the directory / home - folders of all users fall under the rule "+ / home / *" and will be recreated in a copy (empty for now). The following rule "+ /home/*/.config" copies the .config directories that are attached to the home folders of each user, and "- / home / * / * /" excludes all other directories (the rule comes after the "rescue", so for /home/*/.config does not work). About the files themselves from the home directory says nothing, so they are completely transferred. Files from excluded subdirectories are not copied, because these directories were not created. The "- / var / cache / * / *" rule retains the entire directory structure in / var / cache, but their contents are not migrated. The remaining rules are similar.
Remarks
You can change the rsync rules in the external / etc / default / live_filter file as you like, without having to re-create the initramfs. I would be glad to see your version of the rules in the comments.
There are a lot of opportunities for rsync (man rsync - almost 3000 lines). In the comments, suggest some exotic way to use rsync inside initramfs?
Theoretically, rsync can be replaced with some kind of torrent, and build the root file system with it.
There are a lot of opportunities for rsync (man rsync - almost 3000 lines). In the comments, suggest some exotic way to use rsync inside initramfs?
Theoretically, rsync can be replaced with some kind of torrent, and build the root file system with it.
Add a handler to the initramfs:
cat $root/etc/mkinitcpio.conf ... HOOKS="base udev net_nfs4 live" Generate initramfs:
arch-chroot $root mkinitcpio -p habr nfs4 + live
The server and client work in VirtualBox.
Source File System:
The state of the file system on the downloaded client after the execution of the live processor:
During client download, the following data was collected on the server:
Source File System:
cat $root/etc/fstab # <file system> <dir> <type> <options> <dump> <pass> 192.168.1.100:/diskless / nfs4 defaults,noatime 0 0 The state of the file system on the downloaded client after the execution of the live processor:
mount ... none on / type ramfs (rw,relatime) 192.168.1.100://diskless on /srv/new_root type nfs4 (ro,noatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.131,local_lock=none,addr=192.168.1.100) 192.168.1.100://diskless/usr on /usr type nfs4 (ro,noatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.131,local_lock=none,addr=192.168.1.100) ... During client download, the following data was collected on the server:
vnstat -l ... eth0 / traffic statistics rx | tx --------------------------------------+------------------ bytes 7,23 MiB | 252,33 MiB --------------------------------------+------------------ max 5,11 Mbit/s | 235,23 Mbit/s average 1,48 Mbit/s | 51,68 Mbit/s min 0 kbit/s | 1 kbit/s --------------------------------------+------------------ packets 82060 | 199036 --------------------------------------+------------------ max 6550 p/s | 21385 p/s average 2051 p/s | 4975 p/s min 0 p/s | 0 p/s --------------------------------------+------------------ time 40 seconds We overclock the network
Physically, of course, network overclocking is now impossible without changing equipment, but software optimizations are not prohibited. We need to transfer the contents of the associated / usr folder over the network. We are not able to send this data, but we are able to reduce the amount of space they occupy - to archive. , — , .
squashfs fstab, . — ( ) — :
pacman -S squashfs-tools && mksquashfs $root/usr $root/srv/source_usr.sfs -b 4096 -comp xz :
nano $root/etc/fstab # <file system> <dir> <type> <options> <dump> <pass> 192.168.1.100:/diskless / nfs4 defaults,noatime 0 0 /srv/new_root/srv/source_usr.sfs /usr squashfs loop,compress=xz 0 0 initramfs /usr usr, :
cp $root/{usr/lib,etc}/initcpio/install/usr && cp $root/{usr/lib,etc}/initcpio/hooks/usr , :
nano $root/etc/initcpio/hooks/usr mount "/new_root$usr_source" /new_root/usr -o "$mountopts" usr?
«file system» fstab "/new_root/srv/new_root/usr/source_usr.sfs". systemd , . /new_root c initrams, systemd . , .
cat $root/etc/mkinitcpio.conf HOOKS="base udev net_nfs4 live usr" arch-chroot $root mkinitcpio -p habr nfs4+live+squashed /usr
:
live usr:
:
cat $root/etc/fstab # <file system> <dir> <type> <options> <dump> <pass> 192.168.1.100:/diskless / nfs4 defaults,noatime 0 0 /srv/new_root/srv/source_usr.sfs /usr squashfs ro,loop,compress=xz 0 0 live usr:
mount ... none on / type ramfs (rw,relatime) 192.168.1.100://diskless on /srv/new_root type nfs4 (ro,noatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.131,local_lock=none,addr=192.168.1.100) /srv/new_root/srv/source_usr.sfs on /usr type squashfs (ro,relatime) ... :
vnstat -l ... eth0 / traffic statistics rx | tx --------------------------------------+------------------ bytes 5,07 MiB | 205,67 MiB --------------------------------------+------------------ max 4,02 Mbit/s | 191,82 Mbit/s average 1,04 Mbit/s | 42,12 Mbit/s min 0 kbit/s | 1 kbit/s --------------------------------------+------------------ packets 65524 | 159941 --------------------------------------+------------------ max 5954 p/s | 17170 p/s average 1638 p/s | 3998 p/s min 0 p/s | 0 p/s --------------------------------------+------------------ time 40 seconds 20% , . , live ramfs .
/srv/source_usr.sfs ramfs rsync, fstab , , RAM, .
We remove too much
, : « ?». , , squashfs NFS ( ), Network Block Device , . « », « », . btrfs zlib, squashfs .
initramfs NBD- AUR mkinitcpio-nbd ( ):
curl -o mkinitcpio-nbd.tar.gz https://aur.archlinux.org/packages/mk/mkinitcpio-nbd/mkinitcpio-nbd.tar.gz tar -xvvzf mkinitcpio-nbd.tar.gz cd mkinitcpio-nbd makepkg -s sudo pacman --root $root --dbpath $root/var/lib/pacman -U mkinitcpio-nbd-0.4.2-1-any.pkg.tar.xz $root/boot/grub/grub.cfg :
cat $root/boot/grub/grub.cfg menuentry "NBD" { load_video set gfxpayload=keep insmod gzio echo " ..." linux vmlinuz-linux \ add_efi_memmap \ ip="$net_default_ip":"$net_default_server":192.168.1.1:255.255.255.0::eth0:none \ nbd_host="$net_default_server" nbd_name=habrahabr root=/dev/nbd0 echo " ..." initrd initramfs-linux.img } , :
nbd_host="$net_default_server" nbd_name=habrahabr root=/dev/nbd0 NBD /dev/nbd0, :
nano $root/etc/fstab # <file system> <dir> <type> <options> <dump> <pass> /dev/nbd0 / squashfs ro,loop,compress=xz 0 0 NBD ( ). NBD , , «» . NBD , , . NBD IP netcat , , IP :
cp $root/{usr/lib,etc}/initcpio/install/nbd cp $root/{usr/lib,etc}/initcpio/hooks/nbd . :
nano $root/etc/initcpio/hooks/nbd modprobe nbd # msg "closing old connections..." echo ${ip} | nc ${nbd_host} 45678 local ready=$(nc -l -p 45678) [ "$ready" -ne 1 ] && reboot msg "connecting..." # initramfs - net_nfs4, nbd:
nano $root/etc/mkinitcpio.conf MODULES="loop squashfs" HOOKS="base udev net_nfs4 keyboard nbd live" initramfs:
arch-chroot $root mkinitcpio -p habr $root/srv/source_usr.sfs $root — /usr , /usr:
mksquashfs $root/* /srv/new_root.sfs -b 4096 -comp xz :
pacman -S nbd NBD :
mv /etc/nbd-server/{config,config.old} && nano /etc/nbd-server/config [generic] user = nbd group = nbd [habrahabr] exportname = /srv/new_root.sfs timeout = 30 readonly = true multifile = false copyonwrite = false . habrahabr, , , « », copyonwrite . Copyonwrite , , . . . NBD , man' .
, :
nano /etc/default/close_passive_NBD_connections.sh #!/bin/sh # PID _kill(){ local PID for PID in $* do kill $PID done } main(){ local rIP PIDs # ip grub.cfg rIP=$(netcat -l -p 45678 | cut -d: -f1) # IP PID PIDs=$(netstat -np | grep $rIP | awk '/^tcp.*nbd-server/{split($NF,a,"/");print a[1]}') _kill $PIDs && echo "1" | netcat -z $rIP 45678 } # while [ 0 ] do main done :
chmod +x /etc/default/close_passive_NBD_connections.sh , netcat netstat:
pacman -S gnu-netcat net-tools NBD:
mkdir -p /etc/systemd/system/nbd.service.d && nano /etc/systemd/system/nbd.service.d/close_passive.conf [Service] Type=oneshot ExecStart=/etc/default/close_passive_NBD_connections.sh , , .
nbd + squashed live
:
live usr:
:
cat $root/etc/fstab # <file system> <dir> <type> <options> <dump> <pass> /dev/nbd0 / squashfs ro,loop,compress=xz 0 0 live usr:
mount ... none on / type ramfs (rw,relatime) /dev/nbd0 on /srv/new_root type squashfs (ro,relatime) /dev/nbd0 on /usr type squashfs (ro,relatime) ... :
vnstat -l ... eth0 / traffic statistics rx | tx --------------------------------------+------------------ bytes 1,97 MiB | 198,92 MiB --------------------------------------+------------------ max 2,81 Mbit/s | 138,60 Mbit/s average 575,63 kbit/s | 58,20 Mbit/s min 2 kbit/s | 1 kbit/s --------------------------------------+------------------ packets 32473 | 100874 --------------------------------------+------------------ max 5991 p/s | 7576 p/s average 1159 p/s | 3602 p/s min 4 p/s | 1 p/s --------------------------------------+------------------ time 28 seconds 3% ( ). , NFS 10 , NBD .
. — TFTP . , iPXE , kvaps .
username.
, :
nano ~/myscript.ipxe #!ipxe ifopen net0 set server_ip 192.168.1.100 set http_path http://${server_ip} set kern_name vmlinuz-linux kernel ${http_path}/${kern_name} || read void initrd ${http_path}/initramfs-linux.img || read void imgargs ${kern_name} add_efi_memmap ip=${net0/ip}:${server_ip}:${net0/gateway}:${net0/netmask}::eth0:none nbd_host=${server_ip} nbd_name=habrahabr root=/dev/nbd0 || read void boot || read void vmlinuz-linux initramfs HTTP. :
sudo pacman -S git && git clone git://git.ipxe.org/ipxe.git cd ipxe/src/ make bin/undionly.kpxe EMBED=/home/username/myscript.ipxe root :
cp {/home/username/ipxe/src/bin,$root/boot}/undionly.kpxe DHCP , :
nano /etc/dhcpd.conf #if option architecture = 7 { # filename "/grub/x86_64-efi/core.efi"; # } else { # filename "/grub/i386-pc/core.0"; #} filename "/undionly.kpxe"; systemctl restart dhcpd4 HTTP :
pacman -S apache :
mount --bind /srv/nfs/diskless/boot/ /srv/http/ « »:
mount -o remount,ro,bind /srv/nfs/diskless/boot/ /srv/http/ :
systemctl start httpd , :
vnstat -l ... rx | tx --------------------------------------+------------------ bytes 1,50 MiB | 206,73 MiB --------------------------------------+------------------ max 2,96 Mbit/s | 191,95 Mbit/s average 684,08 kbit/s | 94,08 Mbit/s min 5 kbit/s | 1 kbit/s --------------------------------------+------------------ packets 22762 | 90737 --------------------------------------+------------------ max 5735 p/s | 9871 p/s average 1264 p/s | 5040 p/s min 3 p/s | 1 p/s --------------------------------------+------------------ time 18 seconds TFTP HTTP iPXE. , , . , .
We return to the server
live . rsync /srv, . systemd:
nano /etc/fstab LABEL=HABR / ext4 rw,relatime,data=ordered 0 1 /srv/new_root/srv /srv none bind 0 1 /srv/new_root/srv /srv , .
, « », , USB . , . , . , VPN , …
( ), VirtualBox ( > USB ). lsblk, . , HABR /mnt.
rsync:
nano /root/clone_filter + /boot/* + /etc/* + /home/* + /srv/* + /usr/* + /var/* - /*/* :
rsync -aAXv /* /mnt --filter="merge /root/clone_filter" — /dev/sdb
arch-chroot /mnt grub-install --target=i386-pc --force --recheck /dev/sdb /mnt .
PS . . . .
.
Source: https://habr.com/ru/post/254415/
All Articles