Expand linux working image with minimal interactivity
This publication may be of interest to those lazy system administrators who use any of the following:
It will be about how to install a ready system with a set of software, customized network services, predefined configs, etc. with minimal effort from the user or the staff enikeyschik. The article focuses primarily on enthusiasts and beginners linuksoidov, but the techniques and scripts may well be used in the combat deployment of a group of dozens of machines daily.
For a start we will decide on the distribution kit. This, of course, depends primarily on the tasks. I chose xubuntu 14.04 for three simple reasons:
1. ubuntu - deb based system with the broadest support and a huge number of forums;
2. xubuntu - a lightweight version without ruches and ponts, which does not slow down on machines from 512MB of RAM (there are such dinosaurs in my park);
3. April 14 is the LTS (Long Term Support) version with support until 2019.
There are, of course, faster and smaller options, but for a number of reasons, the choice was made and does not cause regrets over the course of a year.
')
Looking ahead, I’ll note: to repeat this, you will need a configured vpn server or a “config-monitoring server” in the local network, a pair of virtual or real machines, a couple of hours of free time and attention to detail.
So, first, the preparation of the image. All stages are important, but the first is the foundation. Install the system from the cd / dvd / usb / network and select the disk partitioning. It is important to understand that our image will not support dynamic resizing prior to deployment. And after, most likely such a resize is not required. At the root it makes sense to give 6-10 GB.
Most of the volume is justified in the case when it is known that a) the disks on the client machines will not be less than this volume and b) many different necessary packages will actually be installed.
Mark / home and finally swap partitions. In the aggregate, all three sections of the author do not crawl out for 30GB (in order to comfortably fit on ssd 30 Gb, this is the minimum disk configuration for my clients). In case the minimum disk size of the receiver is known in advance, it makes sense to fit partitions for it. I recommend to mark the swap right after the disk root, / home immediately after the swap, so that in case of a resize, do not “drag them across the disk in the console”, but make it elegantly with one command. Set the locale, username, password, hostname and go to the new Linux. To continue, you will need x11vnc, ssh, rsync, openvpn, mc packages (I love mcedit, it's my darling, like Norton from the nineties).
I added skype, icedove (thunderbird), iceweasel (firefox), chromium-browser, rdesktop calculator, resource monitor, viber, libre office, wine and a couple of windows-based applications to my base to my taste (of course, I’m burning in hell , but there are no analogues and is not expected). We adjust slowly and slowly the whole thing. We check that everything starts and works.
Configure vpn. I have a distributed network, two or three computers per geolocation, more than 50 locations in Russia, you are not hitting. We create an ssh-key to the server and on the server to the client (who needs to enter these passwords, details, for example, here ). On the server, we create a directory of unconditional synchronization (for me these are working scripts, openvpn keys, client cron, service scripts in python, C ++ binaries). In the working example, this is / var / sync / in, / var / sync / in / cron, / var / sync / out. In the out there will be logs and any information we need from clients: screenshots, process reports, etc. In there will be synchronization scripts, cron jobs, heartbeat scripts for monitoring, numerous system service configs (same cups or openvpn).
Preparing this whole thing for replicationdoes not take much longer than installing the system to a single user, but careful study of the details can significantly reduce the time that the admin will spend on scripts “dopilivayuschie” all images in battle in the future. Configure openvpn on the client and make sure that the user key [and] work [by it] t. Not be superfluous and configure grub2.
We will not intentionally create files from Hint2, by their absence it will be clear when the "new ones" connect to our vpn.
So, everything is rustling, the office is running, the calculator is running, for rdp with the application server there is already a tab on the desktop.
It's time to make an image.
Allow myself to be distracted. I tried a lot of ways to make an image with a finished system. This article does not claim to be the best way, I can only say that the method is the best for me and I will comment very briefly on other methods and their disadvantages.
Clonezilla is a great thing, but interactively enters into a stupor any enikeyschik for three hundred miles from the admin.
dd if = / dev / sda of = img.img is a magic thing, but together with the data we get garbage from an empty part of the disk, the volume and speed of the method are terrifying.
Norton ghost - does not support ext4.
Partition copy from gparted is a cool thing, but, alas, also not for beginners.
I have been asking myself for a long time how I see the perfect deployment. Here we come to the topic of the article, from which the order has already been distracted.
It was necessary to invent a small bicycle, or rather, to write an unfolding script. For the above tasks, the PartedMagic distribution was chosen. Unetbootin + partedmagic or any live-cd (usb) with linux and tar + b [g] zip on board will be required to wrap the image. We assume that the image we downloaded PartedMagic “run from ram” and the flash drive is mounted in / media / sdb1 (FAT32).
After booting we mount our disk (most likely / dev / sda1) and click all our system directly from it, and we don’t forget about the / home partition:
Then we will create an archive with all the contents of our disk:
Same for the / home section:
Then dump the partition table into a file on our superflashka:
Thus, on a flash drive, we will have archives and a file with a table of sections of the reference image.
Well, for dessert. Below is a script with a talking name that was assembled in parts from different parts of the Internet literally along the line.
Comments of the author directly in the text of the script. Nothing supernatural, just like in a computer science class, but debugged with bloody tears, even once killed a home system with a flash drive with this script. I do not recommend leaving it as it is. This is his unconditional non-interactive version.
cat /pmagic/pmodules/scripts/kill_your_hdd_data.sh (path for autoloading script in PartedMagic)
This method is used in Urban Software to fill information touch kiosks with web applications running under linux. Most often, these are keyboardless machines that are deployed dozens far from thecivilization of our technical support, so it is important not to create problems initially when monitoring them and putting them to work.
PS The article turned out great. The second part will be written about how to “meet” newly deployed computers, manage a remote cron, synchronize scripts and receive monitoring heartbits, if, of course, the first one ever comes out of the sandbox.
- Linux - based system on client machines;
- Thin / fat clients of the same type on different hardware;
- Network storage of client data;
- The preset regulated software;
- Openvpn / rdp to application servers.
- Plush (soft and fluffy) monitoring of linux clients like Nagios.
It will be about how to install a ready system with a set of software, customized network services, predefined configs, etc. with minimal effort from the user or the staff enikeyschik. The article focuses primarily on enthusiasts and beginners linuksoidov, but the techniques and scripts may well be used in the combat deployment of a group of dozens of machines daily.
For a start we will decide on the distribution kit. This, of course, depends primarily on the tasks. I chose xubuntu 14.04 for three simple reasons:
1. ubuntu - deb based system with the broadest support and a huge number of forums;
2. xubuntu - a lightweight version without ruches and ponts, which does not slow down on machines from 512MB of RAM (there are such dinosaurs in my park);
3. April 14 is the LTS (Long Term Support) version with support until 2019.
There are, of course, faster and smaller options, but for a number of reasons, the choice was made and does not cause regrets over the course of a year.
')
Looking ahead, I’ll note: to repeat this, you will need a configured vpn server or a “config-monitoring server” in the local network, a pair of virtual or real machines, a couple of hours of free time and attention to detail.
So, first, the preparation of the image. All stages are important, but the first is the foundation. Install the system from the cd / dvd / usb / network and select the disk partitioning. It is important to understand that our image will not support dynamic resizing prior to deployment. And after, most likely such a resize is not required. At the root it makes sense to give 6-10 GB.
Most of the volume is justified in the case when it is known that a) the disks on the client machines will not be less than this volume and b) many different necessary packages will actually be installed.
Mark / home and finally swap partitions. In the aggregate, all three sections of the author do not crawl out for 30GB (in order to comfortably fit on ssd 30 Gb, this is the minimum disk configuration for my clients). In case the minimum disk size of the receiver is known in advance, it makes sense to fit partitions for it. I recommend to mark the swap right after the disk root, / home immediately after the swap, so that in case of a resize, do not “drag them across the disk in the console”, but make it elegantly with one command. Set the locale, username, password, hostname and go to the new Linux. To continue, you will need x11vnc, ssh, rsync, openvpn, mc packages (I love mcedit, it's my darling, like Norton from the nineties).
sudo apt-get install x11vnc ssh rsync openvpn mc ntp I added skype, icedove (thunderbird), iceweasel (firefox), chromium-browser, rdesktop calculator, resource monitor, viber, libre office, wine and a couple of windows-based applications to my base to my taste (of course, I’m burning in hell , but there are no analogues and is not expected). We adjust slowly and slowly the whole thing. We check that everything starts and works.
Configure vpn. I have a distributed network, two or three computers per geolocation, more than 50 locations in Russia, you are not hitting. We create an ssh-key to the server and on the server to the client (who needs to enter these passwords, details, for example, here ). On the server, we create a directory of unconditional synchronization (for me these are working scripts, openvpn keys, client cron, service scripts in python, C ++ binaries). In the working example, this is / var / sync / in, / var / sync / in / cron, / var / sync / out. In the out there will be logs and any information we need from clients: screenshots, process reports, etc. In there will be synchronization scripts, cron jobs, heartbeat scripts for monitoring, numerous system service configs (same cups or openvpn).
Preparing this whole thing for replication
- Hint1: It’s good when the hostname, ip in openvpn and user certificate contain their unique id, for example user10.key, xyz10 and vladivostok_10 as the hostname;
- Hint2: Well, when copies of these files are at hand, say / var / sync / hostname, / var / sync / id, / var / sync / location;
- Hint3: Reserve 5-10 addresses for unregistered users will also be useful.
We will not intentionally create files from Hint2, by their absence it will be clear when the "new ones" connect to our vpn.
So, everything is rustling, the office is running, the calculator is running, for rdp with the application server there is already a tab on the desktop.
It's time to make an image.
Allow myself to be distracted. I tried a lot of ways to make an image with a finished system. This article does not claim to be the best way, I can only say that the method is the best for me and I will comment very briefly on other methods and their disadvantages.
Clonezilla is a great thing, but interactively enters into a stupor any enikeyschik for three hundred miles from the admin.
dd if = / dev / sda of = img.img is a magic thing, but together with the data we get garbage from an empty part of the disk, the volume and speed of the method are terrifying.
Norton ghost - does not support ext4.
Partition copy from gparted is a cool thing, but, alas, also not for beginners.
I have been asking myself for a long time how I see the perfect deployment. Here we come to the topic of the article, from which the order has already been distracted.
- The image should fit on a small (in my case 2GB) flash drive;
- The image must be placed by pressing one or two buttons;
- After deployment, the image itself must be synchronized and “report” to the admin that the new machine is “in service”;
- Registering a new car should not take a lot of time;
- (Important!) The image will most likely be written to a USB flash drive from under windows by a non-professional according to the instructions;
- (ArchiveImportant!) The image is likely to develop versionally, and it needs to be periodically completed and quickly distributed.
It was necessary to invent a small bicycle, or rather, to write an unfolding script. For the above tasks, the PartedMagic distribution was chosen. Unetbootin + partedmagic or any live-cd (usb) with linux and tar + b [g] zip on board will be required to wrap the image. We assume that the image we downloaded PartedMagic “run from ram” and the flash drive is mounted in / media / sdb1 (FAT32).
A couple of reminders about fat / ntfs on flash drives
- Achtung1 I remind you that the file section in fat32 should not be more than 4GB, watch the size of the image or select the ext4 partition.
- Akhtung2 NTFS partition of the flash drive under linux though works, but takes a huge amount of CPU time.
After booting we mount our disk (most likely / dev / sda1) and click all our system directly from it, and we don’t forget about the / home partition:
mount /dev/sda1 /media/sda1 Then we will create an archive with all the contents of our disk:
tar -czvpf /media/sdb1/image.tgz /media/sda1 Same for the / home section:
mount /dev/sda5 /media/sda5 tar -czvpf /media/sdb1/home.tgz /media/sda5 Then dump the partition table into a file on our superflashka:
sfdisk /dev/sda1 -d >parts.txt Thus, on a flash drive, we will have archives and a file with a table of sections of the reference image.
Well, for dessert. Below is a script with a talking name that was assembled in parts from different parts of the Internet literally along the line.
Comments of the author directly in the text of the script. Nothing supernatural, just like in a computer science class, but debugged with bloody tears, even once killed a home system with a flash drive with this script. I do not recommend leaving it as it is. This is his unconditional non-interactive version.
cat /pmagic/pmodules/scripts/kill_your_hdd_data.sh (path for autoloading script in PartedMagic)
!#/bin/bash # , , , " " . # // PartedMagic . # / , .. home # , (, ) # , #2014 (c) Urban Software LLC under GPLv3 urbansoftware.ru, usrbb.ru dd if=/dev/zero of=/dev/sda bs=512 count=1 # sleep 1s sfdisk /dev/sda < /media/sdb1/parts.txt # sleep 1s mkfs.ext4 -L "" /dev/sda1 -F # sleep 1s mkdir /media/sda1 fsck.ext4 /dev/sda1 # ( home) sleep 5s mount /dev/sda1 /media/sda1 #mount /dev/sdb1 /media/sdb1 sleep 1s mkdir /media/sda1/tmp tar -xzvpf /media/sdb1/image.tgz -C / # sleep 5s echo "This script damn work!" # , mount --bind /dev /media/sda1/dev mount --bind /sys /media/sda1/sys mount --bind /proc /media/sda1/proc chroot /media/sda1/ /bin/bash -c 'grub-install /dev/sda' # chroot /media/sda1/ /bin/bash -c 'update-grub2' sleep 5s umount /dev/sda1 umount /dev/sdb1 echo 'done, remove media, system will reboot in 30s' sleep 30s reboot exit 0 Instead of an epilogue:
This method is used in Urban Software to fill information touch kiosks with web applications running under linux. Most often, these are keyboardless machines that are deployed dozens far from the
PS The article turned out great. The second part will be written about how to “meet” newly deployed computers, manage a remote cron, synchronize scripts and receive monitoring heartbits, if, of course, the first one ever comes out of the sandbox.
Source: https://habr.com/ru/post/253193/
All Articles