Making Your Time Machine for Linux

After intensive use of the time machine on poppies and a couple of situations when it really came in handy (there were options when I had to install a system from backups and a warrant when I had to roll back after problems), a thought arose that there is actually no such convenient system on Linux. After researching the question and interviewing familiar Linux users, it turned out that:
1. You can make such a system in just a few minutes on your knees.
2. strange, but somehow no one actually knows that it can be raised so quickly.
3. our Linux time machine will be with mahjong and geisha.


There was just an unnecessary appletv of the first generation from which it was decided to make a small server on hardeded linux to collect logs and all sorts of auxiliary purposes. After installing hardened gentoo, the question arose of how to back it up so that the entire disk and immediately - so that you can roll out the backup to a new disk or a completely new appletv (if you find such a course of course), or selectively restore remote or lost files just referring to a specific date.

How does the time machine work? Quite simply, the first image of the system is simply copied as files with all the attributes in Backups.backupdb / hostname / YYYY-MM-dd-hhmmss, + a Latest link is made to indicate the last image. And already at the second cast, the following happens: the file dates are compared and if the file has not changed, instead of a new copy of the file, a hardlink is made to the file from the previous cast. Then, if there are no file changes, then the entire new cast will completely refer to the previous one. Nuggets will differ only in new / modified / deleted files. As a result, you can simply remove any cast (directory type YYYY-MM-dd-hhmmss) and nothing breaks. The closest association is a smart pointer in c ++ for example: when the last link to the file disappears (from all the snapshots) it will be deleted from the disks.
')
A very convenient system from the point of view of viewing previous files, their restoration, deleting old backups, etc. Until the restoration of the entire system from the cast.

After researching all the backup systems I know for Linux, they were discarded as a bit of something not really necessary. But there was the simplest solution with rsync in one line.

rsync has a wonderful option --link-dest which makes all the above described logic compared to previous casts and hardlink creation. The -x option excludes all mounted file systems like / proc / sys / dev, and so on. --delete will delete files that will be lost.

The whole time machine will be one script like:

#! / bin / sh
date = `date" +% Y-% m-% d-% H% M% S "`
Src = /
DST = / mnt / backup-hdd / Backups.backupdb / atv

rsync -ax \
--delete \
--link-dest = .. / Latest \
$ Src $ dst / processing- $ date \
&& cd $ DST \
&& mv Processing- $ date $ date \
&& rm -f Latest \
&& ln -s $ date Latest

Copy the script in /etc/cron.hourly and voila we have backups as in the time machine.
If our system crashes, we can boot from the flash drive, format the partition, and run the same rsync in the opposite direction, then reboot or chroot and the system is working again.

We’ll also put in /etc/cron.daily a simple script that will delete hourly impressions of the previous days, leaving only 24 for the last day, then delete daily impressions for previous months, etc. - like in apple time machine, just choose your sequence.

We want to upload backups over the network to the backup server? Also please. We do auto authorization with ssh keys and slightly change the script:

#! / bin / sh
date = `date" +% Y-% m-% d-% H% M% S "`
Src = /
DST = / mnt / backup-server-hdd / Backups.backupdb / atv

rsync -axz \
--delete \
--link-dest = .. / Latest \
$ Src backup-user@company.com: $ DST / Processing- $ date \
&& ssh backup-user@company.com \
"Cd $ DST \
&& mv Processing- $ date $ date \
&& rm -f Latest »
&& ln -s $ date Latest "

We are testing, the 3GB system on the appletv was compared to the last image on the server in five minutes - the remote server is not on the local network, the connection was about 1.5MB / s. We added "-z" for compression. Not bad, but if it will grow you can do not hourly but with a large range. It is necessary to indicate that rsync over the network does not seem to transfer the “owner: group” files - since it will use the backup-user @ ... user - you can simply dump all file attributes into a text file and put it into backup.

Now what is not Mac users
1. Flexible configuration with cron.
2. The option --exclude = PATTERN / - exclude-from = FILE allows you to specify much more flexible rules for exclusion than simply specifying folders. For example, I am very unhappy that the time machine backs up all * .o files on my poppy - the project compilation is easily scattered over a couple of gigabytes of obj files, and I don’t want to exclude project folders from the backup.
3. It is easy to add extra logic to create database locks if needed
4. It is also easy to make, in addition to the impression, still corresponding to the impression of only those files that have changed - it is easy to monitor later which files were changed. Since hardlink is not a lot of space.
5. Now add what you want, you still have Linux in your hands;)

Update: Probably add a list of backup system requirements that I had in my head when I looked at the available packages / systems:
1. Ease of use (rsync: just one command)
2. Accessibility (perhaps find rsync on any livecd)
3. Dependencies (if you crashed a disk, did you boot from livecd, then now you need to put packages with their dependencies somewhere?)
4. The absence of a config file or their minimum (or is it now necessary to first search for a config in the backup?)