svnconfbackup: a script to backup configuration files

A few years ago, I faced the task of backing up configuration files. But not simple, but such that at any moment it would be possible to see what has changed and when. I knew about the existence of csvbackup , but I wanted my own script, with Subversion and without Perl.

For those who are not familiar with Subversion, I will explain. Subversion is one of the version control systems . Such systems allow you to store multiple versions of the same file. It is incredibly convenient. Imagine that the company has two administrators - you and your partner. Monday. You are the first day you left the holiday, and your partner went on holiday abroad on Saturday, without a phone, of course. And suddenly you find that one of the services, for example, a web server, is not working properly. But before you go on vacation, everything worked! What changed? Having a backup copy of configuration files in the version control system you can not only roll back to the working version of the configs, but also track all changes made in the configuration files during the period of your absence, in fact, identify the problem.

I wanted to make the script simple, cross-platform, if I may say so, and with minimal dependencies. Sh was chosen as the interpreter. goes in the base FreeBSD system. To copy files, pax is used, because pax, when copying a file, recreates the full path to it, that is, after the command

pax -rw /etc/ssh/ssh_config /var/tmp

we get the file / var / tmp / etc / ssh / ssh_config. pax also runs on the base FreeBSD system. Later, when I moved to Debian, it was an unpleasant surprise for me that pax should be installed separately.
We will set the list of files for backup in two ways - as a result of the work of the find command and as a simple list of files separated by spaces, which is convenient for single files.
')
The variables will be moved to the settings file /usr/local/etc/svnconfbackup.conf:

#,
TMPDIR= "/var/tmp"
#
REPDIR= "conf"
#
DESTDIR=$REPDIR.`date +%s`
#
LIST= "/etc /usr/local/etc /var/named"
EXCLUDE_LIST= "-not -path /etc/'*'shadow'*' \
-and -not -path /var/named/internal/slave/'*' \
-and -not -name '*'.pem \
-and -not -name '*'.key \
-and -not -name '*'.crt \
-and -not -path /etc/ssh_host_rsa_key \
-and -not -path /etc/ssh_host_dsa_key"
FILES= "/home/pgsql/data/*.conf"
#
SVN=`which svn`
SVNADMIN=`which svnadmin`
#
SVNROOT= "/usr/local/svnconfbackuproot"

* This source code was highlighted with Source Code Highlighter .

The LIST variable contains a list of directories in which find will look for files, and the FILES variable contains an additional list of files to back up. This settings file can be considered as a full sh-script. Let me remind you that inside single quotation marks, or strict quotation marks, any special character (with the exception of single quotation marks) is interpreted as a simple character. That is, in the EXCLUDE_LIST variable, the asterisks will remain asterisks. And in the FILES variable, we end up with a list of files in the /home/pgsql/data/ directory ending in .conf , since the "*" in this case is the file name pattern.
And the content enclosed between the return quotes `` will be replaced with the result of the command execution, i.e. on her console output. Thus, instead of `date +%s` we get the number of seconds that have elapsed since 00:00:00 on January 1, 1970 ( mktemp was rejected due to the different behavior in Linux and FreeBSD)
Backslash \ overrides the special value of the newline character. This is done just in case.

But back to the script. It will consist of two parts — the initialization of the Subversion repository and the backup procedure. Initialization must be performed manually once, at the very beginning. The backup itself should take place without user intervention.
Command syntax:

svnbackupconfig init|update

Those. svnbackupconfig init will initiate the repository initialization, and svnbackupconfig update will start the backup procedure, in other words, update the data in the repository.

First you need to load the settings and initialize the local variables.

#!/bin/sh
. /usr/local/etc/svnconfbackup.conf

# URL
SVNROOTsvn= "file://$SVNROOT"


Next, we will process the command parameter passed in from the command line.

case "$1" in
init)
#
;;
update)
#
esac

* This source code was highlighted with Source Code Highlighter .

Initialization procedure

cd $TMPDIR
#
mkdir $DESTDIR
chmod 700 $DESTDIR
#
mkdir $DESTDIR/branch
mkdir $DESTDIR/tag
mkdir $DESTDIR/trunk
cd $DESTDIR
#
#
if [ ! -d $SVNROOT ]; then
$SVNADMIN create $SVNROOT
fi
#
$SVN import $SVNROOTsvn/$REPDIR -q -m "Initial repository layout"
#
cd ../
rm -rf $DESTDIR

* This source code was highlighted with Source Code Highlighter .

The backup procedure is not much longer, but I want to dwell on the algorithm in more detail. To make the script simple and straightforward, I solved the Hindu problem head-on.

1. Download the previous backup from the repository to a temporary directory.
2. Copy there all the files specified in the backup lists. Let Subversion know what has changed.
3. Add new files to the repository that are missing in the previous backup.
4. Check every file from backup. If it is not in the system, remove it from the repository.
5. Save result to repository.

This is, without doubt, a resource-intensive algorithm. However, on a typical system, its implementation will take only a few minutes.
New files (p.3) can be found easily - in the output of svn status lines start with the character "?":

$ svn status
? some.file

To get the file names we will use the construction

svn status|grep ^?|awk '{print $2}'

Does this mean that the output of the svn status sent to the grep ? which will return strings with the first character "?". This output, in turn, is passed to the awk '{print $2}' command, which will return the second parameter, i.e. file name.

Actually, the backup procedure.

cd $TMPDIR
#
mkdir $DESTDIR
chmod 700 $DESTDIR
#
$SVN checkout -q $SVNROOTsvn/$REPDIR/trunk $DESTDIR
cd $DESTDIR
# ,
list=`eval find $LIST $EXCLUDE_LIST -and -type f|sort`
#
for p in ${list} $FILES ; do
pax -rw $p . > /dev/ null 2>&1
done;
# ,
list=`eval $SVN status|grep ^?|awk '{print $2}' `
#
for p in ${list} ; do
$SVN add -q $p
done;
#
# .svn
list=`eval find . -name .svn -prune -or -print`
#
#
for p in ${list} ; do
if [ ! -e /$p ]; then
$SVN delete -q $p --force
fi
done;
#
$SVN diff
#
$SVN commit -q -m "svnbackup automatic update"
#
cd ../
rm -rf $DESTDIR

* This source code was highlighted with Source Code Highlighter .

Archive link svnconfbackup.googlecode.com/files/svnconfbackup.tar.gz

Using this script is also easy. After unpacking the archive, place the script file in /usr/local/bin , and the configuration file in /usr/local/etc Adjust settings according to your OS. Now you need to initialize the repository. Run the command

svnbackupconfig init

Now, if you wish, you can make the first backup manually.

svnbackupconfig update

The process will take several minutes and will end with the text of all configuration files on the console, since the repository is still empty. Upon further use, the output will contain only changes between versions.
The autorun configuration will be slightly different for Linux and FreeBSD.

For Linux:

echo -e "/usr/local/bin/svnconfbackup update\n" > /etc/cron.daily/svnconfbackup<br>
chmod 755 /etc/cron.daily/svnconfbackup


For FreeBSD:

echo "/usr/local/bin/svnconfbackup update\n" > /usr/local/etc/periodic/daily/999.svnbackup<br>
chmod 755 /usr/local/etc/periodic/daily/999.svnbackup


Now - how to use backup. By and large, below - a very brief description of the main svn commands.
For definiteness, we will use the default settings. To get the last backup run the command

svn checkout file:///usr/local/svnconfbackup/conf/trunk conf

A conf directory will be created in the current directory, in which a copy of the last backup will be placed. When restoring data, keep in mind that in each directory there are service directories named .svn.

For the following operations, go to the conf directory, which is also called the working directory svn.
We need only two commands - update and diff. Note that the update command updates the working directory svn, not the repository. Thus, to roll back to the previous version, you need to run the command

svn update --revision PREV

In general, as an argument for the --revision key or its short form -r, you can specify both the revision number and date. For example:

svn update -r "{2010-03-31}"

In this case, the backup will be taken on March 31, 2010 00:00:00. Please note that if you have an automatic backup later than this time, for example, at 4 am, then there will be a backup for March 30 in the working directory.

With the diff command you can see the changes between any two versions of files. At the same time, you can explicitly indicate the versions of which files should be compared. For example, the command

svn diff -r "{2010-01-01}:HEAD"

will show what has changed since last year. By the way, the -r option also works with the checkout command! More information about syntax and keys can be found in the help:

svn help < >

Just in case, leave a link to the site of an excellent book on Subversion .

Good luck! And remember - administrators are divided into two types - some data has already been lost, while others will lose!

UPD Transferred to the thematic blog