bash: Backup without extra software
Backup important information - every system administrator is faced with this task. The task would seem trivial and will not cause interest to many readers. But, for example, such an article would help me very strongly at a certain moment, so I think that this article should be.
Task: Backup data to a local directory and to a separate server, using a minimum of third-party software, logging and notifying the administrator in jabber in case of failures. All the basic functions of most software for automatic backup, but without installing it, and therefore without its bugs (which, in fact, led to a similar idea).
And now to the point.
First, create and open a script.
Now add the line in the script
Let's declare some variables.
TN - TASKNAME is the name of the task. It is used to output to the log and determine the file name.
Since there are several tasks (monthly, weekly, daily) and writing a script for each case was lazy, I created a universal one, in which you just need to uncomment the necessary lines. The name of the tasks should be written without spaces, preferably in the Latin alphabet, if you do not want problems with the encoding and incorrect command parameters.
OF - Output File - the name of the output file. It turns out from variable TN, that is a task name.
We declare a variable with the path to the log file, and then all error messages and the rest will be output to the log.
Let's make an entry in the log about the beginning of the backup (date, time, task name)
There is a problem that if you specify in the command parameters (eg tar) the names of directories with spaces, the script is triggered with an error. The solution was found on the Internet - the linux operating system uses the space as the standard command parameter separator. Override the standard delimiter (stored in the $ IFS variable) other than a space, for example, \ n with a newline character.
Remember the old value of the standard delimiter
Replace the standard delimiter with your own.
SRCD - SouRCe Directory - backup data directory
Now you can list several directories, the delimiter will be the line breaks as we ourselves indicated in the line above
TGTD - TarGeT Directory - directory in which backups will be added
Naturally, we understand that keeping important backups at the source only is at least frivolous. Therefore, we will leave a copy on a remote resource, which we will separately mount using mount and fstab. Immediately I will explain why I used mount and fstab, and not one mount - I mount this directory in my other scripts, and as one of my familiar programmers said, a good programmer will not write the same code twice (somehow, literally I do not remember, but I hope the meaning was reported).
The archiving process itself in the option "Create a new archive"
and in the option "Update files in the old archive"
In the second case, it is better to use the specific file name instead of $ OF because, for example, the weekly archive is updated daily for me, and their $ TN (task names) do not match, respectively $ OF.
')
In the variable "?" handles the status of the execution of the last command. Save it to use later.
We return the standard separator to the original value.
Now we add a condition - if the process of packing into the tar archive has ended with an error, send a message to the admin, delete the unsuccessful backup file. Otherwise, we continue further - we mount the network ball and throw a copy of the archive into it. After each operation, we check the result of execution, write logs, and either continue or notify the admin and interrupt the procedure.
In the process, we are copying the archive from a local hooded to a remote one. Naturally, we check that each operation is successfully completed, and we write everything to the logs.
To send a message to the administrator, I use the XMPP message, because the organization has a Jabber server raised, and I prefer to get a quick failure message than to go into the mail, hammering passwords, poking links, and waiting for the browser to display everything to me. In any case, no one bothers you to use sendmail instead of sendxmpp.
The file / usr / local / etc / XMPP_settings reads as follows:
In the fstab file, a string describing the connection of Windows balls
Now you only have to add the task to cron. This can be done using the / etc / crontab file, but I, because of the GUI habits inherited from Windows, use web interfaces for such cases. The command should be executed with the rights of root, I mean, for example, sudo bash backup_script. By adding a command to cron, you can determine that it will be immediately executed as root.
In the course of the discussions they raised the problem of the growth of logs. I went through the simplest (in my opinion) path: we will only store the last N lines of the log, for example 300. Two lines will be added to the script, in which we will save the last 300 lines of the log to a temporary file, then we will log them
I will give the full text of the script:
Thank you all for your attention!
Task: Backup data to a local directory and to a separate server, using a minimum of third-party software, logging and notifying the administrator in jabber in case of failures. All the basic functions of most software for automatic backup, but without installing it, and therefore without its bugs (which, in fact, led to a similar idea).
And now to the point.
First, create and open a script.
nano backup-script Now add the line in the script
#!/bin/bash Let's declare some variables.
TN - TASKNAME is the name of the task. It is used to output to the log and determine the file name.
Since there are several tasks (monthly, weekly, daily) and writing a script for each case was lazy, I created a universal one, in which you just need to uncomment the necessary lines. The name of the tasks should be written without spaces, preferably in the Latin alphabet, if you do not want problems with the encoding and incorrect command parameters.
TN=docs-monthly<br /> #TN=docs-weekly<br /> #TN=docs-daily OF - Output File - the name of the output file. It turns out from variable TN, that is a task name.
OF=$TN.tar.gz We declare a variable with the path to the log file, and then all error messages and the rest will be output to the log.
LOGFILE=/var/log/backup.log Let's make an entry in the log about the beginning of the backup (date, time, task name)
echo >>$LOGFILE echo "=====================================================" >>$LOGFILE echo "$(date +'%d-%b-%Y %R')" >>$LOGFILE echo " \"$TN\" ..." >>$LOGFILE There is a problem that if you specify in the command parameters (eg tar) the names of directories with spaces, the script is triggered with an error. The solution was found on the Internet - the linux operating system uses the space as the standard command parameter separator. Override the standard delimiter (stored in the $ IFS variable) other than a space, for example, \ n with a newline character.
Remember the old value of the standard delimiter
OLD_IFS=$IFS Replace the standard delimiter with your own.
IFS=$'\n' SRCD - SouRCe Directory - backup data directory
Now you can list several directories, the delimiter will be the line breaks as we ourselves indicated in the line above
SRCD="/mnt/source/folder_1 /mnt/source/folder_2 /mnt/source/folder_N" TGTD - TarGeT Directory - directory in which backups will be added
TGTD="/var/backups/" Naturally, we understand that keeping important backups at the source only is at least frivolous. Therefore, we will leave a copy on a remote resource, which we will separately mount using mount and fstab. Immediately I will explain why I used mount and fstab, and not one mount - I mount this directory in my other scripts, and as one of my familiar programmers said, a good programmer will not write the same code twice (somehow, literally I do not remember, but I hope the meaning was reported).
TGTD2="/mnt/archive/" The archiving process itself in the option "Create a new archive"
tar -czf $TGTD$OF $SRCD &>>$LOGFILE and in the option "Update files in the old archive"
tar -u -f $TGTD$OF $SRCD &>>$LOGFILE In the second case, it is better to use the specific file name instead of $ OF because, for example, the weekly archive is updated daily for me, and their $ TN (task names) do not match, respectively $ OF.
')
In the variable "?" handles the status of the execution of the last command. Save it to use later.
STATUS=$? We return the standard separator to the original value.
IFS=$OLD_IFS Now we add a condition - if the process of packing into the tar archive has ended with an error, send a message to the admin, delete the unsuccessful backup file. Otherwise, we continue further - we mount the network ball and throw a copy of the archive into it. After each operation, we check the result of execution, write logs, and either continue or notify the admin and interrupt the procedure.
if [[ $STATUS != 0 ]]; then rm $TGTD$OF &>>$LOGFILE echo "###########################################" >>$LOGFILE echo "### ! . ###" >>$LOGFILE echo "###########################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE else echo " \"$TGTD$OF\"" >>$LOGFILE echo " $(date +'%R %d-%b-%Y')!" >>$LOGFILE echo " $TGTD_archive" >>$LOGFILE mount $TGTD2 &>>$LOGFILE if [[ $? != 0 ]]; then echo "#############################################################" >>$LOGFILE echo "### ###" >>$LOGFILE echo "#############################################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE exit fi echo " " >>$LOGFILE cp -f $TGTD$OF $TGTD_archive$OF &>>$LOGFILE if [[ $? != 0 ]]; then echo "#############################################################" >>$LOGFILE echo "### ###" >>$LOGFILE echo "#############################################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE else echo " $(date +'%R %d-%b-%Y')!" >>$LOGFILE echo " \"$TGTD_archive$OF\"" >>$LOGFILE fi echo " $TGTD_archive" >>$LOGFILE umount $TGTD2 &>>$LOGFILE echo " !" >>$LOGFILE fi exit In the process, we are copying the archive from a local hooded to a remote one. Naturally, we check that each operation is successfully completed, and we write everything to the logs.
To send a message to the administrator, I use the XMPP message, because the organization has a Jabber server raised, and I prefer to get a quick failure message than to go into the mail, hammering passwords, poking links, and waiting for the browser to display everything to me. In any case, no one bothers you to use sendmail instead of sendxmpp.
The file / usr / local / etc / XMPP_settings reads as follows:
#_@;IP_jabber_:_jabber_ _ login@domen;127.0.0.1:5222 password In the fstab file, a string describing the connection of Windows balls
//192.168.0.250/arhiv /mnt/archive cifs noauto,rw,iocharset=utf8,cp866,file_mod=0666,dir_mod=0777,noexec,_netdev,credentials=/root/.passwd_to_archive_directory 0 0 Now you only have to add the task to cron. This can be done using the / etc / crontab file, but I, because of the GUI habits inherited from Windows, use web interfaces for such cases. The command should be executed with the rights of root, I mean, for example, sudo bash backup_script. By adding a command to cron, you can determine that it will be immediately executed as root.
In the course of the discussions they raised the problem of the growth of logs. I went through the simplest (in my opinion) path: we will only store the last N lines of the log, for example 300. Two lines will be added to the script, in which we will save the last 300 lines of the log to a temporary file, then we will log them
tail -n 300 $LOGFILE >/tmp/unique_fantastic_filename.tmp mv -f /tmp/unique_fantastic_filename.tmp $LOGFILE I will give the full text of the script:
#!/bin/bash TN=docs-monthly #TN=docs-weekly #TN=docs-daily OF=$TN.tar.gz LOGFILE=/var/log/backup.log echo >>$LOGFILE echo "=====================================================" >>$LOGFILE echo "$(date +'%d-%b-%Y %R')" >>$LOGFILE echo " \"$TN\" ..." >>$LOGFILE OLD_IFS=$IFS IFS=$'\n' SRCD="/mnt/source/folder_1 /mnt/source/folder_2 /mnt/source/folder_N" TGTD="/var/backups/" TGTD2="/mnt/archive/" tar -czf $TGTD$OF $SRCD &>>$LOGFILE #tar -u -f $TGTD$OF $SRCD &>>$LOGFILE STATUS=$? IFS=$OLD_IFS if [[ $STATUS != 0 ]]; then rm $TGTD$OF &>>$LOGFILE echo "###########################################" >>$LOGFILE echo "### ! . ###" >>$LOGFILE echo "###########################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE else echo " \"$TGTD$OF\"" >>$LOGFILE echo " $(date +'%R %d-%b-%Y')!" >>$LOGFILE echo " $TGTD_archive" >>$LOGFILE mount $TGTD2 &>>$LOGFILE if [[ $? != 0 ]]; then echo "#############################################################" >>$LOGFILE echo "### ###" >>$LOGFILE echo "#############################################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE exit fi echo " " >>$LOGFILE cp -f $TGTD$OF $TGTD_archive$OF &>>$LOGFILE if [[ $? != 0 ]]; then echo "#############################################################" >>$LOGFILE echo "### ###" >>$LOGFILE echo "#############################################################" >>$LOGFILE echo "$(date +'%d-%b-%Y %R%n') $OF " | sendxmpp -t -f /usr/local/etc/XMPP_settings _@ &>>$LOGFILE else echo " $(date +'%R %d-%b-%Y')!" >>$LOGFILE echo " \"$TGTD_archive$OF\"" >>$LOGFILE fi echo " $TGTD_archive" >>$LOGFILE umount $TGTD2 &>>$LOGFILE echo " !" >>$LOGFILE fi tail -n 300 $LOGFILE >/tmp/unique_fantastic_filename.tmp mv -f /tmp/unique_fantastic_filename.tmp $LOGFILE exit Thank you all for your attention!
Source: https://habr.com/ru/post/136313/
All Articles