Backup systems
A few months ago, I started working on / understanding backup systems. I keep all the useful docks / links in my notes.
A lot of things have accumulated, I decided to share records, useful links and personal experience.
Most respected errors when backing up databases (taken from citrin.ru/backup.html )
- These principles are applicable not only to databases, but also to backup in general.
Excerpt from the PostgreSQL book. Developer Guide and Administrator. Geshvinde, Schenig Basically, when working with a database server, they make six errors:
- No backup copies at all.
Needless to say, this will certainly lead to serious problems.
')
- Backup copies are created, but the recovery process has never been tested.
Perhaps this is one of the most serious mistakes made when working with backups. The administrator creates backup copies and is sure that the data is protected from any surprises. However, if it is not known how the recovery works, in emergencies you may encounter serious problems. If recovery has never been verified before, the administrator feels insecure in the recovery process, which, in turn, leads to additional time losses and errors. Make sure that you know well enough how to recover data from backup copies. This is a very important aspect that most users and administrators simply forget.
- You create backup copies, but no one except you knows about them.
Generally speaking, cars are more reliable than people. This should be considered when developing database systems and backup strategies. In many organizations that we have had to deal with over the past few years, we have seen a very dangerous tendency: the person who was responsible for the backup or the entire information system was the only one with all the information about the state of the system. And what will happen when this person goes on vacation or decides to leave the organization? In such situations, many organizations can face serious problems. Regardless of the degree of redundancy and reliability of the information system, the organization will face great trouble if no one knows how it works, and how to act in emergency situations. The ability of the personnel working with the system to substitute each other is no less important than the redundancy of the equipment. There should be documentation of all critical processes in the information system, and several people should know about how the system works. Many organizations, trying to save money, neglect the documentation - however, in most cases, recovery processes are more expensive than creating short documents. In conclusion, it should be noted that the documentation must be maintained in a usable condition.
- Backups are created on the same media as the source data.
Over the past few years, we have come across several more worrying scenarios. We had to see backup systems in which backup copies were stored on the same media as the original data. Imagine a situation when a hard disk stops working. If you still manage to read the backups, you will be very lucky. In most cases this will not be possible. Backups and source data should be on different media. Otherwise, when trying to restore, you will encounter problems.
- Backups and raw data are stored in the same room.
In the event of a fire, it is important that one version of the data is stored in another room. Otherwise it is possible to destroy both data and backups. In this case, the data is lost forever, and their recovery is not possible.
- Having only the latest backup.
The existence of only the latest backup may cause problems. In most cases, the exact time of the problem is unknown, so the latest backups will also contain errors, and therefore the recovery will be difficult.
Useful links:
www.ibm.com/developerworks/ru/library/l-backup/index.html - Automate Backup to Linux
www.freebsd.org/doc/ru_RU.KOI8-R/books/handbook/backup-basics.html - Backup Technology Basics
ru.wikipedia.org/wiki/List_PO_for_rezervogo_kopirovaniya
en.wikipedia.org/wiki/Rsync
rsync.samba.org/examples.html
www.fredshack.com/docs/rsync.html
everythinglinux.org/rsync
howtoforge.org/rsync_incremental_snapshot_backups
www.mikerubel.org/computers/rsync_snapshots
www.debianhelp.co.uk/backup.htm
www.linux-backup.net
My choice:
Flexbackup
flexbackup.sourceforge.net
gentoo-wiki.com/HOWTO_Backup - a very smart dock, what to do and how.
Knows the differential \ incrimental \ full backups for which it was selected.
The syntax is very flexible, inclusion / exception masks. A wide selection of archivers.
Tholes is positioned as a backup system for single hosts (this is what I need.)
Differential backups are a type of backups where all subsequent backups back up only the changes that have occurred since the creation of the full backup.
In this way:
Once a month, a full backup is created, a differential copy is created every 3 days, relative to a full backup.
So that backups do not accumulate too much, but there were at least backups for the current and previous month:
Of course, that in flexbackup.conf it is specified to save files in this directory (/ mnt / backups / `hostname -s` / flexbackup.0 /), and also backup type is set to" tar "
Everything, on a separate host, the backup system is configured.
Next, using ssh keys and rsync, we synchronize the backup directory of the remote server with the centralized backup storage server.
It's simple, from standard tools, but effective;)
Attention:
This is all written and tested on Linux. On FreeBSD, this system also works, but at least in scripts everywhere you need to specify the PATH environment variables.
A lot of things have accumulated, I decided to share records, useful links and personal experience.
Most respected errors when backing up databases (taken from citrin.ru/backup.html )
- These principles are applicable not only to databases, but also to backup in general.
Excerpt from the PostgreSQL book. Developer Guide and Administrator. Geshvinde, Schenig Basically, when working with a database server, they make six errors:
- No backup copies at all.
Needless to say, this will certainly lead to serious problems.
')
- Backup copies are created, but the recovery process has never been tested.
Perhaps this is one of the most serious mistakes made when working with backups. The administrator creates backup copies and is sure that the data is protected from any surprises. However, if it is not known how the recovery works, in emergencies you may encounter serious problems. If recovery has never been verified before, the administrator feels insecure in the recovery process, which, in turn, leads to additional time losses and errors. Make sure that you know well enough how to recover data from backup copies. This is a very important aspect that most users and administrators simply forget.
- You create backup copies, but no one except you knows about them.
Generally speaking, cars are more reliable than people. This should be considered when developing database systems and backup strategies. In many organizations that we have had to deal with over the past few years, we have seen a very dangerous tendency: the person who was responsible for the backup or the entire information system was the only one with all the information about the state of the system. And what will happen when this person goes on vacation or decides to leave the organization? In such situations, many organizations can face serious problems. Regardless of the degree of redundancy and reliability of the information system, the organization will face great trouble if no one knows how it works, and how to act in emergency situations. The ability of the personnel working with the system to substitute each other is no less important than the redundancy of the equipment. There should be documentation of all critical processes in the information system, and several people should know about how the system works. Many organizations, trying to save money, neglect the documentation - however, in most cases, recovery processes are more expensive than creating short documents. In conclusion, it should be noted that the documentation must be maintained in a usable condition.
- Backups are created on the same media as the source data.
Over the past few years, we have come across several more worrying scenarios. We had to see backup systems in which backup copies were stored on the same media as the original data. Imagine a situation when a hard disk stops working. If you still manage to read the backups, you will be very lucky. In most cases this will not be possible. Backups and source data should be on different media. Otherwise, when trying to restore, you will encounter problems.
- Backups and raw data are stored in the same room.
In the event of a fire, it is important that one version of the data is stored in another room. Otherwise it is possible to destroy both data and backups. In this case, the data is lost forever, and their recovery is not possible.
- Having only the latest backup.
The existence of only the latest backup may cause problems. In most cases, the exact time of the problem is unknown, so the latest backups will also contain errors, and therefore the recovery will be difficult.
Useful links:
www.ibm.com/developerworks/ru/library/l-backup/index.html - Automate Backup to Linux
www.freebsd.org/doc/ru_RU.KOI8-R/books/handbook/backup-basics.html - Backup Technology Basics
ru.wikipedia.org/wiki/List_PO_for_rezervogo_kopirovaniya
en.wikipedia.org/wiki/Rsync
rsync.samba.org/examples.html
www.fredshack.com/docs/rsync.html
everythinglinux.org/rsync
howtoforge.org/rsync_incremental_snapshot_backups
www.mikerubel.org/computers/rsync_snapshots
www.debianhelp.co.uk/backup.htm
www.linux-backup.net
My choice:
Flexbackup
flexbackup.sourceforge.net
gentoo-wiki.com/HOWTO_Backup - a very smart dock, what to do and how.
Knows the differential \ incrimental \ full backups for which it was selected.
The syntax is very flexible, inclusion / exception masks. A wide selection of archivers.
Tholes is positioned as a backup system for single hosts (this is what I need.)
Differential backups are a type of backups where all subsequent backups back up only the changes that have occurred since the creation of the full backup.
In this way:
#monthly full backup
30 2 1 * * flexbackup -set all -full
#daily differential backups
0 5 2-31/3 * * flexbackup -set all -differential
Once a month, a full backup is created, a differential copy is created every 3 days, relative to a full backup.
So that backups do not accumulate too much, but there were at least backups for the current and previous month:
# Deleting all backups older then 60 days
30 1 1 * * find /mnt/backups/`hostname -s`/flexbackup.0/ -name "*.tar" -a -mtime +60 -delete
Of course, that in flexbackup.conf it is specified to save files in this directory (/ mnt / backups / `hostname -s` / flexbackup.0 /), and also backup type is set to" tar "
Everything, on a separate host, the backup system is configured.
Next, using ssh keys and rsync, we synchronize the backup directory of the remote server with the centralized backup storage server.
It's simple, from standard tools, but effective;)
Attention:
This is all written and tested on Linux. On FreeBSD, this system also works, but at least in scripts everywhere you need to specify the PATH environment variables.
Source: https://habr.com/ru/post/70649/
All Articles