Backup at the ready: destroying myths in honor of the holiday

Backup does not apply to fashionable technologies, which are shouting from each iron. It just has to be in any serious company, that's all. We have several thousand servers backed up in the bank - this is a difficult, interesting job, some of the intricacies of which, as well as typical misconceptions regarding backups, just want to tell.

I have been dealing with this topic for almost 20 years, of which the last 2 years have been at Promsvyazbank. At the very beginning of the practice, I backed up almost manually, with scripts that simply copied the files. Then, convenient tools appeared in Windows: Robocopy utility for preparing files and NT Backup for copying. And then the time came for specialized software, primarily Veritas Backup Exec, which is now called Symantec Backup Exec. So with backcaps familiar for a long time.

Simply put, backing up is backing up data (virtual machines, applications, databases, and files) just in case with a certain regularity. Every case usually manifests itself in the form of a hardware or logical failure and leads to data loss. The task of the backup system is to reduce losses from information loss. A hardware failure is, for example, a server or storage failure where the database resides. Logical is the loss or change of a part of the data, including due to the human factor: inadvertently deleting a table, file, running a script to execute a curve. There are also requirements of the regulator to store a certain type of information for a long period, for example, up to several years.
')


The most typical appeal to backups is the restoration of a saved copy of databases for deploying various test systems, clones for developers.

There are a few typical myths around backup that it’s time to dispel. Here are the most famous ones.

Myth 1. Backup has long been just a minor function within security or storage systems.

Backup systems are still a separate class of solutions, and very independent. Too important business is entrusted to them. In fact, they are the last line of defense when it comes to data integrity. So the backup works at its own pace, according to its own schedule. The servers generate a daily report, there are events that act as triggers for the monitoring system.



Plus, the role-based model of access to the backup system allows you to delegate part of the authority to administrators of target systems to manage backup copies.

Myth 2. When there is a RAID, backup is no longer needed

Undoubtedly, RAID-arrays and data replication are a good way to protect information systems from hardware failures, and in the presence of a standby-server, quickly organize a switch to it in case of failure of the main machine.

From logical errors that were made by users of the system, redundancy and replication does not save. Here is a standby server with postponed recording - yes, it can help out if an error is detected before it was synchronized. And if the moment is missed? This will help only in time made backup. If you know that the data has changed yesterday, you can restore the system as of the day before yesterday and pull out the necessary data from it. Taking into account that logical errors are the most frequent, the good old backup remains a proven and necessary tool.

Myth 3. Backup is something that is done once a month.

Backup frequency is a configurable parameter, primarily dependent on backup system requirements. It is quite possible to find data that almost never changes and is not particularly important, their loss will not be critical for the company.
They, indeed, can be backed up once a month and even less often. But more critical data is saved more often, depending on the RPO (Recovery point objrective) indicator, which specifies the allowable data loss. This can be once a week, once a day, or even several times an hour. We have transaction logs from the DBMS.



When systems are put into commercial operation, backup documentation is approved, which reflects the main points, the update procedure, the system recovery procedure, the backup storage order, and the like.

Myth 4. The volume of copies is constantly growing and occupies any allocated space completely.

Backups have a limited shelf life. It makes no sense, for example, to store during the year all 365 daily backups. As a rule, it is permissible to keep daily copies for 2 weeks, after which they are replaced with fresh ones, and the version that was made first in the month remains for long-term storage. She, in turn, is also stored for a certain time - each copy has a lifetime.



There is a protection against data loss. The rule is valid: before a backup is deleted, the next one must be formed. Therefore, the data will not be deleted if the backup is not executed, for example, due to the server being unavailable. Not only the time frames are respected, but the number of copies in the set is controlled. If the system contains the fact that there should be two complete backups, there will always be two, and the old one will be deleted only when the new third is successfully written. So the increase in the volume occupied by the backup archive is associated only with an increase in the amount of protected data and does not depend on time.

Myth 5. The backup started - everything hung

It is better to say this: if everything is hanging, then the administrator's hands do not grow from there. In general, the speed of backup depends on many factors. For example, from the speed of the backup system itself: how fast the disk storage there is, tape libraries. From the speed of the backup system servers: do they have time to process data, perform compression and deduplication? And also on the speed of the communication lines between the client and the server.

Backup can go in one or several streams, depending on whether the backup system supports multi-threading. For example, the Oracle DBMS allows you to send several threads, according to the number of available processors, until the transfer rate stops at limiting the network bandwidth.

If you try to back up a large number of threads, then there is a chance to overload the working system, it will really start to slow down. Therefore, the optimal number of threads is chosen to ensure sufficient speed. If even the slightest decrease in speed is critical, then there is a great option when backing up is not from the combat server, but from its clone - standby in database terminology. This process does not load the main production system. Data can be collected through more streams, since the server is not used for maintenance.

In large organizations, a separate network is created for the backup system so that the backup does not affect the prod. In addition, traffic can be transmitted not through a network, but through a SAN.

We try to distribute the load also in time. Backups mostly go after hours: at night, on weekends. In addition, they do not all run at the same time. Virtual machine backups are a special case. The process has almost no effect on the performance of the machine itself, so backup can be smeared by daytime, and not put it all off for the night. There are many subtleties, if everything is taken into account, the backup will not affect the performance of the systems.

Myth # 6. Launched a backup system - here’s fault tolerance

Never forget that the backup system is the last line of defense, which means that there should be five more systems in front of it that ensure continuity, high availability and disaster-resilience of the IT infrastructure and information systems of the enterprise.

Hope that the backup will restore all the data and quickly pick up the fallen service is not worth it. Loss of data from the moment of backup until the moment of failure is guaranteed, and the data on the new server can be uploaded for several hours (or days, as lucky). Therefore, it makes sense to create full-fledged fault-tolerant systems, without shifting everything to backup.

Myth 7. I set up a backup once, checked what works. It remains only to watch the logs

This is one of the most harmful myths, the fake one you realize only during the incident. Logs about successful backups are not a guarantee that everything really went right. The saved copy is important to check in advance for deployability. That is, start the recovery process in a test environment and look at the result.

And a little about the work of the sysadmin

In manual mode, no one has long copied data. Modern SRK can backup almost everything, you just have to configure it. If a new server has been added, prescribe policies: select content to back up, specify storage options, and apply a schedule.



At the same time, there is still a lot of work due to the vast fleet of servers, including databases, mail systems, virtual machine clusters, and file resources on both Windows and Linux / Unix. Employees who support the backup system do not sit idle.

In honor of the holiday, I would like to wish all admins strong nerves, clarity of movements and infinite space for keeping backups!