New features Veeam Backup & Replication 8.0: automatic control of the intensity of read / write operations

Prehistory

When we released Veeam Backup & Replication 6.0 two years ago, scalability was one of the most important innovations: thanks to the distributed architecture based on dedicated “proxy servers” (servers that directly perform backup tasks on data movement) , the product was able to serve the infrastructure of almost any size.

There were several reasons for creating a new architecture, and, accordingly, it solved several problems at once:

1. Provide scalability to infrastructures of any size — to do this, you can create the necessary number of “proxy servers” (servers that perform backup tasks) between which you can optimally distribute the workload.
2. Improve the reliability of the backup process and eliminate the problem of a “single point of failure”. Even if one proxy server falls, the task for it will be completed by another.
3. Reduce the load that backup operations have on the production infrastructure through the use of the mechanism described in this article.
4. Remove the headache "How is the optimal schedule for several tasks?" - Now you can customize their parallel execution.
5. Automatically monitor free space on the storage and warn in advance about the approaching threshold value

Each time the backup task is executed, Veeam Backup & Replication virtual machine selects the optimal proxy server based on its territorial location in the network and its current load.

')

Veeam Backup & Replication v7

In the current version of the product released last year, we improved the load distribution algorithm by introducing new functionality - parallel data processing. The disks of one virtual machine could now be backed up simultaneously using multiple proxy servers.

For each VMware virtual disk, an optimal proxy server is selected based on the possible data access methods (listed below in descending order of priority):

• Direct SAN (direct access to data across a storage area network, does not affect production network hosts)
• Hot Add (direct access to the storage system via the ESXi I / O stack)
• Network Block Device - virtual machine data is transmitted over the network using the ESXi control interface

When setting up a proxy server for the VMware platform, you can choose the most appropriate data access mode, as well as determine the list of storages with which the proxy can work. Or you can specify that the data transfer method should be selected automatically, based on the proxy server configuration.

Here you can also specify the maximum number of tasks that this proxy server can handle at the same time.


When distributing new backup tasks, the current load on the proxy server is taken into account, as well as the indicator mentioned above - the maximum number of tasks that can be performed on it. This helps to avoid overloading the infrastructure as a whole. In version 7, we also reworked the data compression algorithm to reduce the processor load - as a result, the performance of the proxy server increased.

Most customers successfully used automatic load balancing settings, but there were also those who, because of the many tasks of proxy servers and, consequently, because of too much load on production storage systems, had problems with the availability of storage systems. Especially worried were customers who needed to support around-the-clock business applications: these same applications during backup creation could suddenly stop responding over the network, and for some they even received error notifications from monitoring systems. The problem lay in the lack of performance in read / write operations: after all, when backing up these operations can be performed very intensively, and if at the same time any virtual machines perform their read / write operations, expect trouble.

First, we chose the path of least resistance and simply decided to set constraints for the repositories - in one of the early patches for version 7, you could use the parameter “Simultaneous processing of no more than X tasks” (“No more than x tasks per datastore at "). However, a reasonable question arose - what if the database administrator deployed a new SQL server to our repository with such carefully adjusted settings - and all the limiting values ​​would suddenly be exceeded? On the other hand, if you are too cautious when setting a limit on the number of tasks to be processed, you can not keep within the allotted backup window. In general, the drawbacks of this approach are obvious: in dynamic, constantly changing virtual infrastructures, it is not suitable to constantly reconfigure threshold values ​​manually.

What will be new in version 8

In version 8, which will be released at the beginning of the 4th quarter of this year, we present a new feature - automatic control over the intensity of read / write DSS operations - Backup I / O Control (currently we are awaiting a patent for the algorithm of this functionality). For the user, this functionality will be available as a set of settings with which you can set the maximum allowable values ​​for the duration of the data exchange delay for the storage (datastore latency). This functionality is supported for both the VMware platform and Hyper-V.

The values ​​you set can be considered a kind of service level agreement for your storage system .


The new approach is very simple and requires only two parameters to be specified (values ​​are indicated in milliseconds).

• “ Stop assigning new tasks to datastore at: ” now when selecting a proxy server for processing a virtual disk, the delay time when accessing the disk for I / O operations (IOPS latency) will be taken into account. If the storage is too loaded (the delay time is equal to or greater than the specified value), then the backup task will not run until this figure decreases (that is, until the load on the storage decreases).
• “ Throttle I / O of existing tasks at: ” is used if the backup task is already running, and during its execution there is a delay in accessing the disk caused by external load. For example, if the SQL server service process starts on the virtual machine and uses the same storage as the backup task, this task will automatically limit its own disk access rate until the delay time is reduced to the specified value. Perhaps this will slightly increase the task execution time, but it will not adversely affect the operation of applications.

An example of using these settings illustrates the following graph:


The green color shows the delay in reading operations - we see that when using the automatic control settings (in the graph, this is the area between the Enabled and Disabled labels), this figure drops to 20 ms and below, and if the automatic control settings are disabled (in the graph, the area on the left from the Enabled label and to the right of the Disabled label), the value creeps up.

In Veeam Backup & Replication 8, users get access to various features of automatic control of the intensity of read / write operations depending on the version. Thus, in the Enterprise edition, these settings are set at the backup server level. The default values ​​we deliberately set were not very high, because we were guided by actual data from customers using our infrastructure monitoring solutions. Such values ​​are set by users in real infrastructures, setting up alerts such as "warning" and "error message." Of course, you can change the default values ​​to values ​​that are relevant specifically for your infrastructure.

In the Enterprise + edition, the automatic control settings can be set for each storage separately (here they are no longer global). This allows you to customize individual values ​​depending on the workload or criticality of a particular storage. For example, for development and testing tasks, the delay (response) time can be increased.

Summarizing

In the new version of Veeam Backup & Replication v8, we will release the functionality of automatic control of the intensity of read / write storage operations, which will allow users to optimally adjust the backup process in terms of the load on the production network’s storage and avoid interruptions in network applications and services.

Additional links

1. " Veeam Backup & Replication v8: Veeam Explorer for Active Directory and Veeam Explorer for Microsoft SQL Server "
2. " Support for hardware snapshots of NetApp storage in Veeam Backup & Replication v8 "
3. Website with an overview of the functionality of the new version of Veeam Backup & Replication Suite v8
4. " Integration of Veeam Availability Suite v8 and EMC Data Domain Boost "
5. " Storage I / O Control (SIOC) and Veeam " (Knowledge Base Article, English)