Veeam Availability Suite v9: “infinitely” scalable backup repository
The new Veeam Unlimited Scale-out Backup Repository allows you to combine different storage systems into a single scalable backup storage pool. As a result, an additional software level of abstraction is created, which provides a simple and more efficient use of storage systems, as well as backup tasks. I'll start by describing the admin problem.
Familiar picture? All of this is backup repositories.
Tired of manually selecting the target repository for various backup jobs? Want to optimize storage usage? Do you dream once and for all to get rid of manual management of backup tasks and refuse to buy new storage systems, more effectively using existing ones? If yes, this article is for you.
What is the reason for the complexity of the storage system? The fact is that several physical devices are often used as backup storage locations. For example, backups can be stored on the internal disks of physical servers. In some cases, storage devices are limited by the maximum volume size (LUN). Anyway, many people use more than one backup repository, because even the smallest environments quickly outgrow the original repository, which is rarely sent to a landfill.
')
As a result, most users have to create as many backup jobs as they have repositories . Often, against their own will, they have to create dozens or even hundreds of tasks, since otherwise it is impossible to use the available backup storage resources.
In addition, there is another not immediately noticeable problem. Look in the “Free” column, and you will see that significant amounts of unallocated disk space disappear without any sense . When placing backup jobs today, you need to be more careful and take into account the future growth of virtual machine (VM) volumes. Otherwise, you doom yourself to periodic failures in backups due to lack of space and constant reshaping tasks. You literally pay for it by buying additional backup storage devices. Despite the fact that 30% of the available capacity (and sometimes more) remain unused and constitute a reserve for future growth.
Choosing the right backup solution is often a matter of company survival. Depending on the existing work environment, users have to choose a solution that is best suited for a particular scenario. Sometimes this is a high-speed storage array or a large system capable of supporting multiple recovery points, sometimes a deduplication system, etc.
However, all these systems lack one important quality: they do not evolve. The selected solution will always work in the same way, its characteristics will not change. The storage array on flash drives in the short term will not be cheaper than the deduplication system, the disk array will not become faster than the flash array, and the deduplication system will not be able to start the VM from the backup file as a disk array. Each option has its pros and cons, so the system architect has to carefully choose the solution. After all, it costs money and should last long enough for the investment to pay off. No one wants to buy additional storage because of errors in the initial selection. However, we live in a world where data volumes double in a year, so even a correctly chosen solution in a year may turn out to be untenable.
In short, the principle of operation of the new repository can be described as follows: SOBR combines several “simple” repositories into a single system that can be used as a target for any backup copies. It sounds very simple, but in fact users will get a lot of amazing features. I am sure that you have already begun to understand ...
First, now it will be extremely easy for both small and large companies to increase their repositories as they are filled. It is no longer necessary to reconfigure backup chains for a long time (in large companies, they can be quite big). Users can simply expand their scalable backup repository by adding another “simple” repository to it. All existing backup files are saved. After adding a new repository to the group, the backup destination remains the same, but additional free space appears in it, which can be used immediately.
This feature allows you to set up just one backup task to protect the entire environment, even if you have thousands of VMs. This option is especially useful in combination with another great feature of v9: setting up separate backup chains for each VM. Simply specify as a destination for such a task a scalable repository that combines many capacious devices, and you will no longer have to worry about managing the capacity of individual repositories or counting the volume of each task. And while maintaining high backup speeds! In contrast to the “storage pools” that are on the market, a single task will use all available devices at the same time and thereby maintain a high speed of work. Previously, this required creating many backup tasks and running them simultaneously.
A scalable backup repository is not just a group of backup-repositories working as one. Any scalable solution is capable of this: they added a new node, the system recalculated capacity and began to use additional space. Of course, our scalable repositories can do this, but this is only a fraction of their capabilities. Veeam does not offer storage systems, but software solutions using such devices. The customer himself chooses storage systems based on their needs and taking into account their capabilities, performance, volume and cost. Using a scalable backup repository, customers will be able to use simultaneously any Veeam storage systems supported: servers running Windows or Linux with local or DAS storage, network drives, and even storage systems with deduplication. Do you have a lot of small pieces of free disk space, scattered on different systems? Add them to the new scalable backup repository, and you can use all of this volume. Do not buy new storage systems until you use fully existing ones!
More importantly, a scalable backup repository is a software technology that manages physical devices . This means that all the functions of all storage systems are preserved. For example, deduplication systems will continue to significantly reduce data volumes and improve performance through the use of unique APIs (when used with EMC Data Domain Boost systems, HP StoreOnce Catalyst or ExaGrid Accelerated Data Mover). Yes, you understood correctly: you can combine repositories of any type, both existing and newly acquired in your environment. And you can use all their features. Unlike any other scalable general purpose storage systems, we do not limit you to servers with local disks.
This solution again emphasizes Veeam’s main goal: to ensure complete independence from the type of equipment and storage system. We want users to be able to fully utilize the previously acquired storage systems and not pay money for new ones until the available resources are completely exhausted.
All backup repositories have their own characteristics, and we took this into account when developing a scalable repository. Each physical device can be assigned its own “role” . With just a few clicks, you can determine which backups will be stored in a specific group repository: full, incremental, or both. Think about what endless possibilities open up before you. To create a simple scalable backup repository, simply group several backup stores with different characteristics and configure them based on the strengths of each device.
As an example, consider the backup conversion operation in Veeam. When this happens, two I / O operations are used to merge the old incremental file with the full backup. Many low-cost storage systems have problems handling arbitrary I / O operations. As a result, users prefer to make active full copies to reduce the I / O load on the storage. But at the same time they lose the benefits of constantly incremental backup. Now imagine that you added another one to the first set of disks (JBOD). A scalable backup repository can make their work much more efficient. Having assigned one block for storing incremental copies and another for storing full copies, we separate the input-output operations. During the backup conversion, the reading will be performed by the incremental backups repository, and the recording by the repository of full backups. So a scalable backup repository instantly increased the performance of conversion operations at least twice. And all this without the use of flash drives, cache or any other additional mechanism. Not bad, right?
Now imagine that you have several specialized volumes instead of many small clones, like these JBODs. If desired, you can use a high-speed array of flash drives for fast processing of incremental files in parallel with any storage that supports deduplication for long-term storage of full copies (GFS scheme). This combination of fundamentally different solutions allows you to use the strengths of each and at the same time eliminates restrictions.
Each type of storage system, even the most modern, can be perfectly suited for one task and is completely unsuitable for another. A scalable backup repository provides users with complete freedom of choice and retains all the capabilities of the selected repositories.
The level of abstraction created by the scalable repository allows the administrator to become a “storage service provider”. It can provide users with the ability to self-configure backup tasks, and they don’t have to worry about choosing a storage or perform complex calculations of backup sizes and retention periods.
Administrator simply set up a single scalable repository. After that, users will only see it (and not dozens of component devices) and will be able to choose such storage as a target for their backup tasks. Then the scalable repository will start using available devices, depending on the assigned policy and the amount of free space. Like any solution that simulates the cloud structure, a scalable backup repository allows you to completely separate the responsibilities of providers and users.
Familiar picture? All of this is backup repositories.
Tired of manually selecting the target repository for various backup jobs? Want to optimize storage usage? Do you dream once and for all to get rid of manual management of backup tasks and refuse to buy new storage systems, more effectively using existing ones? If yes, this article is for you.
Backup Storage Administration Problem
What is the reason for the complexity of the storage system? The fact is that several physical devices are often used as backup storage locations. For example, backups can be stored on the internal disks of physical servers. In some cases, storage devices are limited by the maximum volume size (LUN). Anyway, many people use more than one backup repository, because even the smallest environments quickly outgrow the original repository, which is rarely sent to a landfill.
')
As a result, most users have to create as many backup jobs as they have repositories . Often, against their own will, they have to create dozens or even hundreds of tasks, since otherwise it is impossible to use the available backup storage resources.
In addition, there is another not immediately noticeable problem. Look in the “Free” column, and you will see that significant amounts of unallocated disk space disappear without any sense . When placing backup jobs today, you need to be more careful and take into account the future growth of virtual machine (VM) volumes. Otherwise, you doom yourself to periodic failures in backups due to lack of space and constant reshaping tasks. You literally pay for it by buying additional backup storage devices. Despite the fact that 30% of the available capacity (and sometimes more) remain unused and constitute a reserve for future growth.
New type of backup repository
Choosing the right backup solution is often a matter of company survival. Depending on the existing work environment, users have to choose a solution that is best suited for a particular scenario. Sometimes this is a high-speed storage array or a large system capable of supporting multiple recovery points, sometimes a deduplication system, etc.
However, all these systems lack one important quality: they do not evolve. The selected solution will always work in the same way, its characteristics will not change. The storage array on flash drives in the short term will not be cheaper than the deduplication system, the disk array will not become faster than the flash array, and the deduplication system will not be able to start the VM from the backup file as a disk array. Each option has its pros and cons, so the system architect has to carefully choose the solution. After all, it costs money and should last long enough for the investment to pay off. No one wants to buy additional storage because of errors in the initial selection. However, we live in a world where data volumes double in a year, so even a correctly chosen solution in a year may turn out to be untenable.
Veeam Unlimited Scale-out Backup Repository
In short, the principle of operation of the new repository can be described as follows: SOBR combines several “simple” repositories into a single system that can be used as a target for any backup copies. It sounds very simple, but in fact users will get a lot of amazing features. I am sure that you have already begun to understand ...
Shared storage pool
First, now it will be extremely easy for both small and large companies to increase their repositories as they are filled. It is no longer necessary to reconfigure backup chains for a long time (in large companies, they can be quite big). Users can simply expand their scalable backup repository by adding another “simple” repository to it. All existing backup files are saved. After adding a new repository to the group, the backup destination remains the same, but additional free space appears in it, which can be used immediately.
This feature allows you to set up just one backup task to protect the entire environment, even if you have thousands of VMs. This option is especially useful in combination with another great feature of v9: setting up separate backup chains for each VM. Simply specify as a destination for such a task a scalable repository that combines many capacious devices, and you will no longer have to worry about managing the capacity of individual repositories or counting the volume of each task. And while maintaining high backup speeds! In contrast to the “storage pools” that are on the market, a single task will use all available devices at the same time and thereby maintain a high speed of work. Previously, this required creating many backup tasks and running them simultaneously.
Efficient use of storage investment
A scalable backup repository is not just a group of backup-repositories working as one. Any scalable solution is capable of this: they added a new node, the system recalculated capacity and began to use additional space. Of course, our scalable repositories can do this, but this is only a fraction of their capabilities. Veeam does not offer storage systems, but software solutions using such devices. The customer himself chooses storage systems based on their needs and taking into account their capabilities, performance, volume and cost. Using a scalable backup repository, customers will be able to use simultaneously any Veeam storage systems supported: servers running Windows or Linux with local or DAS storage, network drives, and even storage systems with deduplication. Do you have a lot of small pieces of free disk space, scattered on different systems? Add them to the new scalable backup repository, and you can use all of this volume. Do not buy new storage systems until you use fully existing ones!
More importantly, a scalable backup repository is a software technology that manages physical devices . This means that all the functions of all storage systems are preserved. For example, deduplication systems will continue to significantly reduce data volumes and improve performance through the use of unique APIs (when used with EMC Data Domain Boost systems, HP StoreOnce Catalyst or ExaGrid Accelerated Data Mover). Yes, you understood correctly: you can combine repositories of any type, both existing and newly acquired in your environment. And you can use all their features. Unlike any other scalable general purpose storage systems, we do not limit you to servers with local disks.
This solution again emphasizes Veeam’s main goal: to ensure complete independence from the type of equipment and storage system. We want users to be able to fully utilize the previously acquired storage systems and not pay money for new ones until the available resources are completely exhausted.
Storage Distribution
All backup repositories have their own characteristics, and we took this into account when developing a scalable repository. Each physical device can be assigned its own “role” . With just a few clicks, you can determine which backups will be stored in a specific group repository: full, incremental, or both. Think about what endless possibilities open up before you. To create a simple scalable backup repository, simply group several backup stores with different characteristics and configure them based on the strengths of each device.
As an example, consider the backup conversion operation in Veeam. When this happens, two I / O operations are used to merge the old incremental file with the full backup. Many low-cost storage systems have problems handling arbitrary I / O operations. As a result, users prefer to make active full copies to reduce the I / O load on the storage. But at the same time they lose the benefits of constantly incremental backup. Now imagine that you added another one to the first set of disks (JBOD). A scalable backup repository can make their work much more efficient. Having assigned one block for storing incremental copies and another for storing full copies, we separate the input-output operations. During the backup conversion, the reading will be performed by the incremental backups repository, and the recording by the repository of full backups. So a scalable backup repository instantly increased the performance of conversion operations at least twice. And all this without the use of flash drives, cache or any other additional mechanism. Not bad, right?
Now imagine that you have several specialized volumes instead of many small clones, like these JBODs. If desired, you can use a high-speed array of flash drives for fast processing of incremental files in parallel with any storage that supports deduplication for long-term storage of full copies (GFS scheme). This combination of fundamentally different solutions allows you to use the strengths of each and at the same time eliminates restrictions.
Each type of storage system, even the most modern, can be perfectly suited for one task and is completely unsuitable for another. A scalable backup repository provides users with complete freedom of choice and retains all the capabilities of the selected repositories.
Storage as a service
The level of abstraction created by the scalable repository allows the administrator to become a “storage service provider”. It can provide users with the ability to self-configure backup tasks, and they don’t have to worry about choosing a storage or perform complex calculations of backup sizes and retention periods.
Administrator simply set up a single scalable repository. After that, users will only see it (and not dozens of component devices) and will be able to choose such storage as a target for their backup tasks. Then the scalable repository will start using available devices, depending on the assigned policy and the amount of free space. Like any solution that simulates the cloud structure, a scalable backup repository allows you to completely separate the responsibilities of providers and users.
Additional links:
Source: https://habr.com/ru/post/275115/
All Articles