How are the backups of customers in our TierIII data center

The simplest is our S3 storage. In this case, we provide only the space, without any logic and organization of copying. Cheaply, simply, reliably, like a blow with a baton in the dark. S3 can copy most backup products - Veritas NetBackup / Backup Exec, CommVault, Veeam, EMC Networker, etc. This solution takes place in 2 hours.

The second approach is a complete backup infrastructure. We actually give the customer to rent a part of the software and hardware complex. And in this case, the customer receives not just a backup, but the entire backup service, including the programmatic organization of the copying logic. For example, if the customer is located in the data center on Volochaevskaya and uses the service, the Avamar complex collects backups from its infrastructure and makes both a copy in the same data center and a copy to another site if necessary. In addition, the system has internal protection for the Avamar itself - the system’s impression is removed twice a day, deduplicated, compressed.
')
Everything, of course, is excellent, but for objective reasons (we all managed to feel the crisis), we began to look for more budgetary, but at the same time functional options.

Let's start with Avamara, once mentioned

To meet the challenges of storing backups outside the local infrastructure, we installed the EMC Avamar PAC. The scheme is classical: a client application is installed on the copied machines, it integrates with the operating system and software, collects data, deduplicates it and sends it to our data center. Copying is always incremental: first, the system makes a complete copy of the data, and then only changing blocks (usually it is 1-3% of the total). This can significantly reduce the load on the channel and reduce the backup window. From the point of view of access, these are synthetic full backups.

Pros:

• fast copying due to deduplication and compression;
• good price;
• the possibility of recovery in the infrastructure of our data center to virtual machines;
• Recovery Guarantee: This is an array on the RAIN architecture with protection against disk / shelf failure and internal consistency checks on data. This is a more reliable medium for storing information than a regular tape cartridge — not least because it was chosen as the main cloud storage;
• backup costs grow as requirements grow, not jumps in capital costs;
• minimum of concern and customer support. We, together with the customer, carry out the initial configuration, as well as monitor the status of the system (it is free);
• suitable for those who cannot buy software and hardware from foreign suppliers.

The infrastructure, including PAK Avamar and integrated with vcloud (one of the shoulders of KROK cloud infrastructure) and KROK cloud (KVM-based cloud solution), is deployed in the Compressor data center.

Minuses:

• Backup and recovery on a local site depends entirely on the quality of the Internet channel.

Customers often need something small in backup, for example, a separate letter deleted yesterday, a specific file or something else. It's pretty silly to roll out the whole image because of the file "1.txt", which the user accidentally overwritten on his desktop. PAK Avamar provides software administrators with a convenient console and a nice GUI for working with application files and users. They themselves can find and restore data without pulling the backup administrator. Naturally, access rights are delimited.

Cheap and angry

Evaluating various implementation options, we gradually came to another solution - when we turn out a turnkey backup solution to the customer’s infrastructure:



CrocBackApp is a server with a large disk capacity and the ability to connect external disk shelves, on which CommVault is installed, which we are OEM. At the request of the customer, the servers can be clustered in one chassis, and in general, almost any architectural solution can be built to the customer’s needs.

As testing progressed, another important advantage of such a solution emerged: ease of customization and integration into the corporate environment. That is, the customer takes the hardware with our installed CrocBackApp, simply puts it on the platform, launches it, installs it with the help of the clients wizard on the protected machines - and it works.

Plus, we achieve tasty discounts due to the volume of purchases of hardware and software - if you try to build or purchase such a solution from large vendors yourself, you will get more expensive.

Along with the task of storing backups on a remote site, there is one more frequently encountered task - organizing not only backups, but also a flexible infrastructure in a remote data center, which you can quickly upgrade in an hour of X. And here we offer the customer no longer “Backup-as-a-Service”, but actually “Disaster Recovery-as-a-Service”. The attractiveness of the service is that in normal mode the customer actually pays only for storing and transferring data to a remote data center, and, if necessary, begins to consume the entire infrastructure, and can restore it in a matter of minutes.

We do this service - “Disaster Recovery-as-a-Service” - in different ways:

1. There is direct replication between our clouds and the private cloud or customer infrastructure. There are VMware, Hyper-V and KVM clouds where data can be directly replicated by means of hypervisors (for example, vSphere Replication). This is the easiest and most affordable replication method.
2. Restoring infrastructure from backups to our cloud infrastructure. There are solutions from different manufacturers. There is Veeam Cloud Connect Replication, which allows you to replicate data or backups over the WAN-channels to our infrastructure. As part of our main backup solution - Avamar - we can copy and restore data from different environments - p2v, v2p, between different clouds (for example, Vmware in KVM, and vice versa).
3. The third solution is more interesting:

Some of our customers have asked to replicate their hardware (OS) into our cloud to our virtual machines. This is exactly the task of the “fallen out” office, when a client can take and run something from his backup directly from the cloud in 10 minutes. Replication using Zerto and Double-Take software allows to mirror the data of the operating system and applications over IP with minimal delay. On the technical side, it looks like a service that runs inside the operating system or the ESX hypervisor and transfers all changes to us in the cloud. On the practical side, this is an opportunity for the client to restore the functionality of the application in the shortest possible time.



With this approach, in addition to the technical solution, we are working with customers and Disaster Recovery plans for disaster recovery. So that the customer himself could, if necessary, test or carry out the restoration of part of their machines or the entire infrastructure.

Practice

The Avamar solution is already 5 years old, and the CrocBackApp is a couple of examples. Somehow, a retail chain came to us, which did not have a centralized backup across the entire infrastructure, and lacked the capacity of a “small” one, designed for individual offices. Iron and software were different everywhere, and when something fell, a circus and a zoo began in an attempt to roll everything back. No, this, in general, worked, but each individual case required the competence of a sysadmin, who has 8 years of experience with this environment and knows exactly what archaeological layer lies on. It was he who ordered CrocBackApp “cubes” from us to get rid of these nightmares in the regions once and for all.
Then came the government agencies: their data is not the largest, but important. They have our "cubes" just stuck in switches FC or Ethernet, and clients are installed on all local machines. Then you just need to set up a backup schedule and storage plan.

As a result, a very simple scheme was formed:

• If a business with a large number of offices or, on the contrary, SMB, and does not want / is not able to install a separate hardware locally, we implement Avamar backup. There is a fairly low storage cost and zero upfront costs. That is, OPEX is small, and CAPEX is zero.
• For those who have the opportunity to place equipment on their sites, we offer CrocBackApp. OPEX is smaller than with Avamar, but there is a CAPEX.
• And for those who already have their own backup infrastructure, we suggest using our S3 for remote storage.

Links

• How does corporate backup work on Avamara?
• My mail is SerVerchenov@croc.ru