How we backed up the client's IT infrastructure in Donetsk
Backing up for a customer is a common service, but this case is distinguished by the fact that the customer, RBC Ukrinvest and its infrastructure are located in Donetsk. Probably it does not make sense to talk about the constant bombardment, shelling of buildings, etc., I think that everything is already up to date. But it was these circumstances that caused the client to think about the need for a full backup of its IT infrastructure in the cloud.
We deployed all our resources remotely.
In this post I will talk about the technical side of the project, the problems that have arisen and the results obtained.
The main goal is to ensure the safety of IT infrastructure and critical corporate information in case of failure of the customer's server and, accordingly, equipment.
')
Note: unfortunately, I do not have the right to publish a full list of equipment and services of the customer in connection with the signing of a non-disclosure agreement. Nevertheless, it can be said that the customer's IT infrastructure corresponds to a medium-sized company: first of all, the MS Exchange mail server, the infrastructure of critical virtual servers for the company.
The main task that faced us was to provide continuous backup of the IT infrastructure and critical corporate information of the customer to our cloud. In case of failure of their own equipment, the customer should be able to work in the same IT infrastructure in the Cloud4Y cloud. The main limitation is the compatibility of virtual environments, on the basis of which the customer’s virtual server infrastructures are synchronized with the Cloud4Y cloud infrastructure. Since we and the customer of the virtualization environment turned out to be compatible, we decided to apply solutions based on Asigra-VDR. For a general view of the amount of synchronized virtual resources: the total amount of data was 1.5 TB.
All work was carried out remotely together with the technical specialists of the customer. Asigra Backup with incremental backup, compression and encryption technology was chosen. The Asigra Remote DS-VDR Incremental Restore scheme was used, which allowed transferring only the modified blocks of virtual machines.
The scheme of Asigra Remote DS-VDR Incremental Restore is as follows:
1) DS-client (Asigra agent) receives only modified blocks of virtual machines.
2) DS-client deduplicates, compresses and encrypts changed blocks and sends them to DS system through WAN.
3) DS-system receives files from DS-Client.
4) The DS system stores files in the backup storage in a compressed, encrypted, and deduplicated form.
5) Remote VDR receives the modified blocks from the DS system, decrypts, unpacks and updates the changed blocks of the backup copy of virtual machines in their original format.
6) Remote DS-VDR performs incremental recovery from virtual machines in idle mode of the ESXi server by writing modified blocks in the appropriate partition of the disk.
7) When a customer encounters equipment failure, he will be able to access the backup IT infrastructure and corporate data that are deployed in the Cloud4Y cloud.
I would like to note a high degree of compression - with a volume of 1.5 TB of useful data, the archive was about 450 GB.
Top 3 specific risks of Donetsk due to hostilities: the threat of getting shells into the room, power outages, the destruction of cable infrastructure. The remaining risks are typical: the removal of servers in case of violation of the law, the transfer of premises to state property - you can summarize this problem as an instant physical destruction of the data center.
The most frequent problem of our client was falling communication channels. The customer constantly lost the Internet in the office due to the shelling of nearby buildings. In addition, the electricity was periodically turned off, while the UPS was only enough for 8 hours. For these reasons, synchronization of the customer's IT infrastructure with the cloud took 3 days.
The key to the success of the task, namely: “Ensure the client’s working capacity, all its critical resources as soon as possible” was the preparatory stage.
With the help of my colleagues from Cloud4Y, the Asigra DS-Client Agent was implemented in the area of ​​responsibility of the customer, which allows to implement the technology of incremental backup of client virtual machines. We didn’t really go into customer data. The intuitive interface of the Asigra DS-Client Agent, after a demo consultation of a client's specialist, made it possible to promptly enter into the backup all critical virtual client servers.
An important component to start successful work is the first backup cycle of a virtual resource that must be completed successfully. Due to the fact that in the city where the client is located, hostilities are constantly being conducted (the connection is interrupted, there is no electricity supply periodically), this stage was completed in several approaches. The efficiency of deduplication and data compression, implemented by Asigra, made it possible to force the first backup cycle and go into incremental mode. For reference, it is possible to report that the efficiency of compression and deduplication reached from 1: 4 to 1:10 (that is, the actual transferred volume even at the first stage of a full backup was several times less than the original infrastructure), which also made it possible not to overload external channels communication client (located at a great distance from the cloud), and do not interfere with the current work of the client infrastructure.
On Cloud4Y side, in parallel, Asigra VDR-Agents were deployed, which automatically restored the customer's virtual machines as soon as the next backup cycle ended.
Summarizing the practical side of backup: a copy of virtual resources in the cloud Cloud4Y unfolded with a maximum delay of 3 hours and in theory was always operational ready to work. But in this "theoretically" there is also a practical side, which requires careful preparation on the side of the cloud provider.
We used the VMWare vCloudDirector cloud infrastructure management interface on our side. The main practical advantage of this platform is the freedom to implement isolated networks both within the infrastructure of the virtual DataCenter of the client (VDC) and within the framework of the vApp infrastructure (virtual infrastructure).
After the next cycle of automatic recovery, virtual machines "come" to us in the configuration and composition, which is directly located at the customer. That is, the first recovery cycle prescribes the network interfaces of the virtual machines to those that are originally located at the customer. To reduce the time of the subsequent emergency lifting of the customer's infrastructure, it is necessary to prepare in advance virtual isolated networks in vCloudDirector and register in them the actual addressing used by the client. This stage was also made in a dialogue with the technical specialists of the customer.
The final touches of preparing for work a copy of the virtual infrastructure is the preparation of a virtual gateway that implements the access of client virtual machines in the Cloud4Y cloud "out", and also implements the firewall of the client's closed infrastructure.
1. The cloud is always a copy of the infrastructure of the customer with a 3-hour delay (max).
2. Disaster recovery of the client's infrastructure occurs by simply launching its infrastructure in the Cloud (which the customer himself can do via the virtual Data Center's vCloudDirector interface).
3. External client services are reconfigured by making changes to dns.
Subsequently, the client, using the same backup technology Asigra, always through its Asigra DS Client Agent, can restore its infrastructure to its original or new location if the need arises.
As a result, the emergency work of the client in the event of extremely adverse events (up to the complete destruction of its infrastructure) is ensured.
Thank you for your attention, we will be happy to answer your questions.
We deployed all our resources remotely.
In this post I will talk about the technical side of the project, the problems that have arisen and the results obtained.
purpose
The main goal is to ensure the safety of IT infrastructure and critical corporate information in case of failure of the customer's server and, accordingly, equipment.
')
Note: unfortunately, I do not have the right to publish a full list of equipment and services of the customer in connection with the signing of a non-disclosure agreement. Nevertheless, it can be said that the customer's IT infrastructure corresponds to a medium-sized company: first of all, the MS Exchange mail server, the infrastructure of critical virtual servers for the company.
Task
The main task that faced us was to provide continuous backup of the IT infrastructure and critical corporate information of the customer to our cloud. In case of failure of their own equipment, the customer should be able to work in the same IT infrastructure in the Cloud4Y cloud. The main limitation is the compatibility of virtual environments, on the basis of which the customer’s virtual server infrastructures are synchronized with the Cloud4Y cloud infrastructure. Since we and the customer of the virtualization environment turned out to be compatible, we decided to apply solutions based on Asigra-VDR. For a general view of the amount of synchronized virtual resources: the total amount of data was 1.5 TB.
Decision
All work was carried out remotely together with the technical specialists of the customer. Asigra Backup with incremental backup, compression and encryption technology was chosen. The Asigra Remote DS-VDR Incremental Restore scheme was used, which allowed transferring only the modified blocks of virtual machines.
How it works
The scheme of Asigra Remote DS-VDR Incremental Restore is as follows:
1) DS-client (Asigra agent) receives only modified blocks of virtual machines.
2) DS-client deduplicates, compresses and encrypts changed blocks and sends them to DS system through WAN.
3) DS-system receives files from DS-Client.
4) The DS system stores files in the backup storage in a compressed, encrypted, and deduplicated form.
5) Remote VDR receives the modified blocks from the DS system, decrypts, unpacks and updates the changed blocks of the backup copy of virtual machines in their original format.
6) Remote DS-VDR performs incremental recovery from virtual machines in idle mode of the ESXi server by writing modified blocks in the appropriate partition of the disk.
7) When a customer encounters equipment failure, he will be able to access the backup IT infrastructure and corporate data that are deployed in the Cloud4Y cloud.
I would like to note a high degree of compression - with a volume of 1.5 TB of useful data, the archive was about 450 GB.
Problems
Top 3 specific risks of Donetsk due to hostilities: the threat of getting shells into the room, power outages, the destruction of cable infrastructure. The remaining risks are typical: the removal of servers in case of violation of the law, the transfer of premises to state property - you can summarize this problem as an instant physical destruction of the data center.
The most frequent problem of our client was falling communication channels. The customer constantly lost the Internet in the office due to the shelling of nearby buildings. In addition, the electricity was periodically turned off, while the UPS was only enough for 8 hours. For these reasons, synchronization of the customer's IT infrastructure with the cloud took 3 days.
The key to the success of the task, namely: “Ensure the client’s working capacity, all its critical resources as soon as possible” was the preparatory stage.
With the help of my colleagues from Cloud4Y, the Asigra DS-Client Agent was implemented in the area of ​​responsibility of the customer, which allows to implement the technology of incremental backup of client virtual machines. We didn’t really go into customer data. The intuitive interface of the Asigra DS-Client Agent, after a demo consultation of a client's specialist, made it possible to promptly enter into the backup all critical virtual client servers.
An important component to start successful work is the first backup cycle of a virtual resource that must be completed successfully. Due to the fact that in the city where the client is located, hostilities are constantly being conducted (the connection is interrupted, there is no electricity supply periodically), this stage was completed in several approaches. The efficiency of deduplication and data compression, implemented by Asigra, made it possible to force the first backup cycle and go into incremental mode. For reference, it is possible to report that the efficiency of compression and deduplication reached from 1: 4 to 1:10 (that is, the actual transferred volume even at the first stage of a full backup was several times less than the original infrastructure), which also made it possible not to overload external channels communication client (located at a great distance from the cloud), and do not interfere with the current work of the client infrastructure.
On Cloud4Y side, in parallel, Asigra VDR-Agents were deployed, which automatically restored the customer's virtual machines as soon as the next backup cycle ended.
Summarizing the practical side of backup: a copy of virtual resources in the cloud Cloud4Y unfolded with a maximum delay of 3 hours and in theory was always operational ready to work. But in this "theoretically" there is also a practical side, which requires careful preparation on the side of the cloud provider.
We used the VMWare vCloudDirector cloud infrastructure management interface on our side. The main practical advantage of this platform is the freedom to implement isolated networks both within the infrastructure of the virtual DataCenter of the client (VDC) and within the framework of the vApp infrastructure (virtual infrastructure).
After the next cycle of automatic recovery, virtual machines "come" to us in the configuration and composition, which is directly located at the customer. That is, the first recovery cycle prescribes the network interfaces of the virtual machines to those that are originally located at the customer. To reduce the time of the subsequent emergency lifting of the customer's infrastructure, it is necessary to prepare in advance virtual isolated networks in vCloudDirector and register in them the actual addressing used by the client. This stage was also made in a dialogue with the technical specialists of the customer.
The final touches of preparing for work a copy of the virtual infrastructure is the preparation of a virtual gateway that implements the access of client virtual machines in the Cloud4Y cloud "out", and also implements the firewall of the client's closed infrastructure.
Result
1. The cloud is always a copy of the infrastructure of the customer with a 3-hour delay (max).
2. Disaster recovery of the client's infrastructure occurs by simply launching its infrastructure in the Cloud (which the customer himself can do via the virtual Data Center's vCloudDirector interface).
3. External client services are reconfigured by making changes to dns.
Subsequently, the client, using the same backup technology Asigra, always through its Asigra DS Client Agent, can restore its infrastructure to its original or new location if the need arises.
As a result, the emergency work of the client in the event of extremely adverse events (up to the complete destruction of its infrastructure) is ensured.
Thank you for your attention, we will be happy to answer your questions.
Source: https://habr.com/ru/post/242919/
All Articles