How to set up backup to S3-compatible cloud from Commvault

A source

Not so long ago, we talked about the launch of our new cloud service - the Technoserv Cloud object storage facility compatible with S3. Today we want to go to specific technical issues in working with our service, and in particular to the settings of the backup system for working with an S3-compatible storage. Now almost all modern RMSs have built-in support for the S3 API protocol, but not everyone knows about it and, accordingly, does not fully use the functionality of the systems.

A recent study by the Taneja Group has prioritized the use of object storage among users, according to which backup storage takes first place. As an example for today's post, we took Commvault SRK. Learn more about the study, about Commvault and how to configure the cloud connection to the system in our new material.

Since the volumes of data accumulated by companies are growing by more than 50% per year, and over 80% are unstructured data, object storage is of undoubted value. This is the only technology that allows you to keep up with the increasing storage capacity, while remaining within budget.

According to the results of the study , file backup and archiving are the most typical tasks (they were noted by 57% of respondents). The reason probably lies in the high demand for scalable file backups and document archiving, as well as cost-effective long-term data storage in order to comply with regulatory requirements. Object storage is ideal for large-scale storage of unstructured data, since it can be easily expanded in capacity (up to petabytes and above) by simply adding storage nodes. This approach removes a bottleneck in performance that limits the single- and dual-controller architectures of traditional file storages. In addition, object storage provides independence from the hardware platform, so companies can increase capacity by adding equipment.

In the second place is the storage of objects (Storage as a service), it was allocated by 44% of the respondents. Object storage provides service providers with a cost-effective way to manage backups using a robust, scalable, and multi-tenant architecture. And since this is a “service,” companies save on staff, equipment, and data center costs. The IT administrator simply rents storage space based on the cost per gigabyte and data transfer costs.

The third place in popularity is occupied by big data analytics (35% of respondents). Object storage is designed for large data sets, making it ideal for big data analytics tasks. In a recent survey, it turned out that about 30% of respondents accumulate 100 TB or more of “big data”, and the amount of this data increases significantly every month. However, in order to support large data analytics and artificial intelligence, for the correlation and interpretation of data, object storage should be closely integrated with low-latency storage and high performance computing systems.

Finally, another common use of object storage is the secure file sharing (it was also indicated by 35% of respondents).

Most object storages also offer very high resiliency with protection against site failures, site and multiple disks.

Therefore, today we will speak about the use of object storage as backup storage, and as an example of a backup system, as already noted, we chose Commvault as one of the leaders in this segment and the most popular solution among corporate clients.

Leaders in the backup solutions segment for the data center (Gartner, July 2017).

A bit about Commvault

Commvault is a scalable software platform for RK databases, applications, files, virtual machines and operating systems.

Backup systems that support S3.

Today, almost all modern backup systems have built-in support for the S3 API protocol, but not everyone knows about it and, accordingly, does not fully use the functionality of the system.

Commvault supports backup to the Amazon S3 cloud and other S3-compatible storage. And the cloud storage core of Technoserv Cloud is not only built using S3 technologies, but also supports all S3 API commands, which ensures compatibility with client applications, both now and in the future.

Further, using our cloud as an example, we will show how to back up data from Commvault to an S3-compatible storage. This instruction applies to any S3-compatible storage.

Configuring the connection of Technoserv Cloud storage to the Commvault backup system

To connect Cloud Storage Technoserv Cloud to the Commvault backup system, you must perform three simple steps.

1. Log in to the cloud resource management console ( https://storage.technoserv.cloud ) and create a container for storing data. To do this, go to the “Baskets and objects” tab, then enter the name of the basket in the “Basket” field and select “Create”, and leave the remaining fields filled out by default.

In this example, we called the bucket.

2. After that, you need to know the access key and password to the cloud storage. To do this, go to the "Password Management" tab.

In the window that opens, you will see the information necessary to connect the cloud storage to the backup system, namely:

• information about the service (Service Host);
• Access Key ID;
• and a secret key (Secret Access Key).

3. Now you need to create a cloud library in the Commvault backup system. To do this, in the Commcell console, right-click on the Libraries tab and select Add -> Cloud Storage Library.

In the window that opens, in the “General” tab, fill in all the fields:

• Name - specify the name of the cloud library being created;
• Type - select Amazon S3 from the drop-down list;
• MediaAgent - select a media server that will connect to the cloud storage;
• Access Information - enter the server address, access key and password mentioned above. In the Bucket field, specify the name of the basket created earlier.

If a proxy server is used to connect to the Internet, then you need to go to the Advanced tab and specify the proxy server settings.

That's all. This completes the integration of Technoserv Cloud cloud storage with Commvault.

To send backups to cloud storage, you need to set a storage policy in the backup system console in which to specify the created cloud library. After that, it remains to assign the storage policy to the backup task and run it.

Use of object cloud storage system Commvault .

In this way, as you can see, in a simple way we connected our cloud storage to the Commvault backup system when we started the backup service in the cloud.

Thus, you can quickly realize the benefits of backing up data to the cloud, using it as an extension of your IT infrastructure. Among the advantages of this solution:

• Fast and reliable backup and data recovery.
• No need for costly gateways and complex interim solutions.
• You can replace the tape with cloud storage.
• You can use the cloud for disaster recovery.

According to IDC forecast , the global cloud storage market (Cloud Storage) will grow by 25% per year.

What do you think? We are interested in your opinion: was the material useful? Do you need a similar backup instruction from another system? Which one? We are waiting for your comments!