Cold Storage in the cloud: Amazon, Google, Microsoft are changing the cloud storage services market

As the volume of data operated by various companies is growing, and this data needs to be stored somewhere, the largest cloud service providers began to offer their customers “cold storage” of data. Apparently, cold storage services may take up some of the traditional storage services market.

At the same time, cold storage as a cloud service can significantly change the way companies work with data, including information storage and delivery. Amazon Web Services, the Google Cloud Platform, and now Microsoft Azure also offer cold storage services to customers. In addition, each of these services has its own characteristics.

Almost all analysts make predictions about the future growth of the cloud services market, and this growth, apparently, will be booming. Analytical agency Gartner recently announced that this year the cost of cloud services and services will be for companies the main part of the cost of IT. In addition, since hybrid clouds are becoming more and more popular, by 2017 about half of large companies will take advantage of this opportunity.
')
How much data is generated annually? According to Cisco, global data center traffic has long exceeded the one zettabyte mark. In 2014, the traffic volume was 3.4 zettabyte, and in 2019, this figure is projected to increase to 10.4 zettabyte. A very fast-growing segment of traffic in data centers is “cloud traffic”, the volume of which in 2019 will be about 8.6 zettabyte.

Understanding the prospects of cold storage services, Google and Amazon have long been offering their customers to take advantage of cold storage services. Microsoft decided to join Google with Amazon and in April launched Cool Blob Storage, a low-cost “cold data” storage service.

What is all this for?

All three companies are positioning their cold storage services as an inexpensive service for storing inactive and unclaimed data, including backups, media content, scientific data, archives. In general, any data that is relatively rarely requested can be considered "cold."

At the same time, the access time to such data in cold storage is much higher than the access time to ordinary information when working with traditional data storage services. The cost of storing “cold data” is lower than the cost of storing information, to which constant on-line access is needed.

Google Nearline : for the first time, Google introduced its archival information storage service in 2015. This solution quickly became popular for several reasons. The main one is quick access to cold data, just a few seconds. This is faster than the same AWS Glacier. According to Google, Nearline is not much different from the company's standard cloud services. Here, access is slightly lower and access is slightly higher. Data access time in the Google Nearline environment ranges from 2 to 5 seconds. This is really nice. But there are a few problems.



The main thing is to limit the channel width of 4 MB / s for each stored terabyte. Accordingly, if you need to download everything at once, it will not work, you will have to wait. True, it is possible to speed up the process using the On-Demand I / O function. This feature allows you to increase the width of the channel with the stipulated 4 MB / s. But this feature is disabled by default.

The cost of storing data is about 1 cent per gigabyte per month. This is relatively small, plus the access time to cold data in 3-5 seconds makes Google Nearline one of the market leaders. The service allows you to store unlimited amounts of data with access to them through the Google Cloud Platform Storage API.

In addition, Nearline also has the ability to schedule data imports from various locations, including Amazon S3, HTTP / HTTPS sites, etc. All this can be automated.

Amazon AWS Glacier : This is one of the first cold storage solutions on the market. The company is positioning AWS Glacier as a safe and very inexpensive service for storing archived data and backups. You can store large amounts of information, the cost of the service is not high, and starts from 1 cent per gigabyte per month. AWS Glacier can be customized to your needs. For some types of stored data, you can configure an access time of several hours. Speeches about a few seconds does not go here, but Glacier has its advantages. For example, 1 TB of data can be downloaded in four hours. A Google Nearline user can download only 5% of a 1 TB data packet for the same time, with a total download time of approximately 70 hours.



The company offers to store IT media resources, health data, scientific data and work with Glacier instead of carriers on magnetic films.

In the Glacier environment, data is stored in “archives”. Any information can be stored, including photos, videos or documents. The maximum amount of one “archive” (this is a peculiar unit of the volume of stored data in Glacier) is 40 TB. You can download and store an unlimited number of such archives. Each of them has a unique ID, with assignment of creation time. After the “archive” has been created, the information stored in it cannot be updated, it can only be downloaded when the need arises. To cut off users who use the service for other purposes, for removing data previously three months of storage charge a certain fee.

You can view and manage stored data using the AWS Management Console and the AWS SDK. For any of the data sets, the following operations are applicable: creating, deleting, viewing content, tagging, establishing a set of policies, and so on.

Microsoft Cool Blob Storage : Azure Storage provides two tiers for storage of blobs (object storage) so that data can be stored most efficiently depending on its usage. Azure's “hot” storage tier is optimized for storing frequently used data. Azure's “cold” storage tier is optimized for storing data that is seldom and long stored.

Microsoft Cool Blob Storage is a cool level, this service is optimized for storing data that is rarely accessed and that should be stored for a long time. The cost of storing information is from 1 to 5 cents per gigabyte per month. It all depends on the region and the total amount of stored data. For comparison, the cost of storing hot data from Microsoft ranges from $ 0.0223 to $ 0.061 per gigabyte per month. According to company representatives, in some situations the storage of cold data is approximately twice cheaper than the hot one.



At the same time, the company allows you to transfer data from cold storage to hot storage and back. True, this service is paid. Within one account, you can store 500 TB. The maximum number of storage accounts for a subscription is 100. The target bandwidth for one file resource is up to 60 MB per second.

In general, cold data storage services from Microsoft, Google, Amazon can be called competitors with a certain stretch. The fact that they differ from each other and a set of functions, and characteristics. The leader is not here, and so far he is unlikely to appear. But users have plenty to choose from - for any requirements there will be a solution. The main thing is that new services are gradually changing the configuration of the market for traditional data storage services, and you can be sure that there will be more and more “cold” cloud services.