Good afternoon, dear colleagues.
I suggest you familiarize yourself with the translation of the next part of the comparison of the services of the storage of two famous cloud platforms. The remaining translations will be posted as they are made.
The first part of the cycle - Comparing Windows Azure Table Storage and Amazon DynamoDBIn this series of articles, I want to compare the core functionality of the
Windows Azure Blob Storage Service and the
Amazon Simple Storage Service (S3) . In this part, we look at the basic concepts, pricing and compare the functionality of blob containers and baskets. In Part II, we will compare blobs and objects. In Part III, I’ll give conclusions, in Part IV, a comparison of Windows Azure Blob Storage and Google Cloud Storage.
Abbreviations:
Windows Azure Blob Storage - WABS, Amazon Simple Storage Service - AS 3.Conceptually, WABS and AS3 are similar in functionality to the system. In short, both systems can be viewed as a file system in the cloud, in which the user stores large amounts of unstructured data (usually in the form of files).
In both systems, you can create one or more containers of blobs or baskets that will contain zero or more blobs or objects.
Both systems provide a REST API for working with queues and messages and other high-level language libraries, which are usually REST API wrappers. Both systems have a temporary version naming convention - at the time of this writing, the WABS API version is
2011-08-18 , AS3 -
2006-03-01 .
In general, both systems provide similar functions, some of which are listed below:
- Both systems are essentially file systems in a cloud with two levels of hierarchy.
- Both systems allow reliable and cheap storage of large amounts of data.
- Both systems allow you to protect content from unauthorized access.
- Both systems allow you to store multiple versions of the same object, but the versioning mechanism works in each system in its own way.
- Both systems allow you to set the contents of the blob and basket containers through the appropriate data delivery network (CDN) to reduce latency and cache content.
- Both systems provide access control mechanisms for data protection. AS3 has several options ( Amazon Identity and Access Management (IAM), Bucket Policies, ACLs and Query String Authentication ), while WABS provides signature-based protection - ACLs and Shared Access Signatures .
There are, of course, differences:
- WABS only supports HTTP (using REST), AS3 - HTTP (REST, SOAP), and BitTorrent for distributing point-to-point content.
- AS3 has public mechanisms for importing and exporting very large volumes ( Amazon Import / Export ) (when you provide Amazon with a huge amount of data, and the vendor loads the data into the storage itself). This is not yet available in WABS.
- In AS3, you can specify the moment when the object will be self-destruct. This is not yet available in WABS.
- AS3 allows you to bill customers based on their resource consumption using Amazon DevPay . This is a great feature for building SaaS applications, which is not yet available in WABS. Another feature not available in WABS, but available in AS3 is Requester Pay Buckets , where users who access the data stored in your baskets pay for data consumption.
- AS3 allows you to encrypt data using Server Side Encryption (SSE). In WABS, this is not yet available (already available - Trust Services) .
- AS3 supports virtual-hosted-style (for example, http://mybucket.s3.amazon.com/myobject ) and path-style (for example, http://s3-eu-west-1.amazonaws.com/mybucket/ myobject ), whereas WABS only supports path-style (eg http://myaccount.blob.core.windows.net/myblobcontainer/myblob ).
- AS3 offers a Reduced Redundancy Storage (RRS) —clients can choose to store data with a lower level of redundancy (99.99% durability and 99.99% availability) than the standard redundancy level provided by AS3 (99.999999999% durability and 99.99% availability) to reduce costs. I think this is convenient in order to reduce data storage costs if they are not very critical and easily reproducible. WABS provides one level of redundancy.
Concepts
Before we talk more about these two services, I find it important to clarify some concepts. If you are familiar with the basic concepts of WABS and AS3, you can skip this section.
Containers of blobs and baskets : If these services are file systems in the cloud, consider the WABS blob container and the AS3 basket as a folder or directory. In a WABS storage account or an AS3 account, you can have zero or more blob containers and baskets that can contain blobs or objects, respectively.
Comments:
- There is no such thing as a nested container blobov or baskets. Both services provide a two-level hierarchy without nesting. However, both systems allow you to create the illusion of a folder hierarchy using prefixes.
- There are no restrictions on the number of containers and baskets.
- Both systems allow you to place a static website in a container or basket, but AS3 also allows you to define the root and document index and edit 4xx error pages (for example, error page 404).
- Both systems provide the ability to log requests for resources - this function is called “logging” in AS, and “Storage Analytics” in WABS. The difference is that in AS3 logging works at the basket level, in WABS, at the storage account level. In AS3, the logging data is placed in a separate user-defined basket, in WABS, in the predefined tables and containers, which are created automatically when logging is enabled.
Blobs and objects : WABS blobs and AS3 objects are files in your cloud file system, located in blob containers and baskets.
Comments:
- There is no limit on the number of stored blobs and objects, while in AS3 this number is simply unknown, in WABS this number is limited by the size of the storage account (100 TB).
- The maximum size of an object in AS3 is 5 TB, in WABS it is 1 TB.
- WABS has two types of blobs - blocky, convenient for streaming (for example, pictures, video, documents) and having a maximum size of 200 GB, and page ones, convenient for random access / recording operations and having a maximum size of 1 TB. A common case of using a page blob is to mount the VHD as a disk in the role of Windows Azure. There is no such separation in AS3.
- Both systems provide rich functionality for managing blobs and objects. You can copy, download, download and perform other operations.
- Both systems provide the ability to protect content from unauthorized access, and the access control list mechanism is configured in more detail in AS3, where you can create your own ACL for each file in the basket. In WABS, everything happens at the blob container level.
Pricing
When using both systems there are no “capital” costs. The pricing model is relatively simple and based on consumption. In both systems, billing is based on usage and it can consist of three components:
- Number of transactions : Payment is made according to the number of transactions made - roughly speaking, one transaction is one function call in the system. There is a significant difference between the two systems - in WABS, the transaction cost is fixed ($ 0.01 for 10,000 transactions), in AS3 it varies depending on the type of transaction. If you perform PUT, COPY, POST, LIST operations, you pay a higher price per transaction ($ 0.01 per 1000 transactions in the US Standard Region), GET and others pay a lower price ($ 0.01 per 10,000 transactions in the US Standard Region). All deletion requests in AS3 are free.
- Storage : You pay for the amount of data stored in each system. In AS3, the price depends on the level of redundancy.
- Traffic : You pay for the amount of data transferred to and from the system. At the time of writing the post, both systems provide free incoming traffic. Data transferred between ADDB and Amazon EC2 within one region is free. Data transferred between ADDB and Amazon EC2 in different regions are paid according to the tariffs. In WATS only outgoing traffic is paid.
A special pricing model is also available, and both systems provide different payment packages. More about pricing -
https://www.windowsazure.com/en-ru/pricing/details/ for WABS and
http://aws.amazon.com/s3/pricing/ for AS3.
Functions
Let us consider these functions in more detail.
| WABS
| AS3
|
Create Container / PUT Bucket
| Yes
| Yes
|
This function creates a new blob container or basket.
An important point to remember is that the blob containers are limited to the storage account, while the AS3 baskets are limited to the Amazon account. When you create a WABS storage account, you determine its location (data center), and your blob containers are located in a specific data center in a specific geographic location. When you create a cart in AS3, you define the region in which this cart will be created, so you can distribute the cart to all data centers in AS3, if necessary. In order to do the same in WABS, you need to create a storage account in each data center where you want to place the containers.
')
There are several rules for naming blob containers and baskets; they are tabulated below.
| WABS
| AS3
|
Minimum / maximum length title
| 3/63
| 3/63
|
Case sensitivity
| lower case
| lower case
|
Allowed characters
| Alphanumeric, hyphen (-)
| Alphanumeric, hyphen (-) and period (.)
|
More naming rules:
- The names of blob containers should begin with a letter or a number, but not a hyphen, while after the hyphen there should again be a letter or a number, several consecutive hyphens are not allowed.
- AS3 basket names must consist of labels, separated by a dot, where each label must begin and end with a lowercase letter or a number, and the basket name should not look like an IP address (for example, 127.0.0.1).
- If you create a shopping cart in the US Standard, AS3 region, you can use less stringent naming conventions, allowing you to use a cart name from 3 to 255 characters and use mixed case, numbers, periods, hyphens, and underscores.
Notes:
- If the basket location in AS3 is not explicitly indicated, the basket will be created in the US Standard region.
- When creating a container or basket, you can set the ACL (optional), but if it is not specified, then the container or basket becomes private, that is, available only to the owner.
- WABS allows you to define your own metadata for a container, which are collections of key-value values and have a maximum size of 8 KB. This functionality is not available in AS3.
| WABS
| AS3
|
List Containers / GET Service
| Yes
| Yes
|
The function returns a list of all containers of blobs or baskets.
Comments:
- One call to this function in WABS will return a maximum of 5000 containers, and if there are more of them in the storage account, then the continuation token will also be returned. By default, WABS returns up to 5000 containers, but you can specify a smaller number.
- In WABS, you can filter on the server side using the prefix from which the names of the containers to be sampled must begin.
- In WABS, you can specify whether to return the metadata for the blob container along with the list.
| WABS
| AS3
|
Delete Container / DELETE Bucket
| Yes
| Yes
|
The function removes the blob container or cart.
Comments:
- It may look like this operation looks like synchronous, but in reality it is not like that. When you send a request to delete a blob container, it is marked for deletion and becomes unavailable, after which it is deleted during the garbage collection process, so the actual time it takes to remove a container may vary depending on the size of the data in that container. In my experience, deleting a very large container may take hours, and at this time an attempt to create a container with the same name will result in an error (Conflict Error - HTTP 409). In this regard, it is necessary to plan what to do at this time.
- In AS 3, the basket must be empty before being removed. You must first remove all objects from the basket, and then delete it.
| WABS
| AS3
|
List Blobs / GET Bucket (List Objects)
| Yes
| Yes
|
The function is used to get a list of blobs and objects in a container or basket. Functions in systems perform the same thing, given:
- Both functions allow you to limit the resulting sample to the desired number of objects.
- Both functions have the maximum number of objects that they can return in one function call - in WABS it is 5000, and in AS3 it is 1000.
- Both functions support separators, which are a character that groups blobs or objects. The most used separator is /. As mentioned above, both systems maintain a two-level hierarchy, and using a separator can create the illusion of a folder-type hierarchy. For example, you have the following objects: images / a.png, images / b.png, images / c.png, logs / 1.txt, logs / 2.txt, files.txt. When you want to call a function and pass the delimiter / to it, both systems will return the following values: images, logs, files.txt.
- Both functions support server-side filtering using prefixes. When your request contains a prefix, both systems will return objects that have a name that starts with that prefix. Using the example above, if we pass the “images” prefix without separators, both systems will return the following values: images / a.png, images / b.png, images / c.png.
- Both functions can use a token, which is essentially a continuation token, and is used to tell both systems to start getting a list of objects, starting with this token.
- Both systems return objects in alphabetical order.
Differences:
- One function call in WABS will return a maximum of 5000 blobs, AS3 - 1000 objects.
- When receiving the list, you can indicate to WABS that it is also necessary to return the blobs snapshots. In AS3, there is a separate function for this ( GET Bucket Object versions ).
- When retrieving the list, you can specify WABS to return metadata for blobs. In AS3, blob metadata is not supported.
- When receiving the list, you can specify WABS to return the list of blobs that are not yet confirmed (commited), i.e. partially loaded, AS3 can only return objects that are already fully loaded.
| WABS
| AS3
|
Set Blob Service Properties / PUT Bucket logging
| Yes
| Yes
|
Both systems offer logging capabilities, and this feature is not enabled by default. In WABS, this function is called
Storage Analytics and is used for all Windows Azure storage services — tables, queues, blobs.
Differences in logging:
- In WABS, logging is enabled at the blob storage level, while in AS3 it is enabled at the basket level. This means that when you enable logging in WABS, it is enabled for all blob containers in the storage account, but if you enable logging in AS3, it is enabled only for a specific basket. AS3 in this case provides a more "low-level" functionality that allows you to enable logging for specific objects, in addition, you can specify a prefix, and logging is enabled only for those objects whose name begins with this prefix. For example, if you have a basket named data and it contains images and errors objects, and you add another siteimages object, you can enable logging using the siteimages prefix, only for objects of siteimages type.
- In WABS, logs are stored in a system-defined table in the table storage ($ MetricsTransactionsBlob) and blob containers ($ logs), while in AS3 logs are stored in a user-defined basket.
- In WABS, you can store up to 20 TB of logs, which can be stored for up to 365 days, after which they will be deleted (this time can be customized). There are no limitations in AS3.
- In WABS, only the account owner has access to this data, while in AS3 you can implement an access mechanism that will control access and user rights (Full / Read / Write) to this data.
| WABS
| AS3
|
Get Blob Service Properties / GET Bucket logging
| Yes
| Yes
|
The function is used to obtain logging information. In AS3, any authorized user can access this information, and the operation will return information about this user.
| WABS
| AS3
|
Set Container ACL / PUT Bucket acl
| Yes
| Yes
|
The function is used to specify ACLs for containers or baskets, and one or more access policies can also be specified in WABS.
For a blob container, the ACL values can be:
- Full public read access (Container) : The container and blob data can be read anonymously. Clients may receive a list of blobs in a container in an anonymous request, but they cannot receive a list of containers in a storage account.
- Public read access for blobs only: Blob data in a container with such access can be read in an anonymous request, but the container data is no longer available. Clients cannot get a list of blobs inside the container in an anonymous request.
- No public read access (Personal) : Container and blob data can only be obtained by the account owner.
For baskets, ACL values can be equal to:
- READ : Allowed to get a list of items in the cart.
- WRITE : Allowed to create, rewrite and delete objects in the basket.
- READ _ ACP : Recycle ACL allowed.
- WRITE _ ACP : Allow entry to the ACL basket.
- FULL _ CONTROL : With this value, permissions are granted READ, WRITE, READ_ACP, WRITE_ACP.
Convenient in AS3 is that you can give users different sets of permissions, for example, user1 can have READ ACL, user2 - WRITE ACL, in WABS there is no such flexibility, permissions are placed only on the blob container.
Convenient in WABS is that in addition to the ACL, you can set up to 5 container access policies that define a temporary set of permissions for this container. For example, you can create an access policy with permission to write to the blob container, which will only be valid for a day. Using policies allows you to generate a special URL with a signature and give it to users (flexible Shared Access Signatures functionality). Signatures allow you to issue access rights to containers and blobs at a more detailed level for a specific time.
| WABS
| AS3
|
Get Container ACL / GET Bucket acl
| Yes
| Yes
|
The function is used to get the ACL for the blob container or the recycle bin, and in WABS this function also returns the access policies defined for the container.
| WABS
| AS3
|
List Blobs / GET Bucket Object versions
| Yes
| Yes
|
Both systems support versioning - in WABS, versioning of blobs (snapshots), in AS3 - objects, but using different mechanisms for this. In AS3, the function returns only versioned objects; in WABS, versioned objects can also be obtained using List Blobs (see above).
Helpful hint : If you want to get a list of only snapshots or versions of a single object in AS3, you can use this function by passing it the full name of the object as a prefix.
| WABS
| AS3
|
List Blobs / List Multipart Uploads
| Yes
| Yes
|
Both systems support loading blobs and objects in parts. In WABS, when loading blob blocks, you can divide blobs into blocks and load these blocks, then confirm them and make the blob available for use. In AS3, you can load an object, divide it into many parts and download them, then confirm and make the object available for use.
List Multipart Uploads to AS3 tells you which parts of the object are being downloaded, how they are separated and the status of the upload, but it does not indicate which were uploaded and which were not. In WABS, you can implement similar functionality using List Blobs (specifying WABS that, together with confirmed blobs, you must also return and unconfirmed.
| WABS
| AS3
|
Set container metadata
| Yes
| Not
|
The function is used to specify a metadata dictionary in the form of a collection of key-value entries for a blob container in WABS.
Comments:
- The function overwrites existing metadata, so you cannot just update one key-value pair.
- The maximum size of metadata is 8 Kb.
- The metadata name must be a valid C # identifier.
| WABS
| AS3
|
Get Container Metadata
| Yes
| Not
|
The function is used to get the metadata defined for the blob container.
| WABS
| AS3
|
PUT Bucket versioning
| Not
| Yes
|
Versioning allows you to manage multiple copies of an object. This function allows you to enable versioning on objects in the basket, and if one of the operations PUT, POST, COPY, DELETE is performed, a new version of the object will be created. By default, this functionality is disabled.
WABS supports similar “
Snapshot Blob ” functionality, the call of which creates a read-only copy of the current version of the blob.
Differences in versioning in the two systems:
- In AS3, versioning is specified at the basket level and, after versioning is enabled, it is applied to all objects in this basket. In WABS, the Snapshot Blob functionality works at the blob level and you must explicitly call it on the blobs to which you want to enable versioning. In this sense, WABS provides a more flexible versioning system, but it also imposes on the developer the responsibility to create blobs snapshots.
- In WABS, when removing a blob, all its snapshots are deleted. In AS3, this is not the case.
- In WABS, you only pay for snapshot blocks that are different from the blocks in the original blob. In AS3, each version of an object is treated as an object, and payment is made for each of them.
| WABS
| AS3
|
GET Bucket versioning
| Not
| Yes
|
The function is used to get the versioned state (Enabled or Suspended) for the cart in AS3. Returns an empty result if the versioning state has never changed.
| WABS
| AS3
|
PUT Bucket lifecycle
| Not
| Yes
|
In AS3, this feature allows you to specify whether it is necessary to delete objects from the recycle bin automatically after a certain time. For example, you keep logs as objects in a basket, and you want them to be automatically deleted after 30 days.
By default, objects are stored in baskets until they are deleted by the user.
This feature is applied at the folder level. For example, you store access and error logs in the “logs” basket in the corresponding “accesslogs” and “errorlogs” folders. Using this function, you can determine that the objects in the accesslogs folder will be deleted after a certain time (using the prefix).
Cannot be used if you have included versioning on a specific basket at least once.
| WABS
| AS3
|
GET Bucket lifecycle
| Not
| Yes
|
The function returns information about the configuration of the life cycle (see above about automatic deletion).
| WABS
| AS3
|
DELET E Bucket lifecycle
| Not
| Yes
|
The function removes the life cycle configuration associated with the basket.
| WABS
| AS3
|
PUT Bucket policy
| Not
| Yes
|
AS3 provides several mechanisms for protecting baskets and objects from unauthorized access, one of which is basket policy. To do this, you just need to use this function and put the basket policy where appropriate. With the basket policy, the basket owner can:
- Specify any of the basket level permissions.
- Specify access permissions on any items in the cart.
More information about the policy baskets
here .
| WABS
| AS3
|
GET Bucket policy
| Not
| Yes
|
The function returns the policy set on the basket.
| WABS
| AS3
|
DELETE Bucket policy
| Not
| Yes
|
The function deletes the policy set on the basket.
| WABS
| AS3
|
PUT Bucket notification
| Not
| Yes
|
AS3 can send messages to
Amazon Simple Notification Service (SNS) if any important event happens in the basket's life. SNS subscribers can receive these messages, which can be delivered to a web server, e-mail, Amazon Simple Queue Service.
In the current implementation, you can use notifications for only one event: “
s 3: ReducedRedundancyLostObject ”, which occurs when all replicas of an object are lost and, consequently, it is impossible to access this object.
The function is used to enable or disable the notification of changes in the state of the basket.
| WABS
| AS3
|
GET Bucket notification
| Not
| Yes
|
The function returns the basket notification configuration.
| WABS
| AS3
|
PUT Bucket requestPayment
| Not
| Yes
|
In a normal situation, the basket owner pays for downloads from the basket, but AS3 offers to shift this responsibility to another user - the basket owner can configure it so that another account pays for downloading from the basket. This feature is used to, among other things, determine that the person requesting the download will have to pay for this download. Read more
here .
| WABS
| AS3
|
GET Bucket requestPayment
| Not
| Yes
|
The function returns the configuration at the request of payment (see above) for the basket.
| WABS
| AS3
|
PUT Bucket website
| Not
| Yes
|
Another handy feature in AS3 is the ability to place static websites in baskets, which can be done in WABS, but in WABS you cannot define the page that opens by default, the error page and link any domain name to the hosted website (what can be done in AS3). Read more
here .
The function allows you to configure the basket for use as a static website and specify default pages and error pages (for example, for 404 error).
| WABS
| AS3
|
GET Bucket website
| Not
| Yes
|
The function returns the basket configuration as a static web site.
| WABS
| AS3
|
DELETE Bucket website
| Not
| Yes
|
The function removes the recycle bin configuration as a static web site.
The function allows you to determine whether the basket exists and whether the user has permissions to access this basket.
Summary
Conceptually, both services provide similar functionality, but Amazon S3 is richer in function, partly because Amazon had 2+ years more on system development. I very much hope that Windows Azure Blob Storage will catch up with S3 and implement the missing very necessary functions. Personally, I love S3 features like:
- The ability to host static websites.
- Flexible access control.
- Automatically delete content.
- More "rich" versioning - even if the source is deleted, its copies remain.
- Something similar to Reduced Redundant Storage.
However, it would be great to see features such as:
- Ability to delete the cart without deleting objects stored in it. I think this imposes a big overhead.
- Combining some functions. For example, Get Bucket, Get Bucket versioning, and List Multipart Uploads could be easily combined into one function.
Note from the translator:As far as I know, a separate website has been created for Windows Azure, called
http://www.mygreatwindowsazureidea.com , where users vote to develop the features they need, and Microsoft really responds to user requests. Unfortunately, for the Amazon platform, I did not find such a platform, which is not a good practice - in the case of cloud platforms, which are gigantic software packages, it is very important to focus on users. Amazon , . – ( ) Amazon . , , .
UPD: Trust Services Windows Azure: —
.