Comparing Windows Azure Blob Storage and Amazon Simple Storage Service (S3) - Part II, Summary
The second and third parts of the comparison of the Windows Azure and Amazon storage services. The last part remains - a comparison of Windows Azure Blob Storage and Google Cloud Storage.
In the first part of this article, we began a comparison of Windows Azure Blob Storage and Amazon Simple Storage Service (S3), looked at the basic concepts, and compared pricing and functionality related to blob containers and baskets. Read the first part .
In this part of the article, we will compare the functionality related to blobs and objects, and close the comparison of the storage services of both systems, summarizing the resulting data into one common table.
The first part of the cycle - Comparing Windows Azure Table Storage and Amazon DynamoDB
The second part of the cycle - Comparing Windows Azure Blob Storage and Amazon Simple Storage Service (S3) - Part I
Abbreviations: Windows Azure Blob Storage - WABS and Amazon Simple Storage Service - AS3 .
')
Before a detailed discussion of the functions, I find it important to clarify some of the concepts behind the blobs and objects.
Blobs and objects : WABS blobs and AS3 objects are files in your cloud file system, located in blob containers and baskets.
Comments:
The two most important functions are upload and download, so I propose to discuss them first, then compare the other functions.
Let's talk about loading blobs and objects into containers and baskets. There are two loading mechanisms - you can load a blob or an object entirely within a single request or divide them into pieces (blocks or WABS pages and parts in AS3).
If the download data is small and you have a good connection speed, you can download this data completely in one request. In WABS, Put Blob is used for this. In AS3, Put Object .
You can share large data that is not efficiently loaded in a single query completely. Both systems allow you to split data into chunks (blocks or pages in WABS and parts in AS3) and load gradually. In WABS, you need to use Put Block and Put Block List for block blobs and Put Page for page blobs. In AS3, the Initiate Multipart Upload , Upload Part and Complete Multipart Upload or Abort Multipart Upload functions are used for this.
There are many reasons why you can decide to load data in chunks:
Now for how to load the data in chunks on each system. For example, you want to download a 100 MB file with chunks. Let's see what needs to be done when using each of the platforms.
Suppose each piece has a size of 1 MB (despite the fact that there is no need to have pieces of the same size) and you need to complete the download of 100 pieces. We take a block blob, each of the blocks (pieces) of which has a unique identifier (BlockId). To load it, use the Put Block function. BlockId is a Base64 encrypted string, the maximum size of which is limited to 64 bytes. All BlockId (100 in our case) must be the same length. In this case, it does not matter in what order you will load the blocks - you can load them in parallel. After loading the block, WABS puts it somewhere in storage and stores it for 7 days. After loading all the blocks, we call Put Block List , commit (commit) these blocks. Until the call of this function, the blob cannot be accessed and, if you have not confirmed the blocks within 7 days, they will be deleted by the system. After calling the function based on the order of the BlockId list, WABS will recreate the blob and mark it as available. It makes no difference what values ​​BlockId will have (all of them can be GUIDs), but it is important in which order you send the BlockId when using the Put Block List.
Limitations:
Suppose the size of each piece is 5 MB. Each piece is assigned a unique identifier (Part Number), which identifies the piece and determines its position in the source object. This number can be any number from 1 to 10,000, but they must be sorted, that is, if we take our situation, then a 0-5MB piece should have the number 1, a 5-10MB piece - 2, and so on. But when you start uploading these parts, you first need to call the Initiate Multipart Upload function , which tells AS3 that the object transfer process has begun. The function will return Upload Id. Next, for each piece, you must call the Upload Part function and transfer the Upload Id and the number of the piece, and it does not matter in what order the pieces will be loaded (can be loaded in parallel). After downloading all the pieces, you need to call the Complete Multipart Upload function, thus confirming the pieces. To cancel the process, you can call Abort Multipart Upload .
Limitations:
There are two mechanisms for downloading blobs and objects - either downloading a blob or an object entirely in one request or in chunks.
Each system has only one download function — Get Blob in WABS and Get Object in AS3.
If the data is small and you have a good connection speed, you can download the entire object using Get Blob in WABS and Get Object in AS3.
If the object is large and you are not sure whether you can download it at one time, you can download the pieces using the same function with the addition of the Range header and defining the range of bytes needed for downloading.
Download process:
The functions provided by both systems are tabulated below.
Let's discuss these functions in more detail.
The function adds a blob to the blob container and an object to the basket.
Comments:
The function adds an object to the specified basket using an HTML form. POST is an alternative to PUT and allows you to implement the download object with a browser. Parameters transmitted by PUT using HTTP headers are transmitted in the case of POST as the body of an encrypted multipart / form-data message.
The function allows you to download blob from a container or basket.
Comments:
AS3 allows you to use objects using BitTorrent, which can also reduce the amount of traffic transferred. Read more about using BitTorrent and AS3 - here .
The function can only be called on objects that are less than 5 GB in size.
In AS3, you can specify ACLs at the level of an individual object (in WABS, ACLs can be specified only at the level of the blob container). This function is used to obtain information about the ACL specified for the object.
An interesting point is that if you use a different version of an object, each version will have a separate ACL. To obtain an ACL for a specific version of an object, you must pass Version Id version functions.
The function allows you to specify the ACL for the object. As stated above, each version of an object has its own ACL. To specify an ACL for a specific version of a function object, you must transfer the Version Id of this version.
The function is used to get the blob properties and object metadata, but does not return the contents of the blob.
Comments:
The function determines the blob's system properties in WABS. Not available in AS3. When creating an object, you can specify only system properties.
Properties available for definition include: Cache Control, Content Type, Content MD5, Content Encoding, Content Language. Please note that you do not have the right to redefine these properties for snapshot blob.
WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
The function returns user-defined metadata. This function can be used to get the properties of a particular version of a blob or object. To obtain this information, you must specify the date / time of the blob snapshot in WABS and Version Id of the required version in AS3. If these parameters are omitted, the current version information is returned.
The function is used to specify a metadata dictionary in the form of a collection of key-value entries for blobs in WABS.
Comments:
The function removes a blob or object from the repository.
Comments:
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
This function allows you to delete several objects from the basket in one HTTP request — to do this you need to know the keys of the objects to be deleted. You can also delete specific versions of one or more objects. Up to 1000 objects can be deleted in one request.
The function copies the blob to somewhere from the original location.
Comments:
· Both systems allow you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ). These conditions can be defined both on the source and on the final copy in WABS and on the source in AS3.
· WABS allows you to copy objects from a container to a container only within one storage account. There is no such limitation in AS3. If the baskets between which the exchange takes place belong to the same account, the object will be copied. However, if you created an object using the API for loading in chunks, you cannot copy the object from region to region.
· Both systems allow you to copy existing metadata or specify metadata for the final copy.
· AS3 implements this function as a combination of GET Object and PUT Object .
Useful tips:
The function creates a read-only copy of the blob - this is how WABS versioning is implemented. As I mentioned in previous articles, AS3 implements versioning itself. In WABS, versioning is implemented at the level of a particular blob, and the versioning of this blob goes into the responsibility of the developer.
Comments:
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
The function allows you to put a one-minute lock on the blob so that it cannot be modified. The function is very convenient in a situation with many workers trying to perform the same operation, but you want only one of them to make the change. Learn more: http://blog.smarx.com/posts/managing-concurrency-in-windows-azure-with-leases .
The function is used to initiate data loading in chunks.
The function is used to load pieces (blocks in WABS and parts in AS3) data.
The function is used to confirm blobs or objects in the corresponding storage. The function can be called after loading all blocks or parts.
The function returns the list of loaded blocks or parts.
Comments:
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
The function is used to cancel the ongoing boot process.
In the form in which this functionality is implemented in WABS, you do not need it. In WABS, you cannot interrupt the boot process. If you started downloading blobs in chunks and did not call up the Put Block List within 7 days, the loaded blocks will be deleted.
If the object you are trying to copy is larger than 5 GB, you will not be able to use the Put Object - Copy function. You will have to copy this object using the download pieces, and this function is intended just for this. The difference between this function and the Upload Part is that instead of sending the data in the request body, you specify the source object in the format source_ bucket / source_ object and the range of bytes to be copied. Thus, you need to deal with all parts of the copied object. Before calling this function, you need to initiate the download in chunks and get the Upload Id, which you need to transfer to the function. To complete copying you need to call “Complete Multipart Upload” or “Abort Multipart Upload”.
When you create a page blob using Put Blob, you only initiate its creation, i.e. create an empty page blob with no content. Using this function, you can insert / update data into a page blob, and you do not need to confirm the data - after calling this function, they will be automatically confirmed.
Comments:
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
An important difference between block and page blob is pricing - you pay for a block blob in full, but in the case of a page blob, only for busy (non-zero) pages. For example, if you created a block blob of 2 GB and a page blob of 2 GB and both of them are empty (contain 0 bytes), you will pay for 2 GB of the block blob, but the page blob will not be charged. Then you write 1024 bytes into a page blob using Put Page and start paying for those 1024 bytes.
This function returns a sorted list of non-zero pages.
This function has another advantage - if you call this function when downloading a page blob, you will be guaranteed to know where your data is stored in the blob, after which you can download not the entire blob, but only the occupied pages using the Range header in the function Get Blob.
Both systems provide a similar set of functions. Each system has its own set of functions, which is absent in another system, but in general the differences in functionality are not the same as in the case of containers of blobs and baskets.
Detailed articles comparing Windows Azure Blob Storage and Amazon Simple Storage Service (S3 can be read here and here . In this article I will summarize.
Abbreviations: Windows Azure Blob Storage - WABS and Amazon Simple Storage Service - AS3 .
The table summarizes the comparison of the functions of WABS and AS3.
Note from the translator
Despite the youth (in relation to the competitor) of the Windows Azure platform, Microsoft did their best - the relative equality in functionality between the two platforms remains, and this is good news. I am very interested to observe the development of both platforms, and I think that it will be even more interesting further. By the way, on June 7, Microsoft will roll out global updates - do not miss, it will be very cool, as I say, familiar with innovations! It seems to me that the competition of cloud platforms is gaining a new round, and then everything will depend on how true Amazon and Microsoft will be chosen. Wait and see.The following translation will be devoted to the comparison of Windows Azure Blob Services and the Google Cloud Service.
UPD: missed the appearance of Trust Services for Windows Azure: now there is the possibility of encryption on the server side - more .
In the first part of this article, we began a comparison of Windows Azure Blob Storage and Amazon Simple Storage Service (S3), looked at the basic concepts, and compared pricing and functionality related to blob containers and baskets. Read the first part .
In this part of the article, we will compare the functionality related to blobs and objects, and close the comparison of the storage services of both systems, summarizing the resulting data into one common table.
The first part of the cycle - Comparing Windows Azure Table Storage and Amazon DynamoDB
The second part of the cycle - Comparing Windows Azure Blob Storage and Amazon Simple Storage Service (S3) - Part I
Abbreviations: Windows Azure Blob Storage - WABS and Amazon Simple Storage Service - AS3 .
')
Concepts
Before a detailed discussion of the functions, I find it important to clarify some of the concepts behind the blobs and objects.
Blobs and objects : WABS blobs and AS3 objects are files in your cloud file system, located in blob containers and baskets.
Comments:
- There is no limit on the number of stored blobs and objects, while in AS3 this number is simply unknown, in WABS this number is limited by the size of the storage account (100 TB).
- The maximum size of an object in AS3 is 5 TB, in WABS it is 1 TB.
- WABS has two types of blobs - blocky, convenient for streaming (for example, pictures, video, documents) and having a maximum size of 200 GB, and page ones, convenient for random access / recording operations and having a maximum size of 1 TB. A common case of using a page blob is to mount the VHD as a disk in the role of Windows Azure. There is no such separation in AS3.
- Both systems provide rich functionality for managing blobs and objects. You can copy, download, download and perform other operations.
- Both systems provide the ability to protect content from unauthorized access, and the access control list mechanism is configured in more detail in AS3, where you can create your own ACL for each file in the basket. In WABS, everything happens at the blob container level.
The two most important functions are upload and download, so I propose to discuss them first, then compare the other functions.
Loading blobs and objects
Let's talk about loading blobs and objects into containers and baskets. There are two loading mechanisms - you can load a blob or an object entirely within a single request or divide them into pieces (blocks or WABS pages and parts in AS3).
Loading in one request
If the download data is small and you have a good connection speed, you can download this data completely in one request. In WABS, Put Blob is used for this. In AS3, Put Object .
Loading chunks
You can share large data that is not efficiently loaded in a single query completely. Both systems allow you to split data into chunks (blocks or pages in WABS and parts in AS3) and load gradually. In WABS, you need to use Put Block and Put Block List for block blobs and Put Page for page blobs. In AS3, the Initiate Multipart Upload , Upload Part and Complete Multipart Upload or Abort Multipart Upload functions are used for this.
There are many reasons why you can decide to load data in chunks:
- It is necessary to download very large data. Please note that in WABS one block blob is limited to 200 GB, the page blob is 1 TB, and in AS3 you can have one object up to 5 TB. Such volumes are impractical to load in one request.
- Low connection speed.
- Both systems are cloud services designed to process the requests of hundreds and thousands of users at the same time, and both systems will limit your requests if they run longer than the set limit - in WABS it is 10 minutes to download 1 MB of data.
- Splitting large data into chunks allows parallel loading (respectively, loading data more quickly).
- In the case of a broken piece of loading, you can repeat its loading, if the loading of big data that loads in one request stops, you will have to download everything again, which is very inefficient.
- System Restrictions: WABS does not allow downloading data in one request if its size exceeds 64 MB. There is no such restriction in AS3, but the company recommends dividing the data into pieces if their size exceeds 100 MB.
Now for how to load the data in chunks on each system. For example, you want to download a 100 MB file with chunks. Let's see what needs to be done when using each of the platforms.
WABS
Suppose each piece has a size of 1 MB (despite the fact that there is no need to have pieces of the same size) and you need to complete the download of 100 pieces. We take a block blob, each of the blocks (pieces) of which has a unique identifier (BlockId). To load it, use the Put Block function. BlockId is a Base64 encrypted string, the maximum size of which is limited to 64 bytes. All BlockId (100 in our case) must be the same length. In this case, it does not matter in what order you will load the blocks - you can load them in parallel. After loading the block, WABS puts it somewhere in storage and stores it for 7 days. After loading all the blocks, we call Put Block List , commit (commit) these blocks. Until the call of this function, the blob cannot be accessed and, if you have not confirmed the blocks within 7 days, they will be deleted by the system. After calling the function based on the order of the BlockId list, WABS will recreate the blob and mark it as available. It makes no difference what values ​​BlockId will have (all of them can be GUIDs), but it is important in which order you send the BlockId when using the Put Block List.
Limitations:
- Blob can be divided into a maximum of 50,000 blocks.
- A blob can have a maximum of 100,000 unconfirmed blocks at any given time.
- A set of unconfirmed blocks cannot have a size larger than 400 GB.
- All BlockId blocks of one blob must be of the same length, i.e. the situation when their values ​​will be equal block8, block9, block11 is unacceptable.
- The maximum length of the BlockId is 64 bytes.
AS3
Suppose the size of each piece is 5 MB. Each piece is assigned a unique identifier (Part Number), which identifies the piece and determines its position in the source object. This number can be any number from 1 to 10,000, but they must be sorted, that is, if we take our situation, then a 0-5MB piece should have the number 1, a 5-10MB piece - 2, and so on. But when you start uploading these parts, you first need to call the Initiate Multipart Upload function , which tells AS3 that the object transfer process has begun. The function will return Upload Id. Next, for each piece, you must call the Upload Part function and transfer the Upload Id and the number of the piece, and it does not matter in what order the pieces will be loaded (can be loaded in parallel). After downloading all the pieces, you need to call the Complete Multipart Upload function, thus confirming the pieces. To cancel the process, you can call Abort Multipart Upload .
Limitations:
- Judging by the fact that the number of the piece should be from 1 to 10,000, I assume that the object can be divided into a maximum of 10,000 parts.
- Since the number determines the position of the piece in the object, its value is important.
- Each piece must be at least 5 MB in size, with the exception of the last piece in a sequence for which there are no size restrictions.
Download blobs and objects
There are two mechanisms for downloading blobs and objects - either downloading a blob or an object entirely in one request or in chunks.
Each system has only one download function — Get Blob in WABS and Get Object in AS3.
Download in one request
If the data is small and you have a good connection speed, you can download the entire object using Get Blob in WABS and Get Object in AS3.
Download in chunks
If the object is large and you are not sure whether you can download it at one time, you can download the pieces using the same function with the addition of the Range header and defining the range of bytes needed for downloading.
Download process:
- Determine the size of the object. For example, it "weighs" 100 MB.
- Determine the size of the pieces. For example, you are comfortable to download pieces of 1 MB.
- Call Get Blob or Get Object and pass them the appropriate values ​​in the Range header. If you download consistently, your first request will have the value of this header “0 - ​​1048575” (0 - 1 Mb), the second request - “1048576 - 2097151” (1 - 2 Mb) and so on.
- After downloading put a piece somewhere.
- After downloading all the pieces, create an empty file of 100 MB in size and fill this file with the downloaded pieces.
Functions
The functions provided by both systems are tabulated below.
Let's discuss these functions in more detail.
| WABS | AS3 | |
| Put Blob / PUT Object | Yes | Yes |
The function adds a blob to the blob container and an object to the basket.
Comments:
- In both systems, the function will overwrite the existing object with the specified name. In AS3, with versioning enabled on the basket, a copy of the object will be created. In WABS, you need to handle versioning on your own using Snapshot Blob .
- Both systems allow you to define properties for objects (cache control, content type, etc.)
- Both systems allow you to send an MD5 hash of content to check the consistency of the data.
- In AS3, when creating an object, you can set an ACL on it, which cannot be done in WABS.
- AS3 allows you to store objects with limited redundancy. When you call this function, you specify in an optional parameter whether to store an object with limited redundancy. If the parameter is omitted, the object is saved with standard redundancy.
- Both systems allow you to specify metadata for blobs and objects in the form of a collection of key-value pairs. In both systems, the maximum size of this metadata is 8 Kb.
- When creating a page blob using this function, you only initiate a page blob, but do not put data into it. To insert data, you must use the Put Page function.
- When creating a block blob or object in AS3, the data is sent in the request.
- The maximum size of a block blob created using this function is 64 MB. If the size is larger, the blob must be divided into blocks and loaded using Put Block and Put Block List.
- WABS allows you to define preconditions that must be met to successfully complete this function ( If - Modified - Since , If - Unmodified - Since , If - Match , If - None - Match ).
- AS3 supports encryption of objects on the server side, respectively, you can determine whether the object should be encrypted when prompted. WABS does not support server-side encryption, so you need to encrypt objects yourself.
| WABS | AS3 | |
| POST Object | Not | Yes |
The function adds an object to the specified basket using an HTML form. POST is an alternative to PUT and allows you to implement the download object with a browser. Parameters transmitted by PUT using HTTP headers are transmitted in the case of POST as the body of an encrypted multipart / form-data message.
| WABS | AS3 | |
| Get Blob / GET Object | Yes | Yes |
The function allows you to download blob from a container or basket.
Comments:
- You can download pieces by specifying the number of bytes in the Range header.
- In WABS, the function also returns metadata for the blob to be downloaded.
- WABS allows you to define preconditions that must be met to successfully complete this function ( If - Modified - Since , If - Unmodified - Since , If - Match , If - None - Match ).
- In AS3, you can override the values ​​of specific response headers using query parameters (except anonymous queries). The request must be signed with the Authorization header or use the written URL.
- You can use this function to get blob and object versions — to get a blob version, you need to specify the blob snapshot date / time, to get the version of an object in AS3, specify Version Id. If these parameters are omitted, the current version of the object is returned.
| WABS | AS3 | |
| GET Object torrent | Not | Yes |
AS3 allows you to use objects using BitTorrent, which can also reduce the amount of traffic transferred. Read more about using BitTorrent and AS3 - here .
The function can only be called on objects that are less than 5 GB in size.
| WABS | AS3 | |
| GET Object ACL | Not | Yes |
In AS3, you can specify ACLs at the level of an individual object (in WABS, ACLs can be specified only at the level of the blob container). This function is used to obtain information about the ACL specified for the object.
An interesting point is that if you use a different version of an object, each version will have a separate ACL. To obtain an ACL for a specific version of an object, you must pass Version Id version functions.
| WABS | AS3 | |
| PUT Object ACL | Not | Yes |
The function allows you to specify the ACL for the object. As stated above, each version of an object has its own ACL. To specify an ACL for a specific version of a function object, you must transfer the Version Id of this version.
| WABS | AS3 | |
| Get Blob Properties / HEAD Object | Yes | Yes |
The function is used to get the blob properties and object metadata, but does not return the contents of the blob.
Comments:
- Get Blob Properties in WABS returns a set of user-defined metadata, standard HTTP properties and system properties for a blob, while HEAD Object in AS3 returns only object metadata.
- WABS allows you to define preconditions that must be met to successfully complete this function ( If - Modified - Since , If - Unmodified - Since , If - Match , If - None - Match ).
- This function can be used to get the properties of a particular version of a blob or object. To obtain this information, you must specify the date / time of the blob snapshot in WABS and Version Id of the required version in AS3. If these parameters are omitted, the current version information is returned.
| WABS | AS3 | |
| Set Blob Properties | Yes | Not |
The function determines the blob's system properties in WABS. Not available in AS3. When creating an object, you can specify only system properties.
Properties available for definition include: Cache Control, Content Type, Content MD5, Content Encoding, Content Language. Please note that you do not have the right to redefine these properties for snapshot blob.
WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
| WABS | AS3 | |
| Get Blob Metadata / Head Object | Yes | Yes |
The function returns user-defined metadata. This function can be used to get the properties of a particular version of a blob or object. To obtain this information, you must specify the date / time of the blob snapshot in WABS and Version Id of the required version in AS3. If these parameters are omitted, the current version information is returned.
| WABS | AS3 | |
| Set blob metadata | Yes | Not |
The function is used to specify a metadata dictionary in the form of a collection of key-value entries for blobs in WABS.
Comments:
- The function overwrites existing metadata, so you cannot just update one key-value pair.
- The maximum size of metadata is 8 Kb.
- The metadata name must be a valid C # identifier.
| WABS | AS3 | |
| Delete Blob / DELETE Object | Yes | Yes |
The function removes a blob or object from the repository.
Comments:
- In AS3, the function deletes the null version (if any) of the object and inserts a marker for deletion, which then becomes the version of the object. If there was no null version, none of the objects are deleted.
- In WABS, you can use this function to remove only snapshots without deleting the source. If the source is deleted, all its snapshots are also deleted.
- As I understand it, in AS3, when an object is deleted, all its versions remain intact, which seems to me to be quite convenient protection against accidental deletion operations.
- This function can be used to delete specific versions of a blob or an object — to do this, specify the date / time of the blob snapshot in WABS and Version Id of the required version in AS3.
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
- In AS3, Multi Factor Authentification (MFA) Delete can be used, which provides an additional level of security. Read more about the MFA - here .
| WABS | AS3 | |
| Delete Multiple Objects | Not | Yes |
This function allows you to delete several objects from the basket in one HTTP request — to do this you need to know the keys of the objects to be deleted. You can also delete specific versions of one or more objects. Up to 1000 objects can be deleted in one request.
| WABS | AS3 | |
| Copy Blob / Put Object - Copy | Yes | Yes |
The function copies the blob to somewhere from the original location.
Comments:
· Both systems allow you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ). These conditions can be defined both on the source and on the final copy in WABS and on the source in AS3.
· WABS allows you to copy objects from a container to a container only within one storage account. There is no such limitation in AS3. If the baskets between which the exchange takes place belong to the same account, the object will be copied. However, if you created an object using the API for loading in chunks, you cannot copy the object from region to region.
· Both systems allow you to copy existing metadata or specify metadata for the final copy.
· AS3 implements this function as a combination of GET Object and PUT Object .
- You can use this function with objects no more than 5 GB in size. If the size exceeds 5 GB, you need to make the download pieces.
- In AS3, when copying, a certain ACL is deleted and the permission as private is set on the object.
Useful tips:
- Both systems do not support object renaming. Renaming an object can be done by first copying the object and then deleting it.
- You can also “upgrade” the version of the blob or object to the “current” version. To do this, you need to specify both the source for copying a versioned blob (pointing to its snapshot) or an object (indicating its Version Id) and the final copy as a unversioned blob or object.
| WABS | AS3 | |
| Snapshot blob | Yes | Not |
The function creates a read-only copy of the blob - this is how WABS versioning is implemented. As I mentioned in previous articles, AS3 implements versioning itself. In WABS, versioning is implemented at the level of a particular blob, and the versioning of this blob goes into the responsibility of the developer.
Comments:
- The function creates a read-only copy of the blob - snapshot. Snapshots can be downloaded, copied or deleted, but not modified.
- Unlike AS3, where a version of an object is assigned a Version Id, each snapshot is assigned a date / time value that uniquely identifies the given snapshot.
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
- You can “enhance” the snapshot to the source blob version using the Copy Blob function.
| WABS | AS3 | |
| Lease blob | Yes | Not |
The function allows you to put a one-minute lock on the blob so that it cannot be modified. The function is very convenient in a situation with many workers trying to perform the same operation, but you want only one of them to make the change. Learn more: http://blog.smarx.com/posts/managing-concurrency-in-windows-azure-with-leases .
| WABS | AS3 | |
| Initiate Multipart Upload | Not | Yes |
The function is used to initiate data loading in chunks.
| WABS | AS3 | |
| Put Block / Upload Part | Yes | Yes |
The function is used to load pieces (blocks in WABS and parts in AS3) data.
| WABS | AS3 | |
| Put Block List / Complete Multipart Upload | Yes | Yes |
The function is used to confirm blobs or objects in the corresponding storage. The function can be called after loading all blocks or parts.
| WABS | AS3 | |
| Get Block List / List Parts | Yes | Yes |
The function returns the list of loaded blocks or parts.
Comments:
- WABS has two blob block lists — a list of successfully confirmed blocks and a list of unacknowledged blocks that were loaded but not confirmed.
- In one request, AS3 will return up to 1000 part numbers. If there are still parts left, AS3 will return continuation token. You can specify how many numbers will be returned.
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
| WABS | AS3 | |
| Abort Multipart Upload | Not | Yes |
The function is used to cancel the ongoing boot process.
In the form in which this functionality is implemented in WABS, you do not need it. In WABS, you cannot interrupt the boot process. If you started downloading blobs in chunks and did not call up the Put Block List within 7 days, the loaded blocks will be deleted.
| WABS | AS3 | |
| Upload Part - Copy | Not | Yes |
If the object you are trying to copy is larger than 5 GB, you will not be able to use the Put Object - Copy function. You will have to copy this object using the download pieces, and this function is intended just for this. The difference between this function and the Upload Part is that instead of sending the data in the request body, you specify the source object in the format source_ bucket / source_ object and the range of bytes to be copied. Thus, you need to deal with all parts of the copied object. Before calling this function, you need to initiate the download in chunks and get the Upload Id, which you need to transfer to the function. To complete copying you need to call “Complete Multipart Upload” or “Abort Multipart Upload”.
| WABS | AS3 | |
| Put page | Yes | Not |
When you create a page blob using Put Blob, you only initiate its creation, i.e. create an empty page blob with no content. Using this function, you can insert / update data into a page blob, and you do not need to confirm the data - after calling this function, they will be automatically confirmed.
Comments:
- To place data in a blob, you need to specify the value of the byte range header “Byte Range”. The initial range shift must be an absolute value of 512-1. Examples of allowed byte ranges are 0-511, 512-1023, and so on.
- With this function, you can either write data to a page blob, or clear a specific range of bytes from the data.
· WABS allows you to define preconditions that must be met in order to successfully complete this function ( If- Modified- Since , If- Unmodified- Since , If- Match , If- None- Match ).
| WABS | AS3 | |
| Get page ranges | Yes | Not |
An important difference between block and page blob is pricing - you pay for a block blob in full, but in the case of a page blob, only for busy (non-zero) pages. For example, if you created a block blob of 2 GB and a page blob of 2 GB and both of them are empty (contain 0 bytes), you will pay for 2 GB of the block blob, but the page blob will not be charged. Then you write 1024 bytes into a page blob using Put Page and start paying for those 1024 bytes.
This function returns a sorted list of non-zero pages.
This function has another advantage - if you call this function when downloading a page blob, you will be guaranteed to know where your data is stored in the blob, after which you can download not the entire blob, but only the occupied pages using the Range header in the function Get Blob.
Summary
Both systems provide a similar set of functions. Each system has its own set of functions, which is absent in another system, but in general the differences in functionality are not the same as in the case of containers of blobs and baskets.
Comparison of Windows Azure Blob Storage and Amazon Simple Storage Service (S3) - Summary
Detailed articles comparing Windows Azure Blob Storage and Amazon Simple Storage Service (S3 can be read here and here . In this article I will summarize.
Abbreviations: Windows Azure Blob Storage - WABS and Amazon Simple Storage Service - AS3 .
The table summarizes the comparison of the functions of WABS and AS3.
| WABS | AS3 | |
| The current version of the service | 2011-08-18 | 2006-03-01 |
| Storage Restriction | Up to 100TB | Not limited |
| Cloud file system | Yes | Yes |
| Ability to transfer discs to transfer huge amounts of data | Not | Yes |
| Supported Protocols | HTTP / HTTPS | HTTP / HTTPS / BitTorrent |
| The ability to bill customers for the data they consume | Not | Yes |
| Server side encryption support | Not | Yes |
| Limited redundancy support | Not | Yes |
| Supported hierarchy level | 2 | 2 |
| Blob containers and baskets | ||
| Create blob containers and baskets | Yes | Yes |
| The number of containers blobov and baskets | Not limited | Not limited |
| Minimum / maximum length of the name of the basket or blob container | 3/63 | 3/63 |
| Case Sensitivity in Blob and Basket Container Names | lower case | lower case |
| Characters allowed in blob and basket containers | Alphanumeric, hyphen (-) | Alphanumeric, hyphen (-), dot (.) |
| Virtual hosting support | Not | Yes |
| Path-style hosting support | Yes | Yes |
| The ability to determine the ACL at the time of creation | Yes | Yes |
| Default ACL | Private | Private |
| Specifying your own metadata | Yes | Not |
| Getting a list of blob containers and baskets | Yes | Yes |
| The maximum number returned by a single function call of blob containers or baskets | 5000 | Not specified |
| Returning a part of the list of blob and basket containers using prefix filtering | Yes | Yes |
| Removing Blob Containers and Baskets | Yes | Yes |
| Container blobov or basket must be empty before removal | Not | Yes |
| Getting a list of blobs or objects | Yes | Yes |
| The maximum number of blobs or objects returned by a single function call | 5000 | 1000 |
| The ability to specify the upper limit on the number of returned blobs or objects in a single function call | Yes | Yes |
| Separator support for creating an illusion of folder hierarchy | Yes | Yes |
| Filtering a return list of blobs or objects by prefix | Yes | Yes |
| Get a list of versioned blobs or objects | Yes | Yes |
| Getting a list of unapproved blobs or objects | Yes | Yes |
| Logging requests to blobs or objects | Yes | Yes |
| Logging requests to specific blobs or objects | Not | Yes |
| Getting a logging configuration for requests to blobs or objects | Yes | Yes |
| Determining ACL on blob container or cart | Yes | Yes |
| Possible ACL Values | Container, Blob, Private | READ, WRITE, READ_ACP, WRITE_ACP, FULL_CONTROL |
| Defining different ACLs for different users | Not | Yes |
| Defining custom metadata for a blob or bucket container | Yes | Not |
| Getting user-defined metadata for container blobs or buckets | Yes | Not |
| Object versioning | Yes | Yes |
| Versioning is controlled by the system. | Not | Yes |
| Setting up versioning for blob or basket containers | Not | Yes |
| Getting configuration of blob container or basket versioning | Not | Yes |
| Automatic removal of the contents of the blob container or basket after a certain period of time | Not | Yes |
| Set up automatic deletion of the contents of the blob container or basket after a certain period of time | Not | Yes |
| Get settings for automatic removal of the contents of the blob container or the basket after a certain period of time | Not | Yes |
| Delete settings for automatically deleting the contents of a blob container or basket after a certain period of time | Not | Yes |
| Defining access policies on a blob container or basket to prevent unauthorized access | Not | Yes |
| Getting access policies on a blob container or basket to prevent unauthorized access | Not | Yes |
| Not | Yes | |
| Not | Yes | |
| Not | Yes | |
| Not | Yes | |
| Not | Yes | |
| , | Not | Yes |
| , | Not | Yes |
| Not | Yes | |
| Not | Yes | |
| Not | Yes | |
| Not | Yes | |
| 200 / 1 | 5 | |
| , | - | |
| Yes | Yes | |
| , | 64 | |
| Yes | Yes | |
| 50000 | 10,000 | |
| Not | 5 | |
| Yes | Yes | |
| ACL | N / A | Yes |
| , , | N / A | Yes |
| Yes | Not | |
| HTML- POST | Not | Yes |
| Yes | Yes | |
| Yes | Yes | |
| Not | Yes | |
| Torrent | Not | Yes |
| ACL | Not | Yes |
| ACL | Not | Yes |
| Yes | Yes | |
| Yes | Not | |
| Yes | Yes | |
| Yes | Not | |
| Yes | Yes | |
| Multi Factor Authentication (MFA) delete | Not | Yes |
| Yes | Not | |
| Not | Yes | |
| Yes | Yes | |
| Yes | Yes | |
| read-only | Yes | Not |
| Getting an exclusive lock on a blob or object | Yes | Not |
Note from the translator
Despite the youth (in relation to the competitor) of the Windows Azure platform, Microsoft did their best - the relative equality in functionality between the two platforms remains, and this is good news. I am very interested to observe the development of both platforms, and I think that it will be even more interesting further. By the way, on June 7, Microsoft will roll out global updates - do not miss, it will be very cool, as I say, familiar with innovations! It seems to me that the competition of cloud platforms is gaining a new round, and then everything will depend on how true Amazon and Microsoft will be chosen. Wait and see.The following translation will be devoted to the comparison of Windows Azure Blob Services and the Google Cloud Service.
UPD: missed the appearance of Trust Services for Windows Azure: now there is the possibility of encryption on the server side - more .
Source: https://habr.com/ru/post/145122/
All Articles