We are testing cloud platforms from Top-3
Hi, Habr! I want to share my results of comparison of AWS, Azure and Google cloud platforms. Under the cut - the results of the performance, which gets a regular user, and their comparison on the criterion of "price / quality"
To begin with, I introduced such an indicator as the number of points obtained by the testing tools divided by the cost of this virtual machine per hour. The following instances were used to compare the performance of virtual machines:
* Current price for promotional positions, in brackets the regular price.
** Price with a 30% discount provided the virtual machine is used 24/7 per month.
update : after the publication of the article, there was news about OS of smaller disks in Azure, which allows to save about $ 2.18 more per month. This does not change the results in any way, but it allows saving about 0.8% of the price of a virtual machine (Standard D3 v2).
')
All instances have been “raised” in Europe - Ireland \ Germany \ Belgium. Creation took place through the corresponding portals of cloud providers. For instances, SSDs of various configurations were used:
In order for the tests to be as objective as possible, I didn’t set up virtual machines. All instances use Windows Server 2016, and for the last test, Ubuntu 04/16/02 LTS. I also refused any OS settings and had to install GeekBench for testing. I used trial versions from the manufacturer's sites. Using apt-get for Ubuntu was installed "benchmark" Hardinfo. These are the only settings made with the operating system before running the tests.
On all virtual machines, I consistently “chased” the following tests 5 times:
The most detailed description I found on the link . In short, the total number of tests is 23, they are divided into several categories - Cryptography, Integer, Floating Point, Memory. But in fact, in the results of 25 tests, perhaps because I have a newer version. The tests use compression operations, work with JPEG, HTML parsing, SQLite. I would call it one of the most comprehensive tests for various types of operations.
I could not find descriptions here, but as far as I understand, tests for sequential write / read and random write \ read + the same tests are run in this program, but with a queue depth of 32.
Just leave it here: CPU Blowfish, CPU CryptoHash, CPU Fibonacci, CPU N-Queens, FPU FFT, FPU Raytracing.
The results of multi-core testing are the most interesting, if we look at the information on the systems under test, we can note that AWS and Google Cloud give the following picture: Intel Xeon @ xxx GHz 1 processor, 2 cores, 4 threads , while Azure gives "honest" Core: Intel Xeon E5-2673 v3 @ xxx GHz 1 processor, 4 cores . I suspect that this is the reason for the almost one and a half performance gap. The higher the number in the test, the better.
Total : Azure , AWS, Google Cloud.
More detailed results can be found in the repository .
The results of CrystalDiskMark are the most ambiguous, but most likely this difference is due to the fact that the Google Cloud \ AWS platform must be further configured for optimal performance, while Azure in the default configuration shows good performance. In addition, I completely ignore the price, since it is quite difficult to calculate. The higher the numbers in the tests, the better.
In this configuration, Azure left competitors far behind in all indicators: sequential read / write, 4k random read / write, 4k random read / write from the depth queue equal to 32. I think the result is interesting, because these are the default machines, therefore, this is the result that the average user will receive.
Total : Azure , AWS, Google
Interpretation of the test results Hardinfo I leave to readers . All participants in the test showed approximately the same result.
All clouds have their strengths and weaknesses:
When recalculating power in monetary terms, the picture changes somewhat and Google starts losing the Azure platform by an average of ~ 10% with a 30% discount, although when comparing the calculators of both platforms in the forehead, the opposite picture may appear. Amazon, unfortunately, is starting to lose 28% in money for performance on GeekBench tests.
Naturally, these tests are synthetic and for each individual solution or application you need to carry out specialized workloads, but this can be a good starting point when choosing.
To begin with, I introduced such an indicator as the number of points obtained by the testing tools divided by the cost of this virtual machine per hour. The following instances were used to compare the performance of virtual machines:
| Cloud | Vm size | Corores | Ram | Price \ Hour on windows |
|---|---|---|---|---|
| AWS | m4.xlarge | four | 16GB | $ 0.406 |
| Azure | Standard D3 v2 | four | 14GB | $ 0.422 (0.488) * |
| n1-standard-4 | four | 15GB | $ 0.306 ** |
* Current price for promotional positions, in brackets the regular price.
** Price with a 30% discount provided the virtual machine is used 24/7 per month.
update : after the publication of the article, there was news about OS of smaller disks in Azure, which allows to save about $ 2.18 more per month. This does not change the results in any way, but it allows saving about 0.8% of the price of a virtual machine (Standard D3 v2).
')
All instances have been “raised” in Europe - Ireland \ Germany \ Belgium. Creation took place through the corresponding portals of cloud providers. For instances, SSDs of various configurations were used:
- Google Cloud - 100GB, maximum for "trial";
- Amazon - 30GB, maximum default;
- Azure - 127GB, the default.
In order for the tests to be as objective as possible, I didn’t set up virtual machines. All instances use Windows Server 2016, and for the last test, Ubuntu 04/16/02 LTS. I also refused any OS settings and had to install GeekBench for testing. I used trial versions from the manufacturer's sites. Using apt-get for Ubuntu was installed "benchmark" Hardinfo. These are the only settings made with the operating system before running the tests.
What tests were used
On all virtual machines, I consistently “chased” the following tests 5 times:
- GeekBench (Windows only).
- CrystalDiskMark (Windows only).
- Hardinfo (Ubuntu only).
Geekbench
The most detailed description I found on the link . In short, the total number of tests is 23, they are divided into several categories - Cryptography, Integer, Floating Point, Memory. But in fact, in the results of 25 tests, perhaps because I have a newer version. The tests use compression operations, work with JPEG, HTML parsing, SQLite. I would call it one of the most comprehensive tests for various types of operations.
Crystaldiskmark
I could not find descriptions here, but as far as I understand, tests for sequential write / read and random write \ read + the same tests are run in this program, but with a queue depth of 32.
Hardinfo
Just leave it here: CPU Blowfish, CPU CryptoHash, CPU Fibonacci, CPU N-Queens, FPU FFT, FPU Raytracing.
results
Geekbench
The results of multi-core testing are the most interesting, if we look at the information on the systems under test, we can note that AWS and Google Cloud give the following picture: Intel Xeon @ xxx GHz 1 processor, 2 cores, 4 threads , while Azure gives "honest" Core: Intel Xeon E5-2673 v3 @ xxx GHz 1 processor, 4 cores . I suspect that this is the reason for the almost one and a half performance gap. The higher the number in the test, the better.
| Cloud | GeekBench Score (1) | Price per hour | Perf Score / Price (2) |
|---|---|---|---|
| AWS | 6568.6 | $ 0.406 | 16 177 |
| Azure | 9508.4 | $ 0.422 | 22 530 |
| 6188.2 | $ 0.306 | 20,222 |
Total : Azure , AWS, Google Cloud.
More detailed results can be found in the repository .
Crystaldiskmark
The results of CrystalDiskMark are the most ambiguous, but most likely this difference is due to the fact that the Google Cloud \ AWS platform must be further configured for optimal performance, while Azure in the default configuration shows good performance. In addition, I completely ignore the price, since it is quite difficult to calculate. The higher the numbers in the tests, the better.
In this configuration, Azure left competitors far behind in all indicators: sequential read / write, 4k random read / write, 4k random read / write from the depth queue equal to 32. I think the result is interesting, because these are the default machines, therefore, this is the result that the average user will receive.
Total : Azure , AWS, Google
Hardinfo
Interpretation of the test results Hardinfo I leave to readers . All participants in the test showed approximately the same result.
findings
All clouds have their strengths and weaknesses:
- Google is an excellent discounting system that turns on automatically and, at first glance, really gives a big bonus.
- Amazon is very strong in terms of brand credibility and the most popular platform.
- Azure - has an advantage in performance due to the sale of real cores, and not virtual hyper-threading threads (vCPU).
When recalculating power in monetary terms, the picture changes somewhat and Google starts losing the Azure platform by an average of ~ 10% with a 30% discount, although when comparing the calculators of both platforms in the forehead, the opposite picture may appear. Amazon, unfortunately, is starting to lose 28% in money for performance on GeekBench tests.
Naturally, these tests are synthetic and for each individual solution or application you need to carry out specialized workloads, but this can be a good starting point when choosing.
Source: https://habr.com/ru/post/328916/
All Articles