Performance comparison of the iron server and the Amazon cloud

Today, developers do not imagine a high-load system without a database. This method of storing information becomes cult. We used to work with the database every day, but still there are dark spots. One of them is performance. Many articles have been written on tuning, database optimization, etc. They are useful if you already have a database (DB) and its performance is not enough. But these articles will not help if you need to run the database in cloud services such as AWS, Azure, Rackspace, etc.

I am sure these questions are familiar to some, because I was asked a lot about this:

• Should I use AWS or a regular iron server?
• What type of server to choose on Amazon?
• Use for Amazon RDS or EC2 database?
• Choose from Amazon dedicated or on demand instances?
• How many transactions can each type of server handle?

The purpose of the article is to resolve these issues. Of course, there is no direct answer to them, it will begin with the words "depends on ...". But I hope my analysis will still help you make the right decision.

Testing environment

In the left corner of the ring there is an amazing dedicated HP DL380 G9 iron server with the following specifications:
')

CPU: 16 cores (Dual Socket Octo Core Intel Xeon E5–2630v3 2.4GHz, #Processors: 2, #Cores per Proc: 8) RAM: 128 GB DISKS: 500 GB RAID 5 SSD 

And in the right corner of the ring there are two Amazon services: EC2 and RDS. To achieve the same characteristics as on the iron server, I use two database instances: DB1 (Memory Optimized) and DB2 (Compute optimized). The specifications are as follows:

DB1:

 r3.4xlarge (memory optimized) 16 cores 122 GB RAM 320 GB SSD Instance Storage 

DB2:

 c3.8xlarge 32 cores 60 GB RAM 750 GB io1 EBS 7500 IOPS 

I will also test instances with different types of leases: dedicated and on demand, plus EBS optimization for an instance in which it is not included by default (for example, r3.4xlarge).

Note:

• I did not tune RDS or EC2 services, I used only standard configurations
• Results may vary by availability zones (AWS) and regions.
• Unlike iron servers, Amazon has some overhead for HVM virtualization.

Test conditions

• Operating system for EC2 and iron server: Ubuntu 14.04 LTS x64
• EC2 instance types: r3.4xlarge and c3.8xlarge
• RDS instance types: db.r3.4xlarge and db.m4.10xlarge (db.c3.8xlarge instance is not available on RDS, so I replaced it with a higher instance level)
• Volumes: 320 GB SSD for DB1 and 750 GB IO1 7500 IOPS for DB2
• MariaDB engine: 10.0.17
• SysBench version: 0.4.2
• SysBench test: OLTP complex test
• The command to start SysBench:
  

   sysbench --db-driver=mysql --num-threads=$THREADS --max-requests=0 --test=oltp --mysql-table-engine=innodb --oltp-table-size=2000000 --max-time=60 --mysql-engine-trx=yes --mysql-user=$USER --mysql-password=$PASSWORD --mysql-host=$HOST run 

  
  
• Maximum number of connections (configuration file my.cnf): 300
• Flows: 1 to 256
• Host to start sysbench: localhost for EC2, for RDS - in the same availability zone
• Results: the number of transactions per second according to the results of the SysBench test
• SysBench tests are conducted in isolation from other tests.

AWS Best Practices

Before proceeding to testing, I want to show you the recommendations of Amazon. In 2015, the company released a detailed Whitepaper on the launch of relational databases on EC2 and RDS. Full documentation can be found at the link .

Here is a brief excerpt from the document:

We recommend that you first consider Amazon RDS . This is the best choice if:

• You focus on high-level tasks such as performance tuning, circuit optimization. You expect Amazon to provide a database, manage backup and recovery, security patches, upgrade SQL Server versions (RDBMS), and manage storage.
  
  
• You are looking for a highly available database solution and want to use Multi-AZ synchronous replication at the click of a button without the need to manually configure and maintain database mirroring, failover clustering or AlwaysOn availability groups.
  
  
• You do not want to manage the backup and, most importantly, restore the database to a specific point in time - prefer AWS to automate this process.

It is better to run SQL Server on EC2 if:

• You need complete control over the database instances, including access to the operating system and software stack.
• You want your own administrators to manage the database, including backup, replication, and clustering.
• Database size and performance exceeds current maxima or other Amazon RDS limits.
• You want to use SQL Server features or options that are not supported by Amazon RDS.
• You want to set up a disaster recovery solution when working with SQL Server on AWS as a source.
• You must use a version of SQL Server that is not supported by Amazon RDS (for example, a version of 2014 that is not supported at the time of this writing).

results

As I said, we have two fighters in the ring, so as a result we will get an iron server against AWS. Since we are using a couple of Amazon services, there will be several tests:

• EC2 and RDS based on DB1 compared to the iron server
• EC2 and RDS based on DB2 compared to the iron server

The results are very interesting, let's try to analyze:

• The values ​​for both EC2 servers are very similar, grow linearly to 16 threads, and then stop and stay at almost the same level even after the number of threads increases.
  EBS optimization slightly increases the number of transactions per second for more threads.
  
  
• Dedicated instances have virtually no effect on performance. The results for selected and on-demand instances are very similar - the difference is literally 2-3%. This means that the selected instances do not speed up the work of the database, but ensure data security. Since they are isolated from other users' instances at the host hardware level.
  
  
• Compute optimized instances show slightly lower performance than Memory Optimized instances on both tests: for EC2 and RDS.
  
  
• RDS with fewer threads behaves somewhat worse than EC2 or iron server, this gap persists up to 16 threads. Starting with 16 threads, RDS performance is growing rapidly with a large margin over EC2. On 256 threads, the value for RDS is three times larger.
  
  
• The iron server performs well for fewer threads and loses RDS starting at 128 threads.
  
  
• The iron server shows its best result at 32 streams, at higher rates the value decreases.

Conclusion

As you can see from the graphs above, EC2 instances do not work well for heavily loaded systems with a huge number of connections. Therefore, to the question “Use for Amazon RDS or EC2 database?” I have to answer: “It depends on ...”. If you are working with a highly loaded database with a huge number of connections, then you should definitely choose RDS. Compared to the iron servers, it shows good results, despite the difference with fewer threads. But if you use a cluster system with a pair of slave nodes and the number of threads less than 16, select EC2 - this service works a little better with a smaller number of threads.
To the question: "Should I use AWS or an iron server for my database," I will also answer: "It depends on ...", just remember that:

	Iron server Physical dedicated server	Cloud IaaS Virtual machines in a public cloud
P l Yu with s	good for intensive workloads full access to the server, no "noisy neighbors" stable performance containers do not need virtualization	suitable for different loads virtual machines are deploying faster better management tools for backups, testing cheaper a variety of VM sizes allows you to choose the right size for specific jobs
M and n at with s	expensive not so flexible, takes longer not so cool management tools (there will be no images of the entire machine for backup) too many resources for most workloads	unstable performance, possible problem of "noisy neighbor" security issues when sharing resources

According to Amazon, dedicated instances mainly provide data security. Test results for different regions are floating - they differ literally by 2-3%. This mainly depends on the load on the region and, as a result, discrepancies of the graphs. I would not consider the selected instances as a way to increase the speed of the database.

And yes, it is still difficult to answer the question: "What type of server should I choose on Amazon?". But I advise you to run the database for high-load systems on RDS. If, however, you decide to use EC2, then EBS-optimized Memory Optimized instances are the best choice.