Cloud Server Performance Test Results with Dwarf Fortress
Dwarf Fortress is a legendary game that simulates the fantasy world in detail, and a player (in one of the modes) can build and manage a gnome (dwarf) settlement (fortress). About the game is written enough, so I will not go into details. It is important that due to the large size of the gaming world and the high details of the simulation, the game is quite demanding of resources - both the processor and memory / cache. The game supports all three major operating systems.
For the game there is a project DFHack, engaged in reverse engineering of data structures of the game and allowing you to create plugins and scripts in C ++, Lua or Ruby.
I am the author of the application and the accompanying server code that fully implements the game user interface on devices running iOS. That is, the user puts the original game, plugin and application, and can play remotely from a mobile device with all the amenities.
')
The server can be installed both at home and rented from one of the cloud providers. This is how the idea of ​​this study was born - first, to find out which cloud service is better suited and can be recommended to users first of all, and second, to compare the performance of servers using something different from web servers and special utilities .
For testing, the Dwarf Fortress / DFHack 0.43.03-r1 version was used. I have a public image for Docker, including the game itself and the minimum version of DFHack (as well as the actual server code for the remote game, but in this case it is not used). A script was written on Lua that runs the tests and sends the results to the web server. Also, the following script was used for automation, which is specified in user-data when configuring servers. That is, it was necessary to simply create a server, wait for the results to arrive and delete it.
The following providers were tested: Digital Ocean, Amazon Lightsail, Amazon EC2, Vultr, Linode, Google Compute Engine, and Macbook Pro Late 2013 2Ghz Core i7.
If the cloud service supports the creation of a server with Docker in one click, then this feature was used, if not, then the Ubuntu version 16.04 was chosen. Thus, only Vultr was used not by Ubuntu, but by CentOS 7. For servers with more than 2GB of memory, the swap file was not used. For Google Compute Engine micro and small, an SSD was chosen.
The cheapest server options were chosen, firstly, because they are more interesting to those who are going to use them for the game, and secondly, because Dwarf Fortress still uses only one processor core for the simulation and (in this version) is 32-bit application.
The process of the game in Dwarf Fortress consists of two parts - the generation of the world and the initial history, in which you can specify the size of the world and the length of the history (and other parameters), and the actual gameplay. The following tests were performed sequentially, without restarting the game:
The last test is most important, firstly, because this is actually the game process, for which everything happens, and secondly, it is less susceptible to chance than the generation of the world (in which civilizations, settlements, landscape, events and etc).
Here is the schedule. Execution time in seconds, respectively, less - better. Prices per month in US dollars.
Immediately visible unnatural variation in the results of the first test, I do not know exactly what this is connected with. I suppose, partly with caching, partly with the low performance of the newly created server for some reason. Therefore, the results are sorted by the last test, which, as noted, the most important. The second test is also generally correlated with the results of the third.
Apart from the expected first place for the EC2 c4.large server, related to the performance-optimized C4 type, Amazon Lightsail was the fastest, which, given the price, is a rather pleasant surprise. It is followed by a Google single-processor server and a laptop with almost the same result. Further, the unknown to me was Vultr, Linode, from which I expected more. The Audience Prize gets the Digital Ocean - slower, but the most simple and fast to use. Quite expensive EC2 m3.medium servers are surprisingly slow, and the cheapest GCE servers are completely bad even with SSD drives, I did not wait for the result of testing a micro server size.
Separately, it is necessary to say about EC2 instances of type T2. Their peculiarity is that when idle the server receives processor credits (up to a certain limit), which are spent under load. Spending loans, the server works with full performance when they run out - performance drops to 20%. On the one hand, it is very well suited for our purposes - people, especially from a mobile device, do not play continuously all day, plus the simulation in the game stops when a person does something on the menu, etc., therefore this is a good option to get high performance for little money. On the other hand, this is “unreal” performance, so I did not include instances of this type in testing, so as not to confuse anyone. By experience, while there are credits, they are quite fast, when they run out - very slow.
That's all.
For the game there is a project DFHack, engaged in reverse engineering of data structures of the game and allowing you to create plugins and scripts in C ++, Lua or Ruby.
I am the author of the application and the accompanying server code that fully implements the game user interface on devices running iOS. That is, the user puts the original game, plugin and application, and can play remotely from a mobile device with all the amenities.
')
The server can be installed both at home and rented from one of the cloud providers. This is how the idea of ​​this study was born - first, to find out which cloud service is better suited and can be recommended to users first of all, and second, to compare the performance of servers using something different from web servers and special utilities .
For testing, the Dwarf Fortress / DFHack 0.43.03-r1 version was used. I have a public image for Docker, including the game itself and the minimum version of DFHack (as well as the actual server code for the remote game, but in this case it is not used). A script was written on Lua that runs the tests and sends the results to the web server. Also, the following script was used for automation, which is specified in user-data when configuring servers. That is, it was necessary to simply create a server, wait for the results to arrive and delete it.
#!/bin/bash fallocate -l 4G /swp ; chmod 600 /swp ; mkswap /swp ; swapon /swp ; echo '/swp none swap sw 0 0' >>/etc/fstab apt-get update apt-get install -fy docker.io apt install unzip wget https://github.com/mifki/dfremote-cloud-benchmark/raw/master/benchmark.lua wget http://mifki.com/a/t/gloveloved.zip unzip gloveloved.zip -d save/ docker run --name=df -dt -v $PWD/save:/df_linux/data/save -v $PWD/benchmark.lua:/df_linux/hack/scripts/benchmark.lua mifki/dfremote sleep 5 docker exec df /df_linux/dfhack-run benchmark server_name The following providers were tested: Digital Ocean, Amazon Lightsail, Amazon EC2, Vultr, Linode, Google Compute Engine, and Macbook Pro Late 2013 2Ghz Core i7.
If the cloud service supports the creation of a server with Docker in one click, then this feature was used, if not, then the Ubuntu version 16.04 was chosen. Thus, only Vultr was used not by Ubuntu, but by CentOS 7. For servers with more than 2GB of memory, the swap file was not used. For Google Compute Engine micro and small, an SSD was chosen.
The cheapest server options were chosen, firstly, because they are more interesting to those who are going to use them for the game, and secondly, because Dwarf Fortress still uses only one processor core for the simulation and (in this version) is 32-bit application.
Tests
The process of the game in Dwarf Fortress consists of two parts - the generation of the world and the initial history, in which you can specify the size of the world and the length of the history (and other parameters), and the actual gameplay. The following tests were performed sequentially, without restarting the game:
- The generation of the world is very small (smaller) with a very long (very long) history.
- Generation of the world of large (large) size with a short (short) history.
- Game process. One of the “group” (when many people play in turns) saves was used - glovedloved (AnethadefĂni, 254). The choice is not so important, as long as there is a large enough fortress in it, and nothing that would automatically stop the game, does not occur during the test, which lasted for 20 in-game days.
The last test is most important, firstly, because this is actually the game process, for which everything happens, and secondly, it is less susceptible to chance than the generation of the world (in which civilizations, settlements, landscape, events and etc).
results
Here is the schedule. Execution time in seconds, respectively, less - better. Prices per month in US dollars.
Immediately visible unnatural variation in the results of the first test, I do not know exactly what this is connected with. I suppose, partly with caching, partly with the low performance of the newly created server for some reason. Therefore, the results are sorted by the last test, which, as noted, the most important. The second test is also generally correlated with the results of the third.
Apart from the expected first place for the EC2 c4.large server, related to the performance-optimized C4 type, Amazon Lightsail was the fastest, which, given the price, is a rather pleasant surprise. It is followed by a Google single-processor server and a laptop with almost the same result. Further, the unknown to me was Vultr, Linode, from which I expected more. The Audience Prize gets the Digital Ocean - slower, but the most simple and fast to use. Quite expensive EC2 m3.medium servers are surprisingly slow, and the cheapest GCE servers are completely bad even with SSD drives, I did not wait for the result of testing a micro server size.
Separately, it is necessary to say about EC2 instances of type T2. Their peculiarity is that when idle the server receives processor credits (up to a certain limit), which are spent under load. Spending loans, the server works with full performance when they run out - performance drops to 20%. On the one hand, it is very well suited for our purposes - people, especially from a mobile device, do not play continuously all day, plus the simulation in the game stops when a person does something on the menu, etc., therefore this is a good option to get high performance for little money. On the other hand, this is “unreal” performance, so I did not include instances of this type in testing, so as not to confuse anyone. By experience, while there are credits, they are quite fast, when they run out - very slow.
That's all.
Source: https://habr.com/ru/post/401485/
All Articles