Working with the in-memory database using Intel Optane
Today we are publishing a guest post, written in collaboration with Konstantin Boyandin - a system administrator from Novosibirsk.
Konstantin used a server with an Intel Optane SSD to work with the in-memory database (memcached, Redis in the appropriate mode, Apache ignite, and so on). The Intel Optane SSD DC P4800X storage device used in the tests was connected as an IMDT (using Intel Memory Drive Technology), available to the OS and applications as random access memory.
')
Another area of ​​application may be the use of a Memory Drive to maintain a large farm of virtual machines. It is assumed that a large amount of RAM in this case will significantly simplify and cheapen the procedure for testing software products.
NVMe alone is not news. NVMe in Memory Drive mode is something new. For one of the projects that Konstantin is involved in, it is important to efficiently process a large amount of data. Test results are the most encouraging, DRAM-mode using Intel Memory Drive Technology will help to significantly reduce processing resources.
We hope that the experience of Constantine will be of interest to our readers.
Task Description
The issue of Intel Optane performance in the IMDT mode was investigated (the Redis and memcached performance tests delivered in the Phoronix testing package were run, as well as the RAM performance test, ramspeed, from the same package). The same tests worked with Intel Optane in IMDT mode, and, for comparison, with Intel Optane media disconnected, on the same server and the same OS.
The testing server is running CentOS 7.4 (64-bit), and on the following hardware components:
- Motherboard: Supermicro X10SRi-F
- RAM: 64Gb (8x8), 2133MHz
- Processor: Intel® Xeon® E5-1650 v4 (3.60GHz)
- Drives (SATA): Intel SSD DC S3700 (480Gb)
- NVMe drive: Intel Optane SSD DC P4800X (375Gb)
The first phase of the tests was carried out without rebooting.
Briefly about these DBMS:
- Memcached is used to store and quickly retrieve data in a key-value format; originally created to speed up the blog service LiveJournal, in 2003.
- Redis (short for “remote dictionary server”, “remote vocabulary server”) is a networked logged data repository focused on high-speed data processing. In this case, configured to work only in RAM without resetting WAL (change log in case of emergency shutdown) to disk.
The purpose of the tests: to compare the performance of the mentioned DBMS using storage in RAM, in the case of using the IMDT mode and without the participation of NVMe in general.
Preparation for work
For tests, ramspeed tests (operations in RAM), redis (Redis performance), mcperf (memcached performance), running CentOS 7.4 (64-bit) were selected.
In both cases, the OS did not use the upload file (swap), so as not to distort the test results.
The Phoronix test suite 7.4 package (and the redis, mcperf included in it) was used, as well as the ramspeed 3.5 package.
In the configuration "Intel Optane in IMDT mode" (hereinafter "IMDT"), the OS and applications have 320Gb RAM available.
In the configuration with disabled Intel Optane (hereinafter "RAM") OS and applications, 64Gb of RAM is available.
In each case, at least three test runs were run, averaged values ​​were used (arithmetic average). All services to be tested rebooted before each test run.
Measurements, practical examples and use cases
For testing the memtest program was used . The memory was taken randomly from the range of 50-75% of the area not used by the test.
Ramfs sequential entry
The first test was to measure the performance of RAM connected as ramfs (sequential write), the following commands were used:
IMDT:
mkdir -p /mnt/ram mount -t ramfs -o size=300G ramfs /mnt/ram dd if=/dev/zero of=/mnt/ram/256gb.dat bs=4096 count=67108864 Total:
274877906944 bytes (275 GB) copied, 184,756 s, 1,5 GB / s
RAM:
mkdir -p /mnt/ram mount -t ramfs -o size=62G ramfs /mnt/ram dd if=/dev/zero of=/mnt/ram/60gb.dat bs=4096 count=15728640 Total:
64424509440 bytes (64 GB) copied, 18.0382 s, 3.6 GB / s
The ramfs filesystems are unmounted before the rest of the tests are run.
Redis test
In testing, it was checked how many DB commands (SADD, LPUSH, SET, GET) were executed per second.
The higher the value, the better the performance:
- IMDT - in the Intel Optane configuration, 320Gb of RAM is available for applications;
- RAM - 64Gb of RAM is available for applications with Intel Optane disabled.
| Redis v3.2.10 | IMDT | Ram |
| SADD | 1,748,516.67 | 1,727,879.00 |
| LPUSH | 1,343,129.58 | 1 310 077.82 |
| SET | 1 562 944.63 | 1,490,965.52 |
| Get | 2,026,167.62 | 2,213,292.12 |
Mcperf test
In testing, it was checked how many commands per second are executed for 1 and 5 simultaneous connections (speed is measured in commands per second for each connection).
| Memcached v1.4.15 | IMDT | Ram | ||
| one | five | one | five | |
| add | 54 931.2 | 55 197.2 | 59 613.9 | 59 174.8 |
| delete | 85 361.3 | 83 259.4 | 89 181.6 | 88 235.4 |
| append | 57 834.2 | 57 666.1 | 59 252.8 | 60 652.4 |
| prepend | 58 441.9 | 58 168.5 | 59 359.3 | 60 115.8 |
| replace | 57,231.2 | 57 046.0 | 59 223.5 | 59 233.0 |
| get | 86 657.0 | 85 842.8 | 89 468.9 | 89 107.4 |
| set | 54 546.9 | 55 570.1 | 58 311.6 | 57 725.3 |
Ramspeed test
Tests were conducted for 4 concurrent threads. Where it matters, the size of the memory block used is 4096 bytes. The higher the value (MB per second), the better the performance.
| Ramspeed v3.5 (smp) | IMDT | Ram |
| Integer & read | 104 548.28 | 112,535.63 |
| Integer & write | 33,238.73 | 34,085.68 |
| Integer copy | 30,063.08 | 30 105.97 |
| Integer scale | 29,916.03 | 29 757.79 |
| Integer add | 33,356.14 | 33,192.47 |
| Integer triad | 33,235.32 | 33,184.95 |
| Floating Point & Reading | 102,796.84 | 106 513.01 |
| Floating point & writing | 33,826.04 | 34 037.25 |
| Floating point copy | 30,061.67 | 30,060.73 |
| Floating point scale | 30,094.47 | 29,964.92 |
| Floating point add | 33,144.85 | 32 979.58 |
| Floating point triad | 33,184.84 | 33,149.31 |
Conclusion
The tests demonstrated that when using Intel Optane in the IMDT mode, despite the noticeable difference in the write speed to the file (using the standard dd command), in general, the performance of the DBMS storing data in memory is comparable.
The advantage of NVMe in the IMDT mode is that it is technically impossible for most servers to install the appropriate number of RAM modules in order to get the same amount of available RAM.
In terms of cost-performance (for 8Gb Kingston DDR4 2133MHz RAM modules), the equivalent Intel Optane carrier will be twice as cheap (note that the Optane used in these tests is not optimal in terms of cost-performance ratio).
For both cases of using Memory Drive (that is, a large amount of RAM) or there is no technical possibility to install as much memory (there are no such motherboards), or the price of this configuration is at least 2 times higher (when calculating the equivalent amount of RAM).
It can be assumed that NVME with Memory Drive support will be cheaper (now it is a relative rarity and has a high cost), so a gain in terms of cost-performance may become even more noticeable.
Using NMVe in IMDT mode seems promising in cases where a large amount of RAM is required (large databases like Redis / memcached, a large number of virtual machines on one host, and so on).
In the above tests, conditional performance was considered sufficient. Building a simulation of the environment where you can run tests that are close to the design ones took too much time.
The author of the article
Konstantin Boyandin, Provide Labs LLC, Novosibirsk.
Engaged in software development and system administration, as well as security issues.
Source: https://habr.com/ru/post/345306/
All Articles