Testing flash storage. EMC XtremIO

In the middle of 2012, EMC paid $ 430 million for an Israeli startup that was opened three years earlier. At the development stage, in fact, half a year before the expected appearance of the first XtremIO device. To order, the first devices became available only at the end of 2013.

The main distinguishing feature of XtremIO lies in its architecture and functionality . First, the architecture initially includes constantly running and non-switchable services, such as inline deduplication, compression, and thin provisioning, which save space on the SSD. Secondly, XtremIO is a horizontally-scalable cluster of modules (X-Bricks), between which data is automatically and evenly distributed. At the same time, the standard x86 equipment and SSD are used, and the functionality is implemented by software. As a result, it turns out not just a fast disk, but an array that allows you to save capacity due to deduplication and compression, especially in such tasks as server virtualization, VDI or databases with multiple copies.


Love for different kinds of tests is not EMC's strength. Nevertheless, thanks to the initiative assistance of the local office, for us, in the depths of the remote laboratory, a stand was assembled that included 2 X-Brick systems. That allowed us to conduct a series of tests as close as possible to the method developed by us .
')
Testing was conducted on version 2.4 code, version 3.0 is now available, in which half the delay is stated.

Testing method

During testing, the following tasks were solved:

• studies of the process of storage degradation during long-term write load (Write Cliff);
• performance study of EMC XtremIO storage systems under various load profiles;

Testbed Configuration

Figure 1. The block diagram of the test bench.

The test bench consists of 4 servers, each of which is connected by four 8Gb connections to 2 FC switches. Each switch has 4 8Gb FC connections to EMC Xtream-IO storage. On the FC switches, zones are created in such a way that each initiator is in the zone with each storage port.
Server ;
Storage system
As additional software, Symantec Storage Foundation 6.1 is installed on the test server, which implements:

• Functional logical volume manager (Veritas Volume Manager);
• Functional fault-tolerant connection to disk arrays (Dynamic Multi Pathing).

Testing program.

The tests were performed by creating a synthetic load simultaneously from four servers using the fio program on a Block Device, which is a logical volume of type stripe, 8 column, stripewidth=1MiB , created using Veritas Volume Manager from 8 LUNs presented from the system under test on each server. The created volumes are previously completely filled with data.
Testing consisted of 2 groups of tests:

Test results

Investigation of disk array performance on synthetic tests.

Group 1: Tests that implement long-term random write.

The test results are presented in the form of graphs (Figure 2 and 3).

Main conclusions:
During long-term load, the performance drop was not recorded with time. The phenomenon of "Write Cliff" is missing. Therefore, when choosing a disk subsystem (sizing), you can count on stable performance regardless of the load duration (usage history of the disk array).

Group 2: Disk array performance tests under various types of load.

The test results are presented in the form of graphs (Fig. 4-9) and summarized in Tables 1-3.

Maximum recorded storage performance parameters:

Record:

• 160000 IOPS with latency 1.2ms (4K block);

Reading:

• 455000 IOPS with latency 1,4ms (4K jobs = 10 qd = 16);

Mixed load (70/30 rw)

• 264000 IOPS with latency 1,15ms (4K jobs = 8 qd = 10);

• The storage system provides declared performance with acceptable latency.
• The delays obtained cannot be considered minimal, since Testing was performed only in an asynchronous manner, in / in with a queue depth greater than 1.
• On the 1MB block, the performance is lower than expected. Further studies showed that with tests of 1MB block on the storage system, 256KB blocks of operations were performed. This means that the data blocks before sending were shared on the server, which led to a drop in performance. The reason for this was the use of the swidth = 1m parameter instead of stripeunit = 1m when creating a Symantec VxVM volume.
• In case of random recording, the performance change with increasing block size occurs linearly, which allows to conclude that the storage bandwidth (recording bandwidth) is limited to approximately 700-750MB / s.

As an added bonus, we were shown the performance of “snapshots”. The 32 snapshot shots taken simultaneously from the 32-max, maximally loaded for recording, the moons did not have a visible impact on performance, which can not but arouse admiration of the system architecture.

findings

Summing up, we can say that the array made a favorable impression and demonstrated the iops declared by the manufacturer. Compared to other flash solutions, it is distinguished by:

1. No performance degradation on write operations (write cliff)
2. Online deduplication, allowing not only to save in volume, but also to win in the recording speed.
3. The snapshot function and their impressive performance.
4. Scalability from 1 to 4 nodes (X-brick)

The unique architecture and data processing algorithms of the array allow you to implement great functionality (deduplication, snapshots, thin moons).

Unfortunately, the testing was not complete. After all, maximum performance is expected from a system consisting of 4 X-Brick. In addition, version 2.4 was provided to us, while version 3.0 was already released, with half the delay stated. As before, the issues of the operation of an array with large blocks (its maximum throughput) and work with synchronous I / O, where latency is critical, remain unclear. We hope that soon we will be able to close all these "white spots" with additional research.


PS The author expresses cordial thanks to Pavel Katasonov, Yuri Rakitin and all other company employees who participated in the preparation of this material.