A small assessment of the impact of Cache levels on I / O performance in the EMC VNXe3200

Introduction

Recently, and not for long, the VNXe3200 storage system (DSS), which was announced by EMC ² for customers on May 5, 2014, came into my hands. The VNXe3200 is the second generation of entry-level Unified storage systems from EMC ² . In this model, there appeared technologies available earlier only on older and more expensive midrange arrays. In particular, FastCaché technology - i.e. the second level cache on SSD disks, which runs into a section between the traditional cache in the RAM of the storage controller (in the terminology of the EMC - Storage Processor) and the disks themselves. I decided to test how this technology affects I / O performance on the youngest EMC ² storage systems.

Unlike older EMC VNX storage systems, in this model both block and file access is implemented on the same controllers (SP). The storage in the database has for each SP 4 copper ports of 10Gbit / s (10GBASE-T) through which the connection of clients / hosts is realized via CIFS / SMB, NFS and iSCSI protocols. These ports are with autonegotiation 10G / 1G / 100Mb per second. Additionally, in each SP you can put a board on 4 8Gb / s FC ports. It was decided to test using IOMETER. This article helped, including, this article from Habra: link .

Description of the stand and tests

I didn’t get wise with the load profile, took the standard Database pattern (Intel / StorageReview.com).


')
For testing, we took Blade BL460c G7 server (one CPU 6 cores + HT, 24GB RAM) connected via FC to the storage system through switches built into the FC blade basket with 4Gbit / s ports. The server was installed with OS Windows 2012 R2 with FC loading from VNXe3200 (boot from SAN). A test volume (LUN) of 1Tb with the NTFS file system was also connected to the server, also on FC. On the storage, a disk pool of 10 disks (SAS 2.5 "10k RPM 600Gb) is assembled with two private raid groups (RG) inside, which have a Raid5 configuration (4 + 1). Also on an array of two SSD disks (2.5 "100Gb) assembled FastCache (Raid1 mirror pair).

Testing was conducted in three stages.

1) With the help of IOMETER, a small test file with a size of 2Gb is created, in the calculation that it completely fits in the Cache SP on the storage system.
2) The previous test file was deleted and a test file of ~ 50Gb in size is created, in the hope that it does not fit in Cache SP, but it will be fully included in FastCache.
3) The previous test file was deleted and a test file of ~ 500Gb in size is created, in view of the fact that it does not fit in any of the caches and with 100% random access it will practically give the performance of the existing spindle disks.
The tests were set up in such a way that before each pass there was a “warming up” at 10 minutes, then a test at 20 minutes. Each test was performed with an exponential growth of input / output flows (1-2-4-8-16) for each of 12 workers (6 cores + HT). At the same time, in addition to the actual IOPS storage systems delivered, it was interesting to have a comfortable average response time <10 milliseconds (ms). Immediately, I’ll make a reservation that I’ll give “pictures” with graphs from the VNXe3200 interface, but the quantitative indicators on them coincide with the results in the IOMETER csv files, which will be cited by links.

Next, some calculations.

If you do not take into account the effect of cache on I / O, then for SAS 10k disks, EMC suggests reading 150 IOPS per disk. We in the amount on the back end with 10 disks should have 150 * 10 = 1500 IOPS. If we take into account our load r / w of 67% / 33% and the loss of work with CRC in Raid5, we get the following equation with one unknown. 1500 = X * 0.33 * 4 + X * 0.67, where X is our IOPS, which will receive hosts from our disks. A 4 is the “penalty” size for Raid5. That is, in Raid5, to perform a single write operation coming from the host, 4 I / O operations are required on the backend (on the storage disks). For Raid1 / Raid10, the penalty value is 2, for Raid6 - 6. As a result, we get X = 1500 / 1.99 = ~ 750 IOPS. In practice, I noticed that the maximum achievable values ​​are more than calculated by 1.5-2 times. So in peak load we can get 1125-1500 IOPS from our 10 SAS disks.

Let's move on to the tests and their results.

Test 1

As expected, the test file almost completely fit in cache SP.



When testing, we got the following picture of IOPS and I / O request hits in the cache. Here you need to make a reservation that this chart actually shows all IO passing through SP. Some of them are processed from SP cache (hit), some “flies through” SP cache on through (miss) and are processed either from FastCache or from spindle SAS disks.



The average response time with the maximum number of threads in IOMETER (12 workers * 16 threads IO = 192 I / O threads) was ~ 8 ms. File with IOMETER results here . The first test was conducted with an increase in the number of threads per worker from 4 to 32 (4-8-16-32). I noticed late what I confess, but there was no time to redo it.

Test 2

During the test, the test file at ~ 50GB almost completely fit into FastCache, as expected.



The result is the following picture, which shows that almost all requests fly past SP cache.



The average response time for 192 streams was ~ 12.5 ms. Comfortable response time was 8 threads per worker ~ 8 ms (96 IO threads). File with IOMETER results here .

Test 3

During the test, the input / output randomly “ran” on all ~ 500Gb and not a single cache in theory could not help here, which is evident in practice from the graphs below.




As a result, as planned, the performance of 10 SAS spindles in Raid5. In this case, the first 4 in the storage disk used in the pool is the so-called VaultPack. That is, a part of these first disks (~ 82.5 GB of each) are “cut off” for system needs.



The average response time for 192 streams was rather long and amounted to ~ 149 ms. Comfortable response time was at 1 thread per worker (12 threads IO) ~ 10 ms. File with IOMETER results here .

A small digression about pools

When designing a disk pool, if you do not know the actual size of the hot and warm data area, EMC recommends that you maintain the following proportions for a three-level pool:
10% - SSD drives
20% - SAS drives
70% - NL-SAS drives
In addition, you need to take into account that when you add flash tier to the pool automatically, all the metadata of the thin moon created in the pool will be placed on the SSD. If there is enough space for them. This allows you to increase the overall performance of thin moons and pool. Under this metadata, you need to plan an additional place on the SSD at the rate of 3Gb volume for every 1Tb actually occupied by thin moons in the pool. At the same time, the moons having the highest available tier tiring policy will take precedence when placed on an SSD dash over any other data.

Using the lowest available tier policy for thin moons results in their metadata being placed on the slowest disks.

Results

Testing has shown that all types of Cache in the storage system as a whole have a positive impact not only on the overall performance of the array. But also on the average response time of I / O operations, especially under high load. And given the above, the cache will get the most "hot" data.

It can be concluded that the FastCache in the EMC VNXe3200 will be quite a good and popular addition, even with small configurations. Considering that the process of its “warming up” (hitting data in the cache) occurs rather quickly.