Features of the work of Auto Tiering in storage Qsan XCubeSAN

Continuing to consider the technology of accelerating I / O operations as applied to storage, begun in the previous article , one cannot help but dwell on such a very popular option as Auto Tiering. Although the ideology of the work of this function is very similar among various manufacturers of storage systems, we consider the features of the implementation of tiring on the example of storage Qsan .

Despite the diversity of data stored on the storage system, these same data can be divided into several groups, based on their relevance (frequency of use). The most popular (“hot”) data is extremely important to organize the fastest possible access, while the processing of less popular (“cold”) data can be performed with a lower priority.

For the organization of such a scheme, it is the tiring functional that is used. The data array in this case consists not of single-type disks, but of several groups of drives, forming different levels (tier) of storage. With the help of a special algorithm, data is automatically moved between levels in order to ensure maximum final performance.

Qsan storage systems support up to three levels of storage:

• Tier 1: SSD, Maximum Performance
• Tier 2: HDD SAS 10K / 15K, high performance
• Tier 3: HDD NL-SAS 7.2K, Maximum Capacity

Auto Tiering pool can contain as all three levels, and only two in any combination. Inside each Tier drives are combined into familiar RAID groups. For maximum flexibility, the RAID level in each Tier may be different. Ie, for example, nothing prevents to organize a structure like 4x SSD RAID10 + 6x HDD 10K RAID5 + 12 HDD 7.2K RAID6

After creating volumes (virtual disks) on the Auto Tiering pool, the background collection of statistics on all I / O operations begins on it. To do this, the space is “sliced” into 1GB blocks (so-called sub LUNs). Each time a block is called, a factor of 1 is assigned to it. Then, over time, this coefficient decreases. After 24 hours, in the absence of I / O requests to this block, it will already be equal to 0.5 and will continue to fall every other hour.

At a certain point in time (by default, every day at midnight), the collected results are ranked by sub LUN activity based on their coefficients. Based on this, it is decided which blocks to move and in which direction. After that, in fact, data is relocated between levels.

In Qsan storage, the management of the tiring process is perfectly implemented using a variety of parameters, which will allow you to very flexibly adjust the final performance of the array.

To determine the initial location of the data and the priority direction of their movement, policies are used that are set separately for each volume:

• Auto Tiering is the default policy, the initial placement and movement direction is determined automatically, i.e. “Hot” data tends to the topmost level, and “cold” data slides down. The initial placement is selected based on the available space at each level. But you need to understand that the system primarily seeks to maximize the use of the fastest drives. Therefore, if there is free space, the data will be placed on the upper levels. This policy is suitable for most scenarios when it is not possible to predict the demand for data in advance.
• Start from high and then Auto Tiering - the difference from the previous one is only in the initial location of the data (at the fastest level)
• The highest level - data always strives to occupy the fastest level. If in the process of work they are shifted down, then at the first opportunity they move back. This policy is suitable for data that requires the fastest possible access.
• Minimum level - data always strives to occupy the lowest level. This policy is perfect for rarely used data (for example, archives).
• No move - the system automatically determines the initial location of the data and does not move it. However, statistics continues to be collected in case their relocation is subsequently required.

It is worth noting that, despite the fact that policies are defined during the creation of each volume, they can be repeatedly changed “on the fly” during the system life cycle.

In addition to the policies for the mechanism of the tiring, the frequency and rate of data movement between levels is also configured. You can set a specific time of movement: daily or on certain days of the week, as well as reduce the interval for collecting statistics to several hours (the minimum frequency is 2 hours). If there is a need to limit the execution time of the data movement operation, you can set a time frame (window for moving). In addition, the relocation rate is also indicated - 3 modes: fast, medium, slow.

In case of a need for immediate data relocation, it is possible to execute it in manual mode at any time by the command of the administrator.

It is clear that the more often and faster data will be moved between levels, the more flexible the storage system will be to adapt to the current operating conditions. But at the same time, it is worth remembering that moving is an additional load (primarily on disks), so you shouldn't “chase” the data without extreme necessity. It is better to plan the movement for the moments of minimum loads. If the work of storage systems constantly requires high performance in 24/7 mode, then it is worthwhile to reduce the relocation rate to a minimum.

The abundance of tiring settings will no doubt please advanced users. However, for those who are faced with such technology for the first time, there is nothing to worry about. It is possible to rely on the default settings (Auto Tiering policy, moving at maximum speed once a day at night) and, as statistics accumulate, adjust certain parameters to achieve the desired result.

Comparing the tiring with such equally popular technology of increasing productivity as SSD caching , one should keep in mind the different principles of their algorithms.

Also, one of the features of tiring is the possibility of its use not only for scenarios like “SSD + HDD”, but also “fast HDD + slow HDD” or all three levels in general, which is impossible in principle when using SSD caching.

Testing

To test the functioning of the tiring algorithms, we performed the simplest test. A pool of two levels of SSD (RAID 1) + HDD 7.2K (RAID1) was created, on which a volume with a "minimum level" policy was placed. Those. data should always be located on slow disks.

The control interface clearly shows the placement of data between levels.

After filling the volume with data, we changed the placement policy to Auto Tiering and ran the IOmeter test.

After several hours of testing, when the system was able to accumulate statistics, the relocation process began.

At the end of the data transfer, our test volume completely “crawled” to the upper level (SSD).

Verdict

Auto Tiering is a great technology that allows you to increase storage system performance with minimal material and time costs due to more intensive use of high-speed drives. With respect to Qsan, the only investment is a license that is purchased once and for all without limiting the size / number of disks / shelves / pr. This functionality is provided with such rich settings that it can satisfy almost any business task. A visualization of processes in the interface will allow you to effectively manage the device.