IBM FlashSystem 820 storage system overview and testing
Today we face an extremely difficult task - to begin a series of articles highlighting network equipment designed to work in data centers. Basically, the materials of the cycle will be devoted to a variety of data storage systems of leading vendors, but we don’t want to limit ourselves exclusively to SAN devices and plan to devote several articles to switching equipment and servers. For whom is this cycle of materials intended? First of all, we want to acquaint our customers with the equipment that they can place on the technological sites of SAFEDATA data centers, to help with the choice of the hardware that is needed. And secondly, we would like to share information obtained as a result of testing and operation, which will certainly prove interesting and useful to administrators serving the corresponding data storage and transmission systems.
It should be immediately noted that SAFEDATA customers do not have to independently acquire and maintain expensive equipment, these tasks can be delegated to managers and engineers of the company, taking the required devices for rent . This cycle of articles will allow you to determine the specific model of the switch, router or storage, the most suitable for solving the task.
We decided to start a series of reviews and tests with the IBM FlashSystem 820 storage system, which demonstrates amazing performance and reliability.
The continuous development of information technology leads to the fact that the performance of servers and storage systems begins to play an increasing role. The storage tasks are no longer limited to simply storing user data; today they are required to provide high-performance access to the information placed on them. High access speeds and low response times can provide devices built on the basis of flash memory. IBM FlashSystem 820 network storage uses eMLC (enterprise MLC) -based flash memory, which is more reliable than MLC (Multi Level Cell) by adding ECC and increasing the number of write cycles to 30,000 (regular MLC supports up to 10,000 erase / dubbing). Also, according to the manufacturer, modules based on SLC (Single Level Cell) memory are supported with the number of erase / rewrite cycles equal to 100,000. There are two versions of FlashSystem 820, differing in the amount of pre-installed flash memory: 10 or 20 TB. Installed flash memory modules can be combined into a RAID0 or RAID5 array. The main technical characteristics of IBM FlashSystem 820 storage systems are presented in the table below.
Perhaps now is the time to explain what we mean by abbreviations KB, MB, GB and so on. By KB, we mean 1024 bytes, MB - 1024 KB ... That is, GB will be equal to 2 ^ 30 bytes, not 10 ^ 9. This time we decided to abandon the formally more correct prefixes of kibi, mebi, bebi and tebi, that is, from the values ​​of KiB, MiB, GiB and TiB, so as not to embarrass some of our readers who are not yet ready for such boxes. In the next article we will immediately use the correct abbreviations.
')
The difference between the modifications lies in the used flash memory modules, the modification of 20 TB is dual.
We also decided to list the functionality and the benefits they provide.
VSR (Variable Stripe RAID) technology provides data protection at the level of a memory page, a block, or even a whole chip. This protection allows you to avoid replacing the entire flash module in situations where a single chip fails. As a result, the frequency of servicing FlashSystem 820 series storage facilities is significantly reduced.
The basic elements of the IBM FlashSystem 820 network storage architecture are presented in the diagrams below.
The FlashSystem 820 model is not the only IBM storage system built on the basis of flash memory modules. This vendor also offers FlashSystem 710, 720 and 810 models. The ratio of performance and availability for all four models is presented below. Speaking of increased availability, it is worth noting that the FlashSystem 720 and 820 models implement 2D Flash RAID data protection technology.
At the time when we were preparing our test bench for the first measurements, IBM released a new data storage system - FlashSystem 900 . This model offers even greater performance: up to 1.1 MIOPS for read operations, 800 KIOPS for mixed mode and 600 KIOPS when performing data recording. IBM FlashSystem 900 uses MLC flash memory, developed in collaboration with Micron. The new flash memory eliminated one of the main drawbacks of MLC chips - the number of supported erase / rewrite cycles: according to the manufacturer, the life cycle of new chips is now not inferior to that for eMLC memory. The company IBM decided in principle to abandon the use of eMLC flash memory in new models of storage.
We now return directly to the model under discussion - FlashSystem 820 - and consider the possibilities for setting it up.
IBM FlashSystem 820 network storage is managed using a java application launched by the browser when accessing the device via HTTP or HTTPS protocols. When you log in, you need to enter your login and password, and also indicate whether you need to use encryption.
After you enter the correct credentials, the administrator opens the main window of the Monitoring Utility program, which allows you to configure and collect statistics from several FlashSystem storage systems simultaneously.
We will not consider in detail all the possibilities of this control system, however, we will dwell on the most interesting ones in our opinion.
The Options menu allows you to add storage systems in manual or automatic mode, connect to several FlashSystem at once, perform multiple firmware updates, display statistics for several storage systems.
The utility settings are mainly related to the choice of the method for detecting storage systems on the local network.
Here it is worth noting that detection refers to the automatic search and addition of control modules of FlashSystem. The Actions menu allows you to turn off or reboot devices, save or restore configuration, break control connection, or remove storage from the list.
The two tabs located at the bottom (“Recent Event Log” and “Task Monitor”) display a list of events occurring with the device, as well as a set of tasks performed by the administrator.
What actions can be performed with the device itself? The “Logical Units” menu group allows you to manage “virtual disks” (LUN): create partitions of the required size, delete unnecessary partitions, and also change access parameters.
The “Storage” group contains all flash drives installed in the storage. You can install flash drives with a volume of 1 TB and 2 TB. All installed flash cards can be combined into a RAID array.
Two types of arrays are supported: maximum capacity (analog RAID0) and RAID5. To increase reliability, it is recommended to use a RAID5 array called IBM 2D Flash RAID, since the RAID5 array inside the module itself is also used, which ensures the fault tolerance of the entire module when one of ten chips fails (nine store user data, one is used for parity). It is worth noting that when creating a RAID5 array, not all modules are used to store user data: ten modules contain data, one is used to store parity information and another is not used, it is designed for hot-swapping a failed module. Thus, user data will remain available even if two modules fail, naturally, not at the same time. The diagram below shows the architecture of such a 2D Flash array.
Interfaces group displays information about installed interface modules and their ports.
Information on air temperature and system components, power settings, fan performance and battery status can be found in the Environmental group. You can make sure that the batteries installed inside the battery case can be used with the help of a periodic test, which can also be scheduled here.
The settings of the management module parameters are collected in the Management group. Here the administrator can specify the IP parameters of both network cards and the address of the DNS server.
Time synchronization via NTP is managed using the Date / Time item, while the Users item allows you to manage local or domain users.
The Firmware item is for updating the firmware. Unfortunately, the new firmware versions are available only to users who have acquired special support (service contract), so we will not be able to show the entire process of updating the firmware. Here, perhaps, it is worth saying that SAFEDATA customers, who rent equipment, have at their disposal devices with the latest and most stable firmware. In addition, an additional equipment support service can be ordered, allowing you to perform all the necessary maintenance work on servers, data storage systems and other network equipment during the entire period of using the data center services, which also includes maintaining firmware in the current state.
The Services group allows the administrator to specify the connection settings to FlashSystem using SNMP, Telnet, and SSH protocols. It is also worth noting that when certain events occur, an alert can be sent to the administrator via e-mail, which is also configured here.
The Statistics group is responsible for collecting and displaying statistics. Here, the administrator can view the current use of the system, build graphs of the load on the network interfaces, display the number of operations performed per second, and so on. A variety of counters are available for each component of the system.
The storage system FlashSystem 820 is equipped with two management modules (MCP - management controller ports). The module can be in active or passive mode. The MCP in active mode controls the LUNs, and also allows you to view the status of system components. A passive module is used to provide fault tolerance in the event of an active MCP failure. The Cluster tab displays information about the MCP control modules and allows you to switch the active MCP, that is, the “cluster” is assembled from two modules installed in one FlashSystem 820.
Access to the log information of the system can be obtained using the Logs item.
This concludes our discussion of the IBM FlashSystem 820 web storage system, and finally we would like to mention that device management is possible not only using a browser and a graphical interface, but also using the command line when connecting using Telnet and SSH protocols.
We are now engaged in testing IBM FlashSystem 820.
Connecting to IBM FlashSystem 820 can be done with two fundamentally different goals: for data acquisition and for management. The instance of FlashSystem 820, which we tested, was equipped with four FibreChannel ports for data transfer, therefore, it is worth talking about device security only in the context of security of the control module. Of course, we understand that the management interface will be connected to some internal secure network, access to which will be limited to access lists on routers or other means. However, we were interested in the availability of available services in principle. For testing, we used Positive Technologies XSpider 7.7 (Demo build 3100) network security scanner. A total of four open ports were detected: TCP-22 (SSH), TCP-23 (Telnet), TCP-80 (HTTP) and TCP-443 (HTTPS). The most interesting information is presented below.
We now proceed directly to the load testing.
Before we begin to describe the methodology of the load testing carried out and provide the results, we would like to especially note that we did not set ourselves the goal of getting the maximum MB / s or IOPS. The only goal we pursued is to find out what real speeds will be available to users when connecting to the storage of one or two servers. As we will show later, the values ​​we obtained are not limitations of FlashSystem 820, but of the equipment used for the tests. Thus, in order to “squeeze” the maximum out of the device, you will need to connect several productive servers to it at the same time. And although this storage system was created for the ultra-fast service of requests from customers' applications, as well as to speed up the existing storage systems with conventional hard drives, we still decided to take measurements of file access rates.
The main parameters of the test bench are listed below.
The first measuring tool was the utility CrystalDiskMark version 3.0.3. First, we made measurements for connecting using a single FC link.
Then they connected the second FC link and enabled balancing by adding the Multi-Path I / O component.
We carried out the following measurements using Intel NASPT version 1.7.1. When testing Intel NASPT using the msconfig utility, the RAM available to the operating system was reduced in accordance with Intel's recommendations in order to reduce the effect of local caching on measurement results. First, we measured the data access speeds with one FC link between the server and the storage for three file systems: NTFS, FAT32 and exFAT.
As can be seen from the diagram above, in some tests we are very close to the performance of the 8GFC environment (including 8b / 10b encoding) - 8 Gbit / s. For the NTFS file system, we decided to compare the performance of the system with single and double server connections to the storage. The speeds obtained with the help of double connection turned out to be lower than the values ​​that we managed to get with a single connection. We must admit that at this stage we were somewhat discouraged. Of course, we later discovered the reason - high CPU usage with MPIO turned on, but first things first.
The next testing tool is the Intel IOMeter utility, which allows both creating an absolutely synthetic load in the form of pure reading or writing, and using patterns that emulate the behavior of various servers and workstations. Synthetic load is represented by three tests: 100% reading, 100% writing, as well as 50% reading and 50% writing. The diagram below shows the results of each of the three synthetic tests when connected using one FC link, depending on the size of the data block. The testing involved two servers connected to the Flash System 820 using an FC switch.
As can be seen from the diagrams presented, as the block size increases, the data access speeds increase until the obvious limit is reached - the performance of a single Fiber Channel interface, that is, 8 Gbit / s.
In addition to measuring storage performance in terms of bandwidth, we also measured the number of operations performed by a device per unit time (IOPS). The diagrams below show the results of the same measurements, expressed in thousands of IOPS.
For any write operations, a peak of performance is seen with a block size of 4 kilobytes, due to the size of the internal data structure.
Naturally, we decided to make the same measurements, but when connected to the storage using two FC links, so that the Fiber Channel interface was not a bottleneck of our test bench.
As expected, we found a significant increase in system performance. However, when reading blocks of 4 and 8 kilobytes in size, we were close to the performance of two FC interfaces, which makes us think that the performance of the IBM FlashSystem 820 network storage is even higher, that is, using all four FC ports, even our two test servers could be obtained greater speed of access to user data. The diagrams below show device performance in thousands of IOPS when connected using two FC links.
As with the connection using a single link, there is a pronounced peak in performance on data blocks of 4 kilobytes.
It is time to move away from synthetic measurements and make our stand simulate the load produced by various servers and workstations. The following four storage scenarios were selected: Database, Fileserver, Workstation, and Webserver. The storage performance when connected using a single FC interface is presented below.
We repeated the same measurements, but using two FC connections already.
The results obtained, as well as for synthetic tests, clearly indicate that connecting to FlashSystem 820 using one FC link, the bottleneck will be the FC connection itself.
In all the following tests in this section, we used Server # 1 and read data blocks of 4 kilobytes in size. First, we compared system performance versus request queue depth using a single thread.
Perhaps, it is worth giving some explanations to the last graph, which shows the time of the request to read a block of data from the storage. While the system is not overloaded, reading is performed in approximately 200 µs and is almost independent of the number of physical connections, which is a very worthy result.
Then we decided to find out the dependence of system performance on the number of threads with a queue depth of 4.
The only thing we would like to mention here is the increased processor load when using MPIO. That is, when load balancing between multiple physical FC interfaces, the Windows operating system spends precious processor cycles. For example, in almost all tests without MPIO, the CPU load did not exceed 10% (ranged from 5-7%), while the use of MPIO led to almost one hundred percent CPU utilization. Looking ahead, we note that a similar effect was not observed when testing FlashSystem 820 with the Oracle database running on the Oracle Linux Server release 6.6 operating system.
This concludes the IBM file storage tests and proceeds to measuring storage performance when working with an Oracle database.
Test Objective
To determine the main performance characteristics of the IBM FlashSystem 820 storage system when used with an Oracle database application.
Testing method
To test the performance of the storage used by the Oracle database, the SLOB2 utility (The Silly Little Oracle Benchmark) from Kevin Closson was used. He is a former performance architect of the Oracle Exadata Machine. More information about the utility is presented in the author's blog.
Testing involves reading / writing multiple threads of execution. A diagnostic AWR (Active Workload Repository) report is collected for each test. It serves as a source of information about the actual performance of the disk subsystem within a test.
Platform
Operating system: Oracle Linux Server release 6.6 with kernel 3.8.13-55.1.5.el6uek.x86_64.
Processor: Intel Xeon CPU E7-4830 @ 2.13 GHz, 16 cores, 32 processors
File system block size: 4 Kbytes
Test parameters
24 measurements were taken. The following varied:
- The number of threads (from 1 to 128 with a double increase).
- The ratio of the number of read operations to write operations (Read Only, Write Only and “70% Read, 30% Write”).
- The size of the block used by the database.
- Use multipath I / O.
Test results
4 KB, without MPIO.
4 KB using MPIO.
8 KB, without MPIO.
8 KB using MPIO.
IOPS and throughput
Maximum reading performance of 62K IOPS was achieved with a block size of 4 KB on 32 threads and Multipath I / O disabled. The throughput at the same time amounted to 242 MB / s. Measurements of blocks of 8 KB in size with Multipath I / O disabled show an average of 5.6% worse reading IOPS. The throughput at the same time increases on average by 88% to 457 MB / s for 32 streams.
Multipath I / O
The inclusion of Multipath I / O adversely affects the results. With a block size of 8 KB, we get a loss of about 20% on reading readings of 16 threads and 25% when testing on 32 threads. In the 30/70 mixed mode, the peak of IOPS is also reduced by 20% and 25% for a block size of 4 KB and 8 KB, respectively. However, with fewer threads, Multipath I / O contributes to improved IOPS and throughput. So, with a block size of 4 Kbytes, measurements on 4 and 8 streams showed 12% better results in mixed read / write mode.
30/70 mode
The case “30% of operations is write, 70% is read” corresponds to the real operation mode of the database as part of a multi-user application. “Saturation” occurs at 16 streams. In this case, the read delay increases from 240 µs on 1 stream to 400 µs on 16. The longest response of the carrier was obtained when testing on 64 threads - 950 µs. At 128 threads, the delay is less due to the fact that a struggle for resources between the execution threads took place on the testing system. This led to the fact that the generated load on the media in read / write mode has decreased.
Delay
The maximum allowable for a database is read latency for modern HDD drives is 4 ms, which is an order of magnitude more than the maximum values ​​obtained for FlashSystem 820. It is worth noting that for relational databases the delay parameter is critical, since the task of obtaining necessary records from tables usually consists of consecutive accesses to data from files. That is, to get a block containing the desired row from the table, you first need to find this information by reading several blocks of the index. The high latency allowed for file storage can have a strong impact on database performance.
Correlation with real exploitation
According to our experience of operating an application for servicing the telecommunications operator's network infrastructure with hundreds of active users during business hours, it generates an average load equivalent to 8 synthetic test streams with a 30% write operation. In this mode, IBM FlashSystem 820 showed a read delay of 4 Kbytes of data block of 270 µs, which is an excellent result.
In conclusion, I would like to note that the IBM FlashSystem 820 network storage fully met our expectations for the speed of access to user data and the introduced delays. The flash memory itself, as well as various devices built on its basis, are increasingly entering the corporate equipment market. Flash memory provides maximum performance, which, coupled with minimal latency, will satisfy even the most demanding customers who have the highest demands on performance and reliability of storage systems. The modern solution, capable of demonstrating unsurpassed performance, is the IBM FlashSystem network storage line, with one of which - FlashSystem 820 - we got to know each other quite closely today. IBM FlashSystem 820
storage systemsdesigned to accelerate the work of a variety of enterprise applications, including databases for online transaction processing (OLTP) and online data analysis (OLAP), virtual desktop infrastructure, technical computing applications and scalable cloud infrastructures. This system provides the highest level of performance per gigabyte of data, allowing organizations to quickly analyze data using traditional tools as well as using new technologies designed to analyze Big Data. However, for storing large amounts of archival information that does not require high-speed access, the IBM FlashSysem 820 storage system is clearly redundant. Regardless of whether you are able to independently determine the necessary equipment or not, the company's specialistsSAFEDATA will help you with the selection, acquisition, placement, configuration and support of any network equipment and storage systems.
We would also like to thank IBM for providing the equipment for testing and assistance in the setup and measurement process.
It should be immediately noted that SAFEDATA customers do not have to independently acquire and maintain expensive equipment, these tasks can be delegated to managers and engineers of the company, taking the required devices for rent . This cycle of articles will allow you to determine the specific model of the switch, router or storage, the most suitable for solving the task.
We decided to start a series of reviews and tests with the IBM FlashSystem 820 storage system, which demonstrates amazing performance and reliability.
Description
The continuous development of information technology leads to the fact that the performance of servers and storage systems begins to play an increasing role. The storage tasks are no longer limited to simply storing user data; today they are required to provide high-performance access to the information placed on them. High access speeds and low response times can provide devices built on the basis of flash memory. IBM FlashSystem 820 network storage uses eMLC (enterprise MLC) -based flash memory, which is more reliable than MLC (Multi Level Cell) by adding ECC and increasing the number of write cycles to 30,000 (regular MLC supports up to 10,000 erase / dubbing). Also, according to the manufacturer, modules based on SLC (Single Level Cell) memory are supported with the number of erase / rewrite cycles equal to 100,000. There are two versions of FlashSystem 820, differing in the amount of pre-installed flash memory: 10 or 20 TB. Installed flash memory modules can be combined into a RAID0 or RAID5 array. The main technical characteristics of IBM FlashSystem 820 storage systems are presented in the table below.
| Characteristic | Value | |
| Model | 9831-AE2 | |
| Type of flash memory | eMLC | |
| Modification | 10 TB | 20 TB |
| Available RAID0 Capacity | 12.4 TB | 24.7 TB |
| Available RAID5 Capacity | 10.3 TB | 20.6 TB |
| Minimum recording delay | 25 µs | |
| Minimum read delay | 110 µs | |
| 100% read 4 KB | 525,000 IOPS | |
| 100% write 4 KB | 280,000 IOPS | |
| 100% read 256 KB | 3.3 GB / s (Fiber Channel), 5 GB / s (InfiniBand) | |
| 100% write 256 KB | 2.8 GB / s (Fiber Channel), 2.8 GB / s (InfiniBand) | |
| Nutrition | 300 W | |
| Cooling | 1023 British thermal units (BTU) / hour | |
| Connection Interfaces | 4 8 Gbps FC ports, 4 40 Gbps QDR IB ports | |
| Maximum number of supported volumes (LUN) | 1024 | |
| Supported RAID Levels | 0, 5 | |
| Body dimensions (W x D x H) | 1U x 432 mm x 638 mm | |
| Weight | 13.3 kg | |
| Management interfaces | HTTP, SSH, Telnet | |
| Reliability | High Availability Hardware Configuration Two-dimensional RAID technology in flash modules Variable Stripe RAID at Module Level RAID 5 system level in each module Hot-swappable flash modules Backup Interfaces | |
Perhaps now is the time to explain what we mean by abbreviations KB, MB, GB and so on. By KB, we mean 1024 bytes, MB - 1024 KB ... That is, GB will be equal to 2 ^ 30 bytes, not 10 ^ 9. This time we decided to abandon the formally more correct prefixes of kibi, mebi, bebi and tebi, that is, from the values ​​of KiB, MiB, GiB and TiB, so as not to embarrass some of our readers who are not yet ready for such boxes. In the next article we will immediately use the correct abbreviations.
')
The difference between the modifications lies in the used flash memory modules, the modification of 20 TB is dual.
We also decided to list the functionality and the benefits they provide.
- IBM Variable Stripe Redundant array of independent disks (RAID) technology allows you to simultaneously maintain performance and reliability without reducing the available capacity.
- An architecture without a single point of failure ensures enterprise-class reliability and maximum efficiency in even the most demanding data centers.
- Superior flash memory technology provides capacity combined with performance when using eMLC flash drives.
- 2D RAID technology, hot-swappable flash modules and redundant components with built-in auxiliary power supplies improve data availability and IT infrastructure productivity.
VSR (Variable Stripe RAID) technology provides data protection at the level of a memory page, a block, or even a whole chip. This protection allows you to avoid replacing the entire flash module in situations where a single chip fails. As a result, the frequency of servicing FlashSystem 820 series storage facilities is significantly reduced.
The basic elements of the IBM FlashSystem 820 network storage architecture are presented in the diagrams below.
The FlashSystem 820 model is not the only IBM storage system built on the basis of flash memory modules. This vendor also offers FlashSystem 710, 720 and 810 models. The ratio of performance and availability for all four models is presented below. Speaking of increased availability, it is worth noting that the FlashSystem 720 and 820 models implement 2D Flash RAID data protection technology.
At the time when we were preparing our test bench for the first measurements, IBM released a new data storage system - FlashSystem 900 . This model offers even greater performance: up to 1.1 MIOPS for read operations, 800 KIOPS for mixed mode and 600 KIOPS when performing data recording. IBM FlashSystem 900 uses MLC flash memory, developed in collaboration with Micron. The new flash memory eliminated one of the main drawbacks of MLC chips - the number of supported erase / rewrite cycles: according to the manufacturer, the life cycle of new chips is now not inferior to that for eMLC memory. The company IBM decided in principle to abandon the use of eMLC flash memory in new models of storage.
We now return directly to the model under discussion - FlashSystem 820 - and consider the possibilities for setting it up.
Customization
IBM FlashSystem 820 network storage is managed using a java application launched by the browser when accessing the device via HTTP or HTTPS protocols. When you log in, you need to enter your login and password, and also indicate whether you need to use encryption.
After you enter the correct credentials, the administrator opens the main window of the Monitoring Utility program, which allows you to configure and collect statistics from several FlashSystem storage systems simultaneously.
We will not consider in detail all the possibilities of this control system, however, we will dwell on the most interesting ones in our opinion.
The Options menu allows you to add storage systems in manual or automatic mode, connect to several FlashSystem at once, perform multiple firmware updates, display statistics for several storage systems.
The utility settings are mainly related to the choice of the method for detecting storage systems on the local network.
Here it is worth noting that detection refers to the automatic search and addition of control modules of FlashSystem. The Actions menu allows you to turn off or reboot devices, save or restore configuration, break control connection, or remove storage from the list.
The two tabs located at the bottom (“Recent Event Log” and “Task Monitor”) display a list of events occurring with the device, as well as a set of tasks performed by the administrator.
What actions can be performed with the device itself? The “Logical Units” menu group allows you to manage “virtual disks” (LUN): create partitions of the required size, delete unnecessary partitions, and also change access parameters.
The “Storage” group contains all flash drives installed in the storage. You can install flash drives with a volume of 1 TB and 2 TB. All installed flash cards can be combined into a RAID array.
Two types of arrays are supported: maximum capacity (analog RAID0) and RAID5. To increase reliability, it is recommended to use a RAID5 array called IBM 2D Flash RAID, since the RAID5 array inside the module itself is also used, which ensures the fault tolerance of the entire module when one of ten chips fails (nine store user data, one is used for parity). It is worth noting that when creating a RAID5 array, not all modules are used to store user data: ten modules contain data, one is used to store parity information and another is not used, it is designed for hot-swapping a failed module. Thus, user data will remain available even if two modules fail, naturally, not at the same time. The diagram below shows the architecture of such a 2D Flash array.
Interfaces group displays information about installed interface modules and their ports.
Information on air temperature and system components, power settings, fan performance and battery status can be found in the Environmental group. You can make sure that the batteries installed inside the battery case can be used with the help of a periodic test, which can also be scheduled here.
The settings of the management module parameters are collected in the Management group. Here the administrator can specify the IP parameters of both network cards and the address of the DNS server.
Time synchronization via NTP is managed using the Date / Time item, while the Users item allows you to manage local or domain users.
The Firmware item is for updating the firmware. Unfortunately, the new firmware versions are available only to users who have acquired special support (service contract), so we will not be able to show the entire process of updating the firmware. Here, perhaps, it is worth saying that SAFEDATA customers, who rent equipment, have at their disposal devices with the latest and most stable firmware. In addition, an additional equipment support service can be ordered, allowing you to perform all the necessary maintenance work on servers, data storage systems and other network equipment during the entire period of using the data center services, which also includes maintaining firmware in the current state.
The Services group allows the administrator to specify the connection settings to FlashSystem using SNMP, Telnet, and SSH protocols. It is also worth noting that when certain events occur, an alert can be sent to the administrator via e-mail, which is also configured here.
The Statistics group is responsible for collecting and displaying statistics. Here, the administrator can view the current use of the system, build graphs of the load on the network interfaces, display the number of operations performed per second, and so on. A variety of counters are available for each component of the system.
The storage system FlashSystem 820 is equipped with two management modules (MCP - management controller ports). The module can be in active or passive mode. The MCP in active mode controls the LUNs, and also allows you to view the status of system components. A passive module is used to provide fault tolerance in the event of an active MCP failure. The Cluster tab displays information about the MCP control modules and allows you to switch the active MCP, that is, the “cluster” is assembled from two modules installed in one FlashSystem 820.
Access to the log information of the system can be obtained using the Logs item.
This concludes our discussion of the IBM FlashSystem 820 web storage system, and finally we would like to mention that device management is possible not only using a browser and a graphical interface, but also using the command line when connecting using Telnet and SSH protocols.
We are now engaged in testing IBM FlashSystem 820.
Security testing
Connecting to IBM FlashSystem 820 can be done with two fundamentally different goals: for data acquisition and for management. The instance of FlashSystem 820, which we tested, was equipped with four FibreChannel ports for data transfer, therefore, it is worth talking about device security only in the context of security of the control module. Of course, we understand that the management interface will be connected to some internal secure network, access to which will be limited to access lists on routers or other means. However, we were interested in the availability of available services in principle. For testing, we used Positive Technologies XSpider 7.7 (Demo build 3100) network security scanner. A total of four open ports were detected: TCP-22 (SSH), TCP-23 (Telnet), TCP-80 (HTTP) and TCP-443 (HTTPS). The most interesting information is presented below.
We now proceed directly to the load testing.
File tests
Before we begin to describe the methodology of the load testing carried out and provide the results, we would like to especially note that we did not set ourselves the goal of getting the maximum MB / s or IOPS. The only goal we pursued is to find out what real speeds will be available to users when connecting to the storage of one or two servers. As we will show later, the values ​​we obtained are not limitations of FlashSystem 820, but of the equipment used for the tests. Thus, in order to “squeeze” the maximum out of the device, you will need to connect several productive servers to it at the same time. And although this storage system was created for the ultra-fast service of requests from customers' applications, as well as to speed up the existing storage systems with conventional hard drives, we still decided to take measurements of file access rates.
The main parameters of the test bench are listed below.
- Server number 1. IBM x3850, 32 cores.
- Server number 2. IBM x3650 M4, 24 cores.
- FC-card QLOGIC QLE2564.
- FC switch Brocade 300.
- IBM FlashSystem 820 9831-AE2 storage.
The first measuring tool was the utility CrystalDiskMark version 3.0.3. First, we made measurements for connecting using a single FC link.
Then they connected the second FC link and enabled balancing by adding the Multi-Path I / O component.
We carried out the following measurements using Intel NASPT version 1.7.1. When testing Intel NASPT using the msconfig utility, the RAM available to the operating system was reduced in accordance with Intel's recommendations in order to reduce the effect of local caching on measurement results. First, we measured the data access speeds with one FC link between the server and the storage for three file systems: NTFS, FAT32 and exFAT.
As can be seen from the diagram above, in some tests we are very close to the performance of the 8GFC environment (including 8b / 10b encoding) - 8 Gbit / s. For the NTFS file system, we decided to compare the performance of the system with single and double server connections to the storage. The speeds obtained with the help of double connection turned out to be lower than the values ​​that we managed to get with a single connection. We must admit that at this stage we were somewhat discouraged. Of course, we later discovered the reason - high CPU usage with MPIO turned on, but first things first.
The next testing tool is the Intel IOMeter utility, which allows both creating an absolutely synthetic load in the form of pure reading or writing, and using patterns that emulate the behavior of various servers and workstations. Synthetic load is represented by three tests: 100% reading, 100% writing, as well as 50% reading and 50% writing. The diagram below shows the results of each of the three synthetic tests when connected using one FC link, depending on the size of the data block. The testing involved two servers connected to the Flash System 820 using an FC switch.
As can be seen from the diagrams presented, as the block size increases, the data access speeds increase until the obvious limit is reached - the performance of a single Fiber Channel interface, that is, 8 Gbit / s.
In addition to measuring storage performance in terms of bandwidth, we also measured the number of operations performed by a device per unit time (IOPS). The diagrams below show the results of the same measurements, expressed in thousands of IOPS.
For any write operations, a peak of performance is seen with a block size of 4 kilobytes, due to the size of the internal data structure.
Naturally, we decided to make the same measurements, but when connected to the storage using two FC links, so that the Fiber Channel interface was not a bottleneck of our test bench.
As expected, we found a significant increase in system performance. However, when reading blocks of 4 and 8 kilobytes in size, we were close to the performance of two FC interfaces, which makes us think that the performance of the IBM FlashSystem 820 network storage is even higher, that is, using all four FC ports, even our two test servers could be obtained greater speed of access to user data. The diagrams below show device performance in thousands of IOPS when connected using two FC links.
As with the connection using a single link, there is a pronounced peak in performance on data blocks of 4 kilobytes.
It is time to move away from synthetic measurements and make our stand simulate the load produced by various servers and workstations. The following four storage scenarios were selected: Database, Fileserver, Workstation, and Webserver. The storage performance when connected using a single FC interface is presented below.
We repeated the same measurements, but using two FC connections already.
The results obtained, as well as for synthetic tests, clearly indicate that connecting to FlashSystem 820 using one FC link, the bottleneck will be the FC connection itself.
In all the following tests in this section, we used Server # 1 and read data blocks of 4 kilobytes in size. First, we compared system performance versus request queue depth using a single thread.
Perhaps, it is worth giving some explanations to the last graph, which shows the time of the request to read a block of data from the storage. While the system is not overloaded, reading is performed in approximately 200 µs and is almost independent of the number of physical connections, which is a very worthy result.
Then we decided to find out the dependence of system performance on the number of threads with a queue depth of 4.
The only thing we would like to mention here is the increased processor load when using MPIO. That is, when load balancing between multiple physical FC interfaces, the Windows operating system spends precious processor cycles. For example, in almost all tests without MPIO, the CPU load did not exceed 10% (ranged from 5-7%), while the use of MPIO led to almost one hundred percent CPU utilization. Looking ahead, we note that a similar effect was not observed when testing FlashSystem 820 with the Oracle database running on the Oracle Linux Server release 6.6 operating system.
This concludes the IBM file storage tests and proceeds to measuring storage performance when working with an Oracle database.
Database Tests
Test Objective
To determine the main performance characteristics of the IBM FlashSystem 820 storage system when used with an Oracle database application.
Testing method
To test the performance of the storage used by the Oracle database, the SLOB2 utility (The Silly Little Oracle Benchmark) from Kevin Closson was used. He is a former performance architect of the Oracle Exadata Machine. More information about the utility is presented in the author's blog.
Testing involves reading / writing multiple threads of execution. A diagnostic AWR (Active Workload Repository) report is collected for each test. It serves as a source of information about the actual performance of the disk subsystem within a test.
Platform
Operating system: Oracle Linux Server release 6.6 with kernel 3.8.13-55.1.5.el6uek.x86_64.
Processor: Intel Xeon CPU E7-4830 @ 2.13 GHz, 16 cores, 32 processors
File system block size: 4 Kbytes
Test parameters
24 measurements were taken. The following varied:
- The number of threads (from 1 to 128 with a double increase).
- The ratio of the number of read operations to write operations (Read Only, Write Only and “70% Read, 30% Write”).
- The size of the block used by the database.
- Use multipath I / O.
Test results
4 KB, without MPIO.
4 KB using MPIO.
8 KB, without MPIO.
8 KB using MPIO.
IOPS and throughput
Maximum reading performance of 62K IOPS was achieved with a block size of 4 KB on 32 threads and Multipath I / O disabled. The throughput at the same time amounted to 242 MB / s. Measurements of blocks of 8 KB in size with Multipath I / O disabled show an average of 5.6% worse reading IOPS. The throughput at the same time increases on average by 88% to 457 MB / s for 32 streams.
Multipath I / O
The inclusion of Multipath I / O adversely affects the results. With a block size of 8 KB, we get a loss of about 20% on reading readings of 16 threads and 25% when testing on 32 threads. In the 30/70 mixed mode, the peak of IOPS is also reduced by 20% and 25% for a block size of 4 KB and 8 KB, respectively. However, with fewer threads, Multipath I / O contributes to improved IOPS and throughput. So, with a block size of 4 Kbytes, measurements on 4 and 8 streams showed 12% better results in mixed read / write mode.
30/70 mode
The case “30% of operations is write, 70% is read” corresponds to the real operation mode of the database as part of a multi-user application. “Saturation” occurs at 16 streams. In this case, the read delay increases from 240 µs on 1 stream to 400 µs on 16. The longest response of the carrier was obtained when testing on 64 threads - 950 µs. At 128 threads, the delay is less due to the fact that a struggle for resources between the execution threads took place on the testing system. This led to the fact that the generated load on the media in read / write mode has decreased.
Delay
The maximum allowable for a database is read latency for modern HDD drives is 4 ms, which is an order of magnitude more than the maximum values ​​obtained for FlashSystem 820. It is worth noting that for relational databases the delay parameter is critical, since the task of obtaining necessary records from tables usually consists of consecutive accesses to data from files. That is, to get a block containing the desired row from the table, you first need to find this information by reading several blocks of the index. The high latency allowed for file storage can have a strong impact on database performance.
Correlation with real exploitation
According to our experience of operating an application for servicing the telecommunications operator's network infrastructure with hundreds of active users during business hours, it generates an average load equivalent to 8 synthetic test streams with a 30% write operation. In this mode, IBM FlashSystem 820 showed a read delay of 4 Kbytes of data block of 270 µs, which is an excellent result.
Conclusion
In conclusion, I would like to note that the IBM FlashSystem 820 network storage fully met our expectations for the speed of access to user data and the introduced delays. The flash memory itself, as well as various devices built on its basis, are increasingly entering the corporate equipment market. Flash memory provides maximum performance, which, coupled with minimal latency, will satisfy even the most demanding customers who have the highest demands on performance and reliability of storage systems. The modern solution, capable of demonstrating unsurpassed performance, is the IBM FlashSystem network storage line, with one of which - FlashSystem 820 - we got to know each other quite closely today. IBM FlashSystem 820
storage systemsdesigned to accelerate the work of a variety of enterprise applications, including databases for online transaction processing (OLTP) and online data analysis (OLAP), virtual desktop infrastructure, technical computing applications and scalable cloud infrastructures. This system provides the highest level of performance per gigabyte of data, allowing organizations to quickly analyze data using traditional tools as well as using new technologies designed to analyze Big Data. However, for storing large amounts of archival information that does not require high-speed access, the IBM FlashSysem 820 storage system is clearly redundant. Regardless of whether you are able to independently determine the necessary equipment or not, the company's specialistsSAFEDATA will help you with the selection, acquisition, placement, configuration and support of any network equipment and storage systems.
We would also like to thank IBM for providing the equipment for testing and assistance in the setup and measurement process.
Source: https://habr.com/ru/post/258701/
All Articles