The future of flash memory: IBM FlashSystem
Time is a key factor in business, especially in business-critical applications. Those who have seen the Apollo 13 film, or older people who remember this ill-fated space flight, know what happens when critical systems fail. According to the plot of the film, three American astronauts are sent to the moon, but the spacecraft explodes on the way and suffers serious damage, but they only survived by a miracle. The astronauts manage to return home only thanks to their heroic efforts and a bit of luck. Fortunately, the consequences of the failure of critical business applications are rarely as serious (compared to, say, applications that support the work of services and institutions, for example, in intensive care units, air traffic control services, etc.). But on the other hand, failures of such applications directly affect the viability of the company: if the systems work flawlessly, the company is economically healthy and develops, if not, it is hardly kept afloat.
Business-critical applications include applications for a variety of tasks, for example: business intelligence (BI - business intelligence), online transaction processing (OLTP), online analytical processing (OLAP), virtual workers' infrastructure tables, high-performance computing systems and content delivery applications (cloud storage, video on demand, etc.). All these important tasks are united by one thing: high requirements for speed of work, because it depends on how quickly management, employees, customers and other key business partners get access to the necessary information.
The inability to quickly access and process sensitive data leads to significant business and financial risks. In particular, such delays cause:
- The inability to promptly and correctly assess the situation - not having access to critical data at the right time and in the right place, people do not see the full picture of what is happening and, as a result, make the wrong decisions, allowing for completely unnecessary mistakes.
')
- A sharp decline in the ability to predict business results - the quality of financial forecasts at the level of an individual project, a separate activity line and the entire company depends on the ability to obtain the most accurate data in a timely manner. Without access to such data, the company cannot analyze them, which negatively affects its activities.
- Growing customer dissatisfaction - in many industries (financial services, health care, tourism, etc.) the quality of service directly depends on the availability of instant access to relevant information, especially due to the fact that a modern customer expects service around the clock for 7 days week and 365 days a year. Under these conditions, the inability to promptly provide customers with the necessary information not only means lost profits, but often leads to the loss of a client, and even market share.
- Too large volumes, diversity and speed of information flow - the explosive growth of data volumes leads to the inevitable decline in performance of mainstream applications in many enterprises. Increased delays become a source of user dissatisfaction, and businesses are missing out on opportunities.
IT managers who are looking for a solution to these problems, which would have the best combination of functionality, implementation time, scalability and cost of the systems, are not to be envied. Ultimately, the right decision may be far from obvious.
Task setting (hint: processor speed - wrong answer)
As a rule, IT slows down the system by first adding or upgrading processors and memory. However, too often, the performance gain from this upgrade is minimal, while the cost is high. The second method, often accompanying the first, at least in relation to databases, is the optimization of SQL statements. Enterprises spend millions of dollars on it and even get some results. But they overlook the fact that the most ideal database will be powerless in front of the low speed of the storage system, and the latter is often the source of the problem.
Indeed, the root cause of problems with application performance is often in the storage system, and not in processor speed or low memory. When the processor is idle while waiting for information from the storage system, precious time and resources are wasted. And given the fact that in the past 20 years the processing power of processors is growing exponentially, this simple one becomes more and more. Simply put, the computing resources of modern processors far exceed the speed at which hard drives can deliver data to them. This performance gap is especially acute in database servers, where the number of I / O operations is usually much larger than in other systems. As a result, ultra-fast processors and huge bandwidth are often not fully utilized, because access to data on hard drives takes several milliseconds, which is an eternity for modern processors. And when servers wait for data from storage systems for a long time, users wait for response from servers for a long time. This is where the real source of the problem with slow I / O operations is.
This problem is due to purely physical reasons. The movement of data on solid-state drives and through the network occurs at a speed comparable to the speed of the current in the electrical circuit, i.e., in fact, the speed of light, while hard drives are mechanical devices that read information through the rotation of the plate. Thus, with all its reliability and performance, hard drives in terms of access speed are clearly the weakest link in the information transfer chain.
Disadvantages of traditional ways to work around HDD speed problems
Performance problems due to the inefficiency of hard drives are tried to be solved in several ways, in particular, by creating JBOD arrays (just a bunch of disks - just a set of disks) or RAID (redundant array of independent disks - a redundant array of independent disks). With the increase in the number of disks, it becomes possible to distribute the I / O operations carried out by the database across several disks. Unfortunately, this gives a too modest increase in HDD performance.
It is also customary to move frequently used files to a separate disk. The speed of I / O operations in one disk at the same time really grows to its maximum, except that even at maximum this speed is still low. At best, one hard disk yields ~ 200, without the use of additional functionality, I / O operations per second (IOPS - input / output operations per second), which is several times lower than the speed that could reduce the performance gap between the processor and HDD.
The third traditional way to improve HDD performance is to replace JBOD with RAID. Performance as a result really grows, especially if you connect disks to a controller with a large cache that will distribute data across different disks. But at the same time, in order to achieve the high speed of input-output operations expected by users, such a number of disks will be required that their cost, occupied space, power consumption, heat generation and weight will simply be unacceptable.
Answer: flash-only storage systems
The performance gap between the processing power of servers and storage systems on hard drives, especially acutely felt in mission-critical business applications, requires fundamentally different, faster storage systems.
Flash-based storage systems are devices with solid-state drives designed to solve the problem of insufficient I / O operations; their speed of access to data and I / O operations is several times better than that of hard drives. These storage systems can complement or completely replace traditional hard disk arrays for many mission-critical business applications, including ERP systems, transactional databases, and analytical applications (business analytics, etc.). Moreover, due to the gradual decline in prices for these new devices, it is quite possible today to build a whole storage system based on a single flash memory.
In the broad sense of the word, SSD (solid state drive - solid-state drive) is understood as any input-output device, devoid of mechanical components. Recently, however, the term SSD is mainly used to refer to solid-state drives that have the traditional form factor of a hard disk and are replacing it. SSDs of such a form factor should not be confused with flash-based storage systems. Interfaces and controllers of such SSD-drives are based on traditional hard disk technologies, which were created based on the inherent HDD access delays and bandwidth limitations. Storage systems based on flash memory are initially developed based on the features of this technology and based on a fast FPGA controller, aiming at minimizing latency and getting maximum throughput.
Modern storage systems based on flash memory I / O latency is reduced due to the fact that access speed is 250 times faster than hard drives (0.2 ms versus 5 s). And the increase in access speed, in turn, allows flash memory to issue 2000 times more I / O operations per second than HDDs (more than 400,000 IOPS versus 200). This improvement can dramatically reduce information access delays due to the slowness of the storage system.
The introduction of flash drive technology or the replacement of existing disks allows servers of any architecture to increase the capabilities of the CPU by the number of tasks performed per unit of time. So, using IBM Power Systems as an example, server performance gains when working with Oracle databases reached up to 40% in open testing, which can significantly increase the efficient utilization of the existing infrastructure. That is, a server without a configuration change will be able to fulfill 40% more requests in the same period of time.
IBM FlashSystem Family
IBM considers flash memory as a strategic storage technology and aims to be at the forefront of developing flash-only storage systems. (One consequence of this policy was the purchase in October 2012 of Texas Memory Systems (TMS), a high-end manufacturer of powerful and reliable flash-based storage systems.)
Targeted use of IBM FlashSystem helps enterprises increase their flexibility and more efficiently use analytics - staff can at any time quickly obtain information that is always up-to-date, as it comes in real time, and not with long delays. In addition, this system helps to optimize data centers and consolidate resources, resulting in increased efficiency of business processes and the operation of mission-critical applications. The stability of the systems also increases, without loss of performance and available capacity.
Flash-only storage systems have more storage capacity than all previous storage technologies. This is because they do not require additional batteries to copy the DDR cache in the event of a power outage, as well as they do without a large number of expensive DDR memory modules. The amount of DDR memory they need is minimal — it serves as a write buffer for flash memory and storage of metadata during operation. The power needed to copy a small cache and metadata to flash memory during power outages is provided by small batteries. A flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size.
The IBM FlashSystem products use FlashCore technology which allows using the storage type of the multi-level class MLC to get storage and performance indicators no worse than SLC.
FlashSystem storage systems are capable of 1,000,000 read operations per second with a delay of less than 100 microseconds and are so compact that in one 2U compartment you can place devices with a capacity of up to 57 TB. Another important advantage is the high level of availability and reliability that is mandatory for enterprises: the lack of components, the failure of which leads to the failure of the entire system, the presence of several levels of data correction, backup chips and components with hot-swappable support.
IBM FlashSystem products have the lowest latency and highest IOPS among competitors and are attractive in terms of total cost of ownership. It is necessary to take into account that it is not the individual parameters that are compared, but the sum of the response time and the number of I / O operations. They can be used as storage systems t. Tier 0 (Tier 0, i.e., superior in capabilities to traditional Tier 1 — storage systems required for enterprise priorities) in combination with SVC (IBM System Storage SAN Volume Controller). IBM FlashSystem will also be particularly useful when the performance of mission-critical business applications is directly dependent on cooling, low power consumption and compact size. Today, there are two main models of IBM FlashSystem 900 and IBM FlashSystem V9000. The first one occupies 2 units and provides high performance while the second system occupies 6U and, in addition to its high performance, has a set of High End functionality. That allows you to optimize the use of volume, more efficiently distribute the load, create storage clusters both within the data center and geographically separated. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line.
Economic Benefits of IBM FlashSystem
Storage systems based only on flash memory have not only technical advantages, but also a number of economic benefits as compared to traditional HDD-based systems. For example, the cost of licensing software for IBM FlashSystem storage systems is 50% lower than for HDD-based systems.
In addition, a much smaller size and higher data storage density can significantly reduce the footprint. As noted above, a flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size. Thus, in one rack can accommodate a system with a capacity of more than 1 petabyte.
Flash-only data storage systems are also much more energy efficient than comparable HDD-based systems — energy costs are 75% lower.
- The cost of licensing software for IBM FlashSystem is 50% lower than that of systems based on HDD.
- IBM FlashSystem takes up much less space and saves up to 75% of power compared to HDD.
- The cost of operational support for systems based on flash memory is 35% lower compared to HDD
- The cumulative cost of storage systems based only on flash memory is 31% less than that of the HDD
Flash transition
The key to maximum impact and cost-effectiveness of flash memory is its strategic use. A flash-only storage system is useful for many applications, but not all; First of all, it is needed by the main enterprise applications. The key to high performance applications, user satisfaction, optimal TCO is the ability to identify those applications that will benefit from switching to flash-based storage systems, and those that are enough for traditional HDD-based systems or hybrid systems based on a combination of these technologies.
Poor application performance is usually associated with a large amount of simultaneous and often highly complex database queries. If the weak link in this case is the I / O subsystem, then it is necessary to identify the components of the database that operate under the highest load and thus cause a delay in I / O operations.
In some cases, entire databases are transferred to flash-based storage systems. This is usually a database with a standardly high amount of simultaneous queries or regular random access to all database tables, when isolating a subset of the most frequently used files is not possible.
Well suited for transfer to storage systems based on flash memory also small and medium-sized databases, because such solutions are extremely attractive in price, especially compared to RAID-systems. The same is true for large databases with a large number of read requests.
Other arguments in favor of flash memory are such disadvantages of large disk arrays from the HDD, such as large footprint and high power consumption.
Business-critical applications include applications for a variety of tasks, for example: business intelligence (BI - business intelligence), online transaction processing (OLTP), online analytical processing (OLAP), virtual workers' infrastructure tables, high-performance computing systems and content delivery applications (cloud storage, video on demand, etc.). All these important tasks are united by one thing: high requirements for speed of work, because it depends on how quickly management, employees, customers and other key business partners get access to the necessary information.
The inability to quickly access and process sensitive data leads to significant business and financial risks. In particular, such delays cause:
- The inability to promptly and correctly assess the situation - not having access to critical data at the right time and in the right place, people do not see the full picture of what is happening and, as a result, make the wrong decisions, allowing for completely unnecessary mistakes.
')
- A sharp decline in the ability to predict business results - the quality of financial forecasts at the level of an individual project, a separate activity line and the entire company depends on the ability to obtain the most accurate data in a timely manner. Without access to such data, the company cannot analyze them, which negatively affects its activities.
- Growing customer dissatisfaction - in many industries (financial services, health care, tourism, etc.) the quality of service directly depends on the availability of instant access to relevant information, especially due to the fact that a modern customer expects service around the clock for 7 days week and 365 days a year. Under these conditions, the inability to promptly provide customers with the necessary information not only means lost profits, but often leads to the loss of a client, and even market share.
- Too large volumes, diversity and speed of information flow - the explosive growth of data volumes leads to the inevitable decline in performance of mainstream applications in many enterprises. Increased delays become a source of user dissatisfaction, and businesses are missing out on opportunities.
IT managers who are looking for a solution to these problems, which would have the best combination of functionality, implementation time, scalability and cost of the systems, are not to be envied. Ultimately, the right decision may be far from obvious.
Task setting (hint: processor speed - wrong answer)
As a rule, IT slows down the system by first adding or upgrading processors and memory. However, too often, the performance gain from this upgrade is minimal, while the cost is high. The second method, often accompanying the first, at least in relation to databases, is the optimization of SQL statements. Enterprises spend millions of dollars on it and even get some results. But they overlook the fact that the most ideal database will be powerless in front of the low speed of the storage system, and the latter is often the source of the problem.
Indeed, the root cause of problems with application performance is often in the storage system, and not in processor speed or low memory. When the processor is idle while waiting for information from the storage system, precious time and resources are wasted. And given the fact that in the past 20 years the processing power of processors is growing exponentially, this simple one becomes more and more. Simply put, the computing resources of modern processors far exceed the speed at which hard drives can deliver data to them. This performance gap is especially acute in database servers, where the number of I / O operations is usually much larger than in other systems. As a result, ultra-fast processors and huge bandwidth are often not fully utilized, because access to data on hard drives takes several milliseconds, which is an eternity for modern processors. And when servers wait for data from storage systems for a long time, users wait for response from servers for a long time. This is where the real source of the problem with slow I / O operations is.
This problem is due to purely physical reasons. The movement of data on solid-state drives and through the network occurs at a speed comparable to the speed of the current in the electrical circuit, i.e., in fact, the speed of light, while hard drives are mechanical devices that read information through the rotation of the plate. Thus, with all its reliability and performance, hard drives in terms of access speed are clearly the weakest link in the information transfer chain.
Disadvantages of traditional ways to work around HDD speed problems
Performance problems due to the inefficiency of hard drives are tried to be solved in several ways, in particular, by creating JBOD arrays (just a bunch of disks - just a set of disks) or RAID (redundant array of independent disks - a redundant array of independent disks). With the increase in the number of disks, it becomes possible to distribute the I / O operations carried out by the database across several disks. Unfortunately, this gives a too modest increase in HDD performance.
It is also customary to move frequently used files to a separate disk. The speed of I / O operations in one disk at the same time really grows to its maximum, except that even at maximum this speed is still low. At best, one hard disk yields ~ 200, without the use of additional functionality, I / O operations per second (IOPS - input / output operations per second), which is several times lower than the speed that could reduce the performance gap between the processor and HDD.
The third traditional way to improve HDD performance is to replace JBOD with RAID. Performance as a result really grows, especially if you connect disks to a controller with a large cache that will distribute data across different disks. But at the same time, in order to achieve the high speed of input-output operations expected by users, such a number of disks will be required that their cost, occupied space, power consumption, heat generation and weight will simply be unacceptable.
Answer: flash-only storage systems
The performance gap between the processing power of servers and storage systems on hard drives, especially acutely felt in mission-critical business applications, requires fundamentally different, faster storage systems.
Flash-based storage systems are devices with solid-state drives designed to solve the problem of insufficient I / O operations; their speed of access to data and I / O operations is several times better than that of hard drives. These storage systems can complement or completely replace traditional hard disk arrays for many mission-critical business applications, including ERP systems, transactional databases, and analytical applications (business analytics, etc.). Moreover, due to the gradual decline in prices for these new devices, it is quite possible today to build a whole storage system based on a single flash memory.
In the broad sense of the word, SSD (solid state drive - solid-state drive) is understood as any input-output device, devoid of mechanical components. Recently, however, the term SSD is mainly used to refer to solid-state drives that have the traditional form factor of a hard disk and are replacing it. SSDs of such a form factor should not be confused with flash-based storage systems. Interfaces and controllers of such SSD-drives are based on traditional hard disk technologies, which were created based on the inherent HDD access delays and bandwidth limitations. Storage systems based on flash memory are initially developed based on the features of this technology and based on a fast FPGA controller, aiming at minimizing latency and getting maximum throughput.
Modern storage systems based on flash memory I / O latency is reduced due to the fact that access speed is 250 times faster than hard drives (0.2 ms versus 5 s). And the increase in access speed, in turn, allows flash memory to issue 2000 times more I / O operations per second than HDDs (more than 400,000 IOPS versus 200). This improvement can dramatically reduce information access delays due to the slowness of the storage system.
The introduction of flash drive technology or the replacement of existing disks allows servers of any architecture to increase the capabilities of the CPU by the number of tasks performed per unit of time. So, using IBM Power Systems as an example, server performance gains when working with Oracle databases reached up to 40% in open testing, which can significantly increase the efficient utilization of the existing infrastructure. That is, a server without a configuration change will be able to fulfill 40% more requests in the same period of time.
IBM FlashSystem Family
IBM considers flash memory as a strategic storage technology and aims to be at the forefront of developing flash-only storage systems. (One consequence of this policy was the purchase in October 2012 of Texas Memory Systems (TMS), a high-end manufacturer of powerful and reliable flash-based storage systems.)
Targeted use of IBM FlashSystem helps enterprises increase their flexibility and more efficiently use analytics - staff can at any time quickly obtain information that is always up-to-date, as it comes in real time, and not with long delays. In addition, this system helps to optimize data centers and consolidate resources, resulting in increased efficiency of business processes and the operation of mission-critical applications. The stability of the systems also increases, without loss of performance and available capacity.
Flash-only storage systems have more storage capacity than all previous storage technologies. This is because they do not require additional batteries to copy the DDR cache in the event of a power outage, as well as they do without a large number of expensive DDR memory modules. The amount of DDR memory they need is minimal — it serves as a write buffer for flash memory and storage of metadata during operation. The power needed to copy a small cache and metadata to flash memory during power outages is provided by small batteries. A flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size.
The IBM FlashSystem products use FlashCore technology which allows using the storage type of the multi-level class MLC to get storage and performance indicators no worse than SLC.
FlashSystem storage systems are capable of 1,000,000 read operations per second with a delay of less than 100 microseconds and are so compact that in one 2U compartment you can place devices with a capacity of up to 57 TB. Another important advantage is the high level of availability and reliability that is mandatory for enterprises: the lack of components, the failure of which leads to the failure of the entire system, the presence of several levels of data correction, backup chips and components with hot-swappable support.
IBM FlashSystem products have the lowest latency and highest IOPS among competitors and are attractive in terms of total cost of ownership. It is necessary to take into account that it is not the individual parameters that are compared, but the sum of the response time and the number of I / O operations. They can be used as storage systems t. Tier 0 (Tier 0, i.e., superior in capabilities to traditional Tier 1 — storage systems required for enterprise priorities) in combination with SVC (IBM System Storage SAN Volume Controller). IBM FlashSystem will also be particularly useful when the performance of mission-critical business applications is directly dependent on cooling, low power consumption and compact size. Today, there are two main models of IBM FlashSystem 900 and IBM FlashSystem V9000. The first one occupies 2 units and provides high performance while the second system occupies 6U and, in addition to its high performance, has a set of High End functionality. That allows you to optimize the use of volume, more efficiently distribute the load, create storage clusters both within the data center and geographically separated. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line.
Economic Benefits of IBM FlashSystem
Storage systems based only on flash memory have not only technical advantages, but also a number of economic benefits as compared to traditional HDD-based systems. For example, the cost of licensing software for IBM FlashSystem storage systems is 50% lower than for HDD-based systems.
In addition, a much smaller size and higher data storage density can significantly reduce the footprint. As noted above, a flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size. Thus, in one rack can accommodate a system with a capacity of more than 1 petabyte.
Flash-only data storage systems are also much more energy efficient than comparable HDD-based systems — energy costs are 75% lower.
- The cost of licensing software for IBM FlashSystem is 50% lower than that of systems based on HDD.
- IBM FlashSystem takes up much less space and saves up to 75% of power compared to HDD.
- The cost of operational support for systems based on flash memory is 35% lower compared to HDD
- The cumulative cost of storage systems based only on flash memory is 31% less than that of the HDD
Flash transition
The key to maximum impact and cost-effectiveness of flash memory is its strategic use. A flash-only storage system is useful for many applications, but not all; First of all, it is needed by the main enterprise applications. The key to high performance applications, user satisfaction, optimal TCO is the ability to identify those applications that will benefit from switching to flash-based storage systems, and those that are enough for traditional HDD-based systems or hybrid systems based on a combination of these technologies.
Poor application performance is usually associated with a large amount of simultaneous and often highly complex database queries. If the weak link in this case is the I / O subsystem, then it is necessary to identify the components of the database that operate under the highest load and thus cause a delay in I / O operations.
In some cases, entire databases are transferred to flash-based storage systems. This is usually a database with a standardly high amount of simultaneous queries or regular random access to all database tables, when isolating a subset of the most frequently used files is not possible.
Well suited for transfer to storage systems based on flash memory also small and medium-sized databases, because such solutions are extremely attractive in price, especially compared to RAID-systems. The same is true for large databases with a large number of read requests.
Other arguments in favor of flash memory are such disadvantages of large disk arrays from the HDD, such as large footprint and high power consumption.
Source: https://habr.com/ru/post/308304/
All Articles