New memory for new storage architecture
Over the past decade, the speed of processors has grown about 10 times, RAM - 8 times, networks - 100 times, the internal bus of computing systems - 20 times, and hard disk drives - just 1.5 times. That is, the increase in the performance of the server storage subsystem significantly lags behind the growth in the performance of the central processor, RAM, internal bus, and network bandwidth. The lack of performance of traditional hard drives (HDD) leads to an increase in application response time. Yesterday, the only solution was to use SSD or PCIe flash. Today we have gone a step further with a new type of drives directly on the RAM bus!
Partly to solve this problem helps an order of magnitude faster flash memory. New protocols and technologies are being developed for it that reveal the potential of a PCIe interface, for example, Non-Volatile Memory Express (NVMe). NVMe flash memory eliminates significant hardware delays and storage costs associated with storage and can be used to improve overall system performance, especially in demanding tasks such as virtualization, big data analytics and high performance computing (HPC). Using NVMe, you can access flash memory bypassing the SCSI layer, which reduces access time, especially in the case of hardware support for NVMe, when driver functions are implemented at the hardware level.
')
NVMe scalable technology was developed specifically for direct connection of NVMe devices (such as SSD) through a PCIe Gen3 interface. It provides fast processor access to data in flash memory and is characterized by high IOPS performance, low power consumption and low latency. Deliveries of the first NVMe products began in 2014. In 2015, many server vendors presented their platforms with NVMe slots.
NVMe allows you to improve system performance by more fully using the parallelism of devices and software. The drive becomes “closer” to the processor, which makes it possible to reduce the delay by more than 20 times.
Currently, solid state drives (SSD) in servers provide high speed data access and are sometimes used as cache memory or for storing intermediate data. Not surprisingly, flash drives are confidently conquering the market. According to the IHS company forecast, by 2017, SSD shipments in real terms will have reached half the volume of hard disk shipments.
In the arsenal of Hewlett Packard Enterprise - a whole range of drives with support for the protocol Non-Volatile Memory Express. These HPE solutions use different drive formats for various types of loads, including 2.5 "SFF and PCIe cards. Standard NVMe drivers are included in the most popular operating systems. However, NVMe is already becoming the best practice of yesterday. It is replaced at the top of the pyramid New technology - Persistent Memory.
HPE Persistent Memory technology (which means “non-volatile” or “permanent memory”) is not only new to the original architectural solution - placing a non-volatile flash drive on the memory bus - but also an integrated approach. It already has a driver for Microsoft Windows, an SDK for Linux, and in the near future plans to teach applications to understand this new media type, which will provide, for example, support for in-memory calculations on these modules without various tricks on data integrity.
What is HPE Persistent Memory? Simply put, this is non-volatile memory with RAM performance. Being non-volatile, this memory provides permanent data storage. HPE Persistent Memory, introduced in December 2015, is used in the 2016 HPE ProLiant product line. Most of these systems are already supplied.
Persistent Memory (or Non-Volatile Memory) is a combination of DRAM with flash memory that avoids data loss during a power outage. Since DRAM (DIMM) and NAND sit on the memory bus, this combination is faster than flash memory on a PCIe card.
According to its characteristics, HPE Persistent Memory (NVDIMM) occupies an intermediate position in the pyramid of different types of memory between DRAM and flash memory. In which applications does this new technology demonstrate its best qualities? First of all, it is analytics and databases.
If there is such a permanent memory in the system, applications can store more data in RAM and rarely access much slower SSD or HDD drives. As shown by the first test results, the new technology increases the performance of database applications and analytics by 2-10 times. For example, when working with SQL Server, NVDIMM memory can double the speed of writing database logs and speed up SPARK analytics by an order of magnitude.
As the demo example uses HPE ProLiant Gen9 servers and Microsoft SQL Server DBMS, the transition from SSD to SSD with an additional layer in the form of NVDIMM memory modules (cache) almost halves the write latency and significantly increases transaction performance in minutes Now it is 1,000,000 transactions per second, but theoretically, values ​​of up to 4-5 million transactions per second are already achievable.
Now ProLiant Gen 9 servers with NVDIMM modules come with Windows or Linux OS (Red Hat and SUSE), support Microsoft DBMS (SQL Server) and Hortonworks analytics.
Together with industry suppliers of operating systems and applications, HPE is working to create an ecosystem of software to support HPE non-volatile memory and simplify its implementation for HPE ProLiant server customers. That is, simultaneously with the release of hardware for the Persistent Memory technology, an ecosystem of hardware and software is being formed. To do this, HPE works with leading developers of operating systems, including Microsoft, Red Hat, SuSE, as well as independent software vendors (ISVs). Together with Microsoft, work is underway to optimize SQL Server. Let's see in more detail what the ProLiant Gen 9 servers are.
In March 2016, HPE announced the release of new computing platforms with built-in support for Persistent Memory . New HPE ProLiant Gen9 servers with Intel Xeon E5-2600 v4 processors accelerate service delivery, improve application performance and ensure business reliability.
They are optimized for the first product in the HPE Persistent Memory non-volatile memory product line — 8 GB HPE NVDIMM modules (16 GB memory modules are expected to be available). This technology is well suited for all applications that are affected by traditional storage bottlenecks.
The 8-gigabyte HPE NVDIMM module is available for order from April 2016 and is already supported by HPE ProLiant DL360 and DL380 Gen9 servers with Intel E5-2600 v4 processors. The HPE 8GB NVDIMM module is based on industry standard NVDIMM-N technology in collaboration with the standardization organizations JEDEC and SNIA and includes backup power from the HPE Smart Storage Battery, supporting up to 16 NVDIMM modules, to move data from DRAM to flash memory loss of power.
At the HPE High Technology Center, the new DL380 Gen9 server platforms have been tested with PostgreSQL. The tests were conducted by HPE specialists together with the developers from Postgres Pro ( https://postgrespro.ru/ ) in various configurations.
Why PostgreSQL? This is a full-featured open source DBMS that allows you to solve a wide range of tasks. During the existence of PostgreSQL, an ecosystem has formed around it, including developers, analysts, and users, so that it is possible to expand the functions of this DBMS depending on market requirements.
PostgreSQL is used both in government and commercial organizations around the world, and its popularity is growing, including precisely because PostgreSQL currently meets the requirements for industrial database management systems and is considered the most sophisticated free-access general-purpose database management system. and the growth in demand for open source solutions is a global trend. In our country, PostgreSQL uses a number of large companies (Yandex, Avito, HeadHunter, Mail.Ru, etc.). Interest in PostgreSQL is growing.
The performance of PostgreSQL DBMSs with different types of internal storage on HPE ProLiant Gen9 servers is shown below.
As the table shows, the results are truly outstanding. And the prospects open up very interesting.
From all this it follows that the “best practice” of storage should now look something like this:
In this layered storage architecture, Tier 0 zero represents the fastest, and Tier 3 the slowest memory. Storage of hot and cold data takes into account performance, capacity and storage cost. Usually, the hotter the data, the higher the $ / GB and the lower the capacity requirements. The traditional storage levels on the left represent:
On the right - new storage devices. The hierarchy looks like this:
Thus, NVDIMM memory replaces drives on PCIe NAND flash memory as the fastest storage devices, and the computing system architecture is changing.
HPE's Persistent Memory technology changes the rules of the game on the server market. It will be an important milestone on the way to a completely new computer architecture called HPE Machine, developed since 2014. In Machine with memristor memory, the von Neumann architecture, which has dominated the beginning of the computer era, is being reinterpreted.
All this can be called a transition to a new computational model - Memory-Driving Computing, where memory plays a leading role. Its embodiment has already become servers HPE ProLiant Gen9. But Memory-Driving Computing, memory computing, is not just hardware with a new memory architecture, but also a whole software ecosystem. For this, HPE works closely with the developers of operating systems and application software.
And in conclusion - a small bright video:
NVMe comes to the rescue
Partly to solve this problem helps an order of magnitude faster flash memory. New protocols and technologies are being developed for it that reveal the potential of a PCIe interface, for example, Non-Volatile Memory Express (NVMe). NVMe flash memory eliminates significant hardware delays and storage costs associated with storage and can be used to improve overall system performance, especially in demanding tasks such as virtualization, big data analytics and high performance computing (HPC). Using NVMe, you can access flash memory bypassing the SCSI layer, which reduces access time, especially in the case of hardware support for NVMe, when driver functions are implemented at the hardware level.
')
NVMe scalable technology was developed specifically for direct connection of NVMe devices (such as SSD) through a PCIe Gen3 interface. It provides fast processor access to data in flash memory and is characterized by high IOPS performance, low power consumption and low latency. Deliveries of the first NVMe products began in 2014. In 2015, many server vendors presented their platforms with NVMe slots.
NVMe allows you to improve system performance by more fully using the parallelism of devices and software. The drive becomes “closer” to the processor, which makes it possible to reduce the delay by more than 20 times.
Currently, solid state drives (SSD) in servers provide high speed data access and are sometimes used as cache memory or for storing intermediate data. Not surprisingly, flash drives are confidently conquering the market. According to the IHS company forecast, by 2017, SSD shipments in real terms will have reached half the volume of hard disk shipments.
In the arsenal of Hewlett Packard Enterprise - a whole range of drives with support for the protocol Non-Volatile Memory Express. These HPE solutions use different drive formats for various types of loads, including 2.5 "SFF and PCIe cards. Standard NVMe drivers are included in the most popular operating systems. However, NVMe is already becoming the best practice of yesterday. It is replaced at the top of the pyramid New technology - Persistent Memory.
HPE Persistent Memory
HPE Persistent Memory technology (which means “non-volatile” or “permanent memory”) is not only new to the original architectural solution - placing a non-volatile flash drive on the memory bus - but also an integrated approach. It already has a driver for Microsoft Windows, an SDK for Linux, and in the near future plans to teach applications to understand this new media type, which will provide, for example, support for in-memory calculations on these modules without various tricks on data integrity.
What is HPE Persistent Memory? Simply put, this is non-volatile memory with RAM performance. Being non-volatile, this memory provides permanent data storage. HPE Persistent Memory, introduced in December 2015, is used in the 2016 HPE ProLiant product line. Most of these systems are already supplied.
Persistent Memory (or Non-Volatile Memory) is a combination of DRAM with flash memory that avoids data loss during a power outage. Since DRAM (DIMM) and NAND sit on the memory bus, this combination is faster than flash memory on a PCIe card.
According to its characteristics, HPE Persistent Memory (NVDIMM) occupies an intermediate position in the pyramid of different types of memory between DRAM and flash memory. In which applications does this new technology demonstrate its best qualities? First of all, it is analytics and databases.
If there is such a permanent memory in the system, applications can store more data in RAM and rarely access much slower SSD or HDD drives. As shown by the first test results, the new technology increases the performance of database applications and analytics by 2-10 times. For example, when working with SQL Server, NVDIMM memory can double the speed of writing database logs and speed up SPARK analytics by an order of magnitude.
As the demo example uses HPE ProLiant Gen9 servers and Microsoft SQL Server DBMS, the transition from SSD to SSD with an additional layer in the form of NVDIMM memory modules (cache) almost halves the write latency and significantly increases transaction performance in minutes Now it is 1,000,000 transactions per second, but theoretically, values ​​of up to 4-5 million transactions per second are already achievable.
Now ProLiant Gen 9 servers with NVDIMM modules come with Windows or Linux OS (Red Hat and SUSE), support Microsoft DBMS (SQL Server) and Hortonworks analytics.
Together with industry suppliers of operating systems and applications, HPE is working to create an ecosystem of software to support HPE non-volatile memory and simplify its implementation for HPE ProLiant server customers. That is, simultaneously with the release of hardware for the Persistent Memory technology, an ecosystem of hardware and software is being formed. To do this, HPE works with leading developers of operating systems, including Microsoft, Red Hat, SuSE, as well as independent software vendors (ISVs). Together with Microsoft, work is underway to optimize SQL Server. Let's see in more detail what the ProLiant Gen 9 servers are.
New HPE ProLiant Gen9 Servers with Persistent Memory Technology
In March 2016, HPE announced the release of new computing platforms with built-in support for Persistent Memory . New HPE ProLiant Gen9 servers with Intel Xeon E5-2600 v4 processors accelerate service delivery, improve application performance and ensure business reliability.
They are optimized for the first product in the HPE Persistent Memory non-volatile memory product line — 8 GB HPE NVDIMM modules (16 GB memory modules are expected to be available). This technology is well suited for all applications that are affected by traditional storage bottlenecks.
The 8-gigabyte HPE NVDIMM module is available for order from April 2016 and is already supported by HPE ProLiant DL360 and DL380 Gen9 servers with Intel E5-2600 v4 processors. The HPE 8GB NVDIMM module is based on industry standard NVDIMM-N technology in collaboration with the standardization organizations JEDEC and SNIA and includes backup power from the HPE Smart Storage Battery, supporting up to 16 NVDIMM modules, to move data from DRAM to flash memory loss of power.
Test results of HPE ProLiant Gen9 server with PostgreSQL DBMS
At the HPE High Technology Center, the new DL380 Gen9 server platforms have been tested with PostgreSQL. The tests were conducted by HPE specialists together with the developers from Postgres Pro ( https://postgrespro.ru/ ) in various configurations.
Why PostgreSQL? This is a full-featured open source DBMS that allows you to solve a wide range of tasks. During the existence of PostgreSQL, an ecosystem has formed around it, including developers, analysts, and users, so that it is possible to expand the functions of this DBMS depending on market requirements.
PostgreSQL is used both in government and commercial organizations around the world, and its popularity is growing, including precisely because PostgreSQL currently meets the requirements for industrial database management systems and is considered the most sophisticated free-access general-purpose database management system. and the growth in demand for open source solutions is a global trend. In our country, PostgreSQL uses a number of large companies (Yandex, Avito, HeadHunter, Mail.Ru, etc.). Interest in PostgreSQL is growing.
The performance of PostgreSQL DBMSs with different types of internal storage on HPE ProLiant Gen9 servers is shown below.
As the table shows, the results are truly outstanding. And the prospects open up very interesting.
New "rules of the game"
From all this it follows that the “best practice” of storage should now look something like this:
In this layered storage architecture, Tier 0 zero represents the fastest, and Tier 3 the slowest memory. Storage of hot and cold data takes into account performance, capacity and storage cost. Usually, the hotter the data, the higher the $ / GB and the lower the capacity requirements. The traditional storage levels on the left represent:
Tier 0 = PCIe SSD and PCIe Workload Accelerator (NAND flash on SAS / SATA or PCIe), 100 microseconds latency.
Tier 1 = SAS HDD (high performance drives), the delay is tens of milliseconds.
Tier 2 = SATA HDD (large capacity drives), delayed sleepy milliseconds.
Tier 3 = tape or archive media, delay - seconds and minutes.
On the right - new storage devices. The hierarchy looks like this:
Tier 0 = NVDIMM, delay - nanoseconds.
Tier 1 = PCIe SSD and PCIe Workload Accelerators (NAND flash on SAS / SATA or PCIe), 100 microseconds latency.
Tier 2 = SAS HDD (high performance drives), the delay is tens of milliseconds.
Tier 3 = SATA HDD (large capacity drives), the delay is hundreds of milliseconds.
Thus, NVDIMM memory replaces drives on PCIe NAND flash memory as the fastest storage devices, and the computing system architecture is changing.
HPE's Persistent Memory technology changes the rules of the game on the server market. It will be an important milestone on the way to a completely new computer architecture called HPE Machine, developed since 2014. In Machine with memristor memory, the von Neumann architecture, which has dominated the beginning of the computer era, is being reinterpreted.
All this can be called a transition to a new computational model - Memory-Driving Computing, where memory plays a leading role. Its embodiment has already become servers HPE ProLiant Gen9. But Memory-Driving Computing, memory computing, is not just hardware with a new memory architecture, but also a whole software ecosystem. For this, HPE works closely with the developers of operating systems and application software.
And in conclusion - a small bright video:
Source: https://habr.com/ru/post/282395/
All Articles