What will the new storages for AI and MO systems offer?
MAX Data will combine with Optane DC to work effectively with AI and MO systems.
Photos - Hitesh Choudhary - Unsplash
According to a MIT Sloan Management Review and The Boston Consulting Group study, 85% of the three thousand executives surveyed believe that AI systems will help their companies gain a competitive advantage in the market. However, something like that tried to put into practice only 39% of companies.
')
One of the reasons for this situation is that efficient work with data and optimization of capacity utilization for machine learning tasks is not an easy job. The IDC notes that a new permanent memory technology (Persistent Memory, PMEM) could resolve the situation.
This technology was proposed by NetApp and Intel, combining NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory to get a product for local data storage based on non-volatile memory.
MAX Data is a server technology that allows you to increase application performance by using PMEM or DRAM, but does not require re-engineering of software architecture.
It implements the principles of automated multi-level storage, distributing data across tiers and vaults depending on the frequency of use — for “cold” data, more accessible storage is used, and frequently used ones are “at hand” - in Persistent Memory, which minimizes latency when working with such data .
Version 1.1 uses DRAM and NVDIMM memory . Both implementations have their drawbacks — relative loss of efficiency and high memory cost, respectively — compared to Optane DCPMM. A chart giving a comparative assessment of the delay is presented here (p. 4).
The technology supports both POSIX and work with semantics of block or file systems. Data protection and data recovery at the storage level is implemented using MAX Snap and MAX Recovery. These technologies use snapshots, the SnapMirror tool and other ONTAP defense mechanisms.
Schematically, the implementation is as follows:
There is no PMEM on this scheme yet, but the developers promise to add support for this type of memory in the end of the year. So far, Max Data works with DRAM and DIMM.
IDC argues that in the coming years, developments like MAX Data will become more, because the volume of corporate data is constantly growing, and companies do not have enough capacity to process them effectively. The technology can be useful in a large-scale cloud environment and for working with resource-intensive tasks like learning neural networks. It will find application on trading platforms, in information security systems and any other software products where constant and fast access to large volumes of information is required.
There is also a certain probability that the technology will not immediately take root in the market. As we noted above, only one third of companies around the world work with AI systems in one form or another. From this point of view, the appearance of MAX Data can be considered by many to be premature and will focus their attention on a more accessible infrastructure that allows solving current tasks.
Our other materials about IT infrastructure:
Photos - Hitesh Choudhary - Unsplash
According to a MIT Sloan Management Review and The Boston Consulting Group study, 85% of the three thousand executives surveyed believe that AI systems will help their companies gain a competitive advantage in the market. However, something like that tried to put into practice only 39% of companies.
')
One of the reasons for this situation is that efficient work with data and optimization of capacity utilization for machine learning tasks is not an easy job. The IDC notes that a new permanent memory technology (Persistent Memory, PMEM) could resolve the situation.
This technology was proposed by NetApp and Intel, combining NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory to get a product for local data storage based on non-volatile memory.
How it works
MAX Data is a server technology that allows you to increase application performance by using PMEM or DRAM, but does not require re-engineering of software architecture.
It implements the principles of automated multi-level storage, distributing data across tiers and vaults depending on the frequency of use — for “cold” data, more accessible storage is used, and frequently used ones are “at hand” - in Persistent Memory, which minimizes latency when working with such data .
Version 1.1 uses DRAM and NVDIMM memory . Both implementations have their drawbacks — relative loss of efficiency and high memory cost, respectively — compared to Optane DCPMM. A chart giving a comparative assessment of the delay is presented here (p. 4).
The technology supports both POSIX and work with semantics of block or file systems. Data protection and data recovery at the storage level is implemented using MAX Snap and MAX Recovery. These technologies use snapshots, the SnapMirror tool and other ONTAP defense mechanisms.
Schematically, the implementation is as follows:
There is no PMEM on this scheme yet, but the developers promise to add support for this type of memory in the end of the year. So far, Max Data works with DRAM and DIMM.
Solution potential
IDC argues that in the coming years, developments like MAX Data will become more, because the volume of corporate data is constantly growing, and companies do not have enough capacity to process them effectively. The technology can be useful in a large-scale cloud environment and for working with resource-intensive tasks like learning neural networks. It will find application on trading platforms, in information security systems and any other software products where constant and fast access to large volumes of information is required.
There is also a certain probability that the technology will not immediately take root in the market. As we noted above, only one third of companies around the world work with AI systems in one form or another. From this point of view, the appearance of MAX Data can be considered by many to be premature and will focus their attention on a more accessible infrastructure that allows solving current tasks.
Our other materials about IT infrastructure:
Source: https://habr.com/ru/post/451166/
All Articles