Quantum leap

The term " quantum jump ", also known as "quantum transition", describes a jump-like change in the state of a quantum system. And it is this physical term that is associated with the current situation, in which the cheapening of flash memory allowed the creation of super-productive storage systems that are not inferior in capacity to systems on hard drives and compete with them at the total cost of one terabyte. This year, the total cost of one terabyte for the first time will make flash-systems more profitable.


Source: http://wikibon.org/wiki/v/Evolution_of_All-Flash_Array_Architectures
')
The importance of this moment is difficult to overestimate, it can be compared with the rapid change of historical formation. The era of HDD dominion is rapidly becoming a thing of the past. And one of the heralds of this process was DSSD D5 - a high-performance storage rack class on flash-drives (RSF, Rack-Scale Flash).

An RSF system is a shared repository designed for solving tasks that require the highest performance and processing huge amounts of data. Solutions of this class have become popular because, despite the rapid development of traditional storage systems, they did not have time to keep up with the requirements for the performance of workloads of new types. The world of software-defined infrastructures has become necessary as fast as possible electronic storage, which can only allow modern technological process. DSSD D5, in fact, is the system created to solve this problem.

In the maximum configuration, DSSD D5 indicators are as follows:

• Bandwidth up to 100 Gb / s.
• Up to 10 million IOPS.
• Delay ~ 100 microseconds (for applications with an integrated native API).
• Total capacity up to 144 TB, used - 100 TB.

Spheres of application

In high-performance computing systems, storage bottlenecks are the number of operations / input / output, throughput, and capacity. The main task of RSF-systems is to level these restrictions.

From the point of view of the nature of workloads, there are three ways to apply D5:

• High performance repositories / databases . These include traditional databases like Oracle, as well as databases with massively parallel processing. Such systems are used to handle financial transactions, risk management, fraud prevention, supply management, etc.
  
  • For transaction processing applications, ultra low latency means more transactions per second, and therefore more profit.
  • A high level of IOPS and bandwidth allows you to quickly generate business solutions or analyze more data, and therefore be more competitive.
  • Performing all the analytics on one D5 allows you to significantly save time on data movement.
• Hadoop High Performance Applications . Unlike traditional Hadoop batch processing, it requires high speed data reception and processing. In particular, for the analysis of real-time streaming information in the areas of stock trading, advertising systems, business reporting and forecasting modeling based on extensive data sets.
  
  • Independent scaling of computing power and storage in the Hadoop infrastructure.
• High performance user applications . Workloads that imply high storage performance and data processing through a fast API. In such cases, traditional file systems like XFS or parallel cluster file systems are used. Applications: trade modeling, weather prediction, engineering, medical research, pharmacology, climate analysis, etc.
  
  • High throughput and large amounts of IOPS allow conclusions to be drawn based on data processing within minutes after receiving them, rather than hours or days.
  • When using parallel file systems, you can reduce the amount of infrastructure by a factor of 20 or more.
  • Analytics in real time allows you to quickly bring new products to the market.

On the other hand, DSSD D5, in its current release, is not designed to work with SAP HANA, since this requires appropriate technological integration at the memory level and subsequent certification to SAP.

Architecture

DSSD D5 is based on a new architecture that provides an order of magnitude better performance, minimal latency and extremely high memory density. This ensures the level of reliability and availability typical of corporate systems.

What are the features of D5 architecture?

• For the first time in the industry, direct access to flash-memory (DMA) with segmented control and data levels is applied.
• Flexible data access (block, object, key-value).
• Integrated switches (PCIe Mesh Fabric) provide extremely low latency.
• The system can be simultaneously used by several computing nodes.
• There is no single point of failure.

DSSD D5 block diagram.

Ability to integrate into user applications direct memory access API. High system performance is achieved due to:

• API with asynchronous architecture without locks,
• the absence of software intermediaries in the data transmission path,
• no interaction of the kernel with the used APIs and plugins,
• and intensive parallelization of processes.

Initially, the D5 will be available in four configurations: with 18 or 36 flash-modules of 2 or 4 TB (thousands of NAND-memory chips). Thus, the total amount of storage can be 36, 72 or 144 TB. All modules are combined into a RAID array. Available for use capacity will be approximately 25, 50 or 100 TB for “typical workload”. The exact amount of memory will be determined by the size of FLEN: the smaller it is, the more metadata will be in storage. According to our measurements, the highest IOPS level is achieved with FLEN objects of 4 KB in size, and the best throughput with FLEN objects of 32 KB.

All flash-modules are non-volatile and protected from power failures. Each module is connected to a PCIe network through two separate lane connections for PCIe Gen3 x4, which allows bandwidth up to 8 Gb / s.


Flash module DSSD D5.

Thanks to the multiply connected PCIe network, each client has direct access to each of the NAND chips. This made it possible to abandon redundant copying when writing data to the final part of memory. And the use of the NVMe protocol ensures parallel access to any parts of the D5 memory.

Unlike traditional architectures, in which the system is controlled by the processor, in DSSD D5, the control and data transmission paths are separated and have their own control module. These modules control I / O operations, but the data itself goes through I / O modules directly between the connected servers and flash-modules.


Controller module DSSD D5.

D5 is connected to servers using I / O modules operating according to the active-active scheme. Each module allows you to connect with redundancy up to 48 nodes via lane ports PCIe Gen3 x4 (96 ports in total). Cards for connecting nodes are installed in standard PCIe Gen3 x8 slots and are connected to I / O modules via dual PCIe Gen3 x4 copper cables. In the future, it is planned to introduce support for active optical cables (Active Optical Cable, AOC).


DSSD D5 I / O Module.

Bidirectional throughput of a single port reaches 3.5 Gb / s, and dual ports - 6.5 Gb / s. So the theoretical bandwidth of the entire system is 6.5 * 48 = 312 Gb / s. But due to the limitations inherent in some components, 100 Gb / s is now provided.

To access DSSD D5, client software must be installed on client servers:

• client card driver,
• block device driver
• client CLI,
• the libflood library used by applications to call our APIs
• API and CLI documentation.

NVMe protocol.

Finally, one of the very important properties of the DSSD D5 architecture is its “maintainability”: all major components — from cards for connecting clients (servers) to flash modules — are reserved and can be replaced by the customer on their own. This ensures a very high level of availability and the absence of a single point of failure.

In the dry residue

Let's sum up all of the above. DSSD D5 is the first solution in the class RSF (Rack-Scale Flash). It is designed to meet the challenges of extremely demanding storage performance, latency and throughput. At the same time, the total cost of 1 TB of DSSD D5 memory is already comparable with HDD-based systems, and the new solution surpasses most of all all flash systems with “traditional” architectures on the market for a number of metrics.