Scaling up to 24 controllers in the new ETERNUS DX storage system

In September, Fujitsu Corporation is starting to supply the next generation of its high-end ETERNUS DX8700 S3 and DX8900 S3 disk arrays, built on the new Quad Star modular architecture. The entry into the market of these storage systems completes Fujitsu’s third generation generation of ETERNUS DX S3 arrays, which began at the end of 2013, significantly surpassing their predecessors in scalability and performance thanks to the use of the latest technologies, including a 12-gigabit SAS interface, 16-gigabit Fiber Channel and PCI Express Gen3 buses, as well as the latest models of multi-core Intel Xeon processors, hard drives up to 6 TB, and solid-state drives (read more about the third-generation ETERNUS DX midrange innovations I am here and there ). Note that the history of Fujitsu disk arrays began in the 1970s, but until the end of the last decade its arrays were not sold outside of Japan, so Fujitsu debuted on the European storage systems market in 2009 when the ETERNUS DX S1 series was released.

The use of Quad Star's quadrangular star architecture, based on the use of ultra-fast switches for interconnecting array controllers, has greatly improved the scalability of arrays. If in ETERNUS DX S3 of the middle class, the array controllers are connected directly, in the new ETERNUS DX8000, even in the minimum configuration of two controllers, they are connected via array switches. To ensure high availability and protection against failures, ETERNUS DX8000 S3 has four switches, which provides multiple redundancy of data transfer routes on the PCIe 3.0 bus between controllers (note that multiple redundancy applies to other main components of the array, which almost completely eliminates the risk of access loss to data due to array hardware failure).

Let's take a closer look at the system architecture of the new ETERNUS DX8000.
')

Picture 1.

Basic system architecture

Figure 1 shows the basic system architecture in which the four Controller Module (CM) modules are interconnected via PCI Express through the Front-end Router (FRT) front switches installed in the Front-end Enclosure shelf (FE). Each CM is equipped with one or more adapters for connecting to the Channel Adapter (CA) hosts and one or more disk interfaces. As mentioned above, to ensure maximum protection against failures, each controller connects to all four FRT routers. This is a fourfold reservation of connections and gave the name of the "quadrilateral star" of the architecture of the third generation of eight-thousand-meter ETERNUS DX. In addition, the Drive Enclosure (DE) disk shelves, in which the array drives are installed, have two independent SAS-3 ports that are connected directly to CM controllers located in the Controller Enclosure (CE) controllers shelf, which also provides redundancy for data transfer paths. In multi-controller array configurations, adjacent controllers access the disk shelf through FRT switches.


Figure 2.

Controller circuit

A simplified diagram of the CM controller is shown in Figure 2. It consists of a Xeon processor running VxWorks hard real-time operating system, DRAM memory, including data caches, and CA host adapters. Each CM can be connected to a maximum of 16 DE disk shelves via the SAS Expanders (EXP) expander. The PCIe 3.0 Interface (INF) four-port interface card provides high-speed controller connectivity to FRT switches. As an option, each CM controller is equipped with one or two PCI Express Flash Modules (PFM) flash memory modules, which form the second level cache memory Extreme Cache.


Figure 3.

CE controller shelf wiring diagram

Controllers are installed in pairs in the CE shelf and interconnected via PCIe 3.0 through four redundant FRT switches installed in the FE shelf. The minimum ETERNUS DX8000 S3 configuration consists of two controllers and one shelf of CE controllers installed in one 42-unit rack and expands to 24 controllers and 12 shelves of CE controllers (Figure 4), which can occupy up to 12 42U racks.


Figure 4.

Maximum configuration ETERNUS DX8000 S3

Thus, if the previous generation of high-end ETERNUS DX8000 S2 arrays expanded to a maximum of 8 controllers and 3072 disks with a total capacity of about 6 PB, the new flagship of the DX8900 S3 corporation disk array supports configurations of 24 controllers and 4608 disks with a total capacity of almost 14 PB, equipped with 384 ports of front-end (younger model DX8700 S3 is scaled to 8 controllers and 1536 disks with a total capacity of 4.6 PB). Such gigantic capacity expansion indicators will allow array owners to successfully cope with unpredictable data growth while at the same time using the modular architecture online to flexibly scale their storage resources by adding controllers, disk shelves or individual disks to an array as their performance needs increase. capacities starting with a minimum configuration of two controllers supporting up to 384 disks. If necessary, the younger model can be upgraded to the level of the older one by simple replacement of controllers and without data rewriting (data-in-place). In addition, Fujitsu offers a data migration service from the ETERNUS DX middle class to the Eternus DX8000 S3, which guarantees the safety of information.

ETERNUS DX8000 S3 array controllers are equipped with DRAM cache memory up to 6 TB and second-level cache memory on solid-state drives (SSD) with a capacity up to 67 TB. They support SAN storage connectivity over 16 Gbps Fiber Channel interfaces, FCoE and iSCSI. Thanks to the use of the “quadrilateral star”, new interfaces and Xeon processors, as well as the equipment of the Extreme Cache second level cache memory, which was absent from the previous generation systems, the new arrays exceed the DX8000 S2 by eight times in performance and provide I / O performance over 4 Million IOPS.

To ensure maximum availability of business critical data at the level of six nines (99.9999%), they use “snapshots”, synchronous and asynchronous replication between arrays to protect data from major accidents, Fujitsu Storage Cluster package for fully automated online switching to a backup platform without stopping the operation of applications, as well as accelerated data recovery from failed disks of Fast Recovery RAID arrays (about Storage Cluster and Fast Recovery, which Fujitsu introduced last year, can be read here ).

Another advantage of the new ETERNUS DX8000 is the use of the same ETERNUS storage management software package of the ETERNUS SF software package, which are used in previously released ETERNUS DX S3 models of the junior and middle class. This unification of the management software of different ETERNUS DX S3 models simplifies the work of the storage administrator, who with the help of the same tools can manage storage systems of different classes. In addition, using the Storage Cluster package, you can replicate data from ETERNUS DX8000 S3 to a middle-class model, for example, ETERNUS DX600 S3, which significantly reduces the cost of purchasing backup data center equipment if the older ETERNUS DX model is installed in the main data center. Along with the release of the DX8000 S3, automatic QoS functions have been added to the ETERNUS SF package, with which the system administrator can set the maximum allowable data access delays. Another new feature of the DX8000 S3 is support for VMware vSphere Virtual Volumes virtual volumes that increase storage efficiency for a virtualized VMware vSphere infrastructure.

It is worth emphasizing that Fujitsu is one of the few players on the storage market who develops all of its disk arrays, ranging from entry-level models to high-end arrays. As a result, unlike other large vendors, in which the storage product line often combines both proprietary products, systems of newly acquired companies and OEM versions of arrays from other manufacturers, Fujitsu arrays of different classes and generations are easily integrated without using virtualization solutions storage, which significantly reduces the cost to the customer to build a corporate storage infrastructure. In addition, the same disk shelves, racks and cables are used in all ETERNUS DX S3, therefore, when upgrading the array to the level of a more powerful model, customer's investment in the hardware of the storage system will be protected.

With record-breaking scalability among high-end disk arrays, Fujitsu ETERNUS DX8000 S3 systems can successfully cope with the most demanding applications used in corporate IT infrastructure and ensure their availability at the level of six nines.