About HPE Synergy. Part III - D3940 Disk Storage and SAS Switches

Continued, beginning - Part I (Introduction)
Continued, beginning - Part II (Chassis and server)

Disk storage SY D3940 - one of the key elements of Synergy; the module is optimized for use either as DAS (Direct Attached Storage), i.e., disks are connected directly to the server, and some RAID level is organized by the controller installed in the server), or as software-defined storage, such as HPE StoreVirtual VSA (or similar). About HPE StoreVirtual VSA can be read here , Alex described everything in great detail. Personally, VSA is confused by the restriction of 50 TB and the memory requirements. I thought for a long time where to start telling about the D3940, then I found a photo of a real piece of iron on the Internet and decided to start the story with an error that the guys in the photo made:


What is the mistake if the arrangement of the disks seems to be correct? After all, according to the scheme, we insert the first disk into the compartment №1, the second disc into the compartment №2 and so on?


So yes, not so. The “HPE Synergy Configuration and Compatibility Guide” document contains a completely different filling procedure: “For proper air flow, it must be populated from back to front. “Pop up the bays 33 through 40, and continue to populate the bays 1 through 8.” In the figure, the arrows indicate a more correct disc layout.
')

What is the reason for the illogical numbering - is unclear. Now about the module. As already mentioned, it supports installing 40 pieces of SFF SAS or SATA disks into it. It is impossible (at least for the time being) to put into it an SFF module with two uFF disks, described in the second part of “Chassis and Server” . I suspect that this function will never be implemented, since the adapter box 2 pcs. uFF -> SFF has a RAID adapter inside it that supports RAID levels 0 and 1, and the D3940 has no adapters at all, all drives from the module are presented to the HPE Smart Array P542D Controller with 2GB FBWC adapter installed in the server. Connection and power cables are placed in a flexible segmented sleeve, which is located at the bottom of the module.


Connecting disks to SAS switches are performed by SAS I / O modules, by default one is installed in the I / O compartment 1, which works with the HPE Synergy 12Gb SAS switchboard SAS Connection Module installed in ICM 1 slot. If fault tolerance is required, in the compartment I / O 2 installs the second I / O module, and the second SAS switch is installed in the ICM 4, in addition to each server that will work with D3940 drives, you must install the HPE Array P542D controller in the mezzanine slot 1. The module itself it looks boring, you can see it either in the photo in the first part (installed in ICM 1), or below:

Just in case, once again, the ICM layout (InterConnect modules):


The same switching scheme of internal components, which explains why the connections are made this way and not otherwise (The storage module is supported for connectivity with fabric 1):


Compute module 1-12, meaning the server SY 480 Gen9 HH (Half Height), as already said, it has three compartments for installing mezzanine cards.

Thus, up to 10 half-size servers can be connected to a single disk module D3940 in a complete set in the chassis, or vice versa - to install five disk modules and two servers.

If you take a non-fault-tolerant configuration, the options are as follows:

• The SAS Connection Module in the ICM 1 bay connects to the main I / O adapters for bay 1, 3, and 5;
• The SAS Connection Module in the ICM 4 bay connects to the primary I / O adapters of bay 7, 9, and 11;

It turns out that if there are more than three planned disk modules, you will have to install the second I / O modules and the second SAS switch. According to the documentation, for three or more full-size servers (FH - full height) and two disk modules, a second SAS switch is also required.

The SAS switch has 12 internal SAS ports of 4x12 Gbps per port per port and creates a dynamic virtual JBOD that can connect to servers inside the chassis. Separately, it should be noted that Factory 1 is non-blocking, which allows you to fill SSD disk modules and enjoy the enormous IOPs and the minimum time for reading. SAS switch is managed via Synergy Composer or server with OneView version 3.0 or higher.

It is impossible to connect disk modules in one chassis to servers in another chassis, this is a very big sadness. Perhaps something will change.

Returning to the disk module D3940. As already mentioned, you can install up to 40 SFF disks per module and up to five modules per chassis, for a total of 200 disks; inside the module, you can install any of the disks listed, but we must remember that a logical array can be created only on disks of the same type. At the moment, the list of supported disks is:


It supports 44 types of disks, with the type of HDD about 45%, and about 55% - SSD. A maximum of one disk module can provide 153.6 TB (in the form of 40 pieces HPE 3.84TB 12G SAS RI-3 SFF SC SSD in RAID0), or 768 TB - five modules (maximum) per chassis.

Separately, it is necessary to mention the controller HPE Smart Array P542D, which is installed in the server and manages drives connected to it. It has onboard 2 GB DDR3-1866 MHz cache module, 16 ports 12 Gbit / s SAS (2x4 internal, and 2x4 external), but there are the following limitations - 404 TB per logical drive and 64 logical drives per controller. On the host, the connected controller eats up 4.4 GB of memory. Supports RAID levels 0, 1, 10, 5, 50, 6, 60, 1 ADM, 10 ADM.

This controller has one option, purchased separately - HPE SmartCache - allows you to organize multi-level data storage (tiering), i.e. when data is frequently requested, it moves them to faster SSDs. It consists, respectively, of slow HDDs and fast SSDs connected to the controller, and metadata about the demand for blocks of information from the array held by the FBWC (Flash-backed write cache, recording flash buffer, the same 2 GB DDR3 module). Personally, I have not tested this functionality, it would be interesting to see how the data “temperature” levels are set and how the controller copes with this level “jitter”.

All this splendor is managed with the help of the SY Composer and on this review we will consider the part complete.
At the end there is a description of the network part (the most difficult for me), the control modules (the most incomprehensible) and the principles of construction and control (the most interesting).