Rasking through the parts of a particularly hardy iron bullion S line, where 768 GB of RAM

We detach from the front of the Ultra capacitor module - the thing is designed for the operation of the power supply units in the Active / Passive mode, as well as for the guaranteed switching without power drawdowns to the backup power supply unit. It turns out that it can also be used to protect the server against a potential adversary, quickly converting it into a powerful stun gun. And you know what? They let him on the plane when you say "yes, this server is like an apartment opposite the Kremlin!"

Looking around what else to put in the crowbar, we became interested in the Bull server hardware and decided to take one of the servers for testing. In general, there is the Atos Bull product line. Bullion S is a high-performance server with a sea of memory on board based on x86 architecture servers. All that needs to be considered inside the RAM is their diocese. The declared feature is high fault tolerance combined with power and relative cheapness.

In general, we got one. The first surprise was that it was loaded for 15 minutes (the vendor says: in full calculation up to half an hour), 768 GB of operatives are being tested. Loaded - we started pulling everything we could. Yeah, right on the go. The piece of iron behaved adequately. For all I / O experienced normally, just crossed the path, and the disks remained available with some delay. When pulling out other pieces, short rebuilds were sometimes made (performance dropped), then everything was restored. Restored - we continued to unscrew the details further. By the way, if you tear off this kapasiti module with a photo or a power supply, the server will also survive this without incident.
')
The only time is if you remove the memory plate “hot” with your hands from it during the load, it will reboot under certain conditions (I’ll tell you about them below).

Meet the bullion S server

So, we set ourselves the task of checking these statements, to get acquainted and in every way to torment the equipment provided. For 2 weeks, we were allocated a bullion S4 server, in fact, these are two x86 nodes combined into one high-performance server. The server came as a ready-made stand in a 20U-rack with a Cisco 2960 switch, a console with a KVM switch, and a PDU.

Configuration and features

The feature of the server is the VMWare ESXi OS pre-installed from the box. That is, in fact, this is a ready-made solution: I took a server out of the box, connected and deployed the virtuals. If necessary, the ESXi version can be updated.

Server configuration is presented in the table:

bullion S4
Form factor	19 inches, 6U
PROCESSORS	4xCPU Intel Xeon E7v3
Chipset	Intel C602J
XQPI interface	Max. 9.6 GT / s
Memory	4 blades for 6 slats 32GB DDR4 RDIMM
Embedded NICs	1Gb Ethernet: 2 × 2 ports
Nic	1Gb Ethernet: 4 ports per node, 10GbE 2 ports per node
HBA	8/16 Gbit / s: 2 modules with 2 ports per node
Flush	SSD, 1.6 TB
SAS	12 Gbps: 2 external connectors per blade module
Controller	1 I / O blade with RAID 0.1 (12 Gb / s) Raid controller 9380 4i4e for RAID 5/6 on bullion disks
Power Supply	2 per node
Fan	6 per node
HDD	2 × 600 GB per node

All server components are reserved, like any server in a good configuration.

Each bullion S node is a server with two sockets, 48 memory slots, at the maximum the server can be scaled up to 8 such nodes, which makes it possible to get a bold 24U server with 16 sockets, 24Tb of memory and 56 8xPCI-e slots. Plus possible Node Partitioning.

The OS is loaded from the master node, from two disks in RAID1. The server has a modular structure, all I / O cards, memory cards, not to mention the PSU and fans, are designed so that they can be removed to the hot from the front or back of each node.

The scalability of bullion is his thing. For communication of processors between nodes, eXtended Quick Path Interconnect is used, which was developed by Bull and works 2 times faster and reduces the average delay between the local and remote processor memory compared to Intel QPI. Physically, this is the built-in coherent Bull switch (abbreviated as BCS).

The architecture is traced below.

S4

S16

Excess star Distinguishes bullion from KunLun and SDX

Monitoring and Management

The server has a rather strange external display, which is a bit confusing, but that is the design.

In fact, the orange light indicates that the UCM (ultra capacitor module) is charged.

The status of the server and its components can be monitored through the Server Hardware Console.

Vendor says that iCare (Bull insight care) tool exists to monitor a large number of servers.

There is also the possibility of integration with vRealize. There is also a powerful CLI "BSMHW_NG_140" for management - it is installed separately, it can work together with BPM (Bull Platform Manager).

Testing

Turned on the server. Initialization when restarting or turning on takes 15–20 minutes, mainly due to the memory check: the more, the longer. The boot process can be monitored by connecting, for example, with the help of Putty (terminal) to the SHC, where the process of controller initialization, memory checking, etc. is detailed.

We made a couple of virtual locks, we connected the server with both nodes to our entry level of the DS3500 storage system and started pulling the modules. Here we have a blade with memory. Taken out? Removed!

Reboot caught when installing memory back, although it is stated that the server supports hot plug memory, in small print (or in parentheses) add that only certain operating systems.

“Look, another detail. Now I pull out, shoot: "

Interestingly, a stun gun get? It turns out!

After that, the manual read that it is impossible to do so, it is dangerous - you must first defuse!

While tugging I / O cards - the moon remained alive, throwing ways.

They wanted to understand if it can in the single-mode config - they jerked the whole node. The entire server has disconnected (well, the reverse vendor did not declare, but suddenly) . In this regard, a software cluster is preferable to a hardware cluster.

A colleague says: let MSM (Management Switch Module) take a closer picture. No question, hold on, buddy:

The server is still alive. With the help of the program Perfomance test we conducted several performance tests, on a small virtual machine and a virtual machine with maximum resources. Although considering that these are virtual machines, I don’t see much point in this, the main purpose of the server is Oracle, SAP HANA, virtualization. Test data - at the end.

There was a thought to demolish the hypervisor and install something else, but the vendor was very requested not to do it. Most likely, because the car goes "hands-on" on the tests. In general, the zoo arrived, we played, the zoo went on tour.

According to materials published on www.spec.org , bullion is the most powerful corporate x86 server. Bullion still offers up to 16 processors (compared to eight for the so-called "direct" models - an architecture in which up to 8 processors are connected to each other without an additional switch), that is, up to 384 cores on one server.

The results of our tests

CPU Test 1/1

MemTest 1/1

CPU Test 2

Mem test 2

Impressions

It’s quite a regular dvuhnodovy hardware cluster, Bull is not the first to unite Q86 servers using QPI, IBM and HPE did it, although there is such power there. Of course. was not, you can grow here: S4-S6-S8-S10. The OS can be any: Linux, Windows, VMware ... Although here, in my opinion, the chassis with blade servers and vMotion is more reliable than all the virtual machines in one box (but here it is necessary to take into account the cost of licenses for sokety).

Why is he so beautiful need

The main scenario of using servers is a platform for in-memory computing. As the most common example of such calculations is the SAP HANA solution. The server can also be used under the database Oracle, SQL Server, virtualization, a good option for the transition from the P-series and mainframe. Price list in the Russian Federation for some hardware configurations certified for SAP HANA:

Name	Model	CPU	DB	DIMMs	Russian list Price
S4-80V4-1TB-32GB	S4	8880V4	1TB	32GB	166 872 €

S4-80V4-2TB-32GB	S4	8880V4	2TB	32GB	211,104 €

S4-90V4-4TB-64GB	S4	8890V4	4TB	64GB	356,820 €
S6-80V4-4.5-32GB	S6	8880V4	4.5TB	32GB	366 567 €
S6-90V4-6TB-64GB	S6	8890V4	6TB	64GB	529,161 €
S8-80V4-3TB-32GB	S8	8880V4	3TB	32GB	417 102 €
S8-80V4-6TB-32GB-CIS-Op	S8	8880V4	6TB	32GB	570 437 €
S8-90V4-6TB-32GB-CIS-Op	S8	8890V4	6TB	32GB	617 785 €
S8-90V4-8TB-64GB	S8	8890V4	8TB	64GB	816 410 €
S8-90V4-8TB-64GB-CIS-Cu	S8	8890V4	8TB	64GB	820 337 €

This, of course, is the street price, almost no one ever buys it. But if you know how much to usually deduct, gives an idea.