Intel strengthens its position in HPC

10 years ago, Intel’s presence in HPC was limited mainly by powerful processors and a set of mathematical libraries.

It is interesting to see what other products for HPC Intel has grown over time. It is equally interesting to predict where the company will move on. Below I will set out my vision. I invite everyone to supplement it with comments and polls at the bottom of the post.

I will divide all aspects of HPC into segments, and then we will see how Intel is represented in each of them. This division is conditional and reflects only the way I used to look at HPC. Perhaps there are more successful schemes.

So HPC is (I will use English names):

1. Compute
2. Storage
3. Interconnect
4. Development tools
5. Administrative tools
6. Services

At the moment, I do not know about the services that Intel would sell directly in the HPC sector. If you know something about this, welcome to the comments!

In all other segments, Intel, as it turns out, is very well represented:

1. Compute:
   
   • powerful xeon processors
   • (not clear how powerful) accelerators Xeon Phi
2. Storage:
   
   • Intel Luster Parallel File System Distribution
   • SSD-drives and flash cards (it seems that the use of Intel drives in the HPC segment is limited).
     Just a few days ago, the first 3D XPoint technology based cards went on sale.
3. Interconnect:
   
   • own factory Omni-Path
4. Development tools:
   
   • Intel Parallel Studio
   • either separately: math libraries, MPI library (based on the MPICH library), compilers, profilers
5. Administrative tools:
   
   • Intel HPC Orchestrator (in fact, otherwise packaged OpenHPC)

If I missed something - please add in the comments.

Things like HPC Orchestrator and Omni-Path appeared only in the last 1.5-2 years and allowed Intel to cover 2 additional segments.

Intel's interest in HPC is easy to understand. Now a lot of computing-intensive areas are developing, like genomics or the Internet of things. The human genome sequencing laboratories can generate hundreds of gigabytes of data per hour. The same can be said about the turbines hung with sensors. And this is not just data. This is the data that you need to "thresh" almost in real time. And on the spot, where they are generated. For sending them for processing to the cloud (for example, from Amazon) requires unrealistically thick channels and wallets.

According to IDC, every 4th server sold in 2018 will be sold to HPC.

So, Intel's motivation is quite understandable. But where will he go next?

Here you can only guess. As it seems to me, Intel will take the most radical steps in the Storage and Interconnect segments.

Currently, HPC can distinguish two basic tastes of storage: very fast local (but small) and basic (slower, but large). The first is called the Burst Buffer.

I suspect that it is Burst Buffer that will be of interest to Intel in the near future. And here it is quite possible to shoot products based on 3D XPoint. In particular, the declared, but not yet released, “memory” Optane (which is not quite memory).

However, the hardware itself in the Storage segment is not as interesting as the general pattern of user / application interaction with it. At the moment, one cannot say that the concept of Burst Buffer in HPC is clearly outlined. So far, there is no common "standard" idea of ​​what it is. How do different levels of storage hierarchy relate? How do HPC applications interact with data warehouses? How do you integrate with HPC Task Schedulers?
Not the fact that these and similar questions should be given a standard answer. But most likely it will be. And it seems that Intel has its own vision of this problem. At least, some leaks / hints on how Intel will act here on the sidelines of the Supercomputing'16 conference held at the end of last year. While this is just a rumor. Wait and see.

The topic of data storage is very interesting. However, I think that the biggest changes await us in the Interconnect segment, which Intel suddenly attended to.

Many experts expect that high-speed interconnects like InfiniBand or Omni-Path will eventually push out the I / O bus. Now the bus is an unnecessary intermediary between them and the devices involved in the exchange of data.

In order to remove the bus from the data transfer chain, direct integration of the processor and the memory subsystem with the factory will be required. It is very interesting to see how this will be done. And since Intel started playing with its own factory, now it has a direct path to integrating it with the processor. Not to mention the fact that there is freedom to choose a solution that will put a stick in the wheels of alternative factories like InfiniBand.

That is why it seems to me that the Omni-Path is not so much interesting as the short-term alternative to InfiniBand, but rather as a springboard for Intel's further research.

I have, perhaps, everything. What do you think about Intel, HPC and " all that "?