We put new trends in the IT infrastructure on the shelves: what is scale-out, web-scale, hyperconvergence

Can we expect open source in the "iron"?

Vladimir Alekseev, IBM







In the past few years, active discussions have been going on in the hardware market on the topic of hyperconvergence, the departure towards horizontal-scalable systems and IT web-scale, software-defined solutions. Below we will try to figure out how all these things are interconnected, and what approaches can be used for which tasks.

We understand the terminology

Scale-up and scale-out systems

')

So let's start with the separation of all systems into two types, depending on scalability: scale-out and scale-up. Vertically scalable systems (scale-up) appeared on the market earlier and were initially presented with mainframes, and then the systems of the RISC architecture of the processor (IBM Power systems, Oracle SPARC systems, etc.). Scale-out systems (scale-out) appeared much later, and their dawn is associated with the proliferation of the x86 architecture. Scale-out architecture is characterized by the presence of a large number of small servers on which the application task is distributed. Despite fundamentally opposing ideas, scale-up and scale-out approaches are often combined within a single system, for example, the database is located on a vertically scalable hardware, and the application server is horizontally scalable.



Hyperconvergence



But all these methods and approaches have long been known to the market and are used almost everywhere. Despite the ongoing development of vertically scalable systems, it was within the scale-out approach that a large number of different directions emerged, which include both the discussed and the more and more hyperconvergent systems used. What are these systems and what properties do they have?



First of all, hyperconvergence means the integration or integration of all hardware components (servers, data storage systems and networks) into one construct. Secondly, hyperconvergence implies the presence of software components in the solution for the implementation of fault tolerance and reliability issues. This software layer is also called the level of hardware abstractions or, in simpler terms, it is the level of virtualization of hardware resources. Here, the concept of a “software-defined environment” or software-defined environments is closely associated with hyperconvergence. The concept means the presence of virtualized resources at each level (servers, storage systems, network), and the management functions are fully automated at the software level.

Thus, by using the approaches of software-defined environments, it is possible to combine physically separated hyper-convergent systems into a single virtual pool of resources.



No need to assume that the software-defined infrastructure can be built only on the basis of hyper-convergent systems - this is not so. On the contrary, it is impossible to build a reliable IT infrastructure based on hyperconvergent solutions without using the approaches of software-defined environments.



The list of vendors producing hyperconvergent systems is wide, it includes both well-known brands and companies that have made a name for themselves on such decisions.



Why is it worth using hyperconvergent systems?



One of the key features of IT infrastructure built on hyperconvergent systems is the unification and standardization of the components used. It turns out that the infrastructure is built on the same type of pre-configured blocks, which are connected with each other at the level of resource virtualization. Since the blocks are universal, they are easy to replace in case of failures or to supplement with an increase in workload.



It turns out that hyperconvergent solutions are efficiently used for the mass of small non-critical business tasks, where deep optimization of systems is not required, but it is necessary to ensure fault tolerance. In this case, it is convenient to have a unified infrastructure consisting of identical blocks.



Disaggregation and options for its use



Disaggregation is the direct opposition of hyperconvergence. If convergence means the integration of components, then disaggregation separates them. What is it done for? Hyper-convergence does not allow creating the optimal infrastructure for each task separately. Disaggregation is aimed precisely at solving this optimization problem. Within this paradigm, solutions are created where storage resources (memory and disks), for example, can be structurally separated from the computing resources (processors). In a number of tasks, for example, for in-memory computing, this approach can be justified and provide more advantages than hyperconvergence.



Web-scale IT



This term was introduced by Gartner in 2014 and means a new approach to building an IT system architecture that allows the capabilities of large cloud-based service providers to be realized within the framework of a classic organization ¹ . Analysts say that the IT web-scale is the implementation of the approaches of the leading web-companies to build IT infrastructure.



In fact, the approach should include all the advantages that these companies have: low time to market, the possibility of making a large number of changes in a short time, the use of open hardware and software, the concepts of Agile and DevOps.



The main difference from hyper-convergent solutions, to which many refer to the web-scale, is the implementation of software-level fault tolerance. The web-scale approach also necessitates deep code optimization for hardware. This means the abandonment of any virtualization technology, since it introduces an additional layer of interaction (and, potentially, delay) with the hardware. The hardware, in turn, is created taking into account the software architecture. Since the web-scale includes, first of all, the giants of the web-industry (Google and Facebook), their scale allows you to create special constructs of servers specifically for them. For example, Google uses in its servers an additional twelve-volt battery ² .



The hardware used within the web-scale can be integrated (as is the case with hyperconvergent systems), or it can be divided into components (the idea of ​​disaggregation) - it all depends on the application. It is important that the use of integrated constructs does not mean a hyper-convergent infrastructure within the framework of the web-scale. The reason for this is the differences in ensuring reliability: for hyper-convergent systems, this is virtualization, for web-scale - implementation at the software level and lack of virtualization.

Accelerated IT - a new industry trend

If approaches to building horizontally scalable systems can be called the third wave of the IT infrastructure revolution (the first is mainframes, then the client-server architecture), then accelerated computing can rightfully become the fourth revolution in the market. The meaning of accelerated IT (direct translation "accelerated") is to use hardware accelerators (for example, GPU or FPGA boards) to improve system performance.



Why use accelerators? First, not all tasks can be parallelized: for example, exchange systems work in one stream. And in most cases, exchange systems have reached the ceiling of the performance of current hardware systems. GPU accelerators can give them a new level of computation speed, exceeding several times the current performance. Secondly, a significant part of increasing the performance of servers from generation to generation lies on the processor. And the technological process of manufacturing processors has its limitations: 14 nm is now used, IBM announced the possibility of using 7 nm ³ . What will happen next? Obviously, the limit will not come tomorrow, but in the foreseeable future.



Thus, companies lose the standard productivity gain of 20-30% with the release of a new generation of processors. And the load on the system continues to grow. And what to do next? The answer is to use boosters.



Despite the fact that now GPUs are used, as a rule, more for tasks from the HPC (High Performance Computing) field, there are a number of areas where such systems can be claimed by commercial organizations even now. First of all - analytical systems, especially deep learning. These are the tasks of building multilevel neural networks, where their application uses can be image processing, speech recognition and text processing. GPUs can also be extremely effective for desktop virtualization (VDI) tasks.

OpenPOWER Consortium

It is worth noting that the creation of hardware solutions in the field of “accelerated” IT was one of the main ideas of the formation of the OpenPOWER consortium. What is this organization? In 2013, Google, IBM, NVIDIA, Mellanox, Tyan created a global open consortium of solution developers based on the Power processor architecture. The goal of the consortium is to create an open ecosystem using the Power architecture to share experience, investment and intellectual property in building server solutions. In many ways, this idea is similar to the concept of open source software - opensource software. Currently, the OpenPOWER consortium includes more than 200 organizations from more than 20 countries, including four Russian participants.



The idea of ​​OpenPOWER has created a certain paradox of modernity: the Power processor is becoming open, Intel - on the contrary, more closed. How is it shown? OpenPOWER creates an open ecosystem of partners that work closely with each other to create custom solutions using co-processors, graphics accelerators and integration tools with them. Intel, on the other hand, acquired Altera, a company that manufactures programmable chips (FPGA) ⁴ , the company also released its co-processor technology ⁵ , which in fact is a competitor to NVIDIA's graphics accelerator technology. The idea of ​​OpenPOWER is fundamentally opposite - it is the joint work of several companies. As part of the consortium, Power processors can be released based on specifications provided by IBM. For example, the Chinese company Suzhou PowerCore Technology is already planning to release a CP1 processor, made on the basis of IBM Power 8 ⁶ .

In any case, whichever of these approaches is more effective, the idea of ​​accelerated computing (accelerated IT) is implemented in both cases.

Where is the industry going? findings

What can be said at the end? Despite the large number of conversations about web-scale IT, this concept is not revolutionary in the market. This is already present: many organizations are already working on how to combine different approaches in their IT environment: vertical and horizontal-scalable systems. Accelerated IT is the future of the industry, which is only just beginning to feel the interest of the corporate sector.