New MIT CPU Load Balancer

System Shenango plan to use in data centers.

/ photo by Marco Verch CC BY

According to one of the providers, data centers use only 20–40% of the available computing power. With high loads, this figure can reach 60% . Such a distribution of resources leads to the emergence of so-called "zombie servers." These are machines that idle most of the time, wasting energy. Today, 30% of servers in the world are out of work , consuming electricity for $ 30 billion a year.

It was decided to fight the inefficient use of computing resources at MIT.
')
A team of engineers developed a processor load balancing system called Shenango. Its goal is to monitor the status of the task buffer and redistribute “stuck” processes (which cannot get CPU time) to free machines.

How does Shenango work

Shenango is a C-language Linux library with Rust and C ++ commands. The project code and test applications are published in the repository on GitHub.

The solution is based on the IOKernel algorithm, which runs on a dedicated core of a multiprocessor system. It manages requests to the CPU using the DPDK framework, which allows applications to interact directly with network devices.

IOKernel decides which kernels to transfer a specific task to. The algorithm also decides how many cores will be needed. For each process, the main cores (guaranteed) and additional (burstable) are defined - the latter are launched in case of a sharp increase in the number of requests to the CPU.

The IOKernel request queue is organized as a ring buffer . Every five microseconds, the algorithm checks whether all tasks assigned to the kernel have been completed. To do this, he compares the current location of the “head” of the buffer with the previous position of its “tail”. If it turns out that the tail was already in the queue at the time of the previous check, the system notes buffer overload and allocates an additional core for the process.

During load distribution, priority is given to the cores on which the same process was performed earlier and partially remained in the cache memory, or to any inactive cores.

Shenango additionally uses the work stealing approach. Kernels allocated for the work of one application, monitor the number of tasks from each other. If one core completes its list of tasks before the others, then it “removes” part of the load from its neighbors.

Advantages and disadvantages

According to MIT engineers, Shenango is able to process five million requests per second and maintain an average response time of 37 microseconds. Experts say that in some cases, the technology can increase the utilization of processors in data centers to 100%. As a result, data center operators will be able to save on the purchase and maintenance of servers.

The potential solutions are also noted by specialists from other universities. According to a professor from the Korean Institute, the system from MIT will help reduce delays in the work of web services. For example, it is useful in the work of online stores. On sales days, even a second delay in page loading leads to a decrease in the number of site views by 11%. Operational load sharing will help serve more customers.

The technology still has flaws - it does not support multiprocessor NUMA systems, in which the chips are connected to different memory modules and do not “communicate” with each other. In this case, IOKernel can regulate the work of a separate group of processors, but not all server chips.

/ photo by Tim Reckmann CC BY

Similar technologies

Among the other systems load balancing processors can be identified Arachne. It calculates how many cores the application will need at the time of its launch, and distributes the processes in accordance with this indicator. According to the authors, the maximum delay of the application in Arachne is about 10 thousand microseconds.

The technology is implemented as a C ++ library for Linux, and its source code is on GitHub .

Another balancer tool is the ZygOS. Like Shenango, technology uses the work stealing method to redistribute processes. According to the authors of ZygOS, the average delay in the work of applications when using the tool is about 150 microseconds, and the maximum - about 450 microseconds. The project code is also publicly available .

findings

Modern data centers continue to expand. Especially, the tendency to increase is noticeable in the hyperscale data center market: there are now 430 hyper-scalable data centers in the world, but in the coming years their number may increase by 30%. For this reason, processor load balancing technologies will be in high demand. Systems like Shenango are already being implemented by large corporations, and in the future the number of such tools will only grow.

Posts from the First Corporate IaaS Blog:

Source: https://habr.com/ru/post/447058/

All Articles