OS for data center Mesosphere: What is it and for whom

At Geektimes, it was already written that Mesosphere attracted investment (several rounds of funding brought investment to the company to $ 50 million) to create “the first operating system for data centers” [ why DPCs need operating systems can be read in this translation from our blog ].

The OS for data centers ( DCOS , data center operating system) is a new type of operating system that runs across the entire data center and does not control individual machines, but operates at a higher level and allows administrators to work with all the data center resources as with single virtual entity. This will simplify management and will give administrators the opportunity to put into operation as needed or, conversely, to deactivate the server and software much faster than it does now.
')
Virtual machines help to better use the resources of individual computers, but in the Mesosphere team they want to take this concept to a new level and allow users to apply the same principle to the resources of the entire data center. Since today's applications are increasingly focused on working with multiple servers , it becomes unimportant how the resources of an individual machine are allocated, and the current approach simply cannot meet all the emerging needs.

By applying the concept of virtual machines to the entire data center, it is possible to form a single pool of resources that experts can manipulate - this can be extremely useful for system administrators. Moreover, the ability to virtualize the full potential of a data center drastically reduces the time required to deploy server clusters from days and weeks to several hours or even minutes, depending on how large the cluster is to be formed.



A pool of virtual resources created to work with thousands of machines. In essence, the Mesosphere said that their standard consumer could have a data center containing about 50 thousand nodes, and that, according to their assumptions, this standard number could be increased to 500 thousand by the end of next year.

Administrators can work with the OS for data centers via the command line or use the graphical interface to quickly select and drag resources from the library to the command line panel. This gives administrators the ability to quickly create multiple instances of any software in the library. The default library includes popular open source products, such as Apache Spark, Apache Cassandra, Apache Hadoop, and Google Kubernetes, to which users can add their own internal software packages.

The data center OS is compatible with some versions of Linux, including Redhat, CentOS, Ubuntu, and CoreOS, as well as public cloud infrastructure from companies like Amazon, Google, and Microsoft. Finally, it can be used in conjunction with services from a private cloud infrastructure, including virtual machines, bare metal and OpenStack.

Administrators can also install triggers using specialized tools so that when certain conditions are met, the system automatically runs a predefined set of commands.



For example, administrators can form a set of commands based on an understanding of the cost of external services, so that it is launched only when the required cost threshold is exceeded. Moreover, administrators can take advantage of the API to create applications at once for a single pool of data center resources available from the Mesosphere operating system.

Who needs an OS for data centers?

The editors of Readwrite asked Brad Silverberg, the man behind the super-successful launch of Windows 95, who received the title “Mr. Windows "and recently invested in the Mesosphere.

According to the expert, the creation of a real operating system for data centers was considered one of the main purposes of cloud computing. However, the existing model of such a system does not work, since it is difficult to scale it effectively. It is too complicated and costly. The fact is that there are several technological trends that increase the scale of this problem, in particular the explosive growth in the number of mobile devices that require access to data center resources and the massive migration of applications to the cloud.

In addition, at the present time, radical new models of behavior are being observed, both for people (first of all, using smartphones) and machines (Internet of Things). The number of smartphones on the planet is about to exceed two billion, and this number will soon double.

In the meantime, more and more machines are getting IP addresses and are beginning to interact with data centers. Applications turn into large distributed collections of microservices that run from thousands of servers, and data continues to grow.

At the top of the iceberg are the expectations of the user who wants to get an adaptive application with the most fresh content. But in the usual ways to ensure the satisfaction of all these needs is very difficult.

Previously it was thought that virtualization could help create scalable solutions, but Silverberg believes that this did not happen.

This is just a transitional technology, not a fundamental paradigm shift. Think about all this in terms of system complexity. Instead of breaking one physical machine into several virtual ones (which you still need to manage), would it not be better to combine all the machines in your data center into one large abstract computer with one operating system? And you know what? This approach is already working.

The concept has already been tested on large-scale projects on Google with their Borg / Omega architecture and on Twitter supporting Apache Mesos.

OS for data centers removes all resource limitations. It creates a single pool of resources, within which it is possible to create applications and dynamically scale them in real time according to the needs for changing workloads.

This system uses the metaphor of a single computer, which is quite simple as a model for programming, and extends it to hundreds of thousands or even millions of machines. Such an approach allows to level all the complications involved in the scaling processes.

[ Full interview can be found here ]

Is everything really good?

Despite the positive attitude towards OS for data centers by venture capitalists and technology experts, not all users currently share their optimism. In the comments to the above topic on the GT, a habrauser (hiktaymsuzer?) Kovyl described his experience with the Mesosphere software:

I apologize for the sharpness. Just for me, the Mesosphere is like a red rag for a bull. Does anyone here present have any positive experience using IT in production? Can you share it? Tried to use it at home. We spent about six months (!!!) and a lot of nerves in the hope of squeezing anything out of it. The impression was that it was not working in principle.

According to the developers, the OS for data centers from the Mesosphere will be available next year (although access to the earlier version of the system already exists) - it is possible investments and the support of the expert community will help the team create a more balanced product.