CoreOS - Linux for minimalist clusters. Short

What is CoreOS?

CoreOS is a Linux-based operating system for building easily and flexibly scalable clusters. CoreOS is a minimalist distribution. The installation ISO image is only 136 MB in size, and in the memory on the final machine after installation and start-up it will take only 114 MB. CoreOS is based on ChromeOS, which in turn is based on Gentoo .

In fact, CoreOS can be divided into the following parts:
')

• Systemd - manages local services on cluster machines
• Docker - provides isolation of services, but its use, in principle, is not necessary
• Etcd - Distributed Cluster Configuration
• Fleet - provides distributed service management ("add-on" above systemd)

CoreOS can run systemd services on the necessary cluster machines, monitor their status, and store their configuration.

Systemd

CoreOS uses the usual systemd , which can now be found in many Linux distributions. This is a local management system for Linux services.

Docker

About Docker more than once wrote on Habré . In short and in a simple way - Docker takes the kernel of the Linux host operating system, mounts the file system image you specify to it and allows you to work inside this image as if you are in a virtualized environment.

Please note that Docker does not use virtualization. To isolate processes, cgroups and namespaces are used , which makes it possible to get rid of a rather serious overhead project, which is provided by tools such as KVM. Of course, Docker can be used both on Mac and Windows, but in this case the host operating system will have to work inside VirtualBox or any other virtualization system, since Docker needs the Linux kernel to work (see Boot2Docker )

Etcd

Etcd is a distributed Key-Value storage that runs on each CoreOS cluster machine and provides general access to virtually all data across the entire cluster. The etcd stores service settings, their current status, the configuration of the cluster itself, etc. Etcd allows you to store data hierarchically (tree storage), subscribe to changes to keys or entire directories, set TTL values ​​for keys and key directories (actually, "expose" them), atomically change or delete keys, store them in an orderly manner (which allows you to implement specific queues ). Since the configuration of services running across the cluster is stored in etcd, you can learn about starting and stopping a service by simply subscribing to changes to the corresponding keys in the repository.

Etcd supports REST HTTP interface, so curl is enough to work with it. From the command line in CoreOS is controlled by the etcdctl utility.

Fleet

Fleet is an add-on over systemd that transfers service management from the local machine to the cluster level. Fleet stores the configuration of services as systemd units (in etcd), automatically delivers it to local machines, starts, restarts (if necessary), and stops the services on the cluster machines. Fleet is able to plan the launch of services based on the workload of specific cluster machines. It can be said that a specific service needs to be run only on certain machines, etc.

CoreOS is very simple

In order to start the hello service in a CoreOS cluster, you just need to do the following:

1. Login to any of the cluster servers via SSH (they are all equal)
2. Create a text file with a description of the service. For example, hello.service:

[Unit] Description=My Service After=docker.service [Service] TimeoutStartSec=0 ExecStartPre=-/usr/bin/docker kill hello ExecStartPre=-/usr/bin/docker rm hello ExecStartPre=/usr/bin/docker pull busybox ExecStart=/usr/bin/docker run --name hello busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done" ExecStop=/usr/bin/docker stop hello 

3. Run the following commands:

 fleetctl submit hello.service fleetctl load hello.service fleetctl start hello.service 

4. Everything! Having received the last command, fleet, because we did not give him any instructions on this matter, determine the least loaded cluster machine, take our unit file, transfer it to the selected machine, put it where systemd stores its services and asks systemd to start it. And after the launch will monitor the status. If the machine on which the service was performed suddenly “crashes,” fleet will restart our service on another machine.

Consider our example in more detail.

Service Configuration File:

 [Unit] #    Description=My Service #       docker.service After=docker.service [Service] #       #    ,  docker         TimeoutStartSec=0 #       . #       ,      . ExecStartPre=-/usr/bin/docker kill hello ExecStartPre=-/usr/bin/docker rm hello #          docker    # fleet ,          ExecStartPre=/usr/bin/docker pull busybox #    .      Bash. ExecStart=/usr/bin/docker run --name hello busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done" #     ExecStop=/usr/bin/docker stop hello 

Commands to execute:

 #               fleetctl submit hello.service # ,             systemd    fleetctl load hello.service #   fleetctl start hello.service 

You can view the status of services using the fleetctl list-unit-files command (a short summary of all services) or fleetctl status hello.service (detailed information about the service).

Install CoreOS

CoreOS can be installed in various ways . But if you decide to experiment, I recommend installing through Vagrant.

1. Install Git , VirtualBox , then the latest versions of Vagrant .
1a Or, run in any folder git clone github.com/coreos/coreos-vagrant.git && cd coreos-vagrant
1b. Or, if you didn’t want to download Git, unpack it somewhere.

2. Go to the folder, rename the file user-data.sample there to user-data , config.rb.sample to config.rb . Go to discovery.etcd.io/new , copy the URL that appears in the buffer, uncomment the line with the discovery instruction in user-data and replace the URL there.

3. Go to the folder and run vagrant up there
4. Wait for the base image to download.
5. Run vagrant ssh.

You inside :)

CoreOS limitations, weaknesses and surprises

CoreOS does not provide anything except what it provides :) In other words, “out of the box”, it doesn’t know anything except what we discussed. And, if you are going to build clusters based on CoreOS, you will encounter at least the following problems:

Batch Manager and Updates

CoreOS is missing a batch manager. And any system for building or installing additional software, too (except for wget and curl :)). Alternatively, I can suggest using the Docker integrated toolbox container . To activate it, simply run / usr / bin / toolbox from the command line. This will launch the privileged Fedora container, where you can access yum.

CoreOS updates are performed for the entire OS using a system of active / passive partitions. In fact, there are two system partitions in CoreOS. Only one of them is active at each moment. The update of the entire OS is installed on the second one, after which a reboot and a change of the active partition are performed. The operating system is updated only as a whole. Accordingly, it is quite easy to roll back to the previous version of the OS. If you, like me;), have already wanted to have a package manager in CoreOS out of the box, read the Developer Documentation .

Service discovery

The "trick" of CoreOS is that it easily transfers services between cluster machines, stopping and starting them on their own. Roughly speaking, this means that your web server, initially running on machine X, in five minutes can easily end up on machine Y, which has a completely different IP address on the internal network. Of course, with the help of special instructions for Fleet, you can easily “chain” the service to a specific machine, but ... then why do you need CoreOS at all? :) This creates an IP address resolution problem for your services. In principle, it is easy enough to request etcd and find out which machine is currently running the service. But this will not happen completely automatically. Service Discovery is a serious question and for its solution a huge amount of software is offered , from which SkyDNS appeals to me the most. You can use SkyDNS with the help of sidekick-units .

Container persistence and data per se

If you notice, CoreOS is actually built around stateless docker containers that are easy to transfer to another machine and run there. Indeed, it’s pretty easy to containerize, say, nginx and php-fpm. Configuration files for them can also be distributed as a container. Even the source code of your application can be distributed in the form of data-volume (this is a docker-container that contains only data) and connected to various services. What do you want to do with databases?

There are several outputs. First, services like Flocker, which migrate data containers after the service containers that use this data. Second, run DBMS-like services in cluster mode. NoSQL solutions can do this very well, relational solutions are noticeably worse, but still it is possible. Thirdly, it is possible to raise within a cluster any distributed file system like GlusterFS or Ceph. However, such a solution needs to be assessed in terms of speed. These issues were discussed here .

Conclusion

CoreOS is experiencing, I’m not afraid of these words, explosive growth. Like the Docker, in general. The number of different projects for CoreOS is increasing every day . So, if now you lack something for solving a specific task, tomorrow this opportunity may appear.

As part of the CoreOS project, an alternative standard for containerization of Rocket is being developed. However ... for now, apparently, it is better to use Docker. Moreover, CoreOS is not going to abandon Docker in the future.

What to read?

Of course, you can read the documentation . However, I really liked the series of articles on Digital Ocean about CoreOS . As a matter of fact, this article was written on its basis and under its influence :)