Why do you need containerd and why it was separated from Docker

In mid-March, it became known that the Docker company proposed its containerd project to the independent fund of the Cloud Native Computing Foundation (by the way, this happened simultaneously with CoreOS rkt) . The event followed the promise of the company last December, when containerd was officially separated from the Docker Engine. What is this component and why was it separated?

How containerd works

containerd is the former part of Docker, and now an independent solution that implements the executable environment for running containers . When it was created, according to the developers, they were striving for simplicity, reliability and portability.
')
“Physically” is a daemon on the host system that manages the entire life cycle of the container: from receiving and storing an image to launching the container (via runC — see below for details) and controlling its operation. You can interact with the containerd daemon via a low-level gRPC API via a local UNIX socket, and the console utility ctr is also available for experiments and debugging (it also uses the gRPC API) . The source code is written in Go and is available on GitHub under the Apache License 2.0 license.



The main primitives that containerd works with are bundles and containers. The containers are run by runC , a utility written in Go, using libcontainer and separated from Docker in 2015. It works according to the OCI Runtime Specification specification and runs containers as its child processes. To start it requires only the root file system and configuration (everything else: obtaining an image, unpacking it, etc. - remains “behind the scenes” for it) .

Bundles contain configuration, metadata, and root file system data. They are a disk representation of a running container (in the simplest case, it is a regular directory in the file system) , which can be transferred to other systems, packaged and distributed. Essentially the entire containerd device is to coordinate the creation and launch of the bundles.

More about architecture

The components of the containerd form the following subsystems:

1. Distribution - a service that provides images of containers.
2. Bundle is a service that allows you to extract (and package) bundles from disk images.
3. Runtime - a service for running bundles (calls runC to start containers with the parameters passed to them).

Each subsystem has one or more components that implement its behavior. It is to the services provided by the subsystems that containerd users access through the gRPC API.

Components of containerd that work simultaneously with different subsystems are called modules and are represented by the following:

• Executor, which implements the immediate launch of the container.
• Supervisor, which controls and reflects the status of the container.
• Metadata storing metadata in a graph database.
• Content, which provides access to the addressed storage of content (permanent data).
• Snapshot, which manages file system snapshots for container images. Analog graphdriver in today's Docker. Layers are unpacked in snapshots.
• Events, which implements event behavior and auditing.
• Metrics, which provides accessibility (by API) of metrics of various components.

All together it looks like this:

How it works

The schema and its description are taken from the containerd / design / architecture document:

• The controller of the distribution subsystem receives a request to pick up the desired image. It places the contents of the image in the content store (Content store) , and pointers to the image name and root manifest are registered in the metadata store (Metadata store) .
• When the image is received, the user can (via the Bundle controller) unpack the image into a bundle. The layers of this image (obtained from the content repository) are unpacked into the Snapshot module.
• When the snapshot of the container's root system is ready, the Bundle controller, using the manifest and configuration of the image, creates a configuration for launching (in particular, a list of mount'ov obtained from the Snapshot module).
• The prepared bundle is transferred to the Runtime subsystem for launch. It reads the finished configuration to create a working container.

Docker role, communication with other components

Delivering containerd to a separate project began about a year ago when Docker developers attempted to “ make the Docker Engine smaller, better, faster and stronger ” and explained their actions: “Having a standalone runtime environment like runc, we needed an even neat point for integration, to add runc to the common stack and manage hundreds of containers. ”

At the time of the allocation of containerd from Docker (1.12) in terms of the project’s development, it was “refactoring the Docker Engine code base to remove most of the deployment logic, networking and storage on a single host into a reusable component that can be used in Docker and use other container orchestration projects and container placement services. ”

The set of functions assigned to containerd is as follows:

• placing images in the Docker Registry;
• network support for creating system interfaces and APIs for managing the container name network space;
• storage (at the host level) for the image and container file systems;
• gRPC API (it is for him and the Docker Engine itself communicates with containerd);
• A new API for Prometheus metrics used inside and at the container level;
• full support for the specification of OCI images (Open Container Initiative) and the runC reference implementation.

So Docker sees the use of the future containerd 1.0 (its release is scheduled for June 2017) inside the Docker Engine:



Summing up, the authors argue that containerd focuses on the opportunities that "any container-oriented platform needs," and nothing superfluous. In fact, this is a fairly low-level solution , which is not designed for end-users and users to interact with, but for larger systems such as Kubernetes (within Kubelet) and Mesos, as well as services like Amazon ECS and Google Container Engine. Embedding into products is offered through the same gRPC API (the ctr utility is intended only for debugging and experiments).

PS Last week, the Cloud Native Computing Foundation formally confirmed the adoption of containerd among its projects. And on the same day, the rkt project, which solves similar containerd tasks in CoreOS, suffered such a fate: it is an implementation of the open App Container (appc) specification and uses the ACI (Application Container Image) format for images. CNCF will support the development of both projects.