CNCF Open Source Solutions Guide (and more) for cloud native

On March 8, CNCF, created by The Linux Foundation, presented a significantly updated version of its “map of the previously uncharted territory of cloud (cloud native) technologies” - Cloud Native Landscape 2.0. The information gathered in this project is aimed at companies that are just starting their way into the world of infrastructure for cloud native applications, and is designed to acquaint them with the many existing solutions with open source and not only.

Path to cloud native

To begin with, let's look at what is called “cloud native” in the fund with a consonant name (Cloud Native Computing Foundation). In the understanding of CNCF, these are applications (and related infrastructure) that are endowed with the following properties:
')

• possibility of exploitation , i.e. application / system lifecycle management with external tools;
• possibility of observation , i.e. availability of mechanisms for obtaining the current status and performance;
• elasticity , which allows to increase and decrease in size within the available resources and depending on the changing load;
• fault tolerance , guaranteeing rapid automatic recovery in the event of a fall;
• dynamic , which includes rapid deployment (deploy), iteration, configuration changes.

Introducing the variety of existing products that help applications meet these requirements is the following description of the main steps towards cloud native (see the translation below the picture) .



So, the recommended process for developing cloud infrastructure in CNCF Trail Map is as follows:

1. Containerization Usually implemented using Docker. You can pack in containers applications and their dependencies of any size (although the code for the PDP-11, run in the emulator). In due course it is recommended to break existing functions into parts, and to implement new ones as microservices.
2. CI / CD . Configure continuous integration and delivery (Continuous Integration / Continuous Delivery) so that changes in the source code are automatically transformed into a container assembled with them, tested and deployed to staging and ultimately, possibly, to production. Configure automated rollbacks, rollbacks and testing.
3. Orchestration . Choose a solution for orchestration. The market leader here is called Kubernetes, and certified versions are recommended as its distributions.
4. Observation and analysis . Choose solutions for monitoring, logging, tracing. CNCF projects include Prometheus (monitoring), Fluentd (logging), Jaeger (tracing).
5. Service mesh . These products address issues of interaction between services and their availability from outside (from the Internet), including the discovery of services, checking their status, routing, load balancing. CNCF offers Envoy, Linkerd, and CoreDNS (we also recently wrote about Conduit , which, however, [for now?] Is not on the CNCF project list) .
6. Network More flexible networking capabilities help provide CNI- compatible solutions: Calico, Flannel , Weave Net.
7. Distributed database . When one DBMS ceases to be enough, the Vitess project is recommended for running MySQL on a large scale.
8. Messaging . To achieve better performance than JSON-REST, CNCF suggests trying gRPC.
9. Container runtime . Alternative environments for launching containers that are compatible with OCI are containerd , rkt, and CRI-O .
10. Software distribution . Notary can be used to safely distribute collected software (we wrote about it at the end of this article ) .

The first 3 points are basic, and the rest are optional. Depending on the situation.

Cloud Native Landscape

The “landscape” itself is a rather extensive catalog of products, not limited to the projects of the CNCF itself and even free licenses (although most of them belong to Open Source). For convenience, they are divided into categories:

• Application development : databases and data warehousing (Vitess, CockroachDB ...), queues (RabbitMQ, Kafka ...), source code management (GitLab, GitHub ...), application modeling / definition (Docker Compose, Bitnami ...), continuous integration and delivery ( GitLab Runner, Jenkins ...);
• Orchestration and management : planning / orchestration (Kubernetes, Mesos ...), interaction and discovery services (CoreDNS, Consul ...), service management (gRPC, Linkerd ...);
• Environment for execution : cloud storage ( Rook , Ceph ...), runtime for containers (containerd, rkt ...), cloud network solutions (CNI, open vSwitch ...);
• Provisioning : tooling and management for hosts (Ansible, Chef ...), infrastructure automation (Helm, Terraform ...), container registries (Docker Registry, Quay ...), security of images (Notary, Clair ...), key management (Vault, Spire ...) ;
• Platforms : Kubernetes certified distributions (OpenShift, Tectonic ...), Kubernetes certified platforms (Google Kubernetes Engine, Azure Container Service ...), non-certified Kubernetes implementations (Amazon EKS, ContainerShip ...), PaaS and container services (Heroku, Hyper.sh ...);
• Observation and analysis : monitoring (Prometheus, Datadog ...), logging (fluentd, Graylog ...), tracing (Jaeger, Zipkin ...), serverless (many subcategories are in a separate table );
• Clouds : public (AWS, Google Cloud ...) and private (OpenStack, MAAS ...);
• Certified Kubernetes Service Providers (Heptio, Huawei ...).

This entire collection is presented graphically:


( Full image in github .)

( Note: in this table you can also see that Kubernetes was the first project in the history of CNCF that achieved the status of graduated , i.e. “graduate.” This was announced quite recently - on March 6. All other CNCF projects fall into the categories of incubating and inception.)

Interactive Landscape

In addition, with the second version of Cloud Native Landscape, CNCF launched an interactive web version of the catalog, available as landscape.cncf.io (for serverless, s.cncf.io ).

Content for Landscape

Cloud Native Landscape catalogs are generated based on data obtained from a special YAML , to which information from external services is added (information about the project in GitHub, financial indicators from Crunchbase and Yahoo Finance), resulting in a new YAML , as well as JSON , which and is used by the application for data output.

CNCF welcomes the addition of new projects, which is done through a pull request to the landscape.yml mentioned. The project must have at least 250 stars on GitHub and it must match one of the available categories (for details on the requirements, see here ).

The main table of the Cloud Native Landscape, as well as its counterpart for serverless, and the previously mentioned Trail Map, are available in different formats . All project data is distributed under the terms of the free Apache License 2.0 except for graphic files and YAML (they are licensed under Creative Commons Attribution 4.0), information from Crunchbase and project / product logos.

PS

Read also in our blog:

• “ CNCF offered a free open source cloud for DevOps / microservices ”;
• “ How many developers think Continuous Integration is not needed? ";
• “ What famous companies use Docker in production and for what?” ";
• “ Why do you need Kubernetes and why is it more than PaaS? ";
• Statistics The New Stack on the Difficulties of Implementing Kubernetes ;
• " Infrastructure with Kubernetes as an affordable service ."