Four releases of 1.0 from CNCF and major announcements about Kubernetes with KubeCon 2017

On these days (December 6-8), a local version of the KubeCon + CloudNativeCon 2017 conferences is being held in the USA, which focuses on numerous projects of the non-profit organization CNCF led by Kubernetes. The event gathered more than 4,100 visitors, 77% of whom use K8s, and in 75% of cases we are talking about production. The event was not only rich in announcements from various start-ups and giants of the industry, but also became an occasion for summarizing the Open Source-community from the cloud native world - it’s enough to note that in December, four of the CNCF projects had a sign release 1.0. So what's new?

A series of releases

CoreDNS 1.0.0

December 1, CoreDNS 1.0.0 was released . More about this project, we have already told in a separate article . In short, CoreDNS is a DNS server written in Go and similar to Caddy (moreover, it originated as its fork) in its key architectural pattern — using a set of handlers linked together. As a backend (storage) for data used in DNS records, CoreDNS supports etcd, Kubernetes, and a zone file in RFC 1035 format.
')

CoreDNS development plans for April 2017

The main area of developer activity in preparing CoreDNS 1.0.0 has been improvements in the plugin support for Kubernetes. And this is logical, if we recall that the authors of the project are seeking to replace kube-dns . They reinforce their intention with evidence that CoreDNS functionality is wider, performance is better, and memory consumption is less. An example of testing Kubernetes cluster with 5000 services is given, in which CoreDNS was able to process 18000 requests per second using 73 MB of RAM, against the figure of 7000 qps for kube-dns with a consumption of 97 MB of RAM.

Among the opportunities in which CoreDNS exceeds kube-dns is filtering records by namespace (namespace) and label selector; pods verified mode for checking the dough into existence before responding to the request pod.cluster.local ; endpoint_pod_names to use pod names when hostname is not set; autopath to autocomplete the server-side search path.

CoreDNS support has already been added to various Kubernetes tools (alpha feature for 1.9): kubeadm, kops, minikube and kubespray.

containerd 1.0.0

About containerd we also had a separate article . The history of this project is such that, initially being a part of Docker, containerd survived separation from a common code base and becoming an independent project under the wing of the CNCF at the same time as its competitor implementing the same functions (i.e., the executable environment for containers) - rkt from CoreOS.

The further destiny of containerd is its integration into Kubernetes via the CRI interface (“Container Runtime Interface”) through a connecting layer called cri-containerd :

More about this and the next main competitor of the project already on the new “ground” - CRI-O - we wrote here . By the way, at the end of November, a noticeable progress in the development of rkt was announced: CNCF announced the first release of rktlet , the rkt implementation on top of the same CRI.

But returning to the topic: containerd 1.0.0 was released on December 5th. And he introduced a lot of innovations in the capabilities of this product, and in particular:

A new model of the executable environment based on tasks (tasks)
client-defined push / pull for images and the ability to use third-party image distribution systems,
snapshot container storage system,
arbitrary storage support for object metadata
extended gRPC API with namespaces.

Fluentd 1.0

Fluentd is a data collector written in Ruby designed to unify the logging layer between log sources ( data sources : syslog, web server logs, etc.) and their storage systems ( data outputs : various DBMS, queue systems, AWS, etc. .). His broad adaptation of "thousands of companies" allows CNCF to call fluentd an "industry standard for logging." (By the way, we ourselves use it now in our logging system for Kubernetes - loghouse .)

In release 1.0 of December 6, appeared:

multiprocessor workers for better performance with SMP,
storage of exact time (with accuracy less than a second) for all records,
Windows support
A new API for plug-ins (and there are already more than 700),
compressing disk-stored data using built-in buffers,
Native support for TLS and new protocol Fluentd Forward Protocol v1 with support for key authentication and authorization by login / password.

Jaeger 1.0

Jaeger joined the ranks of CNCF projects more recently (in September). This is a distributed trace system written in Go, created by Uber and compatible with OpenTracing (this is also a CNCF project). The purpose of Jaeger is to conveniently monitor a complex microservice architecture designed to help in identifying the causes of problems that arise (taking into account all dependencies) and to help optimize performance.

Released on December 6th, Jaeger 1.0 brought this project:

support for new data storage backends (ElasticSearch has been added to Cassandra, as well as work on MySQL + is being prepared in the community experimenting with ScyllaDB and InfluxDB),
numerous improvements in the web user interface,
full integration of all components with Prometheus as the default metrics system,
improvements in integration with Kubernetes (K8s launch patterns and Helm chart ),
early version of client in C ++,
significant progress in backward compatibility with Zipkin.

New CNCF members

Of course, it was not without numerous replenishments in the ranks of the participating companies of CNCF.

Along with the announcement of the start of KubeCon + CloudNativeCon, the fund immediately presented 31 new members, among which Datadog (this SaaS monitoring solution had excellent Docker statistics ), Grafana Labs (as is easy to guess, by Grafana), HPE (Hewlett Packard Enterprise ), InfluxData (authors of the DBMS for working with time series - InfluxDB), NGINX, Pinterest (see details below), SAP Concur (wrote about them in this success story ).

And in separate news, new members of the “higher categories” at the CNCF were announced:

On December 4, Gold Member received the status of JFrog, offering an infrastructure for managing software development; the press release notes that the company “uses technologies like Kubernetes to help 4000+ customers collect and release software quickly, reliably and safely”;
On December 6, the CNCF ranks were replenished with a platinum member (there are only 21 of them now) in the form of Alibaba Cloud — a division of the large Chinese group of companies Alibaba Group (its chief architect entered the managing board of CNCF) responsible for cloud technologies; We have written about their experience with using Docker in production here ;
On December 6, another Chinese company, Baidu , became the gold member of CNCF; their main use is Kubernetes in the field of machine and in-depth learning (in particular, last year the code for the PaddlePaddle framework supporting the launch in the K8s cluster was discovered);
On December 8, Salesforce became the gold member of CNCF, which “adapted many CNCF technologies”; in particular, Kubernetes is used as a basis for internal CI / CD processes, as well as the active use of gRPC in the service mesh team.

Other events

Other news from prominent IT market participants was announced at KubeCon 2017. Among them:

Brendan Burns, one of the founders of the Kubernetes project and a current Microsoft employee, introduced Metaparticle , the “standard library for cloud native applications on Kubernetes,” which aims to simplify the development of distributed systems by providing “simple and powerful building blocks created from containers and Kubernetes; only JavaScript / Node.js, Java and .NET are declared as supported programming languages, but they promise to expand the list soon;

An example of implementing a JavaScript master election in Metaparticle

Oracle has released two Open Source products related to Kubernetes: an installer for its serverless platform, the Fn project, and a set of utilities for distributed clusters (Global Multi-Cluster Management);
Microsoft brought even more news, because:
1. opened the code for Virtual Kubelet , an alternative implementation of the Kubernetes component of the same name, specifically designed to connect K8s to third-party APIs (for examples, Azure Container Instances, Hyper.sh, AWS are given);
2. Opened the Open Service Broker for Azure (OSBA) code created on the basis of the Open Service Broker API to make it easier to work with Azure Data Services through the Kubernetes API: “For example, using OSBA and Helm, you can now easily install a Wordpress instance from Azure Database for MySQL instead of launching the DBMS in the container "; the project has the status of an alpha version;
3. introduced Kashti - dashboard and visualization tool for the Brigade pipelines ( demonstration on YouTube );
another announcement from Microsoft, already with the participation of Heptio, says that the company is working on implementing a backup system for Kubernetes clusters (as part of its recently introduced Azure Kubernetes Service) using the Open Source-utility Heptio Ark ;
Product Manager from the online service Pinterest , which was already mentioned, told about the migration of their production infrastructure to containers, which began in 2016 and is close to completion, as well as plans for its further transfer to Kubernetes (by the middle of next year);
AWS Vice President, Cloud Architecture Strategy, made a presentation in which he announced the company's plan to use CNI as a network basis for containers and promised that all changes made by AWS in the code of open source projects will go upstream (for this, the company is already "working with various open source-projects, communities and foundations").

Finally, the upcoming release of Kubernetes 1.9 is also tied to KubeCon 2017 dates: literally beta2 was released that night, all the documentation is expected to be completed over the next 24 hours, and the release itself is scheduled for the next Monday (December 11th).

PS CNCF collected statistics on KubeCon + CloudNativeCon North America 2017 visitors and how they use containers and various foundation projects in their infrastructure. Here, for example, the main difficulties encountered by the respondents and how they have changed over the past year: