Geekly Articles each Day
📜 ⬆️ ⬇️

Kubernetes 1.14: a review of major innovations



This night will be the next release of Kubernetes - 1.14 . Following the tradition that has arisen for our blog, we talk about the key changes in the new version of this wonderful Open Source product.

The information used to prepare this material is taken from the Kubernetes enhancements tracking table , CHANGELOG-1.14 and the corresponding issues, pull requests, Kubernetes Enhancement Proposals (KEP). UPDATE (March 26): the official announcement of the release appeared on the K8s blog.
')
Let's start with an important introduction from cluster-lifecycle SIG: Kubernetes dynamic fault-tolerant clusters (and more precisely, self-hosted HA deployments) can now be created using the usual (in the context of single-node clusters) kubeadm ( init and join ) commands. In short, for this:



Kubernetes HA-cluster architecture created with kubeadm

Implementation details can be found in the design proposal . This feature was really long-awaited: the alpha version was expected back in K8s 1.9, but it appeared only now.

API


The apply command and generally the declarative management of objects are moved from kubectl to apiserver. The developers themselves briefly explain their decision by the fact that kubectl apply is a fundamental part of working with configurations in Kubernetes, but “is full of bugs and difficult to fix,” and therefore this functionality must be reduced to normal form and transferred to the control plane. Simple and clear examples of existing problems today:



Details on implementation - in KEP . Current readiness - alpha version (advance to beta is scheduled for the next release of Kubernetes).

In the alpha version, it became possible to use the OpenAPI v3 schema to create and publish OpenAPI documentation on CustomResources (CR) used for validating (on the server side) K8s user-defined resources (CustomResourceDefinition, CRD). The OpenAPI publication for CRD allows clients (for example, kubectl ) to perform validation on their side (within the framework of kubectl create and kubectl apply ) and to issue documentation according to the scheme ( kubectl explain ). Details - in KEP .

Previously existing logs are now opened with the O_APPEND flag (and not O_TRUNC ) in order to avoid losing the logs in some situations and for the convenience of truncate logs with external utilities for rotation.

Also in the context of the Kubernetes API, it can be noted that the PodSandbox field has been added to the PodSandbox and PodSandboxStatus to take into account information about the RuntimeClass in the pod Admission Webhooks implemented the ability to determine which versions of AdmissionReview they support. Finally, in the Admission Webhooks rules, you can now limit the scope of their use by namespaces and cluster boundaries.

Storage


PersistentLocalVolumes , which had beta status since K8s 1.10 , have been declared stable (GA): this feature gate will no longer be disabled and will be removed in Kubernetes 1.17.

The ability to use environment variables of the so-called Downward API (for example, the name of the pod) for directory names that are mounted as subPath has evolved - in the form of a new field subPathExpr , which now determines the desired directory name. Initially, the feature appeared in Kubernetes 1.11, but for 1.14 it remained in the status of the alpha version.

As in the previous release of Kubernetes, many significant changes are presented for the actively developing CSI (Container Storage Interface):

CSI


Support for resizing for CSI volumes has become available (in the alpha version). To use it, you will need to enable a feature gate called ExpandCSIVolumes , as well as the presence of support for this operation in a specific CSI driver.

Another feature for CSI in the alpha version is the ability to refer directly (that is, without using PV / PVC) to CSI volumes within the scope of the pod specification. This removes the restriction on using CSI as an exclusively remote data repository , opening the door to the world of local ephemeral volumes . To use ( example from the documentation ) you need to enable the CSIInlineVolume feature gate.

Progress has also been made in Kubernetes-related CSI internals, which are not so noticeable to end users (system administrators) ... At the moment, developers are forced to maintain two versions of each storage plugin: one - in the old way, inside the K8s codebase (in -tree), and the second - in the framework of the new CSI (read more about it, for example, in here ) . This causes understandable inconveniences that need to be addressed as CSI stabilizes as such. Simply taking and declaring the deprecated API of internal (in-tree) plug-ins is not possible due to the corresponding policy of Kubernetes .

All this has led to the fact that alpha versions have reached the process of migration of the internal code of plug-ins implemented as in-tree in the CSI plugins, thanks to which the concerns of developers will be reduced to supporting one version of their plug-ins, and compatibility with the old APIs will be preserved and they will be declare obsolete in the usual way. It is expected that by the next release of Kubernetes (1.15) all plug-ins of cloud providers will be migrated, the implementation will receive beta status and will be activated in K8s installations by default. See the design proposal for details. The consequence of this migration was also the waiver of restrictions for volumes defined by specific cloud providers (AWS, Azure, GCE, Cinder).

In addition, support for block devices with CSI ( CSIBlockVolume ) is translated into beta.

Nodes / Kubelet


Introduced alpha version of the new endpoint in Kubelet, designed to return metrics on the main resources . Generally speaking, if earlier Kubelet received statistics on the use of containers from cAdvisor, now this data comes from the container's executable environment via the CRI (Container Runtime Interface), however, compatibility for working with old Docker versions is also preserved. Previously, statistics collected in Kubelet were given through the REST API, and now the endpoint located at /metrics/resource/v1alpha1 used for this. The long-term strategy of developers is to minimize the set of metrics provided by Kubelet. By the way, these metrics themselves are now called not “core metrics”, but “resource metrics”, and are described as “first-class resources, such as cpu, and memory”.

A very interesting point: despite the clear advantage in gRPC endpoint performance compared to different uses of the Prometheus format (the result of one of the benchmarks see below) , the authors preferred the Prometheus text format because of the clear leadership of this monitoring system in the community.

“GRPC is not compatible with major monitoring pipelines. Endpoint will only be useful for delivering metrics to Metrics Server or monitoring components that integrate directly with it. When using caching in Metrics Server, the Prometheus text format performance is good enough for us to prefer Prometheus over gRPC, given the widespread prevalence of Prometheus in the community. When the OpenMetrics format becomes more stable, we can get closer to the performance of gRPC using a proto-based format. ”


One of the benchmarks for using gRPC and Prometheus in the new Kubelet endpoint for metrics. More charts and other details can be found in KEP .

Among other changes:


CLI


In cli-runtime and kubectl the -k flag has been added for integration with kustomize (by the way, its development is now carried out in a separate repository), i.e. to process additional YAML files from the kustomization special directories (see KEP for details on their use):


An example of a simple use of the kustomization file (perhaps a more complex use of kustomize in overlays )

Besides:


Other


Stable (GA) status received the following features:


Other changes introduced in Kubernetes 1.14:


PS


Read also in our blog:

Source: https://habr.com/ru/post/445196/

More articles:


All Articles