A new foundation for DevOps projects from the Linux Foundation began with Jenkins and Spinnaker

Last week, The Linux Foundation organized a new fund for open source projects during its Open Source Leadership Summit event. The next independent institute for the development of open [and demanded in the industry] technologies is designed to combine tools for DevOps engineers, or more precisely, for organizing and implementing continuous delivery processes, the CI / CD pipelines. The organization was called: The Continuous Delivery Foundation (CDF).

To better understand why such funds are being created at the parent organization of the Linux Foundation, just look at the more well-known example, CNCF (Cloud Native Computing Foundation). This fund appeared in 2015 and since then it has accepted into its ranks many open source projects that truly define the modern landscape of the cloud IT infrastructure: Kubernetes, containerd, Prometheus, etc.
')
The organization itself acts as an independent platform, on the basis of which these projects are managed and developed in the interests of various market participants. To do this, CNCF created technical and marketing committees, adopted certain standards and rules (if details are interesting, then we recommend reading, for example, CNCF TOC Principles ) ... And, as we see in the "living" examples, the scheme works: projects that are in CNCF, are becoming more mature, are gaining popularity in the industry, both among end users and among the developers involved in their development.

Following this success (after all, many CNCF cloud projects have already entered the daily life of DevOps engineers) , general trends in IT and their manifestations in the world of Open Source, The Linux Foundation decided to “take” (and it would be more correct to say that ") A new niche:

“The Continuous Delivery Foundation (CDF) will become a vendor-independent home for the most important open source projects designed for continuous delivery and specifications that accelerate the pipeline processes. CDF will facilitate the interaction of leading developers, end users and vendors from the industry, promote CI / CD and DevOps methodologies, identify and document best practices, create manuals and training materials that will allow the software development team from anywhere in the world to implement the best CI / CD practices. ” .

Idea

The core values and principles that guide the CDF are currently formulated in such a way that the organization:

... believes in the power of continuous delivery and in the fact that it gives new opportunities to developers and teams, allows us to produce high-quality software more often;
... believes in open source solutions, the joint application of which covers the entire cycle of software delivery;
... cultivates and maintains an ecosystem of Open Source-projects that are not dependent on vendors, through joint activities and mutual compatibility;
... promotes and stimulates among professionals engaged in continuous delivery in practice, teamwork, the spread of their practices and their improvement.

Participants and projects

But beautiful words - marketers, which does not always coincide with what is actually happening. And in this sense, the first impression about the organization can be made up by who formed it from the companies and which projects became its “firstborn”.

The main members of the CDF are 8 companies , namely Capital Capital, one of the top 10 banks in the US, and industry representatives much more familiar to IT engineers, such as CircleCI, CloudBees, Google, Huawei, IBM, JFrog and Netflix. Some of them have already talked about such a significant event in their blogs, but more on that - just below.

CDF participants also include end users of its projects - CNCF has a similar category, where you can meet eBay, Pinterest, Twitter, Wikimedia and many others. In the case of a new fund, there are only 15 such participants, but among them there are already some interesting and well-known names: Autodesk, GitLab, Puppet, Rancher, Red Hat, SAP and Sysdig joined just yesterday .

Now, perhaps, the main thing is about the projects, the care of which was entrusted to the CDF. There were four of those at the time of the creation of the organization:

Jenkins and Jenkins X

Jenkins is hardly a system for CI / CD, written in Java, which has hardly existed for many years (think only: the first release - in the form of Hudson - took place 14 years ago!) , Over which it was overgrown with an endless army of plug-ins.

The main commercial structure behind Jenkins today can be considered CloudBees , the technical director of which is the original author of the project (Kohsuke Kawaguchi) and who became one of the founders of the foundation.

Jenkins X - this project is also largely obliged to CloudBees (as it is easy to guess, its main developers are listed in the staff of the same company), however, unlike Jenkins itself, the solution is completely new - it is only a year old.

Jenkins X offers a complete solution for organizing CI / CD for modern cloud applications deployed in Kubernetes clusters. For this, JX offers automated pipelines, a built-in implementation of GitOps, a preview release environment, and other features. The Jenkins X architecture is represented as follows:

Product stack - Jenkins, Knative Build, Prow, Skaffold and Helm. More about the project, we have already written on Habré.

Spinnaker

Spinnaker is a platform for continuous delivery, created in Netflix, the source code of which was opened in 2015. Currently, Google is actively involved in its development: together, the product is being developed as a solution for large organizations whose DevOps teams serve many development teams.

The key concepts in Spinnaker for describing services are applications (applications), clusters (clusters) and server groups (server groups), and load balancers and firewalls are involved in their accessibility to the outside world:

More information about the basic Spinnaker device can be found in the project documentation .

The platform allows you to work with a variety of cloud environments including Kubernetes, OpenStack and various cloud providers (AWS EC2, GCE, GKE, GAE, Azure, Oracle Cloud Infrastructure), as well as integrate with various products and services:

with CI systems (Jenkins, Travis CI) in pipelines;
with Datadog, Prometheus, Stackdriver and SignalFx - for monitoring events;
with Slack, HipChat and Twilio - for notifications;
with Packer, Chef and Puppet - for virtual machines.

Here is what Netflix wrote about incorporating Spinnaker into a new foundation:

“Spinnaker’s success was largely due to a wonderful community of companies and people who use it and contribute to development. Passing Spinnaker to CDF will strengthen this community. This step will contribute to the emergence of changes and investments from other companies that have been watching from the outside. Open doors for new companies will bring new innovations to Spinnaker that everyone will benefit from. ”

And in a Google publication on the occasion of the creation of the Continuous Delivery Foundation, it is specifically noted that "Spinnaker is a multi-component system that is conceptually consistent with Tekton." So we come to the latest project, which is included in the new fund.

Tekton

Tekton is a framework presented in the form of common components for the creation and standardization of CI / CD systems, implying the work of the pipelines in various environments including conventional virtual machines, serverless and Kubernetes.

These components themselves are “Kubernetes-style” resources (implemented in K8s itself as CRD), acting as building blocks for defining pipelines. A brief illustration of their use in the K8s cluster is presented here .

The product stack supported by Tekton will seem familiar: Jenkins, Jenkins X, Skaffold and Knative. In Google Cloud, they believe that Tekton is solving the "problem of the joint work of the Open Source-community and leading vendors to upgrade the infrastructure for CI / CD."

...

By analogy with the CNCF, a technical committee (Technical Oversight Committee, TOC) has been established in the CDF, whose responsibilities include the consideration of issues (and decisions) on the inclusion of new projects in the fund. There is not so much other information about the organization itself on the CDF website , but this is normal and only a matter of time.

Let's finish the quote from the JFrog announcement :

“Now, becoming one of the companies of the only created fund of the Continuous Delivery Foundation, we will withdraw our commitment [to create a technology that is universal in its support of other CI / CD solutions] to a new level. This new organization will, following the ideas of collaboration and openness, manage future standards for continuous delivery, which will accelerate the software release cycle. With the adoption of Jenkins, Jenkins X, Spinnaker and other technologies under the wing of this foundation, we see a bright future for CI / CD! ”