Kapitan at the helm Kubernetes

Meet Kapitan . It will help you to restore beauty and order in the Kubernetes configuration.

Kapitan earns a reputation on the feedback of satisfied users, and therefore does without extensive documentation and expensive marketing. We have enough stars and a couple of mentions from bloggers and preachers Kubernetes. Kapitan even became the protagonist of an entire chapter in the book . Most importantly, he drew the attention of several promising companies, because Kapitan, like no one else, is able to unravel the configuration tied by a sea knot .

At kubernetes.slack.com, #kapitan managed to gather a small but dedicated community (join us!), So we are proud of our work :)

Many still believe that Kapitan is a mixture of jsonnet and jinja, but they are missing the point.

In this post I will tell you how Kapitan manages Kubernetes deployments, but in general he is not only capable of this. This is important: Kapitan is universal and not fixated on Kubernetes. Kubernetes is simply one of many uses.

This is not a guide (although I promise guides too). I just want to tell you why we made it and what problems it should deal with with the deployment of Kubernetes configurations.

What i didn't like

I started experimenting with Kubernetes in 2015 and immediately fell in love.

True, there are several shortcomings with which I do not want to put up:

• Context . As for me, this is one of the most difficult to understand Kubernetes concepts - and one of the most dangerous. The context refers to the client, it is difficult to understand it, it is called confused and creates confusion when running kubectl commands. I hate contexts in kubectl!
• Verbose nested configuration (yaml!) . I had to sweat to figure out each level of the yaml configuration in the manifest. What is the point of repeating labels two or three times in several places?
• Mess with imperative and declarative teams . Newbies in Kubernetes are encouraged to teach imperative teams, although it’s clear that normal people do not use them. In my opinion, it’s just harder to put Kubernetes in the right deployment strategy for your company. Spoiler: there is no single "right strategy".
• Runtime configuration . Jesse Suen is right when he advises not to pass configuration parameters to the helm command line (or kubectl and its ilk). With parameters, the same command can be executed differently each time.
• Application configuration We learned to manage the yaml manifestos at Kubernetes, we are great. That's just under and warming - this is an empty vessel. It still has to run the application with all its configuration.
• Developers want a holiday , and the workflow at Kubernetes is still kind of unintelligible. Kubernetes fans make everyone develop right there. Is it necessary? Obey Kelsey Hightower ( Kelsey Hightower )!
• Operators . I have mixed feelings for them, so for now let's leave this topic :) I can only say that they are often abused.
• Idempotency . Rather, its absence. If we add up all the flaws above, we get insufficiently idempotent workflows, which is sad for Kubernetes.

What to do?

I tried to solve these problems and put together a small template system that used j2cli and a couple of bash scripts to manage Kubernetes configurations.

The system thrust everything into the environmentA.yaml file and used it in the Jinja2 template. Deploit applications in the style of microservices from several components could be a simple command:

bin/apply.sh environments/environmentA.yaml 

Cool! Yaml was all about deploying. Very convenient, because I could use the same file as a source of information for something else. Say for ... bash scripts !

I figured out how to import values ​​from yaml into scripts in order to execute such commands:

 bin/create_kafka_topics.sh environments/environmentA.yaml 

And then everything went out of control at once :

• I could do nothing with the structure in the yaml file. The result was a jumble of identical fields, values, tangled configuration.
• You will never know how the deployment will behave in the environment until you try. Often this was due to changes in jinja2 templates due to a new inventory value (say, feature_X), which did not work in environments where this function is not defined.
• The same trouble with the scripts: if you don’t try, you don’t know.
• Sometimes Kubernetes changed so quickly that it took me constantly to change manifests for different versions, especially messing with annotations to values.
• External factor : the development team switched from configuration files to command line parameters. Such a small change confused us all the cards, and had to think about a new solution.
• The most important thing : yaml standardization with Jinja (or Go templates) is NOT FUN! We then had a riddle: “ What looks like text, reads like text, smells like text, but not text? ". Or, as Lee Briggs aptly put it: “ Why the hell do we template yaml? "

Kapitan: becoming

We gathered all our bitter experience and, together with Ricardo Amaro, began to fantasize about the ideal configuration management system. Then we did not have a clear picture, but we knew that we love and that we do not.

We love :

• Git
• Templating in general: data / values ​​separately from templates.
• Separate values ​​for different aspects (app, Kubernetes, runtime ...).
• Object oriented approach.
• Simplified yaml as an interface where to hide the complexity of Kubernetes.
• A clear understanding of what is happening and why.
• Reuse of values ​​in different components.
• And scripts must have access to values.

Do not like

• Contexts kubectl.
• Text template engines for creating yaml.
• {{ toYaml .Values.resources | indent 10 }} counting: {{ toYaml .Values.resources | indent 10 }} {{ toYaml .Values.resources | indent 10 }} .
• Magic: everything must be clear. No tricks.
• Manual management of passwords and secrets of the application.
• Tiller approach: we wanted to control the use of manifests.
• The git-crypt approach: the secrets on the disk are not encrypted.
• Conveyor templates directly to kubectl.
• Pass command line options.

And then two things happened :

1. We discovered Dave Cunningham's jsonnet ( Dave Cunningham ) for yaml / json templating in an object-oriented language.
2. Gustavo Buriola showed us the reclass , and without him we would not have gone far.

Ricardo Amaro took to work, and soon the whole team sat down at Kapitan - some worked on the basic functionality, others worked on its use in our internal projects. Manage secrets, support gpg \ kms, custom functions: now Kapitan is a complete product that does more than promised.

Who is Kapitan?

Kapitan is trying to solve all (well, or almost all) problems that I mentioned.

From a technical point of view, Kapitan is very simple:

• Inventory : a hierarchical collection of values ​​describing the deployment, based on yaml. Based on reclass. Like Hiera.
• Template engines : now it's Jinja2, Jsonnet, Kadet. They take inventory and create files (yaml, json, documentation or bash scripts).
• Secrets : template secrets, and Kapitan will take care of them.

We use jsonnet for template manifests and Jinja for everything else.

Sometimes people complain that the jsonnet file is not at all like the same yaml, so it’s difficult for them to switch to jsonnet.



We tried to solve this problem with Kadet by wrapping yaml in Python. Take as a basis your favorite yaml and add Python to it.



Consider it as a Python exoskeleton for yaml! Somehow talk about it.

In the Kapitan workflow , the character immediately shows:

• Freedom of choice : we do not impose any working processes and technologies, but usually we work according to the principles described below. In general, Kapitan can be used as you like. You do not have to use git, do not have to compile files in it, and you can even do without jsonnet! Do what you want.
• GitOps to the bone: all in git, all in the master branch, which reflects our intent.
• Declarativity : Kapitan welcomes the compilation of manifest templates in concrete representations. And you compile your scripts.
• Controlled context : we use compiled scripts to simplify our work, for example, when we set contexts and configure clusters.
  
  Kubernetes configuration: compiled/target_A/setup/setup.sh
  
  Applying changes: compiled/target_A/setup/apply.sh
• Idempotency : Kapitan allows you to change patterns and inventory for code refactoring. Compiled manifests and code will not change without your command, so you have nothing to fear when refactoring.
• Cause and effect : we are for the workflow, where changes in inventory or templates and compiled files are included in one merge request. So the reviewer will be able to evaluate your changes and their actual consequences. It is useful to know whether the change in the template will affect one, two or more targets.
• And finally : Kapitan is not tied to Kubernetes. It simply creates files. Deploy changes kubectl . We only give a shell for the commands so that they are executed consistently.

Do I need it?

Let's be clear : you probably Kapitan (still) is not needed.

But it all depends on what you are trying to do and how complicated your system is.

Kapitan is a powerful tool that requires investment. Use it in complex scenarios where you have to deploy a bunch of applications in a bunch of clusters.

If you have standard applications, you are just learning Kubernetes or are already satisfied with your workflow, then Helm or its current alternative will do.

I imagine Helm as apt-get for Kubernetes , and Kapitan is something like Puppet .

In the next post I will give specific examples and describe the inventory in detail. Write about what you want to know or with which you agree / disagree in this post.

Thanks to Jacek Gruzewski .