Creating packages for Kubernetes with Helm: chart structure and templating

We talked about Helm and working with him “in general” in the last article . Now we come to the practice from the other side - from the point of view of the creator of the charts (ie, packages for Helm). And although this article came from the world of exploitation, it turned out more similar to materials about programming languages - such is the fate of the authors of the charts. So, the chart is a set of files ...

Chart files can be divided into two groups:
')

Files needed to generate Kubernetes resource manifests. These include templates from the templates directory and files with values (default values are stored in values.yaml ). This group also includes the requirements.yaml file and the charts directory - all this is used to organize nested charts.
Accompanying files containing information that can be useful when searching for charts, getting to know them and using them. Most of the files in this group are optional.

Details about the files of both groups:

Chart.yaml - file with information about the chart;
LICENSE - optional text file with a license of the chart;
README.md - optional documentation file;
requirements.yaml - optional file with a list of charts-dependencies;
values.yaml - file with default values for templates;
charts/ - optional directory with nested charts;
templates/ - directory with Kubernetes resource manifest templates;
templates/NOTES.txt is an optional text file with a note that is displayed to the user during installation and updating.

To better understand the contents of these files, you can refer to the official guide of the chart developer or look for relevant examples in the official repository .

Creating a chart in the long run boils down to organizing a well-formed set of files. And the main difficulty in this “design” is the use of a sufficiently advanced template system to achieve the desired result. For rendering Kubernetes resource manifests, the standard Go template template is extended with Helm functions .

Reminder : Helm developers announced that in the next major version of the project - Helm 3 - there will be support for Lua scripts that can be used simultaneously with Go-templates. I will not dwell on this point in more detail - you can read about this (and other changes in Helm 3) here .

For example, this is how the Deployment 's Kubernetes-manifest template in Helm 2 looks like a WordPress blog from the previous article :

deployment.yaml

 apiVersion: extensions/v1beta1 kind: Deployment metadata: name: {{ template "fullname" . }} labels: app: {{ template "fullname" . }} chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" spec: replicas: {{ .Values.replicaCount }} template: metadata: labels: app: {{ template "fullname" . }} chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" release: "{{ .Release.Name }}" spec: {{- if .Values.image.pullSecrets }} imagePullSecrets: {{- range .Values.image.pullSecrets }} - name: {{ . }} {{- end}} {{- end }} containers: - name: {{ template "fullname" . }} image: "{{ .Values.image.registry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}" imagePullPolicy: {{ .Values.image.pullPolicy | quote }} env: - name: ALLOW_EMPTY_PASSWORD {{- if .Values.allowEmptyPassword }} value: "yes" {{- else }} value: "no" {{- end }} - name: MARIADB_HOST {{- if .Values.mariadb.enabled }} value: {{ template "mariadb.fullname" . }} {{- else }} value: {{ .Values.externalDatabase.host | quote }} {{- end }} - name: MARIADB_PORT_NUMBER {{- if .Values.mariadb.enabled }} value: "3306" {{- else }} value: {{ .Values.externalDatabase.port | quote }} {{- end }} - name: WORDPRESS_DATABASE_NAME {{- if .Values.mariadb.enabled }} value: {{ .Values.mariadb.db.name | quote }} {{- else }} value: {{ .Values.externalDatabase.database | quote }} {{- end }} - name: WORDPRESS_DATABASE_USER {{- if .Values.mariadb.enabled }} value: {{ .Values.mariadb.db.user | quote }} {{- else }} value: {{ .Values.externalDatabase.user | quote }} {{- end }} - name: WORDPRESS_DATABASE_PASSWORD valueFrom: secretKeyRef: {{- if .Values.mariadb.enabled }} name: {{ template "mariadb.fullname" . }} key: mariadb-password {{- else }} name: {{ printf "%s-%s" .Release.Name "externaldb" }} key: db-password {{- end }} - name: WORDPRESS_USERNAME value: {{ .Values.wordpressUsername | quote }} - name: WORDPRESS_PASSWORD valueFrom: secretKeyRef: name: {{ template "fullname" . }} key: wordpress-password - name: WORDPRESS_EMAIL value: {{ .Values.wordpressEmail | quote }} - name: WORDPRESS_FIRST_NAME value: {{ .Values.wordpressFirstName | quote }} - name: WORDPRESS_LAST_NAME value: {{ .Values.wordpressLastName | quote }} - name: WORDPRESS_BLOG_NAME value: {{ .Values.wordpressBlogName | quote }} - name: WORDPRESS_TABLE_PREFIX value: {{ .Values.wordpressTablePrefix | quote }} - name: SMTP_HOST value: {{ .Values.smtpHost | quote }} - name: SMTP_PORT value: {{ .Values.smtpPort | quote }} - name: SMTP_USER value: {{ .Values.smtpUser | quote }} - name: SMTP_PASSWORD valueFrom: secretKeyRef: name: {{ template "fullname" . }} key: smtp-password - name: SMTP_USERNAME value: {{ .Values.smtpUsername | quote }} - name: SMTP_PROTOCOL value: {{ .Values.smtpProtocol | quote }} ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: httpGet: path: /wp-login.php {{- if not .Values.healthcheckHttps }} port: http {{- else }} port: https scheme: HTTPS {{- end }} {{ toYaml .Values.livenessProbe | indent 10 }} readinessProbe: httpGet: path: /wp-login.php {{- if not .Values.healthcheckHttps }} port: http {{- else }} port: https scheme: HTTPS {{- end }} {{ toYaml .Values.readinessProbe | indent 10 }} volumeMounts: - mountPath: /bitnami/apache name: wordpress-data subPath: apache - mountPath: /bitnami/wordpress name: wordpress-data subPath: wordpress - mountPath: /bitnami/php name: wordpress-data subPath: php resources: {{ toYaml .Values.resources | indent 10 }} volumes: - name: wordpress-data {{- if .Values.persistence.enabled }} persistentVolumeClaim: claimName: {{ .Values.persistence.existingClaim | default (include "fullname" .) }} {{- else }} emptyDir: {} {{ end }} {{- if .Values.nodeSelector }} nodeSelector: {{ toYaml .Values.nodeSelector | indent 8 }} {{- end -}} {{- with .Values.affinity }} affinity: {{ toYaml . | indent 8 }} {{- end }} {{- with .Values.tolerations }} tolerations: {{ toYaml . | indent 8 }} {{- end }}

Now - about the basic principles and features of templating in Helm. Most of the examples below are taken from the official repository charts.

Templates

Templates: `{{ }}`

Everything related to templating turns into double curly braces. The text outside the curly brackets when rendering remains unchanged.

Context value:.

When rendering a file or a partial (for more details on reusing templates, see the next sections of the article) , the value is accessed, which becomes available internally through the context variable - the period. When passed as a structure argument, a point is used to access the fields and methods of this structure.

The value of a variable changes during the rendering process depending on the context in which it is used. Most block statements override the context variable inside the main block. The main operators and their features will be discussed below, after becoming acquainted with the basic structure of Helm.

Helm basic structure

When rendering manifests into templates, a structure is thrown with the following fields:

Field .Values - to access the parameters that are defined during installation and update release. These include the values of the --set , --set-string and --set-file options, as well as the parameters of the files with values, the values.yaml file and files corresponding to the values of the --values options:
```
 containers: - name: main image: "{{ .Values.image }}:{{ .Values.imageTag }}" imagePullPolicy: {{ .Values.imagePullPolicy }} 
```
.Release - to use release data about rollout, installation or update, release name, namespace and values of several more fields that can be useful when generating manifests:
```
 metadata: labels: heritage: "{{ .Release.Service }}" release: "{{ .Release.Name }}" subjects: - namespace: {{ .Release.Namespace }} 
```
.Chart - to access information about the chart . The fields correspond to the contents of the Chart.yaml file:
```
 labels: chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" 
```

The .Files structure is for working with files stored in the chart directory ; The structure and available methods can be found at the link . Examples:

 data: openssl.conf: | {{ .Files.Get "config/openssl.conf" | indent 4 }}

 data: {{ (.Files.Glob "files/docker-entrypoint-initdb.d/*").AsConfig | indent 2 }}

.Capabilities - to access information about the cluster in which .Capabilities is performed:

 {{- if .Capabilities.APIVersions.Has "apps/v1beta2" }} apiVersion: apps/v1beta2 {{- else }} apiVersion: extensions/v1beta1 {{- end }}

 {{- if semverCompare "^1.9-0" .Capabilities.KubeVersion.GitVersion }} apiVersion: apps/v1 {{- else }}

Operators

We begin, of course, with the if , else if and else :

 {{- if .Values.agent.image.tag }} image: "{{ .Values.agent.image.repository }}:{{ .Values.agent.image.tag }}" {{- else }} image: "{{ .Values.agent.image.repository }}:v{{ .Chart.AppVersion }}" {{- end }}

The range operator is designed to work with arrays and maps. If an array is passed as an argument and it contains elements, then a block is sequentially executed for each element (and the value inside the block becomes available through the context variable):

 {{- range .Values.ports }} - name: {{ .name }} port: {{ .containerPort }} targetPort: {{ .containerPort}} {{- else }} ... {{- end}}

 {{ range .Values.tolerations -}} - {{ toYaml . | indent 8 | trim }} {{ end }}

Syntax with variables is provided for working with maps:

 {{- range $key, $value := .Values.credentials.secretContents }} {{ $key }}: {{ $value | b64enc | quote }} {{- end }}

Similar behavior is with the with : operator. If the passed argument exists, then the block is executed, and the context variable in the block corresponds to the value of the argument. For example:

 {{- with .config }} config: {{- with .region }} region: {{ . }} {{- end }} {{- with .s3ForcePathStyle }} s3ForcePathStyle: {{ . }} {{- end }} {{- with .s3Url }} s3Url: {{ . }} {{- end }} {{- with .kmsKeyId }} kmsKeyId: {{ . }} {{- end }} {{- end }}

To reuse templates, a link from define [name] and template [name] [variable] can be used, where the passed value is available via the context variable in the define block:

 apiVersion: v1 kind: ServiceAccount metadata: name: {{ template "kiam.serviceAccountName.agent" . }} ... {{- define "kiam.serviceAccountName.agent" -}} {{- if .Values.serviceAccounts.agent.create -}} {{ default (include "kiam.agent.fullname" .) .Values.serviceAccounts.agent.name }} {{- else -}} {{ default "default" .Values.serviceAccounts.agent.name }} {{- end -}} {{- end -}}

A couple of features to consider when using define , or, more simply, partial'ov:

The declared partials are global and can be used in all files in the templates directory.
The main chart is compiled together with the dependent charts, so if two partial partial of the same name exist, the last loaded one will be used. When naming a partial, it is customary to add a chart name to avoid such conflicts: define "chart_name.partial_name" .

Variables: `$`

In addition to working with the context, you can store, change and reuse data using variables:

 {{ $provider := .Values.configuration.backupStorageProvider.name }} ... {{ if eq $provider "azure" }} envFrom: - secretRef: name: {{ template "ark.secretName" . }} {{ end }}

When rendering a file or partial, $ has the same meaning as the period. But unlike the context variable (point), the value of $ does not change in the context of block operators , which allows you to simultaneously work with the context value of the block operator and the basic structure Helm (or the value passed in partial, if we talk about using $ inside a partial) . Illustration of the difference:

 context: {{ . }} dollar: {{ $ }} with: {{- with .Chart }} context: {{ . }} dollar: {{ $ }} {{- end }} template: {{- template "flant" .Chart -}} {{ define "flant" }} context: {{ . }} dollar: {{ $ }} with: {{- with .Name }} context: {{ . }} dollar: {{ $ }} {{- end }} {{- end -}}

As a result of processing this pattern, the following will be obtained (for clarity, in the output of the structure, they are replaced by the corresponding pseudo names):

 context: #  helm dollar: #  helm with: context: #.Chart dollar: #  helm template: context: #.Chart dollar: #.Chart with: context: habr dollar: #.Chart

And here is a real example of using this feature:

 {{- if .Values.ingress.enabled -}} {{- range .Values.ingress.hosts }} apiVersion: extensions/v1beta1 kind: Ingress metadata: name: {{ template "nats.fullname" $ }}-monitoring labels: app: "{{ template "nats.name" $ }}" chart: "{{ template "nats.chart" $ }}" release: {{ $.Release.Name | quote }} heritage: {{ $.Release.Service | quote }} annotations: {{- if .tls }} ingress.kubernetes.io/secure-backends: "true" {{- end }} {{- range $key, $value := .annotations }} {{ $key }}: {{ $value | quote }} {{- end }} spec: rules: - host: {{ .name }} http: paths: - path: {{ default "/" .path }} backend: serviceName: {{ template "nats.fullname" $ }}-monitoring servicePort: monitoring {{- if .tls }} tls: - hosts: - {{ .name }} secretName: {{ .tlsSecret }} {{- end }} --- {{- end }} {{- end }}

Indentation

When developing templates, extra spaces can be left: spaces, tabs, line breaks. With them, the file simply looks more readable. You can either opt out of them, or use a special syntax to remove indents around the templates used:

{{- variable }} truncates leading spaces;
{{ variable -}} truncates subsequent spaces;
{{- variable -}} - both options.

An example of a file whose processing will result in the string habr flant helm :

 habr {{- " flant " -}} helm

Built-in functions

All the features built into the template can be found at the following link . Here I will tell only about some of them.

The index function is used to access elements of an array or maps:

 definitions.json: | { "users": [ { "name": "{{ index .Values "rabbitmq-ha" "rabbitmqUsername" }}", "password": "{{ index .Values "rabbitmq-ha" "rabbitmqPassword" }}", "tags": "administrator" } ] }

The function takes an arbitrary number of arguments, which allows you to work with nested elements:

$map["key1"]["key2"]["key3"] => index $map "key1" "key2" "key3"

For example:

 httpGet: {{- if (index .Values "pushgateway" "extraArgs" "web.route-prefix") }} path: /{{ index .Values "pushgateway" "extraArgs" "web.route-prefix" }}/#/status {{- end }}

Boolean operations are implemented in the template engine as functions (and not as operators). All arguments for them are calculated during transmission:

 {{ if and (index .Values field) (eq (len .Values.field) 10) }} ... {{ end }}

In the absence of a field rendering of the template will end with an error ( error calling len: len of untyped nil ): the second condition is checked, even though the first one has not been fulfilled. It is worth taking note of this, and it is up to such requests to be resolved by splitting into several checks:

 {{ if index . field }} {{ if eq (len .field) 10 }} ... {{ end }} {{ end }}

Pipeline is a unique feature of Go templates that allows you to declare expressions that run like a pipeline in a shell. Formally, the pipeline is a chain of commands, separated by the symbol | . A command can be a simple value or a function call . The result of each command is passed as the last argument to the next command , and the result of the final command in the pipeline is the value of the entire pipeline. Examples:

 data: openssl.conf: | {{ .Files.Get "config/openssl.conf" | indent 4 }}

 data: db-password: {{ .Values.externalDatabase.password | b64enc | quote }}

Additional functions

Sprig is a library of 70 useful functions for solving a wide range of tasks. For security reasons, Helm excluded the functions env and expandenv , which would provide access to Tiller environment variables.

The include function, like the standard template function, is used to reuse templates. Unlike the template , the function can be used in the pipeline, i.e. transfer the result to another function:

 metadata: labels: {{ include "labels.standard" . | indent 4 }} {{- define "labels.standard" -}} app: {{ include "hlf-couchdb.name" . }} heritage: {{ .Release.Service | quote }} release: {{ .Release.Name | quote }} chart: {{ include "hlf-couchdb.chart" . }} {{- end -}}

The required function allows developers to declare the required values required for rendering a template: if a value exists, when a template is rendered, it is used, otherwise the rendering is completed with an error message specified by the developer:

 sftp-user: {{ required "Please specify the SFTP user name at .Values.sftp.user" .Values.sftp.user | b64enc | quote }} sftp-password: {{ required "Please specify the SFTP user password at .Values.sftp.password" .Values.sftp.password | b64enc | quote }} {{- end }} {{- if .Values.svn.enabled }} svn-user: {{ required "Please specify the SVN user name at .Values.svn.user" .Values.svn.user | b64enc | quote }} svn-password: {{ required "Please specify the SVN user password at .Values.svn.password" .Values.svn.password | b64enc | quote }} {{- end }} {{- if .Values.webdav.enabled }} webdav-user: {{ required "Please specify the WebDAV user name at .Values.webdav.user" .Values.webdav.user | b64enc | quote }} webdav-password: {{ required "Please specify the WebDAV user password at .Values.webdav.password" .Values.webdav.password | b64enc | quote }} {{- end }}

The tpl function allows you to render a string as a template. Unlike template and include , the function allows you to execute templates that are passed in variables, as well as render templates that are stored not only in the templates directory. What does this look like?

Execution of templates from variables:

 containers: {{- with .Values.keycloak.extraContainers }} {{ tpl . $ | indent 2 }} {{- end }}

... and in values.yaml we have the following value:

 keycloak: extraContainers: | - name: cloudsql-proxy image: gcr.io/cloudsql-docker/gce-proxy:1.11 command: - /cloud_sql_proxy args: - -instances={{ .Values.cloudsql.project }}:{{ .Values.cloudsql.region }}:{{ .Values.cloudsql.instance }}=tcp:5432 - -credential_file=/secrets/cloudsql/credentials.json volumeMounts: - name: cloudsql-creds mountPath: /secrets/cloudsql readOnly: true

Render of a file stored outside the templates directory:

 apiVersion: batch/v1 kind: Job metadata: name: {{ template "mysqldump.fullname" . }} labels: app: {{ template "mysqldump.name" . }} chart: {{ template "mysqldump.chart" . }} release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" spec: backoffLimit: 1 template: {{ $file := .Files.Get "files/job.tpl" }} {{ tpl $file . | indent 4 }}

... in the chart, along the path files/job.tpl , there is the following template:

 spec: containers: - name: xtrabackup image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" imagePullPolicy: {{ .Values.image.pullPolicy | quote }} command: ["/bin/bash", "/scripts/backup.sh"] envFrom: - configMapRef: name: "{{ template "mysqldump.fullname" . }}" - secretRef: name: "{{ template "mysqldump.fullname" . }}" volumeMounts: - name: backups mountPath: /backup - name: xtrabackup-script mountPath: /scripts restartPolicy: Never volumes: - name: backups {{- if .Values.persistentVolumeClaim }} persistentVolumeClaim: claimName: {{ .Values.persistentVolumeClaim }} {{- else -}} {{- if .Values.persistence.enabled }} persistentVolumeClaim: claimName: {{ template "mysqldump.fullname" . }} {{- else }} emptyDir: {} {{- end }} {{- end }} - name: xtrabackup-script configMap: name: {{ template "mysqldump.fullname" . }}-script

On this familiarity with the basics of template making in Helm came to an end ...

Conclusion

The article describes the structure of the Helm-charts and details the main difficulty in creating them: templating: the basic principles, syntax, functions and operators of the Go-templating engine, additional functions.

How to start with all this work? Since Helm is already a whole ecosystem, you can always look at examples of charts of similar packages. For example, if you want to pack a new message queue, take a look at the RabbitMQ public chart . Of course, no one promises you perfect implementations in existing packages, but they are perfect as a starting point. The rest comes with practice, in which the helm template and helm lint debugging commands will help you, as well as launching the installation with the --dry-run option.

For a more comprehensive understanding of the development of the Helm charts, best practices and the technologies used, I suggest reading the following links (all in English):

Charts developer guide ;
Best practices ;
Go-template documentation ;
Sprig library documentation ;
The official charts repository (as a knowledge base and best practices implemented).

And at the end of the next material about Helm I attach a survey that will help you better understand what other articles about Helm are waiting for (or not waiting for?) Habr's readers. Thanks for attention!