Build and heat applications in Kubernetes using dapp and GitLab CI

In previous articles about dapp, we talked about building applications and running in Minikube. At the same time, dapp was launched locally on the developer's machine. However, the tool was conceived to support continuous integration (CI) processes and we ourselves use it mainly in conjunction with GitLab. How does dapp help with CI / CD processes?

First, of course, this is an assembly. Dapp allows you to speed up incremental builds of applications by linking build commands to changes between commits in the Git repository (for more details on this and other optimizations during the build phase, see the report “Building Docker images for CI / CD quickly and conveniently with dapp”: article video ).
')
Secondly, dapp helps to roll out the application to the Kubernetes cluster using Helm. In addition to some simplification of working with secrets and checking Helm-patterns, dapp helps waiting for the rollouts of the Deployments specified in the patterns.

Thirdly, dapp implements cache clearing logic in both local and remote Registry with Docker images. Cleaning has been improved in recent versions: now dapp deletes images created on Git branches if these branches are removed from the Git repository. With images on tags a little more difficult: there are no more than 10 images no older than 30 days. In the next versions we plan to make custom cleaning policies.

All of the above is more critical for the build server, so I’ll use the simple GitLab + Minikube stand to show an example of how to integrate dapp into continuous integration and delivery (CI / CD).

Stand GitLab + Minikube + dapp

The stand consists of the installed GitLab, GitLab Runner, Registry and Kubernetes cluster:

The scheme is close to the version that we use in real projects. In short, everything works like this:

The developer is pushing his changes to the git repository.
GitLab runs the build task, GitLab Runner runs a dapp that builds the image and push the image into the Registry.
To roll out the application to the cluster, the deployment task starts: GitLab Runner (with access to kubectl and Helm) performs dapp kube deploy .
When receiving updated resources, the Kubernetes cluster checks if there is a new image in the Registry, downloads it and launches new images with the new image.

The stand is a demo, so in order not to produce virtual machines, along with GitLab you need to install Registry, GitLab Runner, dapp, Docker, kubectl and Helm. As a Kubernetes cluster, Minikube is used as the easiest way to run K8s on a local machine.

Symfony-demo is again used as an application. Building this project in a local version was described in the article “ Practice with dapp. Part 1: Build simple applications, ”and an example of rolling out an application in Minikube was described in the article“ Practice with dapp. Part 2. Deploying Docker images in Kubernetes using Helm . ” The difference from the second article is that the Registry for Minikube becomes external (located in a virtual machine with GitLab) and the dapp kube minikube setup not required.

Host preparation

Before creating virtual machines, it is better to add host names in advance, for example, in /etc/hosts :

192.168.33.20 gitlab.example.com # gitlab
192.168.33.20 registry.gitlab.example.com # registry
192.168.33.100 cluster.example.com # api k8s

GitLab in the virtual machine

Installing GitLab using a ready-made Vagrantfile ( https://github.com/rgl/gitlab-vagrant ) was mentioned in the second part of the "Practice with dapp" , but now I will describe in more detail. To start the virtual machine, you will need to edit the Vagrantfile , change the base image to xenial64 and increase the memory from 2048 to 4096.

After vagrant up you need to install GitLab Runner according to the project instructions . Docker and dapp installation is also required. And for deployment - download the kubectl and Helm binaries.

To start Registry, you need to edit /etc/gitlab/gitlab.rb , uncommenting the lines:

 gitlab_rails['registry_enabled'] = true gitlab_rails['registry_host'] = "registry.gitlab.example.com" gitlab_rails['registry_port'] = "5000" gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_api_url'] = "http://localhost:5000" gitlab_rails['registry_key_path'] = "/var/opt/gitlab/gitlab-rails/certificate.key" gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" registry['registry_http_addr'] = "0.0.0.0:5000"

After saving - run gitlab-ctl reconfigure . Omnibus will reconfigure and restart GitLab.

All of the above is already in the fork of gitlab-vagrant - it is enough to clone it and perform vagrant up .

Minikube

Time to start and configure Minikube. The instruction is quite simple: download the project binary and invoke the command:

 minikube start --insecure-registry registry.gitlab.example.com:5000 --host-only-cidr 192.168.33.1/24

Unfortunately, in order to work as part of the booth, you will have to stop the cluster ( minikube stop ) and manually edit ~/.minikube/machines/minikube/config.json , turning on HostDNSResolver so that the virtual machine accepts the host's /etc/hosts . Also, if Minikube was installed after reading the second part , check the InsecureRegistry key.

 { "Driver": { … "HostDNSResolver": true, … “HostOptions”: { "EngineOptions": { "InsecureRegistry": [ "registry.gitlab.example.com:5000" ], …

To check the availability of a cluster after running minikube start , you can run the command kubectl get all .

Configure GitLab Runner

Cluster started - you need to set up a cluster connection for gitlab-runner. To do this, the local config is copied from the .kube and certificates for access to the Kubernetes API to the virtual machine:

 $ cd ~ $ tar zcf kube-config.tar.gz .kube/config .minikube/ca.crt .minikube/apiserver.* $ cp kube-config.tar.gz $GITLAB_VM_DIR $ cd $GITLAB_VM_DIR $ vagrant ssh ubuntu@gitlab:~$ sudo su - gitlab-runner gitlab-runner@gitlab:~$ tar zxf /vagrant/kube-config.tar.gz

The next step is to edit .kube/config so that the path to the files is correct:

certificate-authority: /home/ gitlab-runner /.minikube/ca.crt
client-certificate: /home/ gitlab-runner /.minikube/apiserver.crt
client-key: /home/ gitlab-runner /.minikube/apiserver.key

Then you can check the connection and configure Helm:

 gitlab-runner@gitlab:~$ kubectl get all

If Minikube has already been installed earlier in the second part , then the tiller already has enough command:

 gitlab-runner@gitlab:~$ helm init --client-only

If minikube has not been previously installed, then you need to install tiller:

 gitlab-runner@gitlab:~$ helm init

For dapp, add the plugin template :

 gitlab-runner@gitlab:~$ helm plugin install https://github.com/technosophos/helm-template

Import project and pipeline

You can now import the symfony-demo repository into GitLab. In the list of projects, click New Project , select Import , then Repo by URL , enter the URL, group and name of the new project.

The project already has a branch with Dappfile and templates in .helm , created when working on previous articles - dapp_deploy_minikube .

To demonstrate how CI / CD works, you need to add the CI configuration - .gitlab-ci.yml . This will be a simple pipeline of two tasks:

Build task

 Build: stage: build script: - dapp --version - dapp dimg build --build-dir ~/dapp_build/${CI_PROJECT_NAME} - dapp dimg push --build-dir ~/dapp_build/${CI_PROJECT_NAME} ${CI_REGISTRY_IMAGE} --tag-ci tags: - build

The task starts dapp 3 times. The first time is informative to see the dapp version. The second launch is an assembly of the images described in the Dappfile . The third one is a push of the collected images in the Registry, with the tags being tagged based on the CI_* variables ( documentation for tagging options ).

Deploy job

 Deploy: stage: deploy script: - dapp --version - dapp kube deploy --tag-ci --namespace ${CI_PROJECT_NAME}-stage --set "global.env=stage" --set "global.git_rev=${CI_COMMIT_SHA}" $CI_REGISTRY_IMAGE tags: - build

Here, too, the first launch is the dapp version. The second launch is rolling out the application to the cluster.

You can see that the stage suffix is used in the namespace - this is done to show that you can specify any namespace name. In the full version of the pipeline will need to create several tasks for the desired environments.

Helm templates

The registry installed in GitLab, unlike the Registry installed by the dapp kube minukube setup , has rights permissions and therefore you need to add a registrysecret with the login and password for the Registry to the Helm-templates. You can create a secret manually using the base64 command, or you can use the hint .

As a result, the .helm/values.yaml will be added to .helm/values.yaml :

 imageCredentials: registry: registry.gitlab.example.com:5000 username: root password: password

And in backend.yaml is such a resource:

 apiVersion: v1 kind: Secret type: kubernetes.io/dockercfg metadata: name: registrysecret data: .dockercfg: {{ printf "{\"%s\": {\"auth\": \"%s\"}}" .Values.imageCredentials.registry (printf "%s:%s" .Values.imageCredentials.username .Values.imageCredentials.password | b64enc) | b64enc }}

... and imagePullSecrets in a container template spec:

 spec: template: ... spec: imagePullSecrets: - name: registrysecret containers: - command: [ '/demo/start.sh' ] image: {{ tuple "symfony-demo-app" . | include "dimg" }}

A rollout on such Helm-templates should already be successful - the result can be seen in the browser at http://cluster.example.com/symfony-demo :

Summarizing

In general, we can assume that we have deployed our stand with the CI / CD process, which is very close to what works for our clients. The next steps will be the complication of the pipeline (see “ GitLab CI for continuous integration and delivery in production. Part 1: our pipeline ” and “ Part 2: overcoming difficulties ”), introducing dynamic environments (a review article of a colleague ), adding a registry cleanup on schedule, adding launch integration tests. Questions about the described booth and about dapp can be asked in the comments and in our Telegram-chat .