I want to tell the story of how the application was launched in Openshift. Also in the course of the play we will consider utilities for managing the application inside Openshift. This is a transcript of a performance on kubernetes SPB meetup # 3 ..

purpose

Typically, customers deploy on separate servers, but then came the task, to test the possibility of running openshift and picking rake.

First you need to talk about our application. A project with a rich history. Used in large organizations and probably each of you indirectly intersected. The application supports multiple databases, integrations, etc. etc.

Prerequisites

The application should work in completely different environments. As a result, our installation documentation is quite extensive. But if you look down, then nothing complicated:

• Apply DB schema.
• Configure the application server.
• Install the license.
• Configure the application and integrate with external systems.

But the world is cruel, we had a number of restrictions:

• The application can be collected only on Jenkins specifically, who deals with the signing. And only there.
• There is no access from the client Openshift to the development environment.
• For a number of ideological reasons, it was not possible to reuse existing Docker images for development.
• We have ansible playbooks for installing and configuring the application on the servers.

Ansible container demo

Ansible container is open source software that aims to automate container assembly, deployment, and process control. As you might guess from the name. Ansible is used to build containers. We already had written Ansible roles for installing and deploying applications on top of servers, so we decided not to reinvent the wheel and reuse them. Not that this is an ideal tool, but a quick re-use of existing roles has proven to be a decisive factor for the demo.

By and large, in order to make a demo we took the existing roles that set up everything and everyone, and made a "monolithic container". What to collect the container there were no special problems, because Openshift has some great recommendations , but I ’ll note separately:

• Roles had to be finalized, because we use systemd.
• By default, for security reasons, openshift is not allowed to use some syscall. As a result, there will be nuances with chroot, sudo. Hi CVE-2019-5736 .
• Similarly, for security reasons, the container is launched from under a user with a random ID, this is also custom behavior.

The main idea at this point is that we made the demo sooo fast.

Multiple containers demo

The demo container fulfilled its role and we sawed it into separate components:

1. Our application.
2. Database.
3. External services, etc ...

The first thing that faced, how to initialize the database? It is clear that we use migrations, but when and how to apply them? There is a link to a wonderful article describing the device POD: PODs life . By and large, there are several approaches:

• Use init-container
• Use orchestration systems that determine the order in which services are deployed and migrate when needed.

We decided to take the Init-container path. Those. in the POD of our application, before the start of our application, the container that rolls the migration starts. But how to configure the application itself and external integrations?

Initialize the application

As I already mentioned, our application can and should work perfectly in different environments, with different databases and integrations. Again, the question is how to set it all up?

• Use orchestration systems that will determine the order of service deployment and apply the configuration after the application starts.
• Pass through the environment variables to the container as configured.
• Use start hook.
• Make a separate container that contains the configuration and applies it to the application. Roughly analog migration for a DB.

We chose the latter approach, because It allows you to make the configuration reproducible and self-sufficient. Only for some reason, this container was originally made in a separate replication controller with a factor of 1.

Ok, read the documentation again.

A pod (as in a pod of whales or pea pod) is a group of containers (such as Docker containers),

POD is a group of containers. As a result, our sub consisted of 3 containers.

1. Init container to initialize a PostgreSQL.
2. Container with an app.
3. Container configuration application.

Tools

We have a scheme of how the application is deployed. Now it’s time to discuss the tools in nature; there are a lot of things already done, I’ll look at some of the list below and make subjective conclusions.

• Openshift templates
• bash / python / ruby ​​+ yml templates
• Terraform k8s provider
• Ansible container
• Ansible + k8s module
• Ansible Playbook Bundle
• Ansible bender
• operator
• source2image
• kustomize
• helm
• Automation broker

Openshift templates

Openshift templates

Pros:

• Natively and they have excellent documentation.

Minuses:

• Another template engine.
• Long and awful YAML files.
• If you have dependencies between services and their order of start, it will be difficult.

Scripts and template

Pros:

• You can use great tools and all the power of OOP.

Minuses:

• Crutches that support. And not only you.

Terraform k8s provider

Terraform k8s provider

Pros:

• You are not worried about the order of creation of infrastructure elements.
• You can reuse the infrastructure code as modules.
• You can add application initialization logic.

Minuses:

• No support for Openshift, only k8s.
• Sometimes outdated dock and modules.
• Another tula in your team.

Ansible container

Ansible container

Pros:

• Make CM, no bash
• You can reuse the code as roles.
• In our case, one tool for everything.

Minuses:

• Huge images, because go one layer.
• Looks abandoned and not supported. Ansible bender has been replaced.

Ansible k8s module

Ansible + k8s module

Pros:

• One playbook to describe all project infrastructures inside Openshift.
• Reuse code in the form of roles.
• You can add application initialization logic.

Minuses:

• No proxy support.
• You care about the removal. If the object is no longer needed, it is necessary to describe its removal.
• You yourself describe the sequence of creation of infrastructure elements.

Ansible Playbook Bundle

The Ansible Playbook Bundle (APB) utility offers an approach: let's package the ansible roles for deploying the application inside k8s / openshift into a container and run inside k8s / openshift.

Pros:

• All my carry with me.
• Testable and reproducible.
• Integration with the Service catalog (friendly web interface for launching applications).

Minuses:

• You need admin level privileges.
• Documentation sometimes leaves much to be desired.

Result

I do not want to be the last instance, but I will share my conclusions:

• If you do not plan to provide the application as a service, then Ansible k8s module is your choice.
• But if you need it, you need to dig about automation broker and Ansible Playbook Bundle .

PS

• Slides (EN)
• It is crosspost
• English Version