Officially present dapp - DevOps utility to maintain CI / CD

Readers of this blog, as well as visitors of the latest HighLoad ++ and RIT ++, have most likely already heard about our utility for daOps DevOps engineers, but now we have decided to officially and finally present it to the “big world”. The formal right to do that gives us the fact that we have been working with dapp to solve problems in production for more than a year, so we believe that the technology is ripe for more mass use.



So, dapp is a Ruby tool created by the Flant company as an Open Source project for implementing and maintaining CI / CD processes . What does he allow?
')

Build Images

When we talked about dapp for the first time ( post , video ), it was about using this utility to build images of Docker containers. I will not retell the entire report, but I will list the main features of dapp, which (using the image configurations described in the Dappfile ) allow you to assemble images quickly and efficiently:

1. four stages of image build ( before_install , install , before_setup , setup ), the results of which are cached (gives a significant increase in the speed of image re-assemblies);
2. "External context" for mounting directories used during builds, but excluded from the final image;
3. thoughtful work with Git, which adds only deltas to the image ( git patch apply ) for new builds and caches the contents of these patches;
4. “Artifacts” - the possibility of using third-party tools at the project assembly stage, but not including them in the final image; for artifacts, the cache is also supported (with the recent release of Docker 17.06, this feature has ceased to be so significant due to the appearance of multi-stage builds) ;
5. Chef support, which allows you to customize systems in Docker-images according to recipes (cookbooks).

We planned to add the latter with the support of Ansible, and the relevance of this issue is only increasing for us, but the actual implementation is still behind the intentions.

Warmth in Kubernetes

But we have advanced in another important direction - expanding the capabilities of dapp beyond the framework of image assembly, in the area of ​​other components of the processes of continuous integration and application delivery. The initial support for deployment in dapp marked the possibility of integrating with the Kubernetes system that we use to orchestrate containers (we’ve read more recently: “ Our experience with Kubernetes in small projects (review and video report) ”) . The essence of this feature is as follows:

1. For Kubernetes, YAML configurations are prepared, which can be different for each desired contour (production, staging, testing ...) and which are placed in a special directory in the Git repository with the application and infrastructure code (for example, the files backend.yaml , frontend.yaml , cron.yaml in a special directory .kube/ ).
2. The created configurations are given to the kubectl utility, which deploys the infrastructure described in them in the required Kubernetes cluster (again, the clusters can be different for each circuit).
3. Created Docker images are deployed in the Kubernetes infrastructure.

Technically, this is implemented using the dapp kube deploy ( see the documentation ), which is essentially a wrapper to the Kubernetes- Helm package manager, allowing:

1. supplement and expand the ability to transfer parameters to Helm (adds secret values ​​and files, has a helper to read these files in templates);
2. integrate the images collected by the Dappfile into Helm (has a helper for composing the image name).

Note : We do the warmth itself through GitLab CI, as described in a recent article “GitLab CI for continuous integration and delivery in production”: “ Part 1: our pipeline ”, “ Part 2: overcoming difficulties ”.

Additional Information

Right now, our colleague is writing an article acquaintance with dapp ( updated : read “ Practice with dapp. Part 1: Building Simple Applications ”). In the meantime, I offer links to already available information:

• The mentioned report “We assemble Docker images for CI / CD quickly and conveniently along with dapp (review and video)”: post and video from the speech of the technical director of JSC “Flant” Dmitry Stolyarov, where the first and more “chamber” presentation of dapp took place;
• official technical documentation in Russian - flant.imtqy.com/dapp ;
• special attention in this documentation deserves the instruction " The first application on dapp ";
• Telegram official support group - dapp_ru .

Yes, it is open source!

The dapp source code is written in Ruby and published on GitHub under the free Apache License 2.0 license (it is also used in projects close to us like Moby / Docker, Kubernetes, Helm, etcd) .

We invite DevOps engineers and Open Source enthusiasts interested in using dapp to participate in the project . Explore it, ask questions (you can right here in the comments) , point out problems , suggest improvements . Thanks for attention!