Build docker containers with docker containers

From the moment we started working with Docker, we encountered a lot of problems. One of them is the organization of the assembly of applications and their packaging in a container. We solved this problem by introducing the concept of an assembly container. About what it is, why it is necessary and how we came to this and will be discussed in this post.

What is a builder container?

The assembly container is a specially assembled docker image, the task of which is to perform some work on the source code of the program, for example, to install dependencies, compile sources, static code analysis, etc. The definition is quite extensive, so let's look at an example.
')
We have a project . To build and run it you need:

1. install all necessary npm packages
2. using bower install all client dependencies
3. then run grunt to bring us all this

We use npm-builder for this - an assembly container with nodejs, npm, g ++ and make inside, for clarity, we will perform all the steps separately:

$ docker run --rm -v `pwd`:/data -w /data leanlabs/npm-builder npm install $ docker run --rm -v `pwd`:/data -w /data leanlabs/npm-builder bower install --allow-root $ docker run --rm -v `pwd`:/data -w /data leanlabs/npm-builder grunt build 

So, in a few teams we got the assembled project.

Let us examine in steps what happens. We run the docker container npm-builder in the root directory of the project and mount the current directory in / data (the root directory of the project with all sources), we specify / data as the working directory of the container so that all commands in the container are executed in it.
The first command, npm install, will install all nodejs modules listed in package.json, including bower, into the node_modules directory.
With the next command, bower install --allow-root, we install into the bower_components directory all the dependencies listed in bower.json.
And the last team we run build the project grunt. Additionally, when launching containers, specify the --rm option so that the container is deleted after stopping.

Using the example of a project in php, we can use the composer assembly container to install dependencies:

 $ docker run --rm -v `pwd`:/data imega/composer:1.1.0 install --no-dev 

Why all this?

The threshold of entry into the project. Using assembly containers eliminates the need to install a variety of software on the developer's machine. All you need to start working on a project is to clone the repository of the project, install docker, docker-compose and run “docker-compose up -d”, all the containers necessary for building and running the project will be pulled from the docker hub. You no longer need to install on the developer’s machine the various software necessary for building and running.

Assembly time If all instructions for the assembly are specified in the Dockerfile, then the container assembly time begins to increase in proportion to the number of assembly steps.

Uniformity of the executable environment. Assembling the application and launching it is now independent things, packaging in a container is just COPY of the assembled application. So, regardless of the environment, the containers in which the applications are executed are always the same, for example, any application on php is executed in a container with php. So, if something suddenly breaks, then it breaks everywhere and equally.

Reuse assembly steps. If the assembly is described in the Dockerfile, then all the assembly steps will have to be re-described each time, it is solved using the ONBUILD instruction, but in this case we lose a certain level of flexibility, for example, if additional assembly steps are needed, they will still have to be described separately, you can also partially lose the transparency assemblies, in order to understand what will happen during assembly, you need to look at the source of the parent container.

Predictable build. When we started using docker, then, in the beginning, we performed some of the project build steps on the host machine, for example, running grunt. Do not do this;)

Uniformity of the build process so that you don’t have to deal with the details of the Dockerfile, so that the docker acts primarily as a means of packaging, delivering and launching applications, and not as a means of configuration management. By highlighting the build process from the Dockerfile, we were able to achieve the fact that most Dockerfiles look like this:

 FROM leanlabs/nginx:1.0.1 COPY . /var/www/client/ COPY ./build/sites-enabled/client.conf /etc/nginx/sites-enabled/client.conf CMD ["nginx", "-g", "daemon off;"] 

Use existing tools. In the ecosystem of the docker, there are already a lot of tools, there are not enough useful and not very good ones. For example, docker-compose, assembly containers are suitable here.

Convention when using assembly containers

In the process of applying the idea of ​​assembly containers, we have identified several agreements for the development and use.

Common “interface” . From the total, we have allocated volumes. In assembly containers there are volume / data - for mounting the application sources and / cache - for mounting the directory with the assembly container caches, these can be, for example, third-party dependencies (composer, npm, bower cache). For assembly containers, we use one basic image , which serves as an interface.

The assembly container is a dependency external to the application. For example, we do not use separate build containers with phpunit, because phpunit connects via composer and both its version and its presence is determined directly by the application itself.

Startup Method According to the launch method, we divide the container into:

• One-shot containers, run once, execute the command and complete their work. It is convenient to use to install dependencies, clone sources , generate documentation for api .
• Long running containers, long-lived, track changes in the source and rebuild the project - rebuilding SCSS with style changes, building JS, running tests with changes in source, running a static analyzer, etc.

Naming containers. We add the suffix “-builder” to the name of the building container, for example, npm-builder, erlnag-builder, etc.

Use with docker-compose

Assembly containers in conjunction with docker-compose are conveniently used in the developer’s environment, for example, when you need to perform some actions when changing source codes.

With docker-compose, everything is simple, but there are nuances. It is possible to use one shot containers only if docker-compose up is executed with the option “-d”, otherwise upon completion of the work of such a container, compose stops all other containers.

Here is the installation of dependencies php application using composer:

 phpbuilder: image: imega/composer:1.1.0 volumes: - "./:/data" - "$HOME/.composer:/cache" command: ["update"] 

Long running containers work without problems. This is how the launch of the nodejs installation, dependency bower and grunt launch starts:

 clientbuilder: image: leanlabs/npm-builder volumes: - "./:/data" - "$HOME/.node_cache:/cache" command: ["/bin/sh", "-c", "npm install && bower install --allow-root && grunt"] 

Use with make

Make in conjunction with the docker and assembly containers we use to build releases, in places we replace them with docker-compose.

Here is an example makefile that uses the build container approach:

 IMAGE = leanlabs/client TAG = 1.1.9 build: @docker run --rm -v $(CURDIR):/data -v $$HOME/node_cache:/cache leanlabs/npm-builder npm install @docker run --rm -v $(CURDIR):/data -v $$HOME/node_cache:/cache leanlabs/npm-builder bower install --allow-root @docker run --rm -v $(CURDIR):/data -v $$HOME/node_cache:/cache leanlabs/npm-builder grunt build release: @docker build -t $(IMAGE) . @docker tag $(IMAGE):latest $(IMAGE):$(TAG) @docker push $(IMAGE):latest @docker push $(IMAGE):$(TAG) .PHONY: build release 

Total

As a result, we got a fairly lightweight, reproducible in any environment in which the docker is installed, the process of assembling and packaging applications in containers, which can be used with existing tools. Having spent time on creating a high-quality assembly container, we can reuse it at any tasks that fit its area of ​​responsibility.

That's all. Thanks for attention.

I thank my colleague, cnam812 , for the help in writing the article.