Docker: Harmless Tips

In the comments to my Docker article : bad advice there were many requests to explain why the Dockerfile described in it was so terrible.

Summary of the previous series : two developers on a hard deadline make up the Dockerfile. In the process Ops Igor Ivanovich comes to them. The final Dockerfile is so bad that the AI ​​is on the verge of a heart attack.

Now let's see what is wrong with this Dockerfile.

So, a week has passed.

Dev Petya meets in the canteen over a cup of coffee with Ops Igor Ivanovich.

P: Igor Ivanovich, are you very busy? I would like to find out where we screwed up.

AI: It's good, you don’t often meet developers who are interested in exploitation.
First, let's agree on some things:

1. Ideology Docker: one container - one process.
2. The smaller the container, the better.
3. The more taken from the cache, the better.

P: Why should there be one process in one container?

AI: Docker, when the container starts, monitors the status of the process with pid 1. If the process dies, the Docker tries to restart the container. Suppose you have several applications running in a container or the main application is not running with pid 1. If the process dies, the Docker will not know about it.

If there are no more questions, show your Dockerfile.

And Peter showed:

FROM ubuntu:latest #    COPY ./ /app WORKDIR /app #    RUN apt-get update #   RUN apt-get upgrade #    RUN apt-get -y install libpq-dev imagemagick gsfonts ruby-full ssh supervisor #  bundler RUN gem install bundler #  nodejs     RUN curl -sL https://deb.nodesource.com/setup_9.x | sudo bash - RUN apt-get install -y nodejs #   RUN bundle install --without development test --path vendor/bundle #     RUN rm -rf /usr/local/bundle/cache/*.gem RUN apt-get clean RUN rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* RUN rake assets:precompile #  ,   ,    . CMD ["/app/init.sh"] 

AI: Oh, let's sort it out in order. Let's start with the first line:

 FROM ubuntu:latest 

You take the latest tag. Using the latest tag leads to unpredictable consequences. Imagine, the image maintainer collects a new version of the image with a different list of software; this image receives the latest tag. And your container stops gathering at best, and at worst you catch bugs that were not there before.

You take an image with a full OS with a lot of unnecessary software that inflates the volume of the container. And the more software, the more holes and vulnerabilities.

In addition, the larger the image, the more it takes up space on the host and in the registry (do you store images somewhere)?

P: Yes, of course, we have a registry, and you set it up.

AI: So, what am I talking about? .. Oh, yes, volumes ... The load on the network is also growing. For a single image it is imperceptible, but when there is a continuous assembly, tests and a warm, it is noticeable. And if you do not have God's mode on AWS, you will also receive a space account.

Therefore, you need to choose the most suitable image, with the exact version and minimum software. For example, take: FROM ruby:2.5.5-stretch

P: Oh, I see. And how and where to see the available images? How to understand what I need?

AI: Usually, images are taken from dokhaba , do not confuse with pornjab :). There are usually several assemblies for an image:
Alpine : images are collected on a minimalist image of Linux, only 5 MB. Its minus: it is compiled with its own libc implementation, standard packages do not work in it. Finding and installing the right package will take a lot of time.
Scratch : base image, not used to build other images. It is intended solely for running binary, prepared data. Ideal for running binary applications that include everything you need, such as go-applications.
Based on any OS, for example, Ubuntu or Debian. Well here, I think, it is not necessary to explain.

AI: Now we need to put all the extras. packages and clean up caches. And immediately you can throw apt-get upgrade . Otherwise, with each assembly, despite the fixed tag of the basic image, different images will be obtained. Updating packages in an image is a task of the maintainer, it is accompanied by a change in the tag.

P: Yes, I tried to do it, I did this:

 WORKDIR /app COPY ./ /app RUN curl -sL https://deb.nodesource.com/setup_9.x | bash - \ && apt-get -y install libpq-dev imagemagick gsfonts ruby-full ssh supervisor nodejs \ && gem install bundler \ && bundle install --without development test --path vendor/bundle RUN rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

AI: Not bad, but here, too, there is something to work on. Look, here is this command:

 RUN rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

... does not remove data from the final image, but only creates an additional layer without this data. Correctly:

 RUN curl -sL https://deb.nodesource.com/setup_9.x | bash - \ && apt-get -y install libpq-dev imagemagick gsfonts nodejs \ && gem install bundler \ && bundle install --without development test --path vendor/bundle \ && rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

But that is not all. What have you got there, Ruby? Then you do not need to copy the entire project at the beginning. It is enough to copy Gemfile and Gemfile.lock.

With this approach, bundle install will not be executed for every source change, but only if Gemfile or Gemfile.lock has changed.

The same methods work for other languages ​​with a dependency manager, such as npm, pip, composer, and other dependency-list-based files.

Finally, remember, at the beginning I spoke about the Docker ideology “one container - one process”? This means that supervisor is not needed. You should also not install systemd for the same reasons. In essence, the Docker is itself a supervisor. And when you try to run multiple processes in it, it’s like running multiple applications in one supervisor process.
When assembling, you will make a single image, and then run the required number of containers so that each process runs one process.

But more about that later.

P: It seems to understand. See what happens:

 FROM ruby:2.5.5-stretch WORKDIR /app COPY Gemfile* /app RUN curl -sL https://deb.nodesource.com/setup_9.x | bash - \ && apt-get -y install libpq-dev imagemagick gsfonts nodejs \ && gem install bundler \ && bundle install --without development test --path vendor/bundle \ && rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* COPY . /app RUN rake assets:precompile CMD ["bundle”, “exec”, “passenger”, “start"] 

And we will redefine start of demons at start of the container?

AI: Yes, that's right. By the way, you can use both CMD and ENTRYPOINT. And to figure out what the difference is, this is your homework. There is a good article on this topic on Habré.

So, come on. You download the file to install the node, but there is no guarantee that it will be what you need. It is necessary to add validation. For example:

 RUN curl -sL https://deb.nodesource.com/setup_9.x > setup_9.x \ && echo "958c9a95c4974c918dca773edf6d18b1d1a41434 setup_9.x" | sha1sum -c - \ && bash setup_9.x \ && rm -rf setup_9.x \ && apt-get -y install libpq-dev imagemagick gsfonts nodejs \ && gem install bundler \ && bundle install --without development test --path vendor/bundle \ && rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

By checksum, you can verify that you have downloaded the correct file.

P: But if the file changes, the build will fail.

II: Yes, and oddly enough, this is also a plus. You will know that the file has changed, and you can see what has been changed there. You never know, they added, say, a script that deletes everything it reaches, or makes a backdoor.

P: Thank you. It turns out, the final Dockerfile will look like this:

 FROM ruby:2.5.5-stretch WORKDIR /app COPY Gemfile* /app RUN curl -sL https://deb.nodesource.com/setup_9.x > setup_9.x \ && echo "958c9a95c4974c918dca773edf6d18b1d1a41434 setup_9.x" | sha1sum -c - \ && bash setup_9.x \ && rm -rf setup_9.x \ && apt-get -y install libpq-dev imagemagick gsfonts nodejs \ && gem install bundler \ && bundle install --without development test --path vendor/bundle \ && rm -rf /usr/local/bundle/cache/*.gem \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* COPY . /app RUN rake assets:precompile CMD ["bundle”, “exec”, “passenger”, “start"] 

P: Igor Ivanovich, thanks for the help. I have to run, I need to make 10 more commits today.

Igor Ivanovich, with his gaze stopping his hurried colleague, takes a sip of strong coffee. After thinking a few seconds about the SLA 99.9% and the code without bugs, he asks a question.

II: Where do you keep logs?

P: Of course, in production.log. By the way, yes, and how do we get access to them without ssh?

AI: If you leave them in files, a solution for you has already been invented. The docker exec command allows you to execute any command in a container. For example, you can make cat for logs. And using the -it switch and running bash (if installed in the container), you will get interactive access to the container.

But storing logs in files is not worth it. At a minimum, this leads to uncontrolled growth of the container, but no one rotates the logs. All logs need to be thrown in stdout. There they can already be viewed using the docker logs command .

P: Igor Ivanovich, and can put out the logs in a mounted directory, on a physical node, as user data?

II: It's good that you didn't forget to transfer the data loaded on the node disk. With logs, it is also possible, just do not forget to configure rooting.
Everything, you can run.

P: Igor Ivanovich, and advise what to read?

AI: To start, read the recommendations from the Docker developers , hardly anyone knows Docker better than them.

And if you want to do an internship, go to intensive . After all, theory without practice is dead.