Manage Docker project with multiple git repositories

The team in which I work uses microservice organization in projects.
Each microservice has its own repository. Each microservice is a docker container.
For the development environment to run everything together, we use docker-compose.

In addition, we use the concept of separating the processes of building an application and packing a container so as not to drag source codes and development utilities into containers.

We encountered two problems:

1. During the initial development of the development environment, it is necessary to explain to the programmer, or to write an initialization script that will clone and create the necessary folder hierarchy from several repositories.
2. docker-compose can not build the application, and then pack it into a display. he can only run docker build .

To solve these problems, we made the docker-project control script, which turned out to be very easy to use.
What we want to share with the open-source community.

There is another way to organize a project, where each micro-service has its own development environment and its own docker-compose.yml, in which it interacts with ready-made containers. Such a system did not suit us, because by breaking the application logically into several microservices, we are simultaneously developing each of them. And to support many similar docker-compose.yml is still a pleasure.

The docker-compose.yml file is stored in a separate project repository, which also stores system tests, documentation of the right level and the rest, which cannot be applied to a specific microservice, but only to the system as a whole.

In order not to produce additional configs, the docker-project expands the functionality of the docker-compose.yml file, adding two new functions to it:

1. Download / update the source code of services from repositories
2. Run specified commands relative to source folders

The docker specification allows you to add meta information in labels to the container definition. Labels do not affect the operation of the container; these are just strings that can always be accessed by working with the container.

An example of such docker-compose.yml:

 version: "2" services: users: image: vendor/users expose: - 80 labels: project.git: "https://github.com/vendor/users.git" #    project.git.branch: cool-feature #  ,   -  project.build: make #      web-ui: image: vendor/web-ui ports: - 80:80 labels: project.git: "https://github.com/vendor/web-ui.git" project.build: make #     ,      mysql: image: mysql expose: - 3306 

Working with the project in this case looks like this:

 git clone ... myproject cd myproject docker-project update #      apps/{vendor}/{repo} docker-project build #       build,       docker-compose up #     . ... docker-project update #    git pull #        . cd apps/vendor/user git add . git commit -m "...." git push 

Inside the docker-project is just a wrapper over git and bash that simplifies the deployment of a project from several repositories, as well as building it and other commands that need to be executed on several repositories at the same time. The command call is interactive.

Do not forget to add /apps/ in .gitignore

If you change the brunch to another within the service, it will remain so. Repeated update does only git pull.

You can add your commands.
For example, if you define the test command for services, you can run unit tests from several services with a single docker-project test .

Call the docker-project command. will only cause services to start where this command is defined.

You can view the available commands for this project by calling the docker-project status

In addition, the docker-project allows:

• Limit the execution of selected services --app
• Run an arbitrary shell command in folders with a shell project
• Add an arbitrary string to the end of the command being executed --extra

A call to the docker-project without parameters shows the cheat sheet by command:

 $ docker-project docker project management tool 0.0.1-alfa Usage: docker-project <command> <arguments> Commands: update - clones or pulls application source shell - uses extra parameter to run shell command for each app status - prints current services with repos and their commands help - prints help your_command - defined as label for the service (example: labels: project.test: make test) Arguments: Full name | Short | Default | Note ------------------------------------------------------- --file -f docker-compose.yml Alternative config file --apps -a apps Applications sources folder --extra -x Extra parameters passed to command 

Installed in the system is quite trivial:

 curl -O -L https://github.com/webreactor/docker-project/releases/download/0.0.1-alfa/docker-project chmod a+x docker-project sudo cp docker-project /usr/local/bin/ 

Or you can compile from sources:

 git clone https://github.com/webreactor/docker-project.git cd docker-project make sudo make install 

Work requires the installed rnp-cli and git.