docker-compose.yml
file.mock
services and tests into separate containers with their own linking and interaction logicN
files into one and launching them as a whole application became the main problem. What became the secondary problems that did not allow us simply to simply merge the existing files one by one?python
preprocessor, which will do all the docker-compose
files.docker-compose
file.python
packaged module. sudo wget https://github.com/paunin/docker-compose-mixer/blob/master/dist/dc-mixer?raw=true -O /usr/local/bin/dc-mixer sudo chmod +x /usr/local/bin/dc-mixer
docker-compose-mixer.yml
file is inherently a small config that describes exactly how two or more projects will start together.project A
and project B
with two docker-compose.yml
files.java-application, redis, rabbitmq, mail
, project A
(new version) also contains a mysql
service. $ cat ../proj_a/docker-compose.yml sources: build: images/sources volumes: - .:/var/application.host application: build: images/java dockerfile: application.yml dns: - 8.8.8.8 - 9.9.9.9 hostname: application.host working_dir: /var/application.host cgroup_parent: m-executor-abcd links: - redis:redis - rabbitmq - mail - mysql:db volumes_from: - sources command: "/start.sh" env_file: - ./env_files/application.env - ./env_files/rabbit.env environment: - DB_DRIVER: mysql - DB_PORT: 3306 ports: - "80:80" #http - "1098:1098" #jmx external_links: - redis_1 - project_db_1:mysql extra_hosts: - "somehost:162.242.195.82" labels: com.example.description: "Accounting webapp" com.example.department: "Finance" redis: labels: - "com.example.description=Accounting webapp" - "com.example.department=Finance" extends: file: ../redis.yml service: redisbase expose: ports: - "6379:6379" - "127.0.0.1:6370:6370" log_driver: "syslog" log_opt: syslog-address: "tcp://192.168.0.42:123" net: "bridge" rabbitmq: build: images/rabbitmq ports: - "15672:15672" volumes_from: - sources command: /start.sh env_file: - ./env_files/rabbit.env mail: build: images/mail hostname: mail domainname: application.host expose: - 25 - 143 ports: - "25:25" - "143:143" volumes: - ./images/mail/spamassassin:/tmp/spamassassin/ - ./images/mail/postfix:/tmp/postfix/ - ./images/mail/mail:/tmp/mail/ mysql: build: images/mysql ports: - 3602
$ cat ./proj_b/docker-compose.yml sources: build: images/sources volumes: - .:/var/application.host application: build: images/java dockerfile: application.yml dns: - 8.8.8.8 - 9.9.9.9 hostname: application.host working_dir: /var/application.host cgroup_parent: m-executor-abcd links: - redis:redis - rabbitmq - mail volumes_from: - sources command: "/start.sh" env_file: - ./env_files/application.env - ./env_files/rabbit.env environment: - VARIABLE: value ports: - "80:80" #http - "1098:1098" #jmx external_links: - redis_1 - project_db_1:mysql extra_hosts: - "somehost:162.242.195.82" labels: com.example.description: "Accounting webapp" com.example.department: "Finance" redis: labels: - "com.example.description=Accounting webapp" - "com.example.department=Finance" extends: file: ../redis.yml service: redisbase expose: ports: - "6379:6379" - "127.0.0.1:6370:6373" log_driver: "syslog" log_opt: syslog-address: "tcp://192.168.0.42:123" net: "bridge" rabbitmq: build: images/rabbitmq ports: - "15672:15672" volumes_from: - sources command: /start.sh env_file: - ./env_files/rabbit.env mail: build: images/mail hostname: mail domainname: application.host expose: - 25 - 143 ports: - "25:25" - "143:143" volumes: - ./images/mail/spamassassin:/tmp/spamassassin/ - ./images/mail/postfix:/tmp/postfix/ - ./images/mail/mail:/tmp/mail/
rabbitmq
service. In addition, you must create a new service pgsql
and connect it to the first project (replacing mysql)./docker-compose-mixer.yml
(the option -i, --input-file
can specify a different name for the file) $ cat ./docker-compose-mixer.yml includes: proja: ../proj_a/docker-compose.yml projb: ./proj_b/docker-compose.yml ignores: - projbrabbitmq overrides: projbapplication: links: - projaredis:redis - projarabbitmq:rabbitmq - projamail:mail projaapplication: links: - projaredis:redis - projarabbitmq:rabbitmq - projamail:mail - pgsql:db environment: - DB_DRIVER: pgsql - DB_PORT: 5432 master_services: pgsql: image: pgsql:latest expose: - 5432 ports: 5432:5432
dc-mixer -v
command runs the preprocessor in verbal mode and saves the result to a file. Startup options help control behavior. $ dc-mixer --help Compile docker-compose from several docker-compose.yml files Usage: dc-mixer [options] Options: -h, --help Print help information -i, --input-file Input file (default `docker-compose-mixer.yml` in current directory) -o, --output-file Output file (default `docker-compose.yml` in current directory) -h, --help Print help information -v, --verbose Enable verbose mode For more information read documentation: https://github.com/paunin/docker-compose-mixer
./docker-compose.yml
. (the -o, --output-file
option can specify a different name for the file) $ dc-mixer -v -o docker-compose.yml DEBUG:root:Start compiling compose file... DEBUG:root:Input file: /Users/paunin/Sites/dc-mixer.local/examples/example2/proj/docker-compose-mixer.yml; output file: docker-compose.yml DEBUG:root:Mixer config is below: {'overrides': {'projbapplication': {'links': ['projaredis:redis', 'projarabbitmq:rabbitmq', 'projamail:mail']}, 'projaapplication': {'environment': {'DB_DRIVER': 'pgsql', 'DB_PORT': 5432}, 'links': ['projaredis:redis', 'projarabbitmq:rabbitmq', 'projamail:mail', 'pgsql:db']}}, 'master_services': {'pgsql': {'image': 'pgsql:latest', 'expose': [5432], 'ports': '5432:5432'}}, 'ignores': ['projbrabbitmq'], 'includes': {'projb': './proj_b/docker-compose.yml', 'proja': '../proj_a/docker-compose.yml'}} DEBUG:root:Creating scope for file: /Users/paunin/Sites/dc-mixer.local/examples/example2/proj/proj_b/docker-compose.yml and prefix: projb DEBUG:root:Creating scope for file: /Users/paunin/Sites/dc-mixer.local/examples/example2/proj_a/docker-compose.yml and prefix: proja DEBUG:root:Resolving services names DEBUG:root:Resolving services paths with DEBUG:root:Resolving services ports DEBUG:root:Redefined ports: {'projaapplication': {80: 81, 1098: 1099}, 'projaredis': {6370: 6371, 6379: 6380}, 'projamail': {25: 26, 143: 144}} DEBUG:root:Result scope is: {'projasources': {'build': '../proj_a/images/sources', 'volumes': ['./../proj_a:/var/application.host']}, 'projaredis': {'log_opt': {'syslog-address': 'tcp://192.168.0.42:123'}, 'log_driver': 'syslog', 'expose': None, 'labels': ['com.example.description=Accounting webapp', 'com.example.department=Finance'], 'extends': {'service': 'redisbase', 'file': '../redis.yml'}, 'net': 'bridge', 'ports': ['6380:6379', '127.0.0.1:6371:6370']}, 'projbmail': {'domainname': 'application.host', 'expose': [25, 143], 'hostname': 'mail', 'build': 'proj_b/images/mail', 'volumes': ['./proj_b/images/mail/spamassassin:/tmp/spamassassin/', './proj_b/images/mail/postfix:/tmp/postfix/', './proj_b/images/mail/mail:/tmp/mail/'], 'ports': ['25:25', '143:143']}, 'projamail': {'domainname': 'application.host', 'expose': [25, 143], 'hostname': 'mail', 'build': '../proj_a/images/mail', 'volumes': ['./../proj_a/images/mail/spamassassin:/tmp/spamassassin/', './../proj_a/images/mail/postfix:/tmp/postfix/', './../proj_a/images/mail/mail:/tmp/mail/'], 'ports': ['26:25', '144:143']}, 'pgsql': {'image': 'pgsql:latest', 'expose': [5432], 'ports': '5432:5432'}, 'projbapplication': {'hostname': 'application.host', 'links': ['projaredis:redis', 'projarabbitmq:rabbitmq', 'projamail:mail'], 'cgroup_parent': 'm-executor-abcd', 'labels': {'com.example.description': 'Accounting webapp', 'com.example.department': 'Finance'}, 'extra_hosts': ['somehost:162.242.195.82'], 'environment': [{'VARIABLE': 'value'}], 'working_dir': '/var/application.host', 'command': '/start.sh', 'build': 'proj_b/images/java', 'dns': ['8.8.8.8', '9.9.9.9'], 'volumes_from': ['projbsources'], 'env_file': ['proj_b/env_files/application.env', 'proj_b/env_files/rabbit.env'], 'dockerfile': 'application.yml', 'ports': ['80:80', '1098:1098'], 'external_links': ['redis_1', 'project_db_1:mysql']}, 'projaapplication': {'hostname': 'application.host', 'links': ['projaredis:redis', 'projarabbitmq:rabbitmq', 'projamail:mail', 'pgsql:db'], 'cgroup_parent': 'm-executor-abcd', 'labels': {'com.example.description': 'Accounting webapp', 'com.example.department': 'Finance'}, 'extra_hosts': ['somehost:162.242.195.82'], 'environment': {'DB_DRIVER': 'pgsql', 'DB_PORT': 5432}, 'working_dir': '/var/application.host', 'command': '/start.sh', 'build': '../proj_a/images/java', 'dns': ['8.8.8.8', '9.9.9.9'], 'volumes_from': ['projasources'], 'env_file': ['../proj_a/env_files/application.env', '../proj_a/env_files/rabbit.env'], 'dockerfile': 'application.yml', 'ports': ['81:80', '1099:1098'], 'external_links': ['redis_1', 'project_db_1:mysql']}, 'projamysql': {'build': '../proj_a/images/mysql', 'ports': []}, 'projarabbitmq': {'volumes_from': ['projasources'], 'env_file': ['../proj_a/env_files/rabbit.env'], 'command': '/start.sh', 'build': '../proj_a/images/rabbitmq', 'ports': ['15672:15672']}, 'projbredis': {'log_opt': {'syslog-address': 'tcp://192.168.0.42:123'}, 'log_driver': 'syslog', 'expose': None, 'labels': ['com.example.description=Accounting webapp', 'com.example.department=Finance'], 'extends': {'service': 'redisbase', 'file': 'redis.yml'}, 'net': 'bridge', 'ports': ['6379:6379', '127.0.0.1:6370:6373']}, 'projbsources': {'build': 'proj_b/images/sources', 'volumes': ['./proj_b:/var/application.host']}} DEBUG:root:Save result scope in the file "docker-compose.yml"
pgsql: expose: - 5432 image: "pgsql:latest" ports: "5432:5432" projaapplication: build: ../proj_a/images/java cgroup_parent: m-executor-abcd command: /start.sh dns: - "8.8.8.8" - "9.9.9.9" dockerfile: application.yml env_file: - ../proj_a/env_files/application.env - ../proj_a/env_files/rabbit.env environment: DB_DRIVER: pgsql DB_PORT: 5432 external_links: - redis_1 - "project_db_1:mysql" extra_hosts: - "somehost:162.242.195.82" hostname: application.host labels: com.example.department: Finance com.example.description: "Accounting webapp" links: - "projaredis:redis" - "projarabbitmq:rabbitmq" - "projamail:mail" - "pgsql:db" ports: - "81:80" - "1099:1098" volumes_from: - projasources working_dir: /var/application.host projamail: build: ../proj_a/images/mail domainname: application.host expose: - 25 - 143 hostname: mail ports: - "26:25" - "144:143" volumes: - "./../proj_a/images/mail/spamassassin:/tmp/spamassassin/" - "./../proj_a/images/mail/postfix:/tmp/postfix/" - "./../proj_a/images/mail/mail:/tmp/mail/" projamysql: build: ../proj_a/images/mysql ports: [] projarabbitmq: build: ../proj_a/images/rabbitmq command: /start.sh env_file: - ../proj_a/env_files/rabbit.env ports: - "15672:15672" volumes_from: - projasources projaredis: expose: ~ extends: file: ../redis.yml service: redisbase labels: - "com.example.description=Accounting webapp" - com.example.department=Finance log_driver: syslog log_opt: syslog-address: "tcp://192.168.0.42:123" net: bridge ports: - "6380:6379" - "127.0.0.1:6371:6370" projasources: build: ../proj_a/images/sources volumes: - "./../proj_a:/var/application.host" projbapplication: build: proj_b/images/java cgroup_parent: m-executor-abcd command: /start.sh dns: - "8.8.8.8" - "9.9.9.9" dockerfile: application.yml env_file: - proj_b/env_files/application.env - proj_b/env_files/rabbit.env environment: - VARIABLE: value external_links: - redis_1 - "project_db_1:mysql" extra_hosts: - "somehost:162.242.195.82" hostname: application.host labels: com.example.department: Finance com.example.description: "Accounting webapp" links: - "projaredis:redis" - "projarabbitmq:rabbitmq" - "projamail:mail" ports: - "80:80" - "1098:1098" volumes_from: - projbsources working_dir: /var/application.host projbmail: build: proj_b/images/mail domainname: application.host expose: - 25 - 143 hostname: mail ports: - "25:25" - "143:143" volumes: - "./proj_b/images/mail/spamassassin:/tmp/spamassassin/" - "./proj_b/images/mail/postfix:/tmp/postfix/" - "./proj_b/images/mail/mail:/tmp/mail/" projbredis: expose: ~ extends: file: redis.yml service: redisbase labels: - "com.example.description=Accounting webapp" - com.example.department=Finance log_driver: syslog log_opt: syslog-address: "tcp://192.168.0.42:123" net: bridge ports: - "6379:6379" - "127.0.0.1:6370:6373" projbsources: build: proj_b/images/sources volumes: - "./proj_b:/var/application.host"
DEBUG:root:Redefined ports
, which provides information about automatically changed ports.Source: https://habr.com/ru/post/274581/
All Articles