Ansible and Rails - flexible replacement for Capistrano while maintaining familiar comfort
Capistrano is a favorite tool for many rails developers that can quickly and easily automate the deployment of your application. Capistrano is the de facto standard for the RoR deployment system, the must-know technology for any self-respecting rubist, the tool that python and PHP developers envied at one time.
Despite the comfort that I don’t want to give up, the more complex tasks I had to solve, the more often Capistrano showed himself not adapted to them.
I noted the following disadvantages:
Many ruby ​​developers have switched to Mina or solve their problems using even more complex configuration management systems like Chef and Puppet . All of them have their own characteristics and disadvantages and solve the problems described above to varying degrees. I managed to solve them with Ansible , without losing the advantages of Capistrano, to which I was used.
Ansible is a tool for managing configurations and its tasks include not only the execution of remote commands described on this article on servers to deploy and manage an individual application, but also the automation of server administration through stored server configurations (Ansible language roles). So, Ansible (as well as Chef and Puppet) allows much more than Capistrano and, ultimately, they all do not go with him in any comparison. However, the goal of this article is to give rails developers a starting point for migration and explain, with this example, the basics of Ansible. At the end of this article, the cap production deploy magic command will turn into ansible-playbook deploy.yml -i inventory / production
Who cares how - I ask under the cat.
Ansible written in python. Not every rubist will like it, but I will dispel fears right away - you will not have to write a single line in the “enemy language”. The drawback of Ansible is that all deployment scripts are configuration files in the well-known yml format with a simple and powerful descriptive syntax.
')
Installation ansible is also simple and fast. Install ansible only on the local machine:
At this, the interaction with the python utilities ends and now the ansible-playbook command is available to us, with the help of which the application is implemented. The team has only one required argument - a relative path to the playbook file.
A playbook file is a list of running tasks or other playbooks. Thanks to nesting, we can effectively isolate tasks by layers and achieve the ability to run only what we need at the moment.
As an example for deployment, let's take myawesomestartup - this is some kind of rails-application with a bunch of passenger 5 standalone and nginx as a web server and sidekiq for background tasks. The physical infrastructure in the example is two production servers:
And one staging:
In the ansible folder, define the master playbook deploy.yml , containing all the other playbooks,
Using the command ansible-playbook deploy.yml , run the entire deployment. However, you can run the playbooks separately, if we need to restart the application without rolling out a new release.
Note the hosts variable in it contains information about the servers on which the deployment will be performed. This variable can be defined in the global configuration ansible, but we will proceed differently using the inventory files.
To store groups of hosts, their hierarchies and settings, inventory files are provided in ansible. These are ini-files with a very simple syntax.
We can describe a group of hosts:
In the group we will announce the hosts themselves:
We declare the variables specific to each specific host:
Notice the curly braces - in ansible all the files are Jinja2 templates. In this example, the environment variables from the machine from which the deployment is performed are interpolated from the template and the lookup command. This is useful in order not to store in the version control system any sensitive information, such as secret keys or database connection strings.
For the example to work, you need to declare the following variables in your ~ / .bashrc or ~ / .zshrc or (which is safer and less convenient) to export them each time before each deployment:
Below are the entire inventory / production and inventory / staging files:
Config templates put in the folder ansible / configs:
For those settings that can be safely stored in the control system, the version I prefer is dotenv .
Create the following file structure in the ansible / environments folder:
Capistrano by default offers a fairly thoughtful file structure on the server.
The releases folder contains the last five recent releases in folders with the names of the type 20150812165952 , which contain the timestamp of the deployment time of this release. Inside each release is a REVISION file containing the hash of the commit from which the release was made.
Simlink current refers to the latest release in the releases folder.
The shared folder contains files common to all releases (for example .pid and .sock ) and those files that are excluded from the version control system (for example, database.yml ). All this allows you to safely roll back the application in case of a failure of the deployment or rolling out the code with unexpected bugs.
Repeat this with Ansible:
Setting some variables was moved to a separate mixin task, since these variables are identical for all playbooks and servers:
Create another playbook to manage the state of the application ansible / app.yml , with which the application can be started, stopped or restarted. Like other playbooks, it can be launched separately or as part of a master playbook.
For more flexibility, add the app_stop and app_start tags . Tags allow you to perform only those parts of tasks that are explicitly indicated during the deployment. If you do not specify the tags during the delay - the playbook will be made entirely.
Here is how it looks in practice:
But the implementation:
The tasks of starting and stopping the application are highlighted separately in the files ansible / tasks / app_start.yml and ansible / tasks / app_stop.yml :
With sidekiq, the situation is similar. For it, we implement a separate playbook, ansible / sidekiq.yml, which supports the corresponding sidekiq_stop and sidekiq_start tags :
The start and stop tasks are also separately allocated to the files ansible / tasks / sidekiq_start.yml and ansible / tasks / sidekiq_stop.yml . In addition to actually starting and stopping sidekiq, these tasks demonstrate how to work with cycles in Ansible and solve the problem of starting / stopping several processes at once:
Now we can use Ansible to deploy rails applications:
Since this article gives only an example (albeit a working one), I will point out the ways in which you can go further:
And the most important thing. Ansible can do much more than roll out application releases and restart servers. After all, I repeat, ansible is not just a deployment tool, but a complete configuration management tool. For example, with the help of roles, you can configure application deployment from scratch, directly to bare server hardware. And the simplicity of the yml-notation makes it easy to modify the solutions found to fit your needs.
All source codes from the article are available on GitHub . Thanks for attention.
Despite the comfort that I don’t want to give up, the more complex tasks I had to solve, the more often Capistrano showed himself not adapted to them.
I noted the following disadvantages:
- Known problems with speed. Due to its versatility, Capistrano deploit slowly, performing extra checks and challenges that you can not always control.
- Consistent warm. The short deployment time must be multiplied by the number of target servers (however, you can configure the parallelization of commands explicitly).
- Strong connectedness with rails. Capistrano configs and dependencies are intertwined with the application, becoming part of it. You cannot create a new deployment environment (for example, a server for early functional rollout) without creating a new rails environment. In difficult situations, Capistrano makes it necessary to keep away from good practice to keep only the development, test and production environment.
- Plugins are double-edged. Giving the opportunity to quickly “screw up” the deployment of an application dependency, plug-ins deprive you of control of the situation, force you to act as the plug-in developer acts. I have written above about the influence of extra “gestures” of plug-ins on the deployment rate.
- Complicated warm heterogeneous applications. The trend of recent years in the rails was the selection of the most difficult (background or network) tasks into separate services, not necessarily written in ruby. In such a situation, capistrano makes you produce a zoo from different deployment systems for different languages ​​and technologies.
Many ruby ​​developers have switched to Mina or solve their problems using even more complex configuration management systems like Chef and Puppet . All of them have their own characteristics and disadvantages and solve the problems described above to varying degrees. I managed to solve them with Ansible , without losing the advantages of Capistrano, to which I was used.
Ansible is a tool for managing configurations and its tasks include not only the execution of remote commands described on this article on servers to deploy and manage an individual application, but also the automation of server administration through stored server configurations (Ansible language roles). So, Ansible (as well as Chef and Puppet) allows much more than Capistrano and, ultimately, they all do not go with him in any comparison. However, the goal of this article is to give rails developers a starting point for migration and explain, with this example, the basics of Ansible. At the end of this article, the cap production deploy magic command will turn into ansible-playbook deploy.yml -i inventory / production
Who cares how - I ask under the cat.
Installation
Ansible written in python. Not every rubist will like it, but I will dispel fears right away - you will not have to write a single line in the “enemy language”. The drawback of Ansible is that all deployment scripts are configuration files in the well-known yml format with a simple and powerful descriptive syntax.
')
Installation ansible is also simple and fast. Install ansible only on the local machine:
sudo easy_isntall pip sudo pip install -U ansible At this, the interaction with the python utilities ends and now the ansible-playbook command is available to us, with the help of which the application is implemented. The team has only one required argument - a relative path to the playbook file.
Ansible-playbook
A playbook file is a list of running tasks or other playbooks. Thanks to nesting, we can effectively isolate tasks by layers and achieve the ability to run only what we need at the moment.
As an example for deployment, let's take myawesomestartup - this is some kind of rails-application with a bunch of passenger 5 standalone and nginx as a web server and sidekiq for background tasks. The physical infrastructure in the example is two production servers:
prima.myawesomestartup.com secunda.myawesomestartup.com And one staging:
plebius.myawesomestartup.com In the ansible folder, define the master playbook deploy.yml , containing all the other playbooks,
--- - hosts: hosts - include: release.yml # - include: app.yml # - - include: sidekiq.yml # sidekiq Using the command ansible-playbook deploy.yml , run the entire deployment. However, you can run the playbooks separately, if we need to restart the application without rolling out a new release.
Note the hosts variable in it contains information about the servers on which the deployment will be performed. This variable can be defined in the global configuration ansible, but we will proceed differently using the inventory files.
Inventory files and application configuration
To store groups of hosts, their hierarchies and settings, inventory files are provided in ansible. These are ini-files with a very simple syntax.
We can describe a group of hosts:
[hosts:children] prima secunda In the group we will announce the hosts themselves:
[prima] prima.myawesomestartup.com [secunda] secunda.myawesomestartup.com We declare the variables specific to each specific host:
[prima:vars] ansible_env_name=production rails_env_name=production database_name={{ lookup('env', 'PRIMA_DB_NAME') }} database_username={{ lookup('env', 'PRIMA_DB_LOGIN') }} database_password={{ lookup('env', 'PRIMA_DB_PASSWORD') }} database_host={{ lookup('env', 'PRIMA_DB_HOST') }} database_port={{ lookup('env', 'PRIMA_DB_PORT') }} Notice the curly braces - in ansible all the files are Jinja2 templates. In this example, the environment variables from the machine from which the deployment is performed are interpolated from the template and the lookup command. This is useful in order not to store in the version control system any sensitive information, such as secret keys or database connection strings.
For the example to work, you need to declare the following variables in your ~ / .bashrc or ~ / .zshrc or (which is safer and less convenient) to export them each time before each deployment:
export PRIMA_DB_NAME=myawesomestartup_production export PRIMA_DB_LOGIN=myawesomestartup export PRIMA_DB_PASSWORD=secret export PRIMA_DB_HOST=db.myawesomestartup.com export PRIMA_DB_PORT=3306 Below are the entire inventory / production and inventory / staging files:
inventory / production
; production [prima] prima.myawesomestartup.com [prima:vars] ansible_env_name=production rails_env_name=production database_name={{ lookup('env', 'PRIMA_DB_NAME') }} database_username={{ lookup('env', 'PRIMA_DB_LOGIN') }} database_password={{ lookup('env', 'PRIMA_DB_PASSWORD') }} database_host{{ lookup('env', 'PRIMA_DB_HOST') }} database_port={{ lookup('env', 'PRIMA_DB_PORT') }} git_branch=master app_path=/srv/www/prima.myawesomestartup.com custom_server_options=--no-friendly-error-pages sidekiq_process_number=4 [secunda] secunda.myawesomestartup.com [secunda:vars] ansible_env_name=production rails_env_name=production database_name={{ lookup('env', 'SECUNDA_DB_NAME') }} database_username={{ lookup('env', 'SECUNDA_DB_LOGIN') }} database_password={{ lookup('env', 'SECUNDA_DB_PASSWORD') }} database_host={{ lookup('env', 'SECUNDA_DB_HOST') }} database_port={{ lookup('env', 'SECUNDA_DB_PORT') }} git_branch=master app_path=/srv/www/secunda.myawesomestartup.com custom_server_options=--no-friendly-error-pages sidekiq_process_number=4 [hosts:children] prima secunda inventory / staging
; staging [plebius] plebius.myawesomestartup.com [plebius:vars] ansible_env_name=staging rails_env_name=production database_name={{ lookup('env', 'PLEBIUS_DB_NAME') }} database_username={{ lookup('env', 'PLEBIUS_DB_LOGIN') }} database_password={{ lookup('env', 'PLEBIUS_DB_PASSWORD') }} database_host={{ lookup('env', 'PLEBIUS_DB_HOST') }} database_port={{ lookup('env', 'PLEBIUS_DB_PORT') }} git_branch=develop app_path=/srv/www/plebius.myawesomestartup.com custom_server_options=--friendly-error-pages sidekiq_process_number=4 [hosts:children] plebius Config templates put in the folder ansible / configs:
configs / database.yml
# configs/database.yml {{rails_env_name}}: adapter: mysql2 database: {{database_name}} username: {{database_username}} password: {{database_password}} host: {{database_host}} port: {{database_port}} secure_auth: false For those settings that can be safely stored in the control system, the version I prefer is dotenv .
Create the following file structure in the ansible / environments folder:
production/ prima.env secunda.env staging/ plebius.env Releases like in capistrano
Capistrano by default offers a fairly thoughtful file structure on the server.
releases/ 20150631130156/ 20150631130233/ 20150631172431/ 20150704162516/ 20150712165952/ current - -> /www/domain/releases/20150712165952/ shared/ The releases folder contains the last five recent releases in folders with the names of the type 20150812165952 , which contain the timestamp of the deployment time of this release. Inside each release is a REVISION file containing the hash of the commit from which the release was made.
Simlink current refers to the latest release in the releases folder.
The shared folder contains files common to all releases (for example .pid and .sock ) and those files that are excluded from the version control system (for example, database.yml ). All this allows you to safely roll back the application in case of a failure of the deployment or rolling out the code with unexpected bugs.
Repeat this with Ansible:
ansible / release.yml
# ansible/release.yml --- - hosts: hosts # inventory- tasks: # app_path shared_path . - include: tasks/_set_vars.yml tags=always # - set_fact: timestamp="{{ lookup('pipe', 'date +%Y%m%d%H%M%S') }}" - set_fact: release_path="{{ app_path }}/releases/{{ timestamp }}" # . ansible - name: Ensure shared directory exists file: path={{ shared_path }} state=directory - name: Ensure shared/assets directory exists file: path={{ shared_path }}/assets state=directory - name: Ensure tmp directory exists file: path={{ shared_path }}/tmp state=directory - name: Ensure log directory exists file: path={{ shared_path }}/log state=directory - name: Ensure bundle directory exists file: path={{ shared_path }}/bundle state=directory # - name: Leave only last releases shell: "cd {{ app_path }}/releases && find ./ -maxdepth 1 | grep -G .............. | sort -r | tail -n +{{ keep_releases }} | xargs rm -rf" - name: Create release directory file: path={{ release_path }} state=directory # - name: Checkout git repo into release directory git: repo={{ git_repo }} dest={{ release_path }} version={{ git_branch }} accept_hostkey=yes # REVISION - name: Get git branch head hash shell: "cd {{ release_path }} && git rev-parse --short HEAD" register: git_head_hash - name: Create REVISION file in the release path copy: content="{{ git_head_hash.stdout }}" dest={{ release_path }}/REVISION # rails - name: Set assets link file: src={{ shared_path }}/assets path={{ release_path }}/public/assets state=link - name: Set tmp link file: src={{ shared_path }}/tmp path={{ release_path }}/tmp state=link - name: Set log link file: src={{ shared_path }}/log path={{ release_path }}/log state=link # .env database.yml . . - name: Copy .env file template: src=environments/{{ansible_env_name}}/{{ansible_hostname}}.env dest={{ release_path }}/.env - name: Copy database.yml template: src=configs/database.yml dest={{ release_path }}/config - set_fact: rvm_wrapper_command="cd {{ release_path }} && RAILS_ENV={{ rails_env_name }} rvm ruby-{{ ruby_version }}@{{ full_gemset_name }} --create do" # Bundle, , ... - name: Run bundle install shell: "{{ rvm_wrapper_command }} bundle install --path {{ shared_path }}/bundle --deployment --without development test" - name: Run db:migrate shell: "{{ rvm_wrapper_command }} rake db:migrate" - name: Precompile assets shell: "{{ rvm_wrapper_command }} rake assets:precompile" # current - name: Update app version file: src={{ release_path }} path={{ app_path }}/current state=link Setting some variables was moved to a separate mixin task, since these variables are identical for all playbooks and servers:
# ansible/tasks/_set_vars.yml --- - set_fact: app_name="myawesomestartup" - set_fact: ruby_version="2.2.2" - set_fact: ruby_gemset="myawesomestartup" - set_fact: git_repo="ilpagency/rails-sidekiq-ansible-sample" - set_fact: keep_releases="5" - set_fact: full_app_name="{{ app_name }}-{{ ansible_env_name }}" - set_fact: full_gemset_name="{{ ruby_gemset }}-{{ ansible_env_name }}" - set_fact: current_path="{{ app_path }}/current" - set_fact: shared_path="{{ app_path }}/shared" Run passenger and sidekiq - tags and cycles Ansible
Create another playbook to manage the state of the application ansible / app.yml , with which the application can be started, stopped or restarted. Like other playbooks, it can be launched separately or as part of a master playbook.
For more flexibility, add the app_stop and app_start tags . Tags allow you to perform only those parts of tasks that are explicitly indicated during the deployment. If you do not specify the tags during the delay - the playbook will be made entirely.
Here is how it looks in practice:
# : ansible-playbook app.yml -i inventory/production # : ansible-playbook app.yml -i inventory/production -t "app_stop" # : ansible-playbook app.yml -i inventory/production -t "app_start" # : ansible-playbook app.yml -i inventory/production -t "app_stop,app_start" But the implementation:
ansible / app.yml
# ansible/app.yml --- - hosts: hosts # inventory- tasks: - include: tasks/_set_vars.yml tags=always # always , , - set_fact: socks_path={{ shared_path }}/tmp/socks tags: always - name: Ensure sockets directory exists file: path={{ socks_path }} state=directory tags: always - set_fact: app_sock={{ socks_path }}/app.sock tags: always - set_fact: pids_path={{ shared_path }}/tmp/pids tags: always - name: Ensure pids directory exists file: path={{ pids_path }} state=directory tags: always - set_fact: app_pid={{ pids_path }}/passenger.pid tags: always - set_fact: rvm_wrapper_command="cd {{ current_path }} && RAILS_ENV={{ rails_env_name }} rvm ruby-{{ ruby_version }}@{{ full_gemset_name }} --create do" tags: always - include: tasks/app_stop.yml tags=app_stop # app_start - include: tasks/app_start.yml tags=app_start # , - app_stop The tasks of starting and stopping the application are highlighted separately in the files ansible / tasks / app_start.yml and ansible / tasks / app_stop.yml :
ansible / tasks / app_start.yml
# ansible/tasks/app_start.yml --- - name: start passenger shell: "{{ rvm_wrapper_command }} bundle exec passenger start -d -S {{ app_sock }} --environment {{ rails_env_name }} --pid-file {{ app_pid }} {{ custom_server_options }}" ansible / tasks / app_stop.yml
# ansible/tasks/app_stop.yml --- - name: stop passenger shell: "{{ rvm_wrapper_command }} bundle exec passenger stop --pid-file {{ app_pid }}" ignore_errors: yes # ... . - . With sidekiq, the situation is similar. For it, we implement a separate playbook, ansible / sidekiq.yml, which supports the corresponding sidekiq_stop and sidekiq_start tags :
ansible / app.yml
# ansible/sidekiq.yml --- - hosts: hosts tasks: - include: tasks/_set_vars.yml tags=always - set_fact: pids_path={{ shared_path }}/tmp/pids tags: always - name: Ensure pids directory exists file: path={{ pids_path }} state=directory tags: always - set_fact: rvm_wrapper_command="cd {{ current_path }} && RAILS_ENV={{ rails_env_name }} rvm ruby-{{ ruby_version }}@{{ full_gemset_name }} --create do" tags: always - include: tasks/sidekiq_stop.yml tags=sidekiq_stop - include: tasks/sidekiq_start.yml tags=sidekiq_start The start and stop tasks are also separately allocated to the files ansible / tasks / sidekiq_start.yml and ansible / tasks / sidekiq_stop.yml . In addition to actually starting and stopping sidekiq, these tasks demonstrate how to work with cycles in Ansible and solve the problem of starting / stopping several processes at once:
ansible / tasks / sidekiq_start.yml
# ansible/tasks/sidekiq_start.yml --- - name: start sidekiq shell: "{{ rvm_wrapper_command }} bundle exec sidekiq --index {{ item }} --pidfile {{ pids_path }}/sidekiq-{{ item }}.pid --environment {{ rails_env_name }} --logfile {{ shared_path }}/log/sidekiq.log --daemon" # item - i . with_sequence 4, item 1,2,3,4 with_sequence: count={{ sidekiq_process_number }} # sidekiq ansible / tasks / sidekiq_stop.yml
# ansible/tasks/sidekiq_stop.yml --- - name: stop sidekiq shell: "{{ rvm_wrapper_command }} bundle exec sidekiqctl stop {{ pids_path }}/sidekiq-{{ item }}.pid 20" ignore_errors: yes # , , , . with_sequence: count={{ sidekiq_process_number }} Conclusion
Now we can use Ansible to deploy rails applications:
cd myawesomestartup/ansible # : ansible-playbook deploy.yml -i inventory/production # : ansible-playbook app.yml -i inventory/production # sidekiq: ansible-playbook sidekiq.yml -i inventory/production # : ansible-playbook deploy.yml -i inventory/staging -e git_branch="hotfix/14082015-777-production_bug" Since this article gives only an example (albeit a working one), I will point out the ways in which you can go further:
- Implement graceful restart for passenger.
- Use the Ansible role mechanism instead of nested playbooks.
- Implement rollback to roll back to previous releases.
- In general, to bring this example more in line with the recommendations of the developers .
And the most important thing. Ansible can do much more than roll out application releases and restart servers. After all, I repeat, ansible is not just a deployment tool, but a complete configuration management tool. For example, with the help of roles, you can configure application deployment from scratch, directly to bare server hardware. And the simplicity of the yml-notation makes it easy to modify the solutions found to fit your needs.
All source codes from the article are available on GitHub . Thanks for attention.
Source: https://habr.com/ru/post/266353/
All Articles