Setting up a server to deploy a Rails application using Ansible

Not so long ago I had to write several ansible playbooks to prepare the server for the deployment of rails application. And, surprisingly, I did not find a simple step by step manual. I didn’t want to copy someone else’s playbook without understanding what was happening, and eventually I had to read the documentation, collecting everything myself. Perhaps someone I can help speed up this process with the help of this article.

The first step is to understand that ansible provides you with a convenient interface for performing a predefined list of actions on a remote server (s) via SSH. There is no magic here, you can’t install a plugin and get your application with docker, monitoring and other stuff out of the box. In order to write a playbook you need to know what exactly you want to do and how to do it. Therefore, I am not satisfied with ready-made playbooks from a github, or articles like: “Copy and run, will work”.

What do we need?

As I said, in order to write a playbook you need to know what you want to do and how to do it. Let's define what we need. For Rails applications, we will need several system packages: nginx, postgresql (redis, etc). In addition, we need a certain version of ruby. It is best to install it via rbenv (rvm, asdf ...). Running it all as a root user is always a bad idea, so you need to create a separate user and set up permissions for it. After this, you need to upload our code to the server, copy the configs for nginx, postgres, etc and run all these services.

As a result, the sequence of actions is as follows:

1. Login as root
2. install system packages
3. create a new user, set up permissions, ssh key
4. configure system packages (nginx etc) and run them
5. Create a user in the database (you can immediately create a database)
6. Login as a new user
7. Install rbenv and ruby
8. Installing a Bandler
9. Fill the application code
10. We start Puma server

Moreover, the last stages can be done with the help of capistrano, at least she is able to copy the code into the release directories from the box, switch the release with a symlink if successful, copy configs from the shared directory, restart puma, etc. All this can be done with Ansible, but why?

File structure

Ansible has a strict file structure for all its files, so it’s best to keep it all in a separate directory. And it is not so important whether it will be in the rails application itself, or separately. You can store files in a separate git repository. Personally, it was most convenient for me to create the ansible directory in the / config directory of the rails application and store everything in one repository.

Simple playbook

A playbook is a yml file in which, using a special syntax, it is described what should be done by ansible. Let's create the first playbook that does nothing:

--- - name: Simple playbook hosts: all 

Here we simply say that our playbook is called Simple Playbook and that its content must be performed for all hosts. We can save it in the / ansible directory with the name playbook.yml and try to run:

 ansible-playbook ./playbook.yml PLAY [Simple Playbook] ************************************************************************************************************************************ skipping: no hosts matched 

Ansible says that it does not know the hosts that would match the all list. They must be listed in a special inventory file .

Let's create it in the same ansible directory:

 123.123.123.123 

So just specify the host (ideally, the host of your VPS for tests, or you can register localhost) and save it under the name of inventory .
You can try to run ansible with the invetory file:

 ansible-playbook ./playbook.yml -i inventory PLAY [Simple Playbook] ************************************************************************************************************************************ TASK [Gathering Facts] ************************************************************************************************************************************ PLAY RECAP ************************************************************************************************************************************ 

If you have ssh access to the specified host, then ansible will connect and collect information about the remote system. (default TASK [Gathering Facts]) then gives a brief report on the implementation (PLAY RECAP).

By default, the connection uses the username under which you are logged in to the system. On the host it probably will not. In the playbook file, you can specify which user to connect to using the remote_user directive. Also, information about a remote system can often be unnecessary and you should not waste time collecting it. This task can also be turned off:

 --- - name: Simple playbook hosts: all remote_user: root become: true gather_facts: no 

Try running the playbook again and make sure the connection is working. (If you specified the root user, then you also need to specify the become: true directive to get elevated rights. As written in the documentation: become set to 'true'/'yes' to activate privilege escalation. although it is not entirely clear why) .

Perhaps you will get an error caused by the fact that the ansible cannot determine the python interpreter, then you can specify it manually:

 ansible_python_interpreter: /usr/bin/python3 

where you have python can be found with the whereis python command.

Installing system packages

The standard Ansible package includes many modules for working with various system packages, so we don’t have to write bash scripts for any reason. Now we need one of these modules to upgrade the system and install system packages. I have Ubuntu Linux on my VPS, respectively, to install packages, I use apt-get and a module for it . If you use another operating system, you may need another module (remember, I said at the beginning that you need to know in advance what we will do and how). However, the syntax is likely to be similar.

Let's complete our playbook with the first tasks:

 --- - name: Simple playbook hosts: all remote_user: root become: true gather_facts: no tasks: - name: Update system apt: update_cache=yes - name: Install system dependencies apt: name: git,nginx,redis,postgresql,postgresql-contrib state: present 

Task - this is just the task that ansible will perform on remote servers. We give the task a name to track its execution in the log. And we describe, using the syntax of a specific module, what it needs to do. In this case, apt: update_cache=yes - says update system packages using the apt module. The second team is somewhat more complicated. We pass a list of packages to the apt module, and say that their state should become present , that is, we say to install these packages. Similarly, we can tell them to remove, or update, simply by changing the state . Please note that in order for rails to work with postgresql, we need the postgresql-contrib package, which we are installing now. On this again, one must know and do, ansible in itself will not do this.

Try running the playbook again and check that the packages are installed.

Create new users.

Ansible also has a user module for working with users. Add one more task (I hid the already known parts of the playbook for comments, so as not to copy it entirely each time):

 --- - name: Simple playbook # ... tasks: # ... - name: Add a new user user: name: my_user shell: /bin/bash password: "{{ 123qweasd | password_hash('sha512') }}" 

We create a new user, set a schell and password for it. And then we face several problems. What if user names have to be different for different hosts? Yes, and store the password in clear form in the playbook is a very bad idea. To begin with, we will put the username and password into variables, and towards the end of the article I will show how to encrypt the password.

 --- - name: Simple playbook # ... tasks: # ... - name: Add a new user user: name: "{{ user }}" shell: /bin/bash password: "{{ user_password | password_hash('sha512') }}" 

using double curly braces in playbooks variables are set.

The values ​​of the variables we specify in the inventory file:

 123.123.123.123 [all:vars] user=my_user user_password=123qweasd 

Pay attention to the directive [all:vars] - it says that the next block of text is variables (vars) and they apply to all hosts (all).

Also interesting is the construction "{{ user_password | password_hash('sha512') }}" . The fact is that ansible does not establish the user through user_add as you would do it manually. And it saves all the data directly, which is why we must also convert the password into a hash in advance, which is what this command does.

Let's add our user to the sudo group. However, before this you need to make sure that such a group exists because nobody will do this for us:

 --- - name: Simple playbook # ... tasks: # ... - name: Ensure a 'sudo' group group: name: sudo state: present - name: Add a new user user: name: "{{ user }}" shell: /bin/bash password: "{{ user_password | password_hash('sha512') }}" groups: "sudo" 

Everything is quite simple, we also have a group module for creating groups, with a syntax very similar to apt. After that, it is enough to register this group to the user ( groups: "sudo" ).
It is also useful to add this ssh user a key so that we can log in under it without a password:

 --- - name: Simple playbook # ... tasks: # ... - name: Ensure a 'sudo' group group: name: sudo state: present - name: Add a new user user: name: "{{ user }}" shell: /bin/bash password: "{{ user_password | password_hash('sha512') }}" groups: "sudo" - name: Deploy SSH Key authorized_key: user: "{{ user }}" key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" state: present 

In this case, the interesting construction is "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" - it copies the contents of the file id_rsa.pub (you may have a different name), that is, the public part of the ssh key and loads it into the list of authorized keys for the user on the server.

Roles

All three tasks for creation can be easily attributed to one group of tasks, and it would be nice to keep this group separate from the main playbook so that it would not grow too much. For this, there are roles in ansible.
According to the file structure specified at the very beginning, the role must be put in a separate roles directory, for each role - a separate directory with the same name, inside the tasks, files, templates, etc directory
Create the file structure: ./ansible/roles/user/tasks/main.yml (main is the main file that will be loaded and executed when the role is connected to the playbook, it is possible to include other role files in it). Now you can transfer to this file all tasks related to the user:

 # Create user and add him to groups - name: Ensure a 'sudo' group group: name: sudo state: present - name: Add a new user user: name: "{{ user }}" shell: /bin/bash password: "{{ user_password | password_hash('sha512') }}" groups: "sudo" - name: Deploy SSH Key authorized_key: user: "{{ user }}" key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" state: present 

Basically, the playbook needs to specify the use of the user role:

 --- - name: Simple playbook hosts: all remote_user: root gather_facts: no tasks: - name: Update system apt: update_cache=yes - name: Install system dependencies apt: name: git,nginx,redis,postgresql,postgresql-contrib state: present roles: - user 

It also probably makes sense to perform a system update before all other tasks, for this you can rename the tasks block in which they are defined in pre_tasks .

Nginx configuration

Nginx should be already installed, you need to configure and run it. Let's do it right away in the role. Create a file structure:

 - ansible - roles - nginx - files - tasks - main.yml - templates 

Now we need files and templates. The difference between them is that the files are ansible to copy directly as is. And the templates must have the extension j2 and in them you can use the values ​​of variables using the same double curly braces.

Let's include nginx in the main.yml file. To do this, we have a systemd module:

 # Copy nginx configs and start it - name: enable service nginx and start systemd: name: nginx state: started enabled: yes 

Here we not only say that nginx should be started (ie, run it), but we immediately say that it should be enabled.
Now copy the configuration files:

 # Copy nginx configs and start it - name: enable service nginx and start systemd: name: nginx state: started enabled: yes - name: Copy the nginx.conf copy: src: nginx.conf dest: /etc/nginx/nginx.conf owner: root group: root mode: '0644' backup: yes - name: Copy template my_app.conf template: src: my_app_conf.j2 dest: /etc/nginx/sites-available/my_app.conf owner: root group: root mode: '0644' 

We create the main nginx configuration file (you can take it directly from the server, or write it yourself). And also the configuration file for our application in the sites_available directory (this is not necessarily but useful). In the first case, we use the copy module to copy files (the file must be in /ansible/roles/nginx/files/nginx.conf ). In the second, we copy the template, substituting the values ​​of the variables. The template should be in /ansible/roles/nginx/templates/my_app.j2 ). And it can look something like this:

 upstream {{ app_name }} { server unix:{{ app_path }}/shared/tmp/sockets/puma.sock; } server { listen 80; server_name {{ server_name }} {{ inventory_hostname }}; root {{ app_path }}/current/public; try_files $uri/index.html $uri.html $uri @{{ app_name }}; .... } 

Pay attention to the inserts {{ app_name }} , {{ app_path }} , {{ server_name }} , {{ inventory_hostname }} - these are all variables, the values ​​of which ansible will insert into the template before copying. This is useful if you use a playbook for different groups of hosts. For example, we can supplement our inventory file:

 [production] 123.123.123.123 [staging] 231.231.231.231 [all:vars] user=my_user user_password=123qweasd [production:vars] server_name=production app_path=/home/www/my_app app_name=my_app [staging:vars] server_name=staging app_path=/home/www/my_stage app_name=my_stage_app 

If we start our playbook now, it will perform the specified tasks for both hosts. But at the same time for staging the host variables will be different from production, and not only in roles and playbooks, but also in the nginx configs. {{ inventory_hostname }} not necessary to specify in the inventory file - this is a special variable ansible and the host for which the playbook is currently running is stored there.
If you want to have an inventory file for several hosts, and run only for one group, you can do this with the following command:

 ansible-playbook -i inventory ./playbook.yml -l "staging" 

Another option is to have separate inventory files for different groups. Or you can combine the two approaches, if you have many different hosts.

Let's return to the nginx setup. After copying the configuration files, we need to create a symlink in sitest_enabled on my_app.conf from sites_available. And restart nginx.

 ... # old code in mail.yml - name: Create symlink to sites-enabled file: src: /etc/nginx/sites-available/my_app.conf dest: /etc/nginx/sites-enabled/my_app.conf state: link - name: restart nginx service: name: nginx state: restarted 

Everything is simple - again, modules are ansible with a fairly standard syntax. But there is one moment. Restarting nginx every time does not make sense. You noticed that we do not write commands like: “to do this like this”, the syntax looks more like “this should have this state”. And most often this is exactly what ansible works. If the group already exists, or the system package is already installed, then ansible will check it and skip the task. The same files will not be copied if they completely coincide with what is already on the server. We can take advantage of this and restart nginx only if the configuration files have been changed. There is a register directive for this:

 # Copy nginx configs and start it - name: enable service nginx and start systemd: name: nginx state: started enabled: yes - name: Copy the nginx.conf copy: src: nginx.conf dest: /etc/nginx/nginx.conf owner: root group: root mode: '0644' backup: yes register: restart_nginx - name: Copy template my_app.conf template: src: my_app_conf.j2 dest: /etc/nginx/sites-available/my_app.conf owner: root group: root mode: '0644' register: restart_nginx - name: Create symlink to sites-enabled file: src: /etc/nginx/sites-available/my_app.conf dest: /etc/nginx/sites-enabled/my_app.conf state: link - name: restart nginx service: name: nginx state: restarted when: restart_nginx.changed 

If one of the configuration files changes, then the restart_nginx variable will be copied and registered. And only if this variable has been registered, will the service restart.

And, of course, you need to add the nginx role to the main playbook.

Configure postgresql

We need to enable postgresql with systemd, just like we did with nginx, and also create a user that we will use to access the database and the database itself.
Create the role /ansible/roles/postgresql/tasks/main.yml :

 # Create user in postgresql - name: enable postgresql and start systemd: name: postgresql state: started enabled: yes - name: Create database user become_user: postgres postgresql_user: name: "{{ db_user }}" password: "{{ db_password }}" role_attr_flags: SUPERUSER - name: Create database become_user: postgres postgresql_db: name: "{{ db_name }}" encoding: UTF-8 owner: "{{ db_user }}" 

I will not describe how to add variables to inventory, this has already been done many times, as well as the syntax of the postgresql_db and postgresql_user modules. More data can be found in the documentation. Here the become_user: postgres directive is most interesting. The fact is that by default only postgres user has access to the postgresql database and only locally. This directive allows us to execute commands on behalf of this user (unless of course we have access).
Also, you may have to add a line in pg_hba.conf to allow the new user access to the database. This can be done just as we changed the nginx config.

And of course, you need to add the postgresql role to the main playbook.

Installing ruby ​​via rbenv

In ansible there are no modules for working with rbenv, and it is installed by cloning the git repository. Therefore, this task becomes the most non-standard. Let's create for it the role /ansible/roles/ruby_rbenv/main.yml and begin to fill it:

 # Install rbenv and ruby - name: Install rbenv become_user: "{{ user }}" git: repo=https://github.com/rbenv/rbenv.git dest=~/.rbenv 

We again use the become_user directive to work from under the user we created for these purposes. Since rbenv is installed in its home directory, not globally. And we also use the git module to clone the repository, specifying repo and dest.

Next, we need to register rbenv init in bashrc and add rbenv to PATH in the same place. For this we have a lineinfile module:

 - name: Add rbenv to PATH become_user: "{{ user }}" lineinfile: path: ~/.bashrc state: present line: 'export PATH="${HOME}/.rbenv/bin:${PATH}"' - name: Add rbenv init to bashrc become_user: "{{ user }}" lineinfile: path: ~/.bashrc state: present line: 'eval "$(rbenv init -)"' 

After that, you need to install ruby_build:

 - name: Install ruby-build become_user: "{{ user }}" git: repo=https://github.com/rbenv/ruby-build.git dest=~/.rbenv/plugins/ruby-build 

And finally, install ruby. This is done via rbenv, simply by bash with the command:

 - name: Install ruby become_user: "{{ user }}" shell: | export PATH="${HOME}/.rbenv/bin:${PATH}" eval "$(rbenv init -)" rbenv install {{ ruby_version }} args: executable: /bin/bash 

We say what command to execute and what. However, here we will come across the fact that ansible does not run the code contained in bashrc before running the commands. So, rbenv will have to be defined directly in the same script.

The next problem is that the shell command has no state in terms of ansible. Ie, an automatic check whether this version of ruby ​​is installed or not will not. We can do it on our own:

 - name: Install ruby become_user: "{{ user }}" shell: | export PATH="${HOME}/.rbenv/bin:${PATH}" eval "$(rbenv init -)" if ! rbenv versions | grep -q {{ ruby_version }} then rbenv install {{ ruby_version }} && rbenv global {{ ruby_version }} fi args: executable: /bin/bash 

And it remains to install the bundler:

 - name: Install bundler become_user: "{{ user }}" shell: | export PATH="${HOME}/.rbenv/bin:${PATH}" eval "$(rbenv init -)" gem install bundler 

And again, add our role of ruby_rbenv to the main playbook.

Shared files.

In general, this setting could be completed. Then it remains to run capistrano and it will copy the code itself, create the necessary directories and start the application (if everything is configured correctly). However, often capistrano requires additional configuration files, such as database.yml or .env They can be copied just like files and templates for nginx. There is only one subtlety. Before copying files, you need to create a directory structure for them, something like this:

 # Copy shared files for deploy - name: Ensure shared dir become_user: "{{ user }}" file: path: "{{ app_path }}/shared/config" state: directory 

we specify only one directory and ansible will automatically create the parent if necessary.

Ansible vault

We have already stumbled upon the fact that the variables may be confidential data such as user password. If you created an .env file for an application, and database.yml then there should be even more such critical data. They would be good to hide from prying eyes. For this is used ansible vault .

Create a file for variable /ansible/vars/all.yml (here you can create different files for different groups of hosts, just like in the inventory file: production.yml, staging.yml, etc).
In this file you need to transfer all the variables that should be encrypted using the standard yml syntax:

 # System vars user_password: 123qweasd db_password: 123qweasd # ENV vars aws_access_key_id: xxxxx aws_secret_access_key: xxxxxx aws_bucket: bucket_name rails_secret_key_base: very_secret_key_base 

Then this file can be encrypted with the command:

 ansible-vault encrypt ./vars/all.yml 

Naturally, when encrypting it will be necessary to set a password for decryption. You can see what is inside the file after calling this command.

Using ansible-vault decrypt file can be decrypted, modified and then encrypted again.

To work to decrypt the file is not necessary. You store it in encrypted form and launch the playbook with the argument --ask-vault-pass . Ansible asks for a password, retrieves variables and performs tasks. All data will remain encrypted.

The complete command for several groups of hosts and the ansible vault will look something like this:

 ansible-playbook -i inventory ./playbook.yml -l "staging" --ask-vault-pass 

And I will not give you the full text of playbooks and roles, write yourself. Because the ansible thing is this - if you do not understand what needs to be done, then he will not do it for you.