Geekly Articles each Day
📜 ⬆️ ⬇️

Ansible where to start

Over the past couple of years, I have increasingly been using Ansible to solve almost any automation-related tasks, be it configuration, backup, or deployment projects. Despite the fact that the system is very well documented, I think I can add some useful information for those who are just starting to use Ansible. To begin, I would like to talk about basic things, such as the project structure which will contain playbooks, roles, variables, templates and files necessary to automate the deployment of servers, code, and everything else that can be done using Ansible.

So, Ansible is a very flexible and easy tool for writing automation scripts of any complexity. You can describe in it as a simple developer environment as well as the complex structure of a large project with several environments (dev / stage / prod).

As I see with Ansible, you can solve the following tasks:


A couple of notes for those who are not familiar with Ansible:
- Before you start writing playbooks / roles, you need to read the Ansible-playbook , Ansible-role documentation and learn to understand YAML-syntax (this is very easy);
- You should also immediately say that it is better to develop in the git-repository, so do not be lazy to get an account on github or get a local git-repo (mercurial, svn, etc). If you have not yet learned how to use git, now is the time.

In my opinion, Ansible is much simpler and easier to “digest” than puppet and chef (I used to use both), even though these are slightly different products. After you have read the introduction and looked into the Module Index , you can already start writing playbooks that will make your life much easier.

Example of a playbook for distributing your ssh pub-key to servers:
# file: keys.yml --- - hosts: app-servers tasks: - name: Set up authorized_keys for the backup user #      ,               . authorized_key: user=backup key="{{ item }}" with_file: - keys/backup.pub ... 

 ansible-playbook playbooks/keys.yml

This is all clear, but what next?
')
Extended project structure

If you want to describe the structure of a more or less complex project with an n-tier architecture, you should immediately determine the list of host groups by roles and then determine which tasks should be common to all hosts (for example, connecting repositories, creating users, etc.) , and which ones are private (configuring nginx, mysql, creating python venv, etc.) and proceeding from this, start writing roles starting from the basic common role gradually moving to the private ones.

An extended project structure may include:

List of host groups (inventory)

If you want to have several environments, you will have to create several inventory files, by default (in the deb package), the hosts (/ etc / ansible / hosts) file is used in the ansible.cfg config. The path to inventory is not necessary to register in the config file, you can specify different files with the -i key .

So that the logic described in your roles or playbooks works in the same way on different environments, start groups with the same name for all environments, for example:

Production inventory
 # file: production [app-servers:children] app-php-servers app-python-servers [app-php-servers] appserv-01.example.com appserv-03.example.com [app-python-servers] appserv-02.example.com appserv-04.example.com [db-servers] dbserv-[01:02].example.com [cdn] cdn-[01:03].example.com [app-servers:vars] env=PROD ansible_ssh_user=admin

Develop inventory
 # file: develop [app-servers:children] app-php-servers app-python-servers [app-php-servers] appserv-01.dev.example.com appserv-03.dev.example.com [app-python-servers] appserv-02.dev.example.com appserv-04.dev.example.com [db-servers] dbserv-[01:02].dev.example.com [cdn] cdn-[01:02].dev.example.com [app-servers:vars] env=DEV ansible_ssh_user=admin

As you can see from the example, groups can include other groups, and you can also declare common variables directly in inventory.

Variables

The directory structure may look like this:
 ansible/ #  ()  - group_vars/ - all/ - common - secret - dev/ - common - secret   - stage/ - common - secret - prod/ - common - secret #    () - host_vars/ - appserv-01.example.com/ - common - secret - dbserv-01.example.com/ - common - secret - cdn-01.example.com/ - common - secret

Each group and each host can have a set of variables in different files (for example, common, secret). The file names in directories are not basic here, the main thing is that the directory name matches the names (group or host) in the corresponding inventory. It is not necessary to create a directory for a group or a single host, you can simply create a file and write all the necessary variables in it. But if you want to store passwords in an encrypted form (not hashes, namely passwords), separately from common variables, then you should start the structure described above, I will tell about it below.

Encrypting Variables with Ansible-vault
Ansible-vault encryption utility (default AES256) of files of group or host variables and, in principle, any files in which you want to store secret variables (passwords, keys, etc.). Thus, you can store in the repository (even in public, although it is still not desirable) any data and not worry about their security.

It makes sense to store user passwords in group files (group_vars) by group name or as all if you have the same users on all environments.

In the structure described above, I specified the secret files in which the encrypted variables are supposed to be stored.

Encrypt file:
 ansible-vault encrypt host_vars/cdn-01.example.com/secret

Edit the encrypted file:
 ansible-vault edit host_vars/cdn-01.example.com/secret

Show encrypted file:
 ansible-vault view host_vars/cdn-01.example.com/secret

Of course, to encrypt and decrypt files you will need a long enough key and you need to keep it in a safe place (for example, KeePass). In order to automatically decrypt files during startup (runtime) of the playbooks, you will need to specify the key --vault-password-file or specify the path to the file via the configible ansc.cfg file , in this case you will also need to take care of the integrity of the key and set it with the necessary rights (0400 ). And of course you should not store it in the repository along with the encrypted files.

Playbooks

All actions (tasks) that you want to perform can be recorded in playbooks (if we say no more than a dozen of them and you do not use files and templates) in other cases you should use roles.

At least the following should be indicated in the playbook:
- Target group of hosts ( hosts ) here you can set exceptions by mask (- hosts: app-servers:! App-04. *), Or several groups separated by a colon (-hosts: db-servers: cdn);
- Actions ( tasks ) or roles ( roles ).
Additionally, you can specify:
- User for ssh-connect ( remote_user );
- sudo user ( become_user );
- Use privilege escalation ( become : True / False);
- Variables ( vars )
- Variable files ( vars_files )
- The number of simultaneous connections ( serial ), if you set 1, then the playlist will be executed sequentially one host at a time.

Roles

The role is a structured playbook containing a set (as well as a minimum) task (task), and additionally - event handlers (variables) (variables) (defaults), files (files), templates (templates), as well as description and dependencies (meta).

Deploy role structure
 - deploy/ - defaults/ - main.yml - files/ - maintenance.html - handlers/ - main.yml - meta/ - main.yml - tasks/ - deploy.yml - main.yml - migrations.yml - tests.yml - templates/ - app_config.j2

Default variable file:
 # file: deploy/defaults/main.yml --- branch_name: master mysql_host: localhost ...

Here it is very convenient to set some common things, for example, the default host for the database is localhost, and group_vars / host_vars you can set the necessary hosts for the respective environments.

Event handler file
 # file: deploy/handlers/main.yml --- - name: reload uwsgi service: name=uwsgi state=reloaded - name: flush cache shell: redis-cli flushall ...

In the handlers, you can specify absolutely any actions that must be performed after certain tasks, whether it is laying out a new code or changing the config.

Task Description File
 # file: deploy/tasks/main.yml --- - include: deploy.yml tags: - common - deploy - include: migrations.yml tags: - common - migrations - include: pytasks.yml when: "'app-python-servers' in group_names" tags: - common - pytasks - include: tests.yml tags: - tests ...

You can set stringent conditions for performing some tasks. In the example above, I set the condition:
 when: "'app-python-servers' in group_names"

which means that the task will only be executed for hosts that are members of the app_python-servers group.

I almost always use tags to decompose large roles. In the role example above, I set the common tag “common” for the first two tasks and a separate tag for tests for the last task. If I need to perform only the first two tasks (i.e., roll out the code), I launch the softbook with the key - tags common (or --skip-tags tests):
 ansible-playbook -i develop playbooks/deploy.yml --tags common

then, you can run the tests separately (usually they take a long time to execute) just by asking - tags tests:
 ansible-playbook -i develop playbooks/deploy.yml --tags tests

Templates
As a template engine Ansible uses jinja2 with all its charms. An example of a simple config uwsgi in the form of an ini file with variables:
 # {{ ansible_managed }} [uwsgi] chdir = {{ app_dir }}/{{ app_name }} home = {{ app_dir }}/{{ app_name }}/env master = True module = {{ app_name }}.wsgi:application chmod-socket = 660 uid = {{ web_user }} gid = {{ web_user }} processes = 8 listen = 256 max-requests = 100 buffer-size = 16384 vacuum = true env DJANGO_SETTINGS_MODULE = {{ env }}.settings {% if env is defined and env == DEV %} # Enable logging req-logger = file:/{{ logs_dir }}/{{ app_name }}/reqlog logger = file:/{{ logs_dir }}/{{ app_name }}/errlog {% endif %}

You can use any of the jinja2 techniques in conjunction with the Ansible variables. For example:
- Set the string, only the host is in one group or another (or set the string only for a specific host) and still use the loop :
 # file: iptables.j2 ... {% if (groups['cdn'] is defined and inventory_hostname in groups['cdn']) %} # Allows HTTP and HTTPS connections from anywhere {% for port in webs_ports %} -A INPUT -p tcp -m tcp --dport {{ port }} -j ACCEPT {% endfor %} {% endif %} ...

Dependencies

Dependencies here mean other roles that need to be fulfilled before fulfilling your final role, in the format - {role: role_name, var_name: value}, for example:
 # file: deploy/meta/main.yml --- dependencies: - { role: base, apt_repo: us }

Having specified all the necessary dependencies for the final role (for example, app-servers), you can create just one playbook in which it suffices to specify the target host group and role. And this will be enough to keep the system in a consistent state, for example launching a playbook on the crown. If you need to update any specific configs in place, then tags will come to your rescue.

If you want to make periodic builds and monitor their implementation, or just give a “button” to developers, I can recommend the Ansible plugin for Jenkins by connecting which you can set the Jenkins task paths to the playbook, inventory, and also tags and extra-vars.

What's next?

The structure described in the article does not pretend to be a reference or universal, but it is certainly suitable for most cases. You can use it as a starting point. In the process of writing your own roles, you will find a better and more convenient way for you to describe the infrastructure you are building. The main thing is not to try to complicate things, the easier you write and the clearer your playbooks for others the better.

Keep an open tab with official documentation on Ansible, it is constantly updated as the project develops. I still find new and interesting things in it.

In this article I did not tell everything I wanted, I have thoughts and ideas for one or two notes. If you are interested in certain questions, write them in the comments or email me, I will try to answer and maybe consider your wishes when writing the next article.

UPD:
A useful example of the separation of environments (production / staging) is here . Thanks shuron for the link.

Source: https://habr.com/ru/post/304732/

More articles:


All Articles