Automate and speed up the process of setting up cloud servers with Ansible. Part 5: local_action, conditions, cycles and roles
In the first part, we began the study of Ansible, a popular tool for automating the configuration and deployment of IT infrastructure. Ansible was successfully installed in InfoboxCloud , described the principles of operation, the basic setting. At the end of the article we showed how to quickly install nginx on multiple servers.
In the second part, we sorted out the output of the playbook, learned how to debug and reuse the Ansible scripts.
In the third part, we learned how to write a single Ansible playbook for different operating systems (for example, with rpm and deb), how to maintain hundreds of hosts and not write them all in inventory, and how to group servers by InfoboxCloud regions. The use of the Ansible variables and the inventory file was examined.
')
In the fourth part, we learned how to use Ansible modules to configure the server: figured out how to run the most common scripts on remote servers in InfoboxCloud , use templating for configuration files, substituting the necessary variables, and how to use version control systems to get the code on the server.
In this part, we will look at how to run a task locally as part of a playbook for remote servers, how to use conditions to perform specific tasks only in a certain situation, how to use cycles to significantly reduce the number of tasks in a playbook. In conclusion, we will discuss how to organize a playbook in a role.
Sometimes tasks need to be run on a local machine as part of a playbook for remote servers. For example, on the Ansible – server, you can register access keys via the API to the cloud and issue commands to the command line utility to create new cloud servers. It may often be necessary to send requests to the REST API via the uri Ansible module. There is an opportunity to do something directly on the Ansible – server for a separate task in the playbook, where the hosts are remote servers.
Suppose you want to run a shell – module on the server from where you run Ansible. For this, the local_action option is useful , which will launch the module locally.
Processes on remote machines.
Processes on the local machine.
We see that the execution of the command is redirected to the local machine.
Thus, you can run any Ansible module with local_action.
Ansible performs all tasks sequentially. However, for a complex playbook with dozens of tasks, you may need, depending on the situation, to run only a part of the tasks. Earlier we considered the situation when using variables we correctly installed Apache on rpm and deb distributions. Similarly, you can specify conditions for performing tasks with when :
If the OS is of the RedHat family, the httpd package will be installed via yum, and if the Debian family is installed, apache2 will be installed via apt. ansible_os_family - Ansible variable obtained at the gather_facts stage.
In the playbook above, we used sudo: yes, meaning that the user has sudo rights. Let's check if this is true:
In the example above, we ran the command on the server / usr / bin / sudo -v and saved its output to a variable through register . The variable captured the output stdout and stderr (rc, return code). In the second task, we checked the contents of the return code variable and if an error occurred, we must complete the execution of the playbook with a message.
For comparison, in Ansible conditions, you can use == (equal),! = (Not equal),> (greater), <(less),> = (greater than), <= (less equal).
If you need to check if a variable contains a character or a string, use the in and not operators.
You can specify multiple conditions using the and (and) and or (or) operators.
You can also check the logical value of a variable. Let's make backup if the backup variable is set to true:
Ansible allows the condition to use information about whether a variable has already been defined. To do this, use when: var is not define (where var is the name of a variable, is not define - has not yet been defined, is defined - has already been defined).
It happens that you need to install several packages at once on the server. But writing many tasks for this can turn into a real nightmare. The problem will solve the use of cycles.
Using standard cycles, you can submit a list of packages for installation and Ansible will launch a task for all specified packages.
In the example above, we used the “with_items:” construct to set variables and used the default variable item. At each iteration, item takes the following value as specified in with_items.
The task runs once, but apt is called for all the specified packages. You can also use with_items as a dictionary instead of strings:
Nested loops are useful when you want to perform several operations on the same resource. For example, if you want to provide access to multiple databases for MySQL users.
In the example above, we use the mysql_user module to set permissions on databases and use nested loops with two lists: a list of users and a list of databases. Ansible will launch the mysql_user module for the alexey user, give rights to all the databases specified in the second list, then launch it for the user alexander and also give rights.
In the previous example, we assigned all specified databases to all specified users. But what to do if for each user you need to assign a specific set of databases? For this we need cycles on subelements.
We have created dictionaries that consist of user names and database names. Instead of adding user data to the playbook, you can put it in a separate variable file and include it in the playbook. Ansible will go through the dictionary using the item variable. Ansible assigns numeric values to the keys represented by the with_subelements construct, starting at 0. In the dictionary 0, the name is the “key-value” pair, so we use item.0.name to refer by user name. Dictionary is a simple list, so item.1 is used for reference.
When designing an architecture, they usually operate on server roles: a web server, a database server, a load balancer, and so on. Each role includes a specific set of software for installation and configuration. As your system grows, components that can be reused will gradually be distinguished. Ansible roles provide a convenient way to organize your playbook. Based on the predefined file structure, role components will be loaded. In fact, the roles are simply magic around include (imports), facilitating the preparation of the playbook.
Typical playbook structure with roles:
The file structure of roles will look like this:
If there is no directory in the role, it will be ignored and the playbook will be executed. It is not necessary that you have all the elements and directories of the playbook.
The rules used for each role are:
In the Ansible configuration file, you can set the roles_path (directory with roles). This can be useful if you have a playbook in one repository, and the roles in another. You can set several paths to roles at once through a colon:
In a role, you can pass variables or use conditions:
Earlier in the articles we did not consider tags. With their help, you can run the marked part of the playbook.
With tasks, the use of tags looks like this:
You can start the playbook part like this: ansible-playbook example.yml --tags "configuration, packages" or skip the part execution like this: ansible-playbook example.yml --skip-tags "notification".
So tags can also be used when specifying roles:
You can specify which tasks should be completed before and after the role:
Role dependencies allow you to automatically execute dependent roles when launching specific roles that have dependencies. Dependencies are stored in roles / x / meta / main.yml. Parameters can be passed along with dependent roles. The path to the roles can be specified both in abbreviated form and in full. A version control repository can also be used.
If the dependencies specify the same role several times, it will start only once. If you need several times, you can explicitly request this in the dependency file.
Ansible Galaxy - Ansible role repository. From this resource, you can use ready-made Ansible roles or add your own.
The book " Learning Ansible " and of course the official documentation helped a lot in writing the article.
It is convenient to conduct all experiments with Ansible in InfoboxCloud , since there is an opportunity for each virtual server to set the exact amount of resources needed for the task (CPU / Ram / disk independently of each other) or to use autoscaling, and not to choose VM from ready-made templates. When experiments are not conducted - you can simply turn off the VM and pay only the cost of the disk.
If you find an error in the article, the author will gladly correct it. Please write in the LAN or in the mail about it. There you can also ask questions about Ansible for coverage in subsequent articles. If you can not write comments on Habré, you can leave them in the InfoboxCloud Community .
Successful work!
In the second part, we sorted out the output of the playbook, learned how to debug and reuse the Ansible scripts.
In the third part, we learned how to write a single Ansible playbook for different operating systems (for example, with rpm and deb), how to maintain hundreds of hosts and not write them all in inventory, and how to group servers by InfoboxCloud regions. The use of the Ansible variables and the inventory file was examined.
')
In the fourth part, we learned how to use Ansible modules to configure the server: figured out how to run the most common scripts on remote servers in InfoboxCloud , use templating for configuration files, substituting the necessary variables, and how to use version control systems to get the code on the server.
In this part, we will look at how to run a task locally as part of a playbook for remote servers, how to use conditions to perform specific tasks only in a certain situation, how to use cycles to significantly reduce the number of tasks in a playbook. In conclusion, we will discuss how to organize a playbook in a role.
Run tasks locally with local_action
Sometimes tasks need to be run on a local machine as part of a playbook for remote servers. For example, on the Ansible – server, you can register access keys via the API to the cloud and issue commands to the command line utility to create new cloud servers. It may often be necessary to send requests to the REST API via the uri Ansible module. There is an opportunity to do something directly on the Ansible – server for a separate task in the playbook, where the hosts are remote servers.
Suppose you want to run a shell – module on the server from where you run Ansible. For this, the local_action option is useful , which will launch the module locally.
--- - hosts: experiments remote_user: root tasks: - name: check running processes on remote system shell: ps register: remote_processes - name: remote running processes debug: msg="{{ remote_processes.stdout }}" - name: check running processes on local system local_action: shell ps register: local_processes - name: local running processes debug: msg="{{ local_processes.stdout }}" Processes on remote machines.
Processes on the local machine.
We see that the execution of the command is redirected to the local machine.
Thus, you can run any Ansible module with local_action.
We work with conditions
Ansible performs all tasks sequentially. However, for a complex playbook with dozens of tasks, you may need, depending on the situation, to run only a part of the tasks. Earlier we considered the situation when using variables we correctly installed Apache on rpm and deb distributions. Similarly, you can specify conditions for performing tasks with when :
--- - hosts: experiments remote_user: root tasks: - name: Install httpd package yum: name=httpd state=latest sudo: yes when: ansible_os_family == "RedHat" - name: Install apache2 package apt: name=apache2 state=latest sudo: yes when: ansible_os_family == "Debian" If the OS is of the RedHat family, the httpd package will be installed via yum, and if the Debian family is installed, apache2 will be installed via apt. ansible_os_family - Ansible variable obtained at the gather_facts stage.
In the playbook above, we used sudo: yes, meaning that the user has sudo rights. Let's check if this is true:
--- - hosts: experiments remote_user: root tasks: - name: Testing user sudo privilege command: /usr/bin/sudo -v register: sudo_response ignore_errors: yes - name: Stop if Users doesn`t have sudo privilege fail: msg="User doesn`t have sudo privilege" when: sudo_response.rc == 1 In the example above, we ran the command on the server / usr / bin / sudo -v and saved its output to a variable through register . The variable captured the output stdout and stderr (rc, return code). In the second task, we checked the contents of the return code variable and if an error occurred, we must complete the execution of the playbook with a message.
For comparison, in Ansible conditions, you can use == (equal),! = (Not equal),> (greater), <(less),> = (greater than), <= (less equal).
If you need to check if a variable contains a character or a string, use the in and not operators.
- name: Querying rpm list for httpd package shell: rpm -qa | grep httpd register: httpd_rpm - name: Check if httpd rpm is installed on the remote host debug: msg="httpd is installed on the remote host" when: "'httpd-2.2.27–1.2.x86_64' in httpd_rpm.stdout" – name: Check if httpd rpm is not installed on the remote host debug: msg="httpd is not installed on the remote host" when: not 'httpd-2.2.27.1.2.x86_64' in httpd_rpm.stdout You can specify multiple conditions using the and (and) and or (or) operators.
– name: Check if httpd rpm is installed on the remote host debug: msg="httpd is installed on the remote host" when: "'httpd-2.2.27–1.2.x86_64' in httpd_rpm.stdout and 'httpd-tools-2.2.27–1.2.x86–64' in httpd_rpm.stdout" You can also check the logical value of a variable. Let's make backup if the backup variable is set to true:
– name: Rsync shell: /usr/bin/rsync -ra /home /backup/{{ inventory_hostname}} sudo: yes when: backup Ansible allows the condition to use information about whether a variable has already been defined. To do this, use when: var is not define (where var is the name of a variable, is not define - has not yet been defined, is defined - has already been defined).
We work with cycles
It happens that you need to install several packages at once on the server. But writing many tasks for this can turn into a real nightmare. The problem will solve the use of cycles.
Standard cycles
Using standard cycles, you can submit a list of packages for installation and Ansible will launch a task for all specified packages.
--- - hosts: experiments remote_user: root tasks: – name: Install nginx package yum: name={{ item }} state=latest with_items: – nginx – htop sudo: yes In the example above, we used the “with_items:” construct to set variables and used the default variable item. At each iteration, item takes the following value as specified in with_items.
The task runs once, but apt is called for all the specified packages. You can also use with_items as a dictionary instead of strings:
with_items: – {name: 'httpd', state: 'latest'} – {name: 'htop', state: 'absent'} Nested loops
Nested loops are useful when you want to perform several operations on the same resource. For example, if you want to provide access to multiple databases for MySQL users.
––– – hosts: experiments remote_user: root tasks: – name: give users access to multiple databases mysql_user: name={{ item[0] }} priv={{ item[1]}}.*:ALL append_privs=yes password=pass login_user=root login_password=root with_nested: – ['alexey', 'alexander'] – ['clientdb', 'providerdb'] In the example above, we use the mysql_user module to set permissions on databases and use nested loops with two lists: a list of users and a list of databases. Ansible will launch the mysql_user module for the alexey user, give rights to all the databases specified in the second list, then launch it for the user alexander and also give rights.
Cycles by subitems
In the previous example, we assigned all specified databases to all specified users. But what to do if for each user you need to assign a specific set of databases? For this we need cycles on subelements.
--- - hosts: experiments remote_user: root vars: users: – name: alexey database: – clientdb – providerdb – name: alexander database: – providerdb tasks: – name: give users access to multiple databases mysql_user: name={{ item.0.name }} priv={{ item.1 }}.*:ALL append_privs=yes password=pass login_user=root login_password=root with_subelements: – users - database We have created dictionaries that consist of user names and database names. Instead of adding user data to the playbook, you can put it in a separate variable file and include it in the playbook. Ansible will go through the dictionary using the item variable. Ansible assigns numeric values to the keys represented by the with_subelements construct, starting at 0. In the dictionary 0, the name is the “key-value” pair, so we use item.0.name to refer by user name. Dictionary is a simple list, so item.1 is used for reference.
Working with roles
When designing an architecture, they usually operate on server roles: a web server, a database server, a load balancer, and so on. Each role includes a specific set of software for installation and configuration. As your system grows, components that can be reused will gradually be distinguished. Ansible roles provide a convenient way to organize your playbook. Based on the predefined file structure, role components will be loaded. In fact, the roles are simply magic around include (imports), facilitating the preparation of the playbook.
Typical playbook structure with roles:
--- - hosts: webservers roles: - common - web – db The file structure of roles will look like this:
site.yml webservers.yml roles/ common/ files/ templates/ tasks/ handlers/ vars/ defaults/ meta/ web/ files/ templates/ tasks/ handlers/ vars/ defaults/ meta/ db/ files/ templates/ tasks/ handlers/ vars/ defaults/ meta/ If there is no directory in the role, it will be ignored and the playbook will be executed. It is not necessary that you have all the elements and directories of the playbook.
The rules used for each role are:
- If roles / x / tasks / main.yml exists, tasks will be added to the playbook execution process.
- If roles / x / handlers / main.yml exists, event handlers will be added to the playbook execution process.
- If roles / x / vars / main.yml exists, variables will be added to the playbook execution process.
- If roles / x / meta / mail.yml exists, any dependency roles will be added to the role list. (In meta, you can specify a list of roles that must be applied before a specific role so that it is applied correctly).
- Any copy task can reference a file in roles / x / files without specifying an absolute or relative path.
- Any script task can reference scripts in roles / x / files without specifying an absolute or relative path.
- Any template task can refer to roles / x / templates without specifying an absolute or relative path.
- Any imported tasks can reference task files in the roles / x / tasks directory without specifying an absolute or relative path.
In the Ansible configuration file, you can set the roles_path (directory with roles). This can be useful if you have a playbook in one repository, and the roles in another. You can set several paths to roles at once through a colon:
roles_path = /opt/mysite/roles:/opt/othersite/roles In a role, you can pass variables or use conditions:
--- - hosts: experiments roles: – common – {role: web, dir: '/var/www', port: 80} – {role: repository, when: "ansible_os_family =='RedHat'"} Earlier in the articles we did not consider tags. With their help, you can run the marked part of the playbook.
With tasks, the use of tags looks like this:
tasks: - apt: name={{ item }} state=installed with_items: - httpd - htop tags: - packages - template: src=templates/src.j2 dest=/var/www/.htaccess tags: - configuration You can start the playbook part like this: ansible-playbook example.yml --tags "configuration, packages" or skip the part execution like this: ansible-playbook example.yml --skip-tags "notification".
So tags can also be used when specifying roles:
--- - hosts: experiments roles: - { role: web, tags: ["apache", "simple"] } You can specify which tasks should be completed before and after the role:
--- - hosts: experiments pre_tasks: - shell: echo 'hello, habr' roles: - { role: web } tasks: - shell: echo 'still busy' post_tasks: - shell: echo 'goodbye, habr' Role dependencies
Role dependencies allow you to automatically execute dependent roles when launching specific roles that have dependencies. Dependencies are stored in roles / x / meta / main.yml. Parameters can be passed along with dependent roles. The path to the roles can be specified both in abbreviated form and in full. A version control repository can also be used.
--- dependencies: - { role: common, some_parameter: 3 } - { role: '/path/to/common/roles/foo', x: 1 } - { role: 'git+http://git.example.com/repos/role-foo,v1.1,foo' } If the dependencies specify the same role several times, it will start only once. If you need several times, you can explicitly request this in the dependency file.
Ansible galaxy
Ansible Galaxy - Ansible role repository. From this resource, you can use ready-made Ansible roles or add your own.
Conclusion
The book " Learning Ansible " and of course the official documentation helped a lot in writing the article.
It is convenient to conduct all experiments with Ansible in InfoboxCloud , since there is an opportunity for each virtual server to set the exact amount of resources needed for the task (CPU / Ram / disk independently of each other) or to use autoscaling, and not to choose VM from ready-made templates. When experiments are not conducted - you can simply turn off the VM and pay only the cost of the disk.
If you find an error in the article, the author will gladly correct it. Please write in the LAN or in the mail about it. There you can also ask questions about Ansible for coverage in subsequent articles. If you can not write comments on Habré, you can leave them in the InfoboxCloud Community .
Successful work!
Source: https://habr.com/ru/post/252461/
All Articles