Expanding Ansible functionality with modules

Under the hood of the d2c.io service , we actively use Ansible — from creating virtual machines in the clouds of providers and installing the necessary software, to managing Docker containers with client applications.

In the article on expanding the functionality of Ansible, we partially considered the difference between plugins and modules. In short, the main difference is that the former are performed on the local machine where Ansible is installed, and the latter are on the target ones.

The main task of plug-ins is to influence the progress of the playbook, add new possibilities of loading and processing data. The task of the modules is to expand the list of systems and services that Ansible can manage. For example, to create a server on the Vultr site - the vultr module, to create a user in a self-made authorization system for an office WiFi network - the mywifiauth_user module.

The principle of the modules

A module is a small program that:

• executed on target host
• can accept parameters as input (via parameter file)
• reports on its work to standard output in JSON format

The process of module execution looks like this:

• From the queue of execution Ansible takes the following task. Specifies the name of the module to use.
• If there is a action-plugin of the same name, executes it ( see article about plugins part 1 ).
  The plugin can do the preparatory work. For example, it will copy files from the management machine to the target host in advance.
• Prepares a parameter file for a module
• Depending on the type of module:
  
  • for modules that do not use “Ansible Framework”: copies the parameters file and the executable file of the module to the target host.
  • for modules based on AnsibleModule : forms and delivers to the target host a self-extracting Python file containing all the necessary Ansible auxiliary classes, a parameter file and a module file itself. Such a “package” is necessary for the operation of the pipelining mode ( see the article about Ansible acceleration ).
• Launches a module or “package” on a remote host.
• The module performs useful work.
• Ansible receives the result of the module as a JSON object from the standard output.

Most often, the target hosts are remote machines, servers and devices: for example, the user module manages users on the same host on which it is running. Some modules, on the contrary, are more often run on the local host using connection: local , local_action or delegate_to: localhost . A striking example of this is cloud management modules, such as ec2 . They require setting the credentials, which are most often available on the management machine. The wait_for module to wait for the opening of TCP ports also often runs on the local machine. It is used, for example, to wait until the remote server restarts and an SSH connection becomes available.

Simplest module

Modules can be created in any language. Starting with Ansible 2.2 modules can be binary executable files. Let's make the simplest module on bash :

 #!/bin/bash echo '{"changed":false,"date":"'$(date)'"}' 

Save this code in the file ./library/bash_mod.sh and check:

 $ ansible localhost -m bash_mod localhost | SUCCESS => { "changed": false, "date": ", 6  2017 . 17:00:32 (MSK)" } 

Ansible receives information about the results of the modules from their standard output. All output must be a valid JSON object (therefore, you cannot debug modules using print statements) . Depending on the value of the service properties, Ansible can make different decisions. One of these properties: changed . For example, you can change its value in bash_mod from false to true and see that Ansible now considers that your module has changed something on the target host, the output has turned yellow.

Input parameters

There are several ways to get input parameters for a module:

• Through a file with key=value pairs, separated by spaces. Used for modules in interpreted languages. The path to the file with parameters is passed to the module as the only command line parameter. For example, for our module on bash .
• Through a file with a JSON object. Used for binary modules, modules based on AnsibleModule and modules in interpreted languages, the body of which contains the word WANT_JSON . For example, in our bash module you can add the comment # WANT_JSON .
• Through injecting a JSON object into a file. Used for modules in interpreted languages, the body of which has a marker <<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>> . Before sending the module to the target host, this string will be replaced with a JSON object with parameters.

If you create a module based on AnsibleModule , then there is no need to take care of how parameters are passed - the base class will do all the underhood work.

Class AnsibleModule

To simplify the development of custom modules in Ansible there is a class AnsibleModule . It takes on the processing of parameters, checking their types and valid values, and also provides a set of auxiliary methods for working with files, checksums and other things.

As a basic example, look at the ping module code:

 from ansible.module_utils.basic import AnsibleModule def main(): module = AnsibleModule( argument_spec=dict( data=dict(required=False, default=None), ), supports_check_mode=True ) result = dict(ping='pong') if module.params['data']: if module.params['data'] == 'crash': raise Exception("boom") result['ping'] = module.params['data'] module.exit_json(**result) if __name__ == '__main__': main() 

A module is described that has the only valid, but optional, parameter data . It supports verification mode, the --check key when starting Ansible. As a result, returns pong or the value of the data parameter. If to transfer crash as data, the module "falls" with an error.

For more complex examples, you can refer to the code of other modules . In this regard, it is convenient that Ansible is an open source project.

Module example

One of the convenient cases for writing a module is a wrapper for shell commands. If you can do some operation on the target host via the command line using the shell / command modules, but you need to use it often in different playbooks and you want to make the code beautiful and readable. I will analyze a little mythical, but no less working, an example of setting the volume level in the operating system:

 #!/usr/bin/python # -*- coding: utf-8 -*- DOCUMENTATION = ''' --- module: osx_volume short_description: Set OS X volume level description: - Set OS X volume level or mute flag options: level: description: - Volume level to be applied aliases: - volume required: false muted: description: - Set mute on/off required: false author: - Konstantin Suvorov ''' EXAMPLES = ''' - name: Set volume to 25 osx_volume: level: 25 - name: Mute osx_volume: muted: yes ''' from ansible.module_utils.basic import AnsibleModule from subprocess import call, check_output def get_volume(): level = check_output(['osascript','-e','output volume of (get volume settings)']).strip() muted = check_output(['osascript','-e','output muted of (get volume settings)']).strip() muted = (muted.lower() == "true") return (int(level), muted) def set_volume(level=None, muted=None): if level is not None: call(['osascript','-e','set volume output volume {}'.format(level)]) if muted is not None: mute_str = 'true' if muted else 'false' call(['osascript','-e','set volume output muted {}'.format(mute_str)]) return get_volume() def main(): module = AnsibleModule( argument_spec=dict( level=dict(type='int', required=False, default=None, aliases=['volume']), muted=dict(type='bool', required=False, default=None) ), supports_check_mode=True ) req_level = module.params['level'] req_muted = module.params['muted'] l, m = get_volume() result = dict(level=(req_level if req_level is not None else l), muted=(req_muted if req_muted is not None else m), changed=False) if req_level is not None and l != req_level: result['changed'] = True elif req_muted is not None and m != req_muted: result['changed'] = True if module.check_mode or not result['changed']: module.exit_json(**result) new_l, new_m = set_volume(level=req_level, muted=req_muted) if req_level is not None and new_l != req_level: module.fail_json(msg="Failed to set requested volume level {} (actual {})!".format(req_level, new_l)) if req_muted is not None and new_m != req_muted: module.fail_json(msg="Failed to set requested mute flag {} (actual {})!".format(req_muted, new_m)) module.exit_json(**result) if __name__ == '__main__': main() 

I will not disassemble the module code in detail - you can see it yourself. The module adjusts the volume level, controls the mute mode in MacOS. Supports dry-run mode. It will display the status changed=true if the values ​​should change. The module is idempotent, and if you apply it twice with the same parameters, it will not do anything and will display the status changed=false .

Debugging modules

An option without any preparatory work - run Ansible with the ANSIBLE_KEEP_REMOTE_FILES=1 setting ANSIBLE_KEEP_REMOTE_FILES=1 and the -vvv logging -vvv . In this case, Ansible will not delete the generated module and parameter files, but will leave them in a temporary folder on the target host. Advanced logging will allow you to see the path of the directory in which the files are located.

We go to the target host on SSH, go to the desired folder, for example, cd /tmp/ansible-tmp-1488291604.43-129413612218427 . Now we can locally run, modify and debug our module. If the module is written based on AnsibleModule , then it can be run with debugging commands:

• explode - unpack the "archive" in the folder for further modification
• execute - launch the module from the folder obtained in the previous step

For example, if we debug the ping module in this way, then:

• ./ping.py - run the module from the "package"
• ./ping.py explode - will unpack the module and parameters in the debug_dir folder
• ./ping.py execute - run the module from the debug_dir debug_dir (with all the changes we make there)

Another option is to use the test-module utility. It allows you to prepare the file with the parameters and the "packaging" of the module in the same way as Ansible does in real work. This option allows you to test modules locally and much faster than through Ansible; easier to connect debugger.

Module Distribution

In order for Ansible to “see” the module, it must be located on the local machine in the module search paths. By default, this is the ./library directory next to the playbook, but this setting can be changed in the configuration file or through environment variables.

If you are using roles, there may also be a library folder with your module inside the role, for example ./roles/myrole/library/mymodule.py . In this case, if the role myrole was applied in the myrole , then mymodule will become available. The role can even be empty, without the tasks/main.yml .

Documenting Modules

Modules are useful for documenting! If the documentation inside Python modules is described in the DOCUMENTATION and EXAMPLES variables in accordance with a specific format (see example above), then information about your module can be conveniently viewed using the ansible-doc utility from the standard Ansible package.

In addition to the reference information about the module, this utility can generate "snippets" for inserting into the code of playbooks, for example:

 $ ansible-doc -s postgresql_db - name: Add or remove PostgreSQL databases from a remote host. action: postgresql_db encoding # Encoding of the database lc_collate # Collation order (LC_COLLATE) to use in the data lc_ctype # Character classification (LC_CTYPE) to use in t login_host # Host running the database login_password # The password used to authenticate with login_unix_socket # Path to a Unix domain socket for local connecti login_user # The username used to authenticate with name= # name of the database to add or remove owner # Name of the role to set as owner of the databas port # Database port to connect to. ssl_mode # Determines whether or with what priority a secu ssl_rootcert # Specifies the name of a file containing SSL cer state # The database state template # Template used to create the database 

Well, it's time to complete the article on modules. Given the previous articles, you can now extend the functionality of Ansible in all directions! If you have any questions, ask in the comments - I will try to answer or prepare another article. Stay tuned!