Hello!

I work as a DevOps-engineer in the hotel reservation service Ostrovok.ru . In this article I want to talk about our experience in testing ansible roles.

In Ostrovok.ru, we use ansible as a configuration manager. Recently, we came to the need for testing roles, but as it turned out, there are not so many tools for this - the most popular is probably the Molecule framework, so we decided to use it. But it turned out that his documentation is silent about many pitfalls. We couldn’t find enough detailed guidance in Russian, so we decided to write this article.

Molecule

Molecule - a framework to assist in testing ansible roles.

Simplified description: The molecule creates an instance on the platform you specified (cloud, virtual machine, container; see the Driver section for details), runs your role on it, then runs tests and deletes the instance. In case of failure at one of the steps, the molecule will inform you about it.

Now more.

A bit of theory

Consider two key entities Molecules: Scenario and Driver.

Scenario

The script contains a description of what, where, how and in what sequence will be executed. A single role can have several scenarios, and each one is a directory along the <role>/molecule/<scenario> path, containing descriptions of the actions required for the test. There must be a default script that will be automatically created if you initialize the role using the Molecule. The names of the following scenarios are chosen at your discretion.

The sequence of testing actions in a script is called matrix , and by default it is:

(Steps marked with ? skipped by default if not described by the user)

• lint - linter run. The default is yamllint and flake8 ,
• destroy - deleting instances from the last launch of the Molecule (if left),
• dependency ? - setting the ansible dependence of the tested role,
• syntax - role syntax checking using ansible-playbook --syntax-check ,
• create - create an instance
• prepare ? - preparation of the instance; for example, check / install python2
• converge - launch the tested playbook,
• idempotence - restarting the playbook for the idempotency test,
• side_effect ? - actions not directly related to the role, but necessary for the tests,
• verify - run tests of the resulting configuration using testinfra (default) / goss / inspec ,
• cleanup ? - (in new versions) - roughly speaking, “cleaning” the external infrastructure, which has been hit by a molecule,
• destroy - delete an instance.

This sequence covers most cases, but, if necessary, it can be changed.

Each of the above steps can be launched separately using a molecule <command> . But it should be understood that for each such cli-command there may exist its own sequence of actions, which you can find out by executing the molecule matrix <command> . For example, when you run the converge command (the test player runs), the following actions will be performed:

 $ molecule matrix converge ... └── default #   ├── dependency #   ├── create #   ├── prepare #   └── converge #   

The sequence of these actions can be edited. If something from the list has already been completed, it will be skipped. Molecule stores the current state, as well as the configuration of the instances, in the $TMPDIR/molecule/<role>/<scenario> directory.

Add steps with ? You can, by describing the desired actions in the format of ansible-playbook, and the file name to do, respectively, step: prepare.yml / side_effect.yml . Expect these files The molecule will be in the folder of the script.

Driver

A driver is an entity where test instances are created.
The list of standard drivers for which Molecules are ready for templates is: Azure, Docker, EC2, GCE, LXC, LXD, OpenStack, Vagrant, Delegated.

In most cases, templates are the create.yml and destroy.yml in the script folder, which describe the creation and deletion of an instance, respectively.
Exceptions are Docker and Vagrant, since interactions with their modules can occur without the above files.

It is worth highlighting the Delegated driver, since if it is used in the creation and deletion of an instance file, only work with instance configuration is described, the rest should be described by the engineer.

The default driver is Docker.

We now turn to practice and consider further features there.

Beginning of work

As the "hello world" we will test the simple role of installing nginx. As a driver, choose the docker - I think it is installed by most of you (and remember that the docker is the default driver).

Prepare virtualenv and set the molecule in it:

 > pip install virtualenv > virtualenv -p `which python2` venv > source venv/bin/activate > pip install molecule docker # molecule  ansible  ; docker   

The next step is to initialize the new role.
A new role, like a new script, is initialized using the command molecule init <params> :

 > molecule init role -r nginx --> Initializing new role nginx... Initialized role in <path>/nginx successfully. > cd nginx > tree -L 1 . ├── README.md ├── defaults ├── handlers ├── meta ├── molecule ├── tasks └── vars 6 directories, 1 file 

The result was a typical ansible role. Further, all interactions with CLI Molecules are made from the root of the role.

Let's see what is in the role directory:

 > tree molecule/default/ molecule/default/ ├── Dockerfile.j2 # Jinja-  Dockerfile ├── INSTALL.rst. #       ├── molecule.yml #   ├── playbook.yml #    └── tests #     verify └── test_default.py 1 directory, 6 files 

Let's analyze the configuration of molecule/default/molecule.yml (replace only docker image):

 --- dependency: name: galaxy driver: name: docker lint: name: yamllint platforms: - name: instance image: centos:7 provisioner: name: ansible lint: name: ansible-lint scenario: name: default verifier: name: testinfra lint: name: flake8 

dependency

This section describes the source of dependencies.

Options: galaxy , gilt , shell.

Shell is just a command shell that is used if galaxy and gilt do not cover your needs.

I will not stay here for a long time, it is described in the documentation sufficiently.

driver

The name of the driver. We have this docker.

lint

Yamllint is used as a linter.

Useful options in this part of the config are the ability to specify the configuration file for yamllint, forward the environment variables or disable the linter:

 lint: name: yamllint options: config-file: foo/bar env: FOO: bar enabled: False 

platforms

Describes the configuration of instances.
In the case of the docker as a driver, the molecule is iterated over this section, and each element of the list is available in Dockerfile.j2 as the item variable.

In the case of the driver, in which create.yml and destroy.yml , the section is available in them as molecule_yml.platforms , and the iterations on it are already described in these files.

Since the Molecule provides instance management for ansible modules, a list of possible settings should be searched there. For the docker, for example, the docker_container_module module is used . What modules are used in other drivers can be found in the documentation .

As well as examples of the use of various drivers can be found in the tests of the Molecule itself .

Replace here centos: 7 with ubuntu .

provisioner

"Provider" - the entity that manages the instances. In the case of the Molecule, it is ansible, the support of others is not planned, therefore this section can be called the advanced configuration ansible with reservation.
Here you can specify a lot of things, I will highlight the main, in my opinion, moments:

• playbooks : you can specify which playbooks should be used at certain stages.

 provisioner: name: ansible playbooks: create: create.yml destroy: ../default/destroy.yml converge: playbook.yml side_effect: side_effect.yml cleanup: cleanup.yml 

• config_options : config ansible

 provisioner: name: ansible config_options: defaults: fact_caching: jsonfile ssh_connection: scp_if_ssh: True 

• connection_options : connection options

 provisioner: name: ansible connection_options: ansible_ssh_common_args: "-o 'UserKnownHostsFile=/dev/null' -o 'ForwardAgent=yes'" 

• options : Ansible parameters and environment variables

 provisioner: name: ansible options: vvv: true diff: true env: FOO: BAR 

scenario

The name and description of the script sequences.
You can change the default action matrix of any command by adding the <command>_sequence and determining the list of steps that we need as a value for it.
Suppose we want to change the sequence of actions when launching the playlist run command: molecule converge

 # : # - dependency # - create # - prepare # - converge scenario: name: default converge_sequence: - create - converge 

verifier

Setting up a framework for tests and linter to it. By default, testinfra and flake8 are used testinfra flake8 . Possible options are similar to the above:

 verifier: name: testinfra additional_files_or_dirs: - ../path/to/test_1.py - ../path/to/test_2.py - ../path/to/directory/* options: n: 1 enabled: False env: FOO: bar lint: name: flake8 options: benchmark: True enabled: False env: FOO: bar 

Let's return to our role. Edit the tasks/main.yml to this:

 --- - name: Install nginx apt: name: nginx state: present - name: Start nginx service: name: nginx state: started 

And add tests to molecule/default/tests/test_default.py

 def test_nginx_is_installed(host): nginx = host.package("nginx") assert nginx.is_installed def test_nginx_running_and_enabled(host): nginx = host.service("nginx") assert nginx.is_running assert nginx.is_enabled def test_nginx_config(host): host.run("nginx -t") 

Done, it remains only to run (from the root of the role, I recall):

 > molecule test 

 --> Validating schema <path>/nginx/molecule/default/molecule.yml. Validation completed successfully. --> Test matrix └── default ├── lint ├── destroy ├── dependency ├── syntax ├── create ├── prepare ├── converge ├── idempotence ├── side_effect ├── verify └── destroy --> Scenario: 'default' --> Action: 'lint' --> Executing Yamllint on files found in <path>/nginx/... Lint completed successfully. --> Executing Flake8 on files found in <path>/nginx/molecule/default/tests/... Lint completed successfully. --> Executing Ansible Lint on <path>/nginx/molecule/default/playbook.yml... Lint completed successfully. --> Scenario: 'default' --> Action: 'destroy' PLAY [Destroy] ***************************************************************** TASK [Destroy molecule instance(s)] ******************************************** changed: [localhost] => (item=None) changed: [localhost] TASK [Wait for instance(s) deletion to complete] ******************************* ok: [localhost] => (item=None) ok: [localhost] TASK [Delete docker network(s)] ************************************************ PLAY RECAP ********************************************************************* localhost : ok=2 changed=1 unreachable=0 failed=0 --> Scenario: 'default' --> Action: 'dependency' Skipping, missing the requirements file. --> Scenario: 'default' --> Action: 'syntax' playbook: <path>/nginx/molecule/default/playbook.yml --> Scenario: 'default' --> Action: 'create' PLAY [Create] ****************************************************************** TASK [Log into a Docker registry] ********************************************** skipping: [localhost] => (item=None) TASK [Create Dockerfiles from image names] ************************************* changed: [localhost] => (item=None) changed: [localhost] TASK [Discover local Docker images] ******************************************** ok: [localhost] => (item=None) ok: [localhost] TASK [Build an Ansible compatible image] *************************************** changed: [localhost] => (item=None) changed: [localhost] TASK [Create docker network(s)] ************************************************ TASK [Create molecule instance(s)] ********************************************* changed: [localhost] => (item=None) changed: [localhost] TASK [Wait for instance(s) creation to complete] ******************************* changed: [localhost] => (item=None) changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=5 changed=4 unreachable=0 failed=0 --> Scenario: 'default' --> Action: 'prepare' Skipping, prepare playbook not configured. --> Scenario: 'default' --> Action: 'converge' PLAY [Converge] **************************************************************** TASK [Gathering Facts] ********************************************************* ok: [instance] TASK [nginx : Install nginx] *************************************************** changed: [instance] TASK [nginx : Start nginx] ***************************************************** changed: [instance] PLAY RECAP ********************************************************************* instance : ok=3 changed=2 unreachable=0 failed=0 --> Scenario: 'default' --> Action: 'idempotence' Idempotence completed successfully. --> Scenario: 'default' --> Action: 'side_effect' Skipping, side effect playbook not configured. --> Scenario: 'default' --> Action: 'verify' --> Executing Testinfra tests found in <path>/nginx/molecule/default/tests/... ============================= test session starts ============================== platform darwin -- Python 2.7.15, pytest-4.3.0, py-1.8.0, pluggy-0.9.0 rootdir: <path>/nginx/molecule/default, inifile: plugins: testinfra-1.16.0 collected 4 items tests/test_default.py .... [100%] ========================== 4 passed in 27.23 seconds =========================== Verifier completed successfully. --> Scenario: 'default' --> Action: 'destroy' PLAY [Destroy] ***************************************************************** TASK [Destroy molecule instance(s)] ******************************************** changed: [localhost] => (item=None) changed: [localhost] TASK [Wait for instance(s) deletion to complete] ******************************* changed: [localhost] => (item=None) changed: [localhost] TASK [Delete docker network(s)] ************************************************ PLAY RECAP ********************************************************************* localhost : ok=2 changed=2 unreachable=0 failed=0 

Our simple role was tested without problems.
It is worth remembering that if there are problems with the molecule test , then if you did not change the standard sequence, the molecule will delete the instance.

The following commands are useful for debugging:

 > molecule --debug <command> # debug info.      . > molecule converge #      . > molecule login #    . > molecule --help #   . 

Existing role

A new script is added to an existing role from the role directory by the following commands:

 #     > molecule init scenarion --help #    > molecule init scenario -r <role_name> -s <scenario_name> 

In case this is the first script in the role, then the -s parameter can be omitted, since the default script will be created.

Conclusion

As you can see, the molecule is not very complicated, and using your own templates, you can reduce the deployment of a new scenario to the editing of variables in the creation and deletion of instances playlists. The molecule seamlessly integrates with CI systems, which allows to increase the speed of development by reducing the time for manual testing of playbooks.

Thank you for your attention. If you have experience in testing ansible roles, and it is not related to the molecule - tell us about it in the comments!