Testing the infrastructure as a code

Hello everyone, we recently started a series of articles on testing in Chef , but today I will talk about more introductory and universal things: why test the infrastructure, what tools are there for this, and how to automate it all. I will also touch upon the topic of publishing open source infrastructure code. The article will be of interest to users of any of the popular configuration management systems - Chef, Puppet, Ansible or SaltStack.

The article is based on my speech at the Ulyanovsk Strike conference; slides are available here . After reading, write about your experience in testing infrastructure, I think everyone will be interested.

Go.

Tests?

If the infrastructure becomes code, good code should be covered with tests. Tests help to not be afraid to make changes, improve the quality and speed of writing code, give feedback. The code covered with tests is easier to maintain, it is easier to operate and easier for beginners. In the concept of “infrastructure as a code,” tests allow to automate manual actions and save a lot of time, as well as documentation, they show how to work with your code.
')

When is it relevant?

Tests are especially relevant when the code is changed, when several people are working on it, and the changes affect complex logic or third-party code. There is no way out of changes, new versions of operating systems and programs come out, APIs change and your infrastructure code becomes outdated and requires changes. When several people make changes to the code, you can no longer be sure that your changes will not affect others, and if you change complicated logic or third-party code, the probability of error is even higher. The standard approach of checking the infrastructure code in Vagrant is no longer working here, since it takes a lot of time and requires manual checking of changes and the final state.

What to do?

Therefore, in the configuration management systems, there appeared their own tools for testing the infrastructure code. I will review the most popular and versatile tools that are suitable for any of the configuration management systems, be it Chef, Puppet, Ansible or SaltStack.

What to test?

But first you need to decide on which parts we will test and what to check.
We will test the main parts of which the infrastructure code consists, that is, kukbuki, modules, roles or formulas, depending on what system you use.

What are we checking?

We will check the following things:

• syntax and coding style
• functional
• the result of the configuration management system

Language style

Popular configuration management systems are written in two languages, Ruby and Python. For them, their own style guides for writing code have been created and it is advisable to follow them so that your infrastructure code is easier to read and maintain. To test the language style, standard code analyzers are used, such as rubocop for Ruby and pep8 for Python. They can be run manually or they can be embedded in your editor or IDE.

Code style

Over time, the configuration management systems developed their own recommendations; there appeared style guides and special tools - linters. They need to be considered to identify known problems. They work on the basis of a set of rules. A list of system linters is provided below:

• Chef foodcritic
• Puppet-lint
• Ansible-lint
• Salt-lint

Functional

The functionality is checked on the test data sets, they are also called fixtures. Fixtures include input data sets to test various uses of the infrastructure code. In the simple case, it is a kukbook, module, role, or formula with default parameters; in a complex case, calls to modules or providers with different parameters. The more different options in fixtures, the more result you can check. Fixtures are also an example of working with your infrastructure code.

Result

To check the result of the configuration management system, use the Serverspec tool. This is a special framework for testing infrastructure. It checks the final state of the virtual machine or server. Written in Ruby and is an RSpec extension, it supports all popular Linux and BSD distributions, as well as Windows. Serverspec contains about 40 built-in resources with which you can verify that the package is installed in the system, the necessary parameters are specified in the config, the service is running, or the port is listening. If this is not enough, then you can call any shell command and check its output. Serverspec is an excellent alternative to shell scripts and allows you to perform checks both locally and remotely. Serverspec is actively used in the infrastructure code community, and Chef recently announced support for Serverspec and recipes.

How to test?

Infrastructure tests can be run manually, locally or remotely, they can be embedded in the editor or IDE, run at any event, such as changing files on the file system or in the repository.

Automating

For automation, we will use a continuous integration system (CI system). Automation eliminates the human factor, improves the quality and speed of testing. Consider the following systems:

• Kitchen ci
• Travis ci
• Other CI systems

Test kitchen

This is a tool for running infrastructure code and testing it. Test Kitchen starts the virtual machine, it runs your infrastructure code and tests to check the result. Test Kitchen allows you to run virtual machines both locally and remotely. It is an excellent alternative to Vagrant for running infrastructure code, as it introduces the concepts test suite (suites), supports all configuration management systems, Chef in the database, the rest through plugins ( Puppet , Ansible , SaltStack ), and supports the Serverspec test framework.

Travis ci

This is a continuous integration SaaS system for GitHub projects. Free for public repositories. Suitable for testing infrastructure code. There is an integration with Chef Supermarket and Puppet Forge, I will tell about them later.

But there are limitations, the test environment in Travis CI is based on Ubuntu 12.04 LTS, so if you use another distribution kit or write cross-platform code, you will not be able to test it. Plus, the test environment comes with pre-installed packages and environment variables that can cause conflicts.

Test Kitchen + Travis CI

To circumvent these limitations, you can use two systems Test Kitchen and Travis CI together and run virtual machines in the cloud, for example, Amazon or DigitalOcean.

Other CI systems

If you already use any CI system, then the process of testing the infrastructure can be built on it, also using Test Kitchen to run the code and test suite.

Sharing with the community

In total, the infrastructure code will be in the same style, convenient for support, modification and training. Do not be ashamed to show others. Therefore, we share with the community and lay out in open source. With the help of the community, you can improve the testing and functionality of the infrastructure code. Due to the expertise and view from the development does not stop.

Share

The developers of configuration management systems have released special platforms for the exchange of infrastructure code:

• Chef supermarket
• Puppet forge
• Ansible galaxy
• Salt Stack Formulas

Do not forget

Laying out the code in open source, do not forget about the following points:

• Delete private information (keys, passwords, internal components)
• Add tests and build status, it increases the credibility of the code
• Keep a history of changes and documentation, you can create them automatically
• Think about compatibility, dependencies and conflicts.
• Sync code with community platform
• Add license

How we do it

How we do it, using the example of our chef kukbuka postgresql_lwrp:

• The code is in the repository on github
• Each change starts a build in Travis CI
• Checks language and code style with Rubocop and Foodcritic
• Next, Test Kitchen launches a virtual machine in the Digital Ocean cloud and runs the code and Serverspec tests.
• If all the tests have passed and there was a release, then the kookbook will be loaded into the Chef Supermarket.
• And we receive notification in the Slack chat and update the status of the assembly on the page kukbuk

findings

Test changes, automate and share with the community.

Links

On GitHub, we have testing examples for Chef, the most popular is the postgresql dollbook . In the near future we will post some more cool kukbuk. We also continue the cycle of articles on Habrahabr about Chef testing, listen to our podcast Devops Deflop and come to DevOps meetings in Moscow.