Gitlab-CI and Ansible-lint syntax checking

Hello! We continue the DevOps series of articles and look for the most efficient ways to manage the configuration, sharing your experience with you. In previous articles, we looked at how to build Ansible configuration management using Jenkins and Serverspec, and now, at your request, consider how to organize configuration management using GitLab-CI.

Ansible-lint is a utility for checking the correctness of the playbook syntax and code style, which can be integrated into the CI-service. In our case, we embed it in gitlab-ci to check playbooks at the stage of accepting a Merge-Request and setting the status of checks.
GitLab (GitLab Community Edition) is a opensource project, a git repository manager, originally developed as an alternative to the paid corporate version of Github.

Installing GitLab CE is described in this article .
')
Install gitlab-ci-multirunner

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.rpm.sh | sudo bash yum install gitlab-ci-multi-runner 

Register runner

 gitlab-ci-multi-runner register 

Now you need to answer the questions:

 Running in system-mode. Please enter the gitlab-ci coordinator URL (eg https://gitlab.com/ci): http://domain.example.com/ci Please enter the gitlab-ci token for this runner: your_token Please enter the gitlab-ci description for this runner: [domain.example.com]: Please enter the gitlab-ci tags for this runner (comma separated): ansible Registering runner... succeeded runner= Please enter the executor: docker-ssh+machine, docker, docker-ssh, parallels, shell, ssh, virtualbox, docker+machine: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 

We take the token and URL from the project settings page.

Features of setting multi-runner.

One runner can handle multiple projects. In order for one runner to process everything, you just need to register a new runner without specifying tags. We take the token for the shared runner in the Admin Area:



You can pack a pack of runners on different servers. And more: in gitlab, as in Jenkins, there is such a thing as tags. The project with the ansible tag will process the runner marked with the ansible tag, and for other projects, without a label or with a different label, this runner will not work.

Also in the admin, you can configure the match runner and project.

domain.example.com/admin/runners

Install Ansible-lint

Ansible-lint is installed via python-pip or from the EPEL repository.

The first way:

First install python-pip, then through it set ansible-lint:

 sudo yum groupinstall @Development Tools sudo yum install python-pip sudo pip install --upgrade pip sudo pip install ansible-lint 

Second way

Everything is simple - we put epel-release and put ansible-lint through the package manager:

 sudo yum install epel-release sudo yum install ansible-lint 

Adjusting the pipeline.

Create a .gitlab-ci.yml file in the root of the repository. It is important to observe the number of spaces, otherwise there will be an error - yaml is not forgiving, yeah.

 stages: [ ]- test test_job: [ ]stage: test [ ]script: [ ]- ansible-lint *.yml [ ]tags: [ ]- ansible 

stages - be sure to describe the assembly stage.

 stages: - stagename 

test_job is an arbitrary job title.
stage: test — you must describe the test stage specified in the stages section.

 jobname: stage: stagename 

script - we carry out the test in the root of the repository
tags - runner tag.

Stage name stage: test can be any, for example: converge, pre-test, post-test, deploy.
The job_name test_job can also be any.

GitLab has built-in lining for pipelines. You can check the correctness of the syntax of the pipeline by URL domain.example.com/ci/lint

Insert pipeline, click Validate.



If an error occurs, the linter will swear and indicate an error.



Must be stages , not stage .

Thus, ansible-lint will automatically check all playbooks with the .yml extension from the repository root at each commit.

We have two roles in the repository:

 roles ├── monit └── openssh 

And two playbooks that roll these roles:

 ├── monit.yml ├── openssh.yml ├── README.md └── roles 

openssh.yml

 --- - hosts: all user: ansible become: yes roles: - openssh 

monit.yml

 --- - hosts: all user: ansible become: yes roles: - monit 

Therefore, by checking the playbook assigning the role, we check all its contents:

 roles/openssh/tasks/ ├── configure_iptables.yml ├── configure_monitoring.yml ├── configure_ssh.yml └── main.ym 

 roles/monit/tasks/ ├── configure_monit.yml ├── configure_monit_checks.yml ├── install_monit.yml └── main.yml 

Now, with a commit, the ansible-lint will start automatically and check all the roles listed in the playbooks.
If you try to commit something and go to the web interface ( pipelines tab), then you can see the job status as failed .



In order to disable the lint checks when pushing to the repository, it is enough to clear all stages related to running the ansible-lint checks in the .gittab-ci.yml file .

Playback check options are also configured:

 ╰─>$ ansible-lint --help Usage: ansible-lint playbook.yml Options: --version show program's version number and exit -h, --help show this help message and exit -L list all the rules -q quieter, although not silent output -p parseable output in the format of pep8 -r RULESDIR specify one or more rules directories using one or more -r arguments. Any -r flags override the default rules in /usr/local/lib/python2.7/dist- packages/ansiblelint/rules, unless -R is also used. -R Use default rules in /usr/local/lib/python2.7/dist- packages/ansiblelint/rules in addition to any extra rules directories specified with -r. There is no need to specify this if no -r flags are used -t TAGS only check rules whose id/tags match these values -T list all the tags -v Increase verbosity level -x SKIP_LIST only check rules whose id/tags do not match these values --nocolor disable colored output --exclude=EXCLUDE_PATHS path to directories or files to skip. This option is repeatable. 

 ╰─>$ ansible-lint -L ANSIBLE0002: Trailing whitespace There should not be any trailing whitespace ANSIBLE0004: Git checkouts must contain explicit version All version control checkouts must point to an explicit commit or tag, not just "latest" ANSIBLE0005: Mercurial checkouts must contain explicit revision All version control checkouts must point to an explicit commit or tag, not just "latest" ANSIBLE0006: Using command rather than module Executing a command when there is an Ansible module is generally a bad idea ANSIBLE0007: Using command rather than an argument to eg file Executing a command when there is are arguments to modules is generally a bad idea ANSIBLE0008: Deprecated sudo Instead of sudo/sudo_user, use become/become_user. ANSIBLE0009: Octal file permissions must contain leading zero Numeric file permissions without leading zero can behavein unexpected ways. See http://docs.ansible.com/ansible/file_module.html ANSIBLE0010: Package installs should not use latest Package installs should use state=present with or without a version ANSIBLE0011: All tasks should be named All tasks should have a distinct name for readability and for --start-at-task to work ANSIBLE0012: Commands should not change things if nothing needs doing Commands should either read information (and thus set changed_when) or not do something if it has already been done (using creates/removes) or only do it if another check has a particular result (when) ANSIBLE0013: Use shell only when shell functionality is required Shell should only be used when piping, redirecting or chaining commands (and Ansible would be preferred for some of those!) ANSIBLE0014: Environment variables don't work as part of command Environment variables should be passed to shell or command through environment argument ANSIBLE0015: Using bare variables is deprecated Using bare variables is deprecated. Update yourplaybooks so that the environment value uses the full variablesyntax ("{{your_variable}}"). 

Some tasks can be skipped when checking ansible-lint doesn’t like command and shell modules very much, since in the ansible ideology it is considered that regular core modules are sufficient for all tasks. In fact, this is not always the case.

For example, we have a task in the role that uses the command module:

 - name: Installing monit command: yum -y install monit tags: monit 

And if a playbook is prolintim with this role, then ansible-lint will quarrel on the fact that we use the command module

 ╰─>$ ansible-lint monit.yml [ANSIBLE0002] Trailing whitespace monit.yml:7 - monit [ANSIBLE0012] Commands should not change things if nothing needs doing /tmp/ansible-lint/roles/monit/tasks/install_monit.yml:8 Task/Handler: Installing monit [ANSIBLE0006] yum used in place of yum module /tmp/ansible-lint/roles/monit/tasks/install_monit.yml:8 Task/Handler: Installing monit 

To avoid this, you can mark the task with the tag skip_ansible_lint :

 - name: Installing monit command: yum -y install monit tags: monit,skip_ansible_lint 

Now, when running lint on a playbook, it will not swear at the used command module:

 ╰─>$ ansible-lint monit.yml [ANSIBLE0002] Trailing whitespace monit.yml:7 - monit 

Features Merge Request.

Looking ahead, a few words about the functionality of checks in MR. By default, merge-request is accepted only upon successful verification. You can disable it in the project settings, in the Merge Requests section:



Ansible-lint can also be run on a local host without committing and not waiting for a CI service check. If you are on a Linux or OS X desktop, just install it to yourself.

Examples of commits with an error like this in gitlab.

1. Open the file in the embedded GitLab editor:



2. Make changes. For example, yaml is very sensitive to spaces, try adding an extra space at the beginning of a line:



Click Commit Changes, back to the modified file. An icon with the check status will appear at the top right:



Now it is in Pending status, as the check is not completed yet. If we poke a pictogram, we will move on to the status of checking our newly created commit:



It is now in the status Failed, since we intentionally made a mistake in the syntax. If we poke the Failed icon, we can see the result of the work of ansible-lint:



You can stick a cute badge with the status of the assembly in README.md

 [![build status](http://domain.example.com/projectname/badges/master/build.svg)](http://domain.example.com/projectname/commits/master) 

You can take it in the project settings, in the CI / CD Pipelines section



Copy-paste marcown and add it to README.md in the project root, commit, now the check status is displayed on the main page of the project:



Green / Passed - if the check was successful.
Red / Failed - if the check ended with an error.

In the Pipelines tab you can see the status of all commits:



Thus, we have built the process of checking the syntax correctness in configuration management. Thank you for your attention and all good automation!

Author: DevOps admin Centos-admin - Victor Batuev.