Hello! I have already written about the sparrowdo tool and its use in developing chef configuration scripts.

Well, during this time a lot of water has flowed under the bridge and I wanted to reveal this topic a little bit again, which is why the title of the article contains version two.

So - sparrowdo is a configuration management system written in the wonderful language Perl6 , which has been actively developing lately. In my personal work, I find sparrowdo very convenient and organically coexisting with a more powerful configuration management platform - opscode chef . In a few specific examples, I will show how I use chef with sparrowdo. This post will not be very long, but I hope will give an understanding of what is at stake.

What we have? By virtue of my main professional activity, I have to write a lot of chef kukbukov . In our project, aws services are actively used, so creating a server and installing a chef client on it is quite cheap and resource-intensive, so I can easily debug chef recipes by running the chef client on a remote ssh server instead of using more traditional vagrant cases

Well, in this way, the development and debugging process of the cookbooks comes down to the fact that while developing the next version of recipes, I update the cookbook on the chef server using the knife upload command and run the test cookbook further on the remote server using the chef-client command (with the specified wound list). Actually, while catching various errors and falls arising during the deployment (if expressed in the language of the documentation chef - convergence node)

In this situation, sparrowdo is very convenient for me - which, unlike chef, is a push based configuration management tool or, quite simply, it can execute its scripts on a remote ssh host.

As you can already begin to guess - the scenario in this case will be like this - run the chef client on the remote machine.

Sparrowdo supports modularity - you can extend its basic functionality through modules written in Perl6, so I specifically wrote a module for this task called Sparrowdo :: Chef :: Client

The interface of the module is quite simple - it provides the necessary minimum for running the client chef - namely, specifying the wounds of the sheet and optionally setting the attributes

Here is how the script for launching the java java with the installation of the attribute specifying the version of the jdk being installed will look like:

 $ cat sparrowfile module_run 'Chef::Client', %( run-list => [ "recipe[java]" ], attributes => %( java => %( jdk_version => 7 ), ), log-level => 'info', force-formatter => True ); 

In our projects, we mainly use jdk version 7, so I expose it through the corresponding chef attribute, which overrides the default value.

Here is what the sparrowdo report will look like when running on a remote server:

 $ sparrowdo --host=remote.server --ssh_user=centos --ssh_private_key=/home/melezhik/.ssh/foo.pem running sparrow tasks on remote.server ... target OS is - centos7 enter module <Chef::Client> ... push task <set up chef run list and attributes> plg <file> OK push task <run chef-client> plg <bash> OK set up task box file - /tmp/sparrowdo/task-box8938.json - OK get index updates from SparrowHub ... OK public@file is uptodate (0.0.5) public@bash is uptodate (0.1.4) running task box from /tmp/sparrow-cache/task-box8938.json ... [t] set up chef run list and attributes at 2017-04-06 13:35:14 set target content touch target target created set target mode to 644 ok scenario succeeded ok text match /target (created|deleted)/ ok text has 'set target content' STATUS SUCCEED [t] run chef-client @ runs bash command [t] run chef-client modules/bash-command/ params: envvars: at 2017-04-06 13:35:14 [2017-04-06T13:35:15+00:00] INFO: Forking chef instance to converge... Starting Chef Client, version 12.19.36 # #   # Chef Client finished, 7/9 resources updated in 38 seconds ok scenario succeeded STATUS SUCCEED 

Well, this is a simple example of how we can manage the settings of the cookbooks that are run, exposing the wound list and various attributes. I will cite below some more interesting cases.

boundary condition testing

It is no secret that a large number of tasks on configuration management and automation is associated with the launch of network services. For example, we have a cookiebook that configures and restarts the nginx server according to some nontrivial rule set. I think those who wrote these kinds of recipes often came up against the fact that their scripts could behave differently with respect to different states of the service at the beginning. Here, for example, I want to see what will happen, and whether my deployment will not fall down if nginx is not started before the chef recipe is executed, using sparrowdo, which itself is also a configuration management tool, is easy to do:

 $ cat sparrowfile service-stop 'nginx' module_run 'Chef::Client', %( run-list => [ "recipe[nginx-app::configure]" ] ); 

You must agree that it would not be good to embed nginx into the chef of recipes, just to test the behavior of our main cookbook. And here with the help of sparrowdo we do it easily and naturally, without changing anything in the code of the chef cookbook itself

embedded post deployment / audit tests

Often it is required to check the work of the client’s chief after he has worked. There are various options for solving this problem - test-kitchen / serverspec , goss , chef minitest-handler , inspec they all have their pros and cons. But here you have another alternative - sparrowdo - the basic meaning is the same - we do checks side by side, or rather inside the sparrowdo script, here’s one of my favorite checks that the tomcat server is running and “visible” in the list of processes - practice shows that Sometimes chef does not catch the situation with the fall of the restarted service and we need to do it ourselves:

 $ cat sparrowfile service-stop 'nginx' module_run 'Chef::Client', %( run-list => [ "recipe[tomcat-app::configure]" ] ); task-run 'check tomcat', 'proc-validate', %( pid_file => '/var/run/tomcat7.pid', footprint => 'java' ); 

optional dependencies

And finally, the last is the simplest and most obvious case (but this does not lose its relevance) - you need to temporarily install additional packages and utilities on the server being tested. You do not want to include this in your chef recipes, because these dependencies do not have a relationship to installing applications, but you need them to debug installed applications, well, just add them through sparrowdo without changing the code of the chef recipes you wrote.

Here, for example, a specialized package, which we hardly need in production, but
It will be necessary if we want to do debugging java code:

 package-install 'java-1.7.0-openjdk-debuginfo' module_run 'Chef::Client', %( run-list => [ "recipe[java]" ] ); 

On this I finish. In conclusion, I would like to say that in my personal work related to the daily development of dozens of dollies for hundreds of different projects, sparrowdo has established itself as an excellent companion tool that significantly speeds up and simplifies the development and testing of chef scripts.

As usual, I will be glad to questions and constructive criticism.