Hello! In this article I would like to talk about a simple configuration management tool - Sparrowdo . I will say right away that the Sparrowdo system is based on using the so-called Sparrow plug-ins, about which I have already written a number of articles on habrahabr.ru

So, the distinctive features of this tool:

• Configuration scripts are written in Perl6 (this explains the appearance of the butterfly logo at the top of the article :)

  
  

• The solution of configuration management tasks occurs by selecting one or another suitable plug-in and launching it on a custom server. Each plugin is a black box for solving a specific task. By collecting a number of plug-ins and running them on the server with the required parameters, you get the necessary system configuration. Plug-ins can solve both elementary tasks, such as creating users , groups , installing system packages , and more complex, complex ones — installing an application from source in the git repository and launching it as a service. You choose the plugin that suits you.

  
  

• Some analogy of plug-ins in known existing configuration management systems can be resources or kukbooks in chef , or modules in ansible .

  
  

• So, Sparrowdo does not provide you with out of the box a basic set of functions, as suggested by chef or ansible, but instead you can always use a ready-made set of plug-ins that solve typical problems. But even if you find the right one, you can easily write your own plugin and immediately use it. Thus, in Sparrowdo / Sparrow, the emphasis is not on the ready-made solution that was originally supplied with the tool, but on the ability to quickly and easily add the required functionality.

  
  

• Sparrowdo scripts define Sparrow plug-in lists with input parameters — in fact, all you need is to describe the required server configuration. The Sparrowdo client walks through the list of plug-ins and sequentially launches them on the target server, the result is output to the console in the form of TAP reports, and the launch of each plug-in generates its own report.

  
  

• Scripts are written in Perl6 , a simple API for running plugins on a remote machine is used, I will give specific examples a little later.

  
  
• For the work of Sparrowdo, push ideology was chosen. You have a master host on which the Sparrowdo client is installed, from which ssh runs plugins on the target server. To run plugins on target servers, the Sparrow client must be installed. Here is how it will look from the point of view of specific commands for installing this system.

Installing Sparrowdo client on master host:

$ ssh master.host $ panda install Sparrowdo 

Installing Sparrow client on target hosts:

 $ ssh target.host $ cpanm Sparrow $ yum install curl 

Sparrowdo system operation scheme:

  +-----------------+ | | ssh | |------------> < host-1 > 192.168.0.1 | <master host> | ssh | {sparrowdo} |------------------> < host-2 > 192.168.0.2 | | ssh | |-----------------------> < host-N > 192.168.0.3 | | +-----------------+ +-------------+ | | | <host> | | | | {sparrow} | | {curl} | | | +-------------+ 

In order for all this to work it is necessary to provide two conditions:

• host master to target machines must be provided with passwordless ssh access
• the user from which the ssh session is created must have sudo without password on the target machine

If both criteria are satisfied, we can create a Sparrowdo script and run it from the master host on the target machine:

  $ ssh master.host $ nano sparrowfile #   $ sparrowdo --host=192.168.0.1 

The client has many additional parameters that you can use, for example, specify the username under which you will run the ssh session or increase the report detailing with the --verbose parameter:

  $ sparrowdo --host=192.168.0.1 --ssh_user=admin --verbose 

Examples

And now a few examples, so that you can see how using Sparrowdo you can automate typical server configuration tasks.

Installing CPAN Modules

Since the Sparrowdo / Sparrow system is written in Perl6 / Perl5, and primarily focused on Perl developers, I can't help but start with this Perl ecosystem-classical case study.

 $ ssh master.host $ cat sparrowfile task_run %( task => 'install some cpan packages', plugin => 'cpan-package', parameters => %( list => 'CGI DBI', install-base => '/opt/perl' ) ); 

In this scenario, we install two CPAN modules - CGI and DBI , and use the path / opt / perl as the install base.

This is how it will look like on my test machine:

 $ sparrowdo --host=192.168.0.1 running sparrow tasks on 192.168.0.1 ... running task <install some cpan packages> plg <cpan-package> parameters: {install-base => /opt/perl, list => CGI DBI} /tmp/.outthentic/5385/opt/sparrow/plugins/public/cpan-package/story.t .. # [/opt/sparrow/plugins/public/cpan-package/modules/cpanm] # install CGI into /opt/perl ... # Successfully installed HTML-Parser-3.72 (upgraded from 3.64) # Successfully installed Test-Deep-1.120 # Successfully installed Sub-Uplevel-0.25 # Successfully installed Carp-1.38 (upgraded from 1.11) # Successfully installed Test-Warn-0.30 # Successfully installed CGI-4.31 (upgraded from 3.51) # 6 distributions installed # install ok ok 1 - output match 'install ok' # [/opt/sparrow/plugins/public/cpan-package/modules/cpanm] # install DBI into /opt/perl ... # Successfully installed DBI-1.636 # 1 distribution installed # install ok ok 2 - output match 'install ok' # [/opt/sparrow/plugins/public/cpan-package] # cpan-package-done ok 3 - output match 'cpan-package-done' 1..3 ok All tests successful. Files=1, Tests=3, 91 wallclock secs ( 0.02 usr 0.01 sys + 72.58 cusr 8.26 csys = 80.87 CPU) Result: PASS 

Installing system packages

For this purpose we will use the package-generic plugin, I would like to note that for now it supports a small number of platforms - Ubuntu / Debian / CentOS, but it is easy to “sharpen” it if necessary. Here's what the nginx, nano, and telnet installation script will look like.

 $ ssh master.host $ cat sparrowfile use v6; use Sparrowdo; task_run %( task => 'install my packages', plugin => 'package-generic', parameters => %( list => 'nginx nano telnet' ) ); 

Create, delete users and groups

This typical problem is solved by plugins with obvious user and group names. Here is what the Sparrowdo script might look like:

 $ ssh master.host $ cat sparrowfile use v6; use Sparrowdo; task_run %( task => 'this is me', plugin => 'user', parameters => %( name => 'melezhik' ) ); task_run %( task => 'admins group', plugin => 'group', parameters => %( name => 'admins' ) ); task_run %( task => 'remove unknown users', plugin => 'user', parameters => %( name => 'Unknown' , action => 'delete' ) ); 

Service Management

Could not miss this classic case. Here is how it will look in Sparrowdo scripts:

 $ ssh master.host $ cat sparrowfile use v6; use Sparrowdo; task_run %( task => 'start nginx ', plugin => 'service', parameters => %( name => 'nginx' ) ); task_run %( task => 'stop nginx ', plugin => 'service', parameters => %( name => 'nginx' , action => 'stop' ) ); task_run %( task => 'add nginx to autostart', plugin => 'service', parameters => %( name => 'nginx' , action => 'enable' ) ); task_run %( task => 'remove nginx from autostart', plugin => 'service', parameters => %( name => 'nginx' , action => 'disable' ) ); 

Using templates to configure config files

And the last example is more complicated. How can you manage config files using templates in the Template-Toolkit format.

Use the templater plugin. In order for everything to work, we will create a template in the same directory as the Sparrowdo script, for example

 $ ssh master.host $ cat foo.conf.tmpl Hello, my name is [% name %]! I speak [% language %] 

Now we will write a script for installing the configuration file based on our template, to which we will pass a couple of variables:

 $ ssh master.host $ cat sparrowfile use v6; use Sparrowdo; task_run %( task => 'install my config', plugin => 'templater', parameters => %( variables => %( name => 'sparrowdo', language => 'perl6' ), target => '/etc/foo.conf', owner => 'user', mode => '644', source => slurp 'foo.conf.tmpl' ) ); 

As a result, after applying this script on the target server, we get the file /etc/foo.conf with the contents

 Hello, my name is sparrowdo! I speak perl6 

Using private plugins

All the plugins just discussed are public . They are all located in the publicly available Sparrow central repository of SparrowHub plugins. If for some reason you don’t want to share your plugins, you can place them in remote git repositories and use them from there. This is called private sparrow plugins.

This is how the example would look like for the system packages installation script, if you used the package-generic private plugin located in the github repository (I omit the details related to authentication):

 $ ssh master.host $ cat sparrowfile use v6; use Sparrowdo; set_spl %( package-generic => 'https://github.com/melezhik/package-generic.git' ); task_run %( task => 'install my packages', plugin => 'package-generic', parameters => %( list => 'nginx nano telnet' ) ); 

This is an overview of the existing features of the Sparrowdo / Sparrow system. A complete list of existing plugins can be found here . New plugins are added constantly, those who are interested in this article can be included in the process, as I have repeatedly said - the writing of new Sparrow plugins is up to everyone who writes in Perl, Bash or Ruby. There is a special environment for developing plugins called documentation for which you can find all the information. The process of "turning" user scripts into Sparrow plugins is also described in the Sparrow documentation.

Summary

So, as stated in the title of the article, Sparrowdo is a simple, but quite effective means of configuration management. Simple because, there are not many features present in known configuration management systems - such as running scripts in dry-run mode, like in chef or managing dependencies between plugins. However, an important feature that distinguishes the Sparrowdo / Sparrow system from a number of others is the ability to quickly develop the plugins you need and their subsequent “inclusion” into the system through a single repository. Compared to the complex LWRP API in chef, writing a new Sparrow plugin will solve your task an order of magnitude faster and easier. And the exit code will be easier to understand, debug and maintain.

It is also important to mention that Sparrow does not impose any DSL on you within which you write plugins, but provides a unified API for all three plug-in development languages ​​(Perl, Bash, Ruby).

As for the system itself, Sparrowdo is a kind of thin client for Sparrow plugins. It has a simple API, with the ability to construct lists of used plugins in a purely imperative style, using the full power of the modern Perl6 language.

Thanks for attention. As always, I look forward to criticism of the case, questions and comments.

Thank.

PS and at the end a simple survey, not directly related to the topic of the article.