Introduction

Ansible modules and sparrow plug-ins are the building blocks for solving the simplest tasks in the field of configuration management and automation of deployment. Ansible modules are used in higher-level scripts - playbooks written in YAML, while sparrow plug-ins are similarly embedded in sparrowdo scripts written in Perl6 .

This article is a free author's translation of its own original from English.

Development language support

ansible - you can choose virtually any development language for writing modules.
Out of the box, the ansible provides an advanced API for Python (so-called shortcuts). When developing modules in other languages, you will need to use supporting libraries (native to each language) to simplify the process of development and integration into ansible.

sparrow - you have a choice of one of the three (*) languages ​​- Perl5, Bash or Ruby. For the plugin developer, sparrow provides a unified API (available for any of the three languages), as well as in the case of ansible, which simplifies the development and integration of plug-ins into the sparrow system. This API provides only basic features and is not as advanced as the ansible API for Python.

(*) In the near future it is planned to add support for Python.

Design

ansible - ansible modules - autonomous blocks of software when (scripts) for solving elementary problems. Ansible modules cannot depend or cause other ansible modules.

sparrow - sparrow plug-ins are similar to ansible modules - autonomous, closed blocks of code for solving elementary tasks. However, sparrow provides an extra degree of freedom at this level of development. Sparrow plugins are suites of scripts. Some scripts can call others, passing them parameters. This kind of design allows you to break any task into scripts that interact with each other. As an example, consider the simple case - installing and removing software packages. We can look at this task as a single block of code (script), but when implemented, break everything into two scripts: one to install the packages, the other to remove. You can also enter the third script, which will parse the input parameters and do dispatching calls (*). This approach is described in detail in the article Evolution sparrow plug-ins .

(*) In fact, package-generic plugin is implemented this way.

Here is a simple illustration of everything said:

Configuration Management Integration

ansible - ansible modules are an integral part of higher-level configuration scripts - ansible playbooks . ansible playbook - written in YAML script, declaring a call to ansible modules with some specified parameters. Such a construction in ansible is called a task.

sparrow - like ansible modules sparrow plugins are an integral part
a more common system - sparrowdo scripts written in Perl6. Like ansible - sparrowdo - configuration management system, but built on sparrow plugins. Sparrow plugins are invoked with parameters inside sparowdo scripts.

End-user interface

ansible - ansible modules, as already mentioned, are invoked via a YAML DSL code, which declares the launch of modules with transmitted parameters. This happens in playbooks. It is possible to launch modules with a console script with parameters passed as command line arguments.

Below is an example of a playable ansible code for installing the httpd web server using the yum package manager (the same name ansible module):

$ cat playbook.yml --- - hosts: webservers tasks: - name: ensure apache is at the latest version yum: name=httpd state=latest 

sparrow - sparrow plugins are invoked inside sparrowdo scripts using the Perl6 API. Plugin parameters are passed as Perl6 hashes. You can also use the console client to run plugins "as is", bypassing sparrowdo (in a similar way as you can do ansible modules). This can be handy when developing and debugging plugins. In this startup mode, you have a lot of additional opportunities to transfer input parameters - the command line, JSON / YAML formats and Config :: General

The following is the equivalent of the previous ansible playbook for installing the web server httpd. There are two code variants (core-dsl and plugin API) of the sparrowdo script:

 $ cat sparrowfile # core-dsl API  ,     #  ""    #         package-install 'httpd'; # ,       core-dsl #      , #  plugin API #     : task-run 'ensure apache is at the latest version', 'package-generic', %( list => 'httpd' ); 

Transmission and processing of input parameters

ansible - the input parameters of the module are passed in the form of pairs key = value (*). In general, when developing a module, you must parse the parameters in order to “extract” the data and assign them to the corresponding variables, with a view to further use in the module code. Under different development languages ​​there are numerous "helpers" that solve this problem.

(*) It is possible to transfer nested data structures.

Also, as already mentioned, ansible provides a high-level API for Python, called the ansible documentation for shortcuts. This API allows you to automatically parse input parameters, generate accessors, determine the types of input parameters and test them, determine default values, and the like.

Below is a slice of an anible module that implements the parsing of input parameters using the Python ansible API:

 $ cat library/greetings.py from ansible.module_utils.basic import * def main(): fields = { "message": {"default": "Hi!", "type": "str" } } module = ansibleModule(argument_spec=fields) module.params['message'] # some other code here to return results if __name__ == '__main__': main() 

sparrow - sparrow implements a unified API (available for any development language) to handle the input parameters of the plugin, so you don’t have to manually parse the data.

Thus, regardless of the language in which you are developing the plugin, you get
out of the box API for accessing input data without actually writing a single line of auxiliary code.

It is also possible to determine default values ​​for parameters, in case the latter are not explicitly specified.

Built-in typing and parameter checking, as in the case of the ansible Python API is missing.

Below is the equivalent of the aforementioned ansible code that parses input parameters. Bash (*) is selected as the plugin implementation language.

(*) to show how even in such a far from the most high-level development language, you can easily write sparrow plug-ins, without “sticking” to the processing of input parameters:

 #    $ cat story.bash message=$(config message) #    : $ cat story.ini message = Hi! 

But how can you use the same Bash to parse nested input parameters:

 #    sparrowdo  $ cat sparrowfile task-run "run my task", 'foo-plugin', %( foo => { bar => { baz => 'BAZ' } } ); #    Bash #    : $ cat story.bash baz=$(config foo.bar.baz) 

Return values

ansible - ansible modules always return the result as a JSON string. And JSON itself should have a special structure. More information about this can be found on the pages of the documentation of ansible module development.

But here's what I want to note about the process of returning values ​​from the ansible modules:

• termination code (script implements the mold) ansible modules ignored

  
  

• the only requirement for returning from a module is the presence of a string in JSON format in STDOUT. The JSON response structure must also meet certain requirements.

  
  

• Thus, if we do not receive valid JSON in the response from the module, this is regarded as an error

  
  

• STDOUT / STDERR generated by the module is not visible (*) in the output of the player
  (*) - under certain conditions or by writing additional code, this output can still be obtained, but here I am talking about the default behavior

  
  
• In order for a module to return a value, it must be represented as a field of JSON structure, which must be returned in STDOUT

I will give an example with the currenttime module for returning the current time:

 $ cat library/currentime.py import datetime import json date = str(datetime.datetime.now()) print json.dumps({ "time" : date }) 

sparrow - sparrow plugins, unlike ansible modules can return anything, sparrow doesn’t regulate the data returned by the plugin. (However, see the following section - "Output Processing").

Here is what is important to know about the output in terms of sparrow:

• The plug-in termination code (resulting, as in the case of sparrow, we can deal not with one script, but with several) matters. If it is non-zero, it is interpreted as a plug-in execution error.

  
  
• The STDOUT generated by the plugin is simply passed on and is visible in the resulting output. No "repacking" or filtering the results, as is the case with ansible. We always see the output as it is, as if we just ran a similar script from the console.

Next, I will give the equivalent of the previous ansible module to display the current time. This time, when writing a plugin, it will be to use Perl5, by virtue of all the plugin code is said to be trivial:

 #   $ cat story.pl print scalar localtime; 

Output Processing

ansible - ansible modules must always return JSON, so it’s very easy to have structured data at the output to turn them into variables and process them somewhere "higher", in fact, in ansible playbooks. Although this is a powerful enough feature, its value in configuration management systems is a controversial one for me. The fact is that by the time (*) of the task list execution on the target server everything should already be definitely in the form of a static, deterministic list, any attempt to add branching and dynamics in such an already formed list only complicates, but does not simplify, the solution of the common task - convergence server to a given form.

(*) it is important to understand here that I’m not against the dynamic generation of such lists of tasks or resources, just here the imperative approach is quite appropriate, as is the case with chef or sparrowdo. But maybe I don’t know so well ansible to reason how the data returned by the modules are used later in playbooks.

Here is an example of a simple module that simply returns the input parameter passed to it, which is then "registered" in the playbook and output to the console:

 $ cat playbook.yml - hosts: localhost tasks: - name: tell me what I say echo: message: "hi there!" register: result - debug: var=result $ cat library/echo.py from ansible.module_utils.basic import * def main(): module = ansibleModule(argument_spec={}) response = {"you_said": module.params['message']} module.exit_json(changed=True, meta=response) if __name__ == '__main__': main() 

sparrow - as already mentioned, sparrow is “all the same” that the plugin reports in STDOUT, there is no way to use this data further. In fact this is not true.

From the point of view that this data will not be processed further - this is true, but you can determine the verification rules with the help of which you can verify the output of the script. This kind of built-in testing is a characteristic feature of sparrow plug-ins. Verification rules are written in the form of constructions of the Outthentic :: DSL language and allow you to do various textual checks - matching the text with different regular expressions, working with individual pieces of text that correspond to our expressions (so-called captures) and much more. All this can be read on the documentation pages Outthenic :: DSL.

If verification is not passed, it is interpreted (along with a non-zero completion code), as a plug-in execution error.

Thus, writing such "self-testing" scripts allows you to write plug-ins not only for tasks from the area of ​​configuration management, but for tasks of testing, monitoring and auditing.

It is difficult to build an analogy with the previous ansible module, in view of all the above, but
here is something roughly "similar" that can be done in sparrow. Pass the parameter to the plugin, output it to STDOUT and write a check rule to make sure that the output is exactly what was at the input:

 #     sparrowdo $ cat sparrowfile run-task "tell me what I say", "echo", %( message => 'hi there!' ) #      $ cat story.bash echo you said $(config message) 

$ cat story.check
generator: config () -> {message}

Installation process

ansible - a large number of ansible are supplied as part of ansible, they are ready for use, you do not need to install them in any special way. Also, users write the so-called custom modules and, as a rule, save them in some version control system - Git / Svn (for example, in github or gitlab). Further, modules in the form of regular files are checked to the master server (the one from which the ansible tasks are launched by ssh on the target servers), again in this case the message is reduced to the usual checkout from the version control system.

So, the ecosystem of modules includes two large subsystems:

• The main repository ansible ( main ansible repository ) - modules that come with ansible

  
  
• Custom modules

Summarizing, we can say that the push-based deployment scheme is implemented in ansible, and with the absence of an agent (agentless) on the target server. Modules in the form of files are simply copied to the target server and are already running there as scripts.

The following is a graphical diagram of the process of deploying custom modules ansible:

sparrow - in sparrow the deployment process is a bit different. Although in general, the same push-based scheme was adopted there. However, in the case of sparrow, there is an agent (or client, to whom, as it is more convenient), who installs plug-ins.

So, sparrow plugins are nothing more than packaged scripts, distributed like a classic distribution system for software packages, such as deb, rpm, CPAN, RubyGems.

sparrow is provided by the plugin installation manager, the same manager is used to launch plugins. The manager must be preinstalled on the target server for everything to work.

sparrowdo compiles scripts placed on the central (master) server into meta data, which is copied into scp as target JSON files, on the target machines, then the manager uses the copied data, downloads, installs, and finally sparrow starts the result is returned back (as in the case of ansible) in the form of a response to the central server.

So, it is important to understand that, unlike ansible, installing plugins is not just copying files to the target server, since in the case of sparrow, not the plugins themselves are copied, but their meta information, then the actual deployment process takes place according to the meta information.

sparrow plugins are located in the central repository of sparrow plugins - sparrowHub . Plugins have author, version, description and documentation.

Here is the meta information file of the above package-generic plugin for installing software packages:

 { "name" : "package-generic", "version" : "0.2.16", "description": "Generic package manager. Installs packages using OS specific package managers (yum,apt-get)", "url" : "https://github.com/melezhik/package-generic" } 

In sparrow there is no hard division into custom and "core" plugins. Any plugin loaded on sparrowHub becomes immediately available to users of the system, and thus it can be used in sparrowdo scripts.

At will, users can place their plugins on remote git repositories, in sparrow such plugins are called private, and these plugins can easily be used in your configuration scripts, along with sparrowHub plugins (so-called public plugins)

The following is a diagram of the deployment of plug-ins in sparrow:

Dependencies

ansible - in ansible there is no possibility to manage software dependencies at the module level. Most likely (I know the ansible so well, I guess) it happens a level higher - in playbooks. Thus, if your module is dependent on a library, you need to take care of resolving dependencies somewhere else, for example, causing the dependencies to be installed elsewhere in the playbook.

sparrow - sparrow provides regular dependency management capabilities (*) at the level of the sparrow plugin. Thus, if the plugin is dependent on any libraries, you can describe these dependencies right in the plugin and the plugin manager will take care of them
during installation of the plugin. For now, dependency management is limited to CPAN modules for Perl5 through the cpanfile file and RubyGems modules for Ruby through the Gemfile file.

(*) - dependencies are limited while CPAN for Perl5 and RubyGems for Ruby

Conclusion

ansible won great popularity, not least due to the large number of ready-made modules. However, in my personal opinion, the process of developing new modules has its drawbacks and the architecture itself is not a good solution for some tasks. I find the individual features of the sparrow more attractive and interesting compared to the ansible. This, of course, IMHO :), I will list some points:

• Playbooks against sparrowdo scripts - sparrowdo provides imperative programming options for configurations using powerful and modern Perl6 language
  While in ansible you are limited by the capabilities of a purely declarative YAML.

  
  
• Script-oriented design - thanks to a script-oriented scheme, sparrow encourages splitting a task into separate pieces and writing a separate script for each piece, and the scripts themselves organically interact with each other, using the parameters that they send to history modules ) In the case of ansible, you are usually limited to a single module scheme = one script. Of course, in the case of ansible you can write in a similar style (several scripts in the module), but ansible, of course, by design is not suitable for this kind of things.

So, in a practical sense, often when solving problems and writing routine scripts, we break the problem into several parts and have a set of scripts interacting with each other, so why not bring this approach here too?

• Life cycle and module and plug-in management - sparrow plug-ins are even more isolated and separate from the configuration management tool itself. As I said, sparrow plugins can be used without sparrowdo. In fact, they are developed, debugged and hosted and managed outside the framework of this system (sparrowdo) and used as black boxes. All this makes the process of developing new plug-ins and their use inside sparrowdo even more flexible and simple.

  
  

• Bash / Shell scripting - sparrow provides better, compared to ansible support for the integration of regular bash / shell scripts. Due to the fact that the latter imposes a restriction on the returned data and does not display STDOUT / STDERR streams by default, in some cases it becomes difficult to debug ordinary bash scripts or to understand what is going wrong. While sparrow actually transparently translates all output from the scripts back to the report. It also plays a significant role ignoring the script completion code in ansible, where the developer is assigned the "override" error at the level of the returned JSON. Sparrow in this case simply generates an error, acting in the convection style of executing unix / linux commands.

  
  

• Software API - sparrow provides a unified interface for all plug-in development languages. This means that virtually every language in the list of supported "has the same rights" from the point of view of the basic API provided by the system. While the modules of ansible are mostly written in python, due to the fact that support for this language is the most complete (ansible modules python API)

  
  
• Testing capabilities - sparrow provides built-in testing capabilities that allow you to look at this tool as a universal automation tool that can be used including (besides those listed) in audit, monitoring and testing tasks.

PS as always interesting to hear the reader’s opinion and constructive criticism. This time I don’t cite the traditional questionnaire, and the article turned out to be voluminous.