Sparrow is a multipurpose scripting system designed to develop and run a variety of automation scripts. Depending on the required tasks - this could be the deployment and configuration of applications, monitoring or testing the northern infrastructure, or the solution of other issues - in other words, the execution of any scripts in automatic mode.

Until recently, the launch of such scenarios was limited to the console client ( sparrow or sparrowdo ), today I would like to present a pilot project for the automatic installation and launch of sparrow scripts on remote servers via a centralized web service - SparrowUP .

Install SparrowUP

SparrowUP is a web application that is installed on a separate server, from which the installation and launch of sparrow scripts on other servers is initiated. Transport in this system is ssh. SparrowUP initiates ssh sessions within which:

• install client sparrow on target server (operation "bootstrap")
• installing the necessary set of sparrow plugins on the target server
• launching installed plug-ins with a set of specified parameters on the target server (parameters of the plug-ins being launched are defined in the so-called sparrow scripts, see below)

In order for all this to work it is necessary to provide password-free access from the SparrowUP server to the target servers, as well as sudo privileges for the user from which ssh commands are executed.

So, install SparrowUP:

Since SparrowUP delegates all the script execution logic to the SparrowDO client, this client must be first.

In turn, SparrowDO is written in Perl6 , so first install the necessary perl6 * dependencies (Perl6, panda), all the details of the installation are described here - http://rakudo.org/how-to-get-rakudo , to save time, I will not give here the entire installation cycle. After installing Perl6 and pand, we install Sparrowdo as a Perl6 module:

$ panda install Sparrowdo 

Now put the SparrowUP web application (it is in turn written in Perl5 / Mojolicious):

 $ git clone https://github.com/melezhik/sparrowup.git $ cd sparrowup $ carton 

Create a database required for SparrowUP:

 $ bash utils/populate_db.bash 

Download the repository with sparrow scripts:

 $ git clone https://github.com/melezhik/sparrowdo-test.git 

The repository structure should be as follows:

 project-dir/sparrowfile project2-dir/sparrowfile project3-dir/sparrowfile    

Thus, each project folder should contain one file - a sparrowfile with a description of the sparrow script, the format and structure of which are described in the SparrowDO documentation . Here's how, for example, the CPAN Moose and DBIx :: Class package installation script might look like

 #!/usr/bin/env perl6 use v6; use Sparrowdo; task_run %( task => 'cpan-package', plugin => 'cpan-package', parameters => %( list => 'Moose DBIx::Class', install-base => '/opt/lib/perl' ), ); 

And here is the list of projects in the repository with scripts https://github.com/melezhik/sparrowdo-test :

 $ ls -1 complex cpan-package cpan-package2 df-check git-base group nano-setup package-generic perl-app proc-validate README.md service templater 

Next, we need to configure the SparrowUP configuration file (/etc/sparrowup.conf), specifying the directory in which we zachakautili repository with scripts and the same directory in which SparrowUP will save the report files (the directory must be accessible to the user from SparrowUP service is launched), this is how it looks on my machine:

 $ /etc/sparrowup.conf { 'repo' => '/home/melezhik/projects/sparrowdo-test/', 'reports_dir' => '/home/melezhik/whatsup-reports' } 

Ok, now you can run the SparrowUP web service, for a simple example I will use the usual "nohup" (for production systems you can of course write a separate init script):

 $ cd sparrowup $ nohup carton exec morbo app.pl & 

Going to the address 127.0.0.1 380, make sure that the interface is available:

Running sparrow scripts

The list of completed tasks is still empty. Let's run some tasks using the scripts repository. For example, check that we have enough disk space:

 $ cat df-check/sparrowfile use v6; use Sparrowdo; task_run %( task => 'check my disk', plugin => 'df-check', parameters => %( threshold => 70 ) ); 

The running task is added to the task queue, which we are notified of.

Task execution occurs asynchronously. In order for the tasks in the queues to start be executed, you need to start the task handler (this could have been done at the very beginning of the installation before starting the service, just at the time of writing this article I forgot about it :)

 $ cd sparrowup $ nohup carton exec ./app.pl minion worker & 

The application SparrowUP is written on the Mojolicious framework and as a result, the Minion module is used as a task scheduler - all who are interested can refer to the documentation to familiarize themselves with these tools.

Okay, after some time the sheduler will start to perform the task:

And we will be able to see the result in the form of a SparrowUP report:

And here is how, for example, the report of the proc-validate script will look, which checks that the nginx server process is running on the system:

 $ cat proc-validate/sparrowfile use v6; use Sparrowdo; task_run %( task => 'check my process', plugin => 'proc-validate', parameters => %( pid_file => '/var/run/nginx.pid', footprint => 'nginx.*master' ) 

);

Note that the launch of a new task is also possible programmatically using the HTTP API:

 $ curl 127.0.0.1:3000/df-check -d server=127.0.0.1 

You can also specify additional parameters, for example, specify a username and port number for an ssh connection:

 $ curl 127.0.0.1:3000/df-check -d server=192.168.0.1 -d ssh_user=sparrow -d ssh_port=23 

Restarting the task can be performed either by re-running the specified project for the same server, or by clicking the "Reschedulle Job" button.

Limitations (what remains to be done or discussed)

• the operation "bootstrap" - installing the client's sparrow on the target server (if the client is not yet installed) - is implemented only for servers running the CentOS distribution. If interested - write - add support for other distributions.

  
  

• There is no possibility to configure the SparrowUP web interface ssh connections for different target servers that run sparrow scripts, now I edit the config ~ / .ssh / config and forward keys (ssh-copy-id) manually on the server running SparrowUP. Not sure, however, that this kind of settings in general, it makes sense to pull out through the web interface.

  
  
• make it possible to customize the color scheme of reports (now a black background is not the most pleasant solution for the eyes;)

Conclusion

On this familiarity with the service SparrowUP can be completed. Thanks for attention!

As usual, I am waiting for questions and constructive comments.

PS All those who are concerned - congratulations on the day of the system administrator!

PS2 At the end of the article is traditionally a survey that is not directly related to the main topic.