Use powershell scripts in icinga2

We continue to raise micromonitoring . We will proceed from the fact that we have a park, mainly consisting of windows machines, and they are located in local networks that are not connected with each other, but with access to the Internet. Let us use native windows - powershell and teach our Icinga2 windows agents to send the information we need without direct access to them.

In the latest versions of Icinga2 agent for windows, the location of configuration files has changed, now they are in the directory: C: \ ProgramData \ icinga2 \ etc \ icinga2. On the client, add a global zone in the zones.conf file (in new versions of the agent, such a zone is already registered, but commented out):

object Zone "global-templates" {        global = true } 

On the server, create the /etc/icinga2/zones.d/global-templates directory, and in it the commands.conf file with the following contents where we define the command to execute the powershell scripts:

 object CheckCommand "powershell" { import "plugin-check-command" timeout = 5m command = [ "powershell.exe" ] arguments = { "-command" = { skip_key = true value = "$ps_command$" order = 0 } "-args" = { skip_key = true value = "$ps_args$" order = 1 } } } 

On the server in the same directory, create a services.conf file in which our agent services will be described. To get started, let's add a script update service from the server.
')

 apply Service "upd-powershell-scripts" {   max_check_attempts = 2 //    60    check_interval = 60m   retry_interval = 30m //       Windows   assign where host.vars.os == "Windows" && host.name == NodeName //    Linux   ignore where host.vars.os == "Linux"   check_command = "powershell"   vars.ps_command = "C:\\Scripts\\Icinga2\\update_icinga2_scripts.ps1" } 

When installing the Windows agent, in the etc / conf.d / hosts.conf file, the default variable vars.os = "Windows" is written based on this and this service will be used and ignored on Linux agents.

Now on the client in the directory c: \ Scripts \ Icinga2 you need to place the powershell script that will perform the update scripts.


After a short time, agents download a new version of the global zone.
We start the update of the node configuration on the server and overload the service:

 icinga2 node update-config service icinga2 reload 

Our service is added to the agent and it works.



For example, add the reboot functionality. Add one more service to the services.conf file, which will reboot the OS at our request. Be sure to turn off the active check of such a service (we don’t want the server we are monitoring to reboot every n minutes).

 apply Service "reboot-system" { //   enable_active_checks = false   max_check_attempts = 2 //       Windows   assign where host.vars.os == "Windows" && host.name == NodeName //    Linux   ignore where host.vars.os == "Linux"   check_command = "powershell"   vars.ps_command = "C:\\Scripts\\Icinga2\\Reboot_System.ps1" } 

Again, we start updating the node configuration on the server and overload the service so that the configuration changes take effect:

 icinga2 node update-config service icinga2 reload 

We will place this script on our web server (which is registered in the download script) and the agent will download it himself after a certain period of time. You can check the serviceability of the service with the following command on the server:

 /bin/echo "[`date +%s`] SCHEDULE_FORCED_SVC_CHECK;;reboot-system;`date +%s`" >> /var/run/icinga2/cmd/icinga2.cmd 

After that, the windows agent machine should reboot.

In order for powershell scripts to run on an agent, they must either be signed ( How to sign ) or set the execution policy correctly (Set-ExecutionPolicy). In order for Icinga2 to correctly determine the state of the service after the check (normal / warning / critical), the script must return the correct return code.

We defined the main return codes in scripts as follows:

 $returnStateOK = 0 $returnStateWarning = 1 $returnStateCritical = 2 $returnStateUnknown = 3 

For example, when the service is in critical condition, we return:

  Write-Host " $ComputerName  ." [System.Environment]::Exit($returnStateCritical) 

Additional performance data from the script can be returned like this:

 Write-Host "OK -  :" "<table><thead><tr><th>_</th><th>Value</th></tr></thead><tbody>" "<tr><td>  :</td><td>" + $catridge_usage_prc + " %</td></tr>" "<tr><td> :</td><td>" + $page_count + "</td></tr></tbody></table>" "|catridge_usage_prc=$catridge_usage_prc;10;3;100;0" "|page_count=$page_count;;;;" [System.Environment]::Exit($returnStateOK) 

Here, after the pipe, we send the performance data in the following format:

 'label'=value[UOM];[warn];[crit];[min];[max] 

In icingaweb2, it will look something like this:



If you have scripts with the output of messages in Cyrillic, the files must be saved in UTF-8 with BOM, otherwise there may be problems with the display of these messages in our web interface or in icingaweb2.

There are already a huge number of plugins for icinga / nagios, but in most cases they are designed to work in linux systems. For windows, of course, there are ready-made solutions that supplement the standard icinga2 commands, for example: nsclient ++ , but these are additional entities, additional configurations. In our solution, everything is done using standard windows tools and at the same time there is an opportunity to receive additional information “on the fly” simply by adding an additional service to the Icinga2 configuration and an additional powershell script to the web server.

Links

We raise micromonitoring on icinga2 with minimal costs
Repository with scripts
Monitoring plugins guidelines