Cacti data queries

A little more than a year ago it was necessary to solve the problem of obtaining and displaying a set of similar indicators from several servers, while the number of servers and the number of indicators removed from them could change over time. zabbix_agent was installed on the nodes, user parameters were configured, templates were made, the data was successfully transferred to Zabbix. For the second system - Cacti, the script found somewhere that zabbix_get data zabbix_get and used Data Queries was zabbix_get . But something in this script soon broke down and Zabbix was left as the only working solution.

To understand what Data Queries is and how to make it work, there was only time for the New Year holidays.

I will repeat the common expression of pro-system administrators who want everything to work by itself. In my case, so that for each monitored node it was possible to immediately add all the necessary graphs and when the number of indicators to be removed was changed, the necessary information was added, if not automatically, then as simply as possible. I wanted to have a mechanism that not only receives and processes data, but also knows what data can be obtained. This is exactly how the Data Queries method works, allowing you to collect values ​​as a Data Input and optionally return a list of all possible parameters.
')
In Cacti, this approach is built out of the box. For nodes using the Generic SNMP template, you can see a list of all network interfaces, select the necessary ones, and build on them the necessary graphs associated with it Data Queries SNMP - Interface Statistics . It would seem that it is easier to see how it works, read the documentation here and here and redo it for yourself. But it is painful, in my opinion, an unsuccessful and complex example of a script selected for description, which I figured out after I did everything. Although the user component, where to click in the interface to earn questions does not cause.

So, Data Queries is represented by three varieties: Get SNMP Data , Get Script Server Data , Get Script Data

1. Get SNMP Data is represented only by XML, specifying which parameters to remove and from where, of course, using the SNMP protocol;
2. The methods Get Script Server Data and Get Script Data are defined by the XML template and the associated script. Only in the case of Get Script Server Data , the execution speed was optimized by using a common PHP process that requires the use of PHP. Get Script Data is the most common approach, we will only consider it later.

From the side of Cacti, it looks like this. The script operation parameters and the arguments to be displayed are read from the XML template. After binding the Data Queries to the node, the script will be requested and stored a list of all possible elements (indices) for which it will return values. If the data changes, it is necessary to reread them again by running the Reload Data Query interface for the associated method. During the next survey, the same script should return the requested data for the specified index and argument. Each time it will be a different script call, for all memorized indexes and for each of the displayed arguments. After that, save to Data Source and draw on the chart.

The task to be solved is to take fping polling results from the servers under control, where the remote node is specified as a parameter, for example, ya.ru or habrahabr.ru . And the answer is the values ​​of delays and losses. Accordingly, we can ping many other nodes from each server. The list of nodes should receive automatically.

First create the XML template.

Common parameters

name and description - what is it and what is it intended for in order not to get confused:

 <name>Get fping stats</name> <description>Remote monitoring fping statistics</description> 

<script_path> - full path to the script file, you can use the variable |path_cacti| :

 <script_path>|path_cacti|/scripts/query-fping-stat.sh</script_path> 

<arg_index> , <arg_num_indexes> , <arg_query> , <arg_get> - the values ​​of the parameters passed to the script, which determine its mode of operation and what values ​​it returns. Only four modes, we do not invent anything complicated:

 <arg_index>index</arg_index> <arg_num_indexes>num_indexes</arg_num_indexes> <arg_query>query</arg_query> <arg_get>get</arg_get> 

<arg_prepend> - parameters that will be transferred first in the script command line. No other parameters will be automatically transferred to the script, except for those that determine its mode of operation, that is, we do not know for which server we perform our actions. Required additional arguments are defined here. It is also allowed to use variables :

 <arg_prepend>|host_hostname|</arg_prepend> 

<output_delimeter> - the delimiter character in the returned values. Nothing too convoluted - use a colon:

 <output_delimeter>:</output_delimeter> 

For us, this is enough, the parameters are not so many, that's what's left .

Data fields

Here we describe what data you need to get at the output. Values ​​are divided into two types of Input and Output .

The Input type is associated with the Query mode of operation and should return values ​​characterizing one or another index. For example, for a network interface: a signature, a physical address, a type — anything that will not change over time. The data obtained in this way will be transferred to Data Source which in turn will connect it with the necessary index. At least one Input element must be required, because it is associated with the Data Source .

In addition, variables are generated |query__| that can be used in graph labels and their values ​​will be displayed in the general list of elements when you call Create Graphs for this Host as separate columns.

Output type - fields whose data will be plotted on graphs.

Directly the field is determined very simply:

name - name, query_name - the value that will be passed to the script and direction - type is Input or Output .

We have one Input field that returns the node name. The variable to be created |query_host| :

 <host> <name>host</name> <direction>input</direction> <query_name>host</query_name> </host> 

And four Output fields, maximum, minimum, average response latency and packet loss percentages:

 <avg> <name>avg</name> <direction>output</direction> <query_name>avg</query_name> </avg> <max> <name>max</name> <direction>output</direction> <query_name>max</query_name> </max> <min> <name>min</name> <direction>output</direction> <query_name>min</query_name> </min> <loss> <name>loss</name> <direction>output</direction> <query_name>loss</query_name> </loss> 

Script

The template we have already described everything that is needed. There are four modes of operation, and therefore four modes of calling.

Index

<script_path> <arg_prepend> <arg_index>

The output of all possible values ​​of the indices, each with a new line. For a solvable problem, this is a list of all possible remote nodes that are monitored on the server under control. All servers give the same set of data, so we define them directly in the script, this is the only place where we need to change something. Of course, the data set can be different and can be taken directly from the server, if necessary. For which server should we return the list we learn from the first parameter, which, |host_hostname| remind you, is specified in the template as |host_hostname| :

 hosts="ya.ru habrahabr.ru" zabbix="/usr/local/bin/zabbix_get -s $1" case $2 in index) for host in $hosts; do echo $host; done ;; esac 

Num_indexes

<script_path> <arg_prepend> <arg_num_indexes>

The number of indexes from the first mode:

 numhosts=`echo $hosts | wc -w` case $2 in num_indexes) echo $numhosts ;; esac 

Query

<script_path> <arg_prepend> <arg_query> <query_name>

It is executed for all data fields defined as Input . The output should receive an index output_delimeter <> for each index and the specified data field, each with a new row. Since only one value is defined in the template, and our index is easily readable, so we use it as a result:

 case $2 in query) for host in $hosts do case $3 in host) echo $host:$host ;; esac done ;; esac 

All the listed modes are executed at the moment of forming the list of node elements. The next mode is called each time the data is polled for each index and each argument.

Get

<script_path> <arg_prepend> <arg_get> <query_name>
Called for all data fields defined as Output . The output should be <> for the specified field and index. What is simpler than in Data Input , where you need to output values ​​in a formatted string, everything here is already defined in the template:

 zabbix="/usr/local/bin/zabbix_get -s $1" get_ping () { res=`$zabbix -k ping[$1,$2]` if [ -z $res ] then res=-1 fi echo $res } case $2 in get) echo $(get_ping $4 $3) ;; esac 

All code

 #!/bin/sh hosts="ya.ru habrahabr.ru" numhosts=`echo $hosts | wc -w` zabbix="/usr/local/bin/zabbix_get -s $1" get_ping () { res=`$zabbix -k ping[$1,$2]` if [ -z $res ] then res=-1 fi echo $res } case $2 in index) for host in $hosts; do echo $host; done ;; num_indexes) echo $numhosts ;; query) for host in $hosts do case $3 in host) echo $host:$host ;; esac done ;; get) echo $(get_ping $4 $3) ;; esac exit 0 

It remains to check - create Data Queries bind it to the site and perform a debug launch of Verbose Query .

Template without errors:

 + Found data query XML file at '/usr/local/cacti/resource/script_queries/fping-stat.xml' + XML file parsed ok. 

Num_indexes mode, number of elements:

 + Executing script for num of indexes '/usr/local/cacti/scripts/query-fping-stat.sh server num_indexes' + Found number of indexes: 2 

Index mode, a list of all items:

 + Executing script for list of indexes '/usr/local/cacti/scripts/query-fping-stat.sh server index' Index Count: 2 + Found index: ya.ru + Found index: habrahabr.ru 

Query mode, Output values ​​of all arguments for all elements:

 + Executing script query '/usr/local/cacti/scripts/query-fping-stat.sh server query host' + Found item [host='ya.ru'] index: ya.ru + Found item [host='habrahabr.ru'] index: habrahabr.ru 

If everything works, then we act as always in Cacti, create a data template, graph templates . We connect together , it should be done directly in the created Data Queries . After that, we should get a list of available parameters automatically when using Create Graphs for this Host .

Now we create a graph, then a script call is added to Poller for each Input argument, each index and each node. When viewing the Poller Cache you can see, among other things, with what parameters the script is called:

 Script: /usr/local/cacti/scripts/query-fping-stat.sh server get avg ya.ru 

This means that you can enjoy the almost automatic addition of new graphs, and the sysadmin to continue to do more important things.

PS Did you fix what was broken?

When the text was completely ready, looking at the graphics, I realized that even a pretty altered last year's version does not fix the problem. On many other graphs that use standard Data Queries Script , some time after re-introducing the solution described, data is not removed and displayed several times a day. Perhaps a feature of the used infrastructure, or configuration errors. In any case, this does not cancel the convenience of the Data Queries , but the reason ...: “We will search” (c)