Geekly Articles each Day
📜 ⬆️ ⬇️

Zabbix: LLD monitoring of IPMI sensors


Hello everyone, my name is Andrew. And I love "iron". And I also love monitoring. And finally, a real server with IPMI and other adult technologies appeared in my life. And I hope there will be more. That is what motivates me to search for universal solutions. So: LLD detection of IPMI sensors.

What do you get out of the box:


Well, it looks like this:


Totally needed:
')
  1. Bash script
  2. Template
  3. Documentation or Internet to rename items

First of all, I’ll clarify that you should have a prepared Zabbix 3.2 working server with the ipmitool utility installed, a configured IPMI node.

Script


It is executed on the server as an external check, therefore it must be located in the externalscripts folder that is specified in the server configuration. In Ubuntu, the default is / usr / lib / zabbix / externalscripts . Do not forget to set the appropriate rights to execute this script.

ipmi.sh
#!/bin/bash
# 2017/06/08 AcidVenom v1
# IPMI Zabbix
#
# :
IFS=$'\n'
JSON="{\"data\":["
SEP=""
sensors=`ipmitool -I lanplus -H $1 -L $2 -U $3 -P $4 sensor | grep "$5"`
for sensor in $sensors
do
ID=`echo $sensor | sed "s/ *|.*//"`
NAME=`echo $ID | sed "s/.*/\U&/" | sed "s/+//g" | sed "s/ /_/g"`
LU=`echo $sensor | cut -c 54-63 | sed "s/ //g"`
LC=`echo $sensor | cut -c 66-75 | sed "s/ //g"`
LNC=`echo $sensor | cut -c 78-87 | sed "s/ //g"`
UNC=`echo $sensor | cut -c 90-99 | sed "s/ //g"`
UC=`echo $sensor | cut -c 102-111 | sed "s/ //g"`
UU=`echo $sensor | cut -c 114-123 | sed "s/ //g"`
STATUS=`echo $sensor | cut -c 46-51 | sed "s/ //g"`
JSON=$JSON"$SEP{\"{#ID}\":\"$ID\", \"{#NAME}\":\"$NAME\", \"{#LU}\":\"$LU\", \"{#LC}\":\"$LC\", \"{#LNC}\":\"$LNC\", \"{#UNC}\":\"$UNC\", \"{#UC}\":\"$UC\", \"{#UU}\":\"$UU\", \"{#STATUS}\":\"$STATUS\"}"
SEP=", "
done
JSON=$JSON"]}"
echo $JSON
view raw ipmi.sh hosted with ❤ by GitHub


Template


For each connected node, you must specify 4 macros, namely: {$ IPMIIP}, {$ IPMIPRIV}, {$ IPMIUSER}, {$ IPMIPASS}. Their meanings are intuitive, except that just now {$ IPMIPRIV} is the user's role (ADMIN, USER, etc.). You need to make them, as there are no standard macros in Zabbiks. Perhaps in the future they will appear.

The peculiarity of the template, as in the previous article, is the double transformation of the {$ {# X}} macros. It allows you to replace sensor names with readable ones. You should agree that the “Battery voltage of the BIOS” looks more pleasant, and not “The voltage of the BB_3.3V_VBAT”.

All you need to do for this is to add the corresponding macro to the list of template in the form:

{$ CPU1_TEMPERATURE} = CPU1

The list already has several conversions for the Intel S1200 and Asus RS300.

Few cuts
BB - BaseBoard
VR - Voltage Regulator
SSB - Server South Bridge

Template
<?xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.2</version>
<date>2017-06-13T07:58:22Z</date>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<templates>
<template>
<template>IPMI - Sensors</template>
<name>IPMI - Sensors</name>
<description> IPMI :&#13;
, , &#13;
&#13;
ipmi.sh&#13;
&#13;
():&#13;
{$IPMIIP} - IP- &#13;
{$IPMIPRIV} - &#13;
{$IPMIUSER} - &#13;
{$IPMIPASS} - &#13;
</description>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<applications>
<application>
<name></name>
</application>
</applications>
<items/>
<discovery_rules>
<discovery_rule>
<name></name>
<type>10</type>
<snmp_community/>
<snmp_oid/>
<key>ipmi.sh[{$IPMIIP},{$IPMIPRIV},{$IPMIUSER},{$IPMIPASS},degrees]</key>
<delay>3600</delay>
<status>0</status>
<allowed_hosts/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<delay_flex/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<filter>
<evaltype>0</evaltype>
<formula/>
<conditions/>
</filter>
<lifetime>30</lifetime>
<description/>
<item_prototypes>
<item_prototype>
<name> {${#NAME}}</name>
<type>12</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>ipmi.temp.[{#NAME}]</key>
<delay>60</delay>
<history>90</history>
<trends>180</trends>
<status>0</status>
<value_type>0</value_type>
<allowed_hosts/>
<units>°C</units>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor>{#ID}</ipmi_sensor>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name></name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
</item_prototypes>
<trigger_prototypes>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&gt;{#UNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&gt;{#UC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&lt;{#LC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&lt;{#LNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>2</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&gt;{#UU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>5</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.temp.[{#NAME}].last()}&lt;{#LU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
</trigger_prototypes>
<graph_prototypes/>
<host_prototypes/>
</discovery_rule>
<discovery_rule>
<name></name>
<type>10</type>
<snmp_community/>
<snmp_oid/>
<key>ipmi.sh[{$IPMIIP},{$IPMIPRIV},{$IPMIUSER},{$IPMIPASS},RPM]</key>
<delay>3600</delay>
<status>0</status>
<allowed_hosts/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<delay_flex/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<filter>
<evaltype>0</evaltype>
<formula/>
<conditions/>
</filter>
<lifetime>30</lifetime>
<description/>
<item_prototypes>
<item_prototype>
<name> {${#NAME}}</name>
<type>12</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>ipmi.fan.[{#NAME}]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor>{#ID}</ipmi_sensor>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name></name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
</item_prototypes>
<trigger_prototypes>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&gt;{#UNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&gt;{#UC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&lt;{#LC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&lt;{#LNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>2</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&lt;{#LU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.fan.[{#NAME}].last()}&gt;{#UU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>5</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
</trigger_prototypes>
<graph_prototypes/>
<host_prototypes/>
</discovery_rule>
<discovery_rule>
<name></name>
<type>10</type>
<snmp_community/>
<snmp_oid/>
<key>ipmi.sh[{$IPMIIP},{$IPMIPRIV},{$IPMIUSER},{$IPMIPASS},Volts]</key>
<delay>3600</delay>
<status>0</status>
<allowed_hosts/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<delay_flex/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<filter>
<evaltype>0</evaltype>
<formula/>
<conditions/>
</filter>
<lifetime>30</lifetime>
<description/>
<item_prototypes>
<item_prototype>
<name> {${#NAME}}</name>
<type>12</type>
<snmp_community/>
<multiplier>0</multiplier>
<snmp_oid/>
<key>ipmi.volt.[{#NAME}]</key>
<delay>60</delay>
<history>90</history>
<trends>365</trends>
<status>0</status>
<value_type>0</value_type>
<allowed_hosts/>
<units>V</units>
<delta>0</delta>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<formula>1</formula>
<delay_flex/>
<params/>
<ipmi_sensor>{#ID}</ipmi_sensor>
<data_type>0</data_type>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name></name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<application_prototypes/>
</item_prototype>
</item_prototypes>
<trigger_prototypes>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&gt;{#UNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&lt;{#LC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&gt;{#UC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&lt;{#LNC}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>2</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&gt;{#UU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>5</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
<trigger_prototype>
<expression>{IPMI - Sensors:ipmi.volt.[{#NAME}].last()}&lt;{#LU}</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name> {${#NAME}} {HOST.NAME}!</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger_prototype>
</trigger_prototypes>
<graph_prototypes/>
<host_prototypes/>
</discovery_rule>
</discovery_rules>
<httptests/>
<macros>
<macro>
<macro>{$3.3V}</macro>
<value>+3.3</value>
</macro>
<macro>
<macro>{$3.3VSB}</macro>
<value>+3.3SB</value>
</macro>
<macro>
<macro>{$5V}</macro>
<value>+5V</value>
</macro>
<macro>
<macro>{$5VSB}</macro>
<value>+5SB</value>
</macro>
<macro>
<macro>{$12V}</macro>
<value>+12V</value>
</macro>
<macro>
<macro>{$BB_3.3V_VBAT}</macro>
<value> BIOS</value>
</macro>
<macro>
<macro>{$BB_12.0V}</macro>
<value>+12V</value>
</macro>
<macro>
<macro>{$BB_BMC_TEMP}</macro>
<value>BMC</value>
</macro>
<macro>
<macro>{$CPU1_TEMPERATURE}</macro>
<value>CPU1</value>
</macro>
<macro>
<macro>{$CPU2_TEMPERATURE}</macro>
<value>CPU2</value>
</macro>
<macro>
<macro>{$CPU_FAN1}</macro>
<value>CPU1</value>
</macro>
<macro>
<macro>{$CPU_FAN2}</macro>
<value>CPU2</value>
</macro>
<macro>
<macro>{$FRNT_FAN1}</macro>
<value>Front1</value>
</macro>
<macro>
<macro>{$FRNT_FAN2}</macro>
<value>Front2</value>
</macro>
<macro>
<macro>{$FRNT_FAN3}</macro>
<value>Front3</value>
</macro>
<macro>
<macro>{$FRNT_FAN4}</macro>
<value>Front4</value>
</macro>
<macro>
<macro>{$FRONT_PANEL_TEMP}</macro>
<value> </value>
</macro>
<macro>
<macro>{$PROCESSOR_1_FAN}</macro>
<value>CPU1</value>
</macro>
<macro>
<macro>{$PROCESSOR_2_FAN}</macro>
<value>CPU2</value>
</macro>
<macro>
<macro>{$PROCESSOR_3_FAN}</macro>
<value>CPU3</value>
</macro>
<macro>
<macro>{$PROCESSOR_4_FAN}</macro>
<value>CPU4</value>
</macro>
<macro>
<macro>{$SSB_TEMP}</macro>
<value>SB</value>
</macro>
<macro>
<macro>{$SYSTEM_FAN_1}</macro>
<value>System1</value>
</macro>
<macro>
<macro>{$SYSTEM_FAN_2}</macro>
<value>System2</value>
</macro>
<macro>
<macro>{$SYSTEM_FAN_3}</macro>
<value>System3</value>
</macro>
<macro>
<macro>{$SYSTEM_FAN_4}</macro>
<value>System4</value>
</macro>
<macro>
<macro>{$VBAT}</macro>
<value> BIOS</value>
</macro>
<macro>
<macro>{$VCORE1}</macro>
<value>CPU1</value>
</macro>
<macro>
<macro>{$VCORE2}</macro>
<value>CPU2</value>
</macro>
</macros>
<templates/>
<screens/>
</template>
</templates>
</zabbix_export>
view raw IPMI - Sensors.xml hosted with ❤ by GitHub


Little about filtering


Not all sensors need to be read, this is a fact. For example, why do I need a total stock (Agg Margin) for temperature? For such cases, each detection has its own filter. But alas, it is impossible to switch it to the "does not match" mode. A possible solution is to use global regular expressions (Result is FALSE). The filter also adds the name of the expression with the @ symbol.

For each of the discoveries I made for myself: IPMI-FAN, IPMI-VOLT, IPMI-TEMP (they do not fit into the template).

IPMI-FAN, IPMI-VOLT, IPMI-TEMP




Little about triggers


Values ​​for trigger conditions are taken from the SDR, that is, from the controller itself. SDR fields contain 6 columns of thresholds: lower dangerous, lower critical, lower non-critical, upper non-critical, upper critical, upper dangerous. If the value of one of the fields is missing, then the trigger is not created. IMHO, the most logical way to change the trigger is to change the device's SDR fields to fit your needs. How to do this - read the instructions for your controller or MP.

Total


My zabbix- bikes solutions and periodically refined. This article presents a starting template. And it was developed with a small margin for scalability. For example, replacing the last variable in the key ipmi.sh [..], you can remove power, air flow, other indicators. The application is limited by your ingenuity.

Source: https://habr.com/ru/post/330604/

More articles:


All Articles