Configuring Munin Notifications

The Munin article - monitoring the network is easy! it was said that monitoring is needed and that the author uses Munin, as well as described how to write a plugin. In this article we will look at how to configure Munin to send problem notifications to the mailbox.

To do this, you need to create a contact in the /etc/munin/munin.conf file and define the parameters \ directives for this contact. This is done like this:

contacts someuser
contact.someuser.command mail -s "Munin notification" somejuser@fnord.comm
contact.someuser.always_send warning critical

This construction means that we create a contact someuser , assign the command to send notifications (in this case, the mail command) and which notifications to send ( warning \ warnings and critical \ critical).

Because Munin collects information every 5 minutes, then notifications (for example, the disk space has run out, the df_ plugin) will be sent every 5 minutes. Theoretically, there is the directive contact.contact.max_messages number , which, as I understand it, should set the maximum number of notifications for the “warning / critical” event. See the list of directives here.
This feature didn't work for me, so I had to write my own script that would resolve this situation.
')
Also, the contact.someuser.command directive allows the use of special variables, instead of which, when sending a message, real values ​​will be substituted. The list of variables is here .

Let us examine all this with an example. Required: send notifications to two users. The first user should receive both critical and warning messages. The second is only critical.

Open /etc/munin/munin.conf and create contacts:

contacts user1 user2
contact.user1.command mail_send_mutt user1mail@gmail.com "Munin notification ${var:host}" "[${var:group};${var:host}] -> ${var:graph_title}: warnings: ${loop<,>:wfields ${var:label}=${var:value}}; criticals: ${loop<,>:cfields ${var:label}=${var:value}}"
contact.user1.always_send warning critical

contact.user2.command mail_send_mutt user2mail@yandex.ru "Munin notification ${var:host}" "[${var:group};${var:host}] -> ${var:graph_title}: warnings: ${loop<,>:wfields ${var:label}=${var:value}}; criticals: ${loop<,>:cfields ${var:label}=${var:value}}"
contact.user2.always_send critical

Here I used the mail_send_mutt self-written script, which takes 3 input parameters:
1. E-mail user
2. Message Header
3. Message text

For sending letters, the console mail client mutt is used, since it works quickly, reliably and does not require any additional settings.

Script text:
#!/bin/bash
# mutt
# :
# 1.
# 2.
# 3.

# -,
# .
# ,
# . " "

email=$1
theme="$2"
ptext="\"$3\""

# : 0 - , 1 - , 2 - .
type_out=0
# .
function out_text () {
dbg_logfile="/var/log/munin/send_mess.log"
if ! test -e "$dbg_logfile"
then
cat "" > "$dbg_logfile"
fi

case $type_out in
1) echo "$1" ;;
2) echo "$1" >> $dbg_logfile ;;
*)
esac
}

#
var_path="/var/log/munin"
# . $email
logfile=""
# ,
if ! test -e "$var_path"
then
mkdir -p $var_path
fi

#
if test -z $email
then
out_text " !"
exit
fi

if test -z "$theme"
then
theme="null"
fi

if test -z "$ptext"
then
ptext="null"
fi

# - $email
#logfile=${email/@/_}
logfile=$var_path"/"${email/@/_}
# , . -
if ! test -e "$logfile"
then
cat "" > "$logfile"
fi

# ,
#
str_found=0
# .
dpos=0
# ,
tdate=""
tmail=""
ttext=""
#
FS="|"
#
FS_d='.'
#
curr_date=$(date +%Y-%m-%d)
out_text "Current date: \"$curr_date\""

# ,
while read fstr
do
# 3 : , e-mail,
# "|"
out_text "str_found begin: $str_found"
out_text "Find symbol: "$FS "; STR:" "$fstr"
dpos=`expr index "$fstr" $FS`
out_text "dpos: $dpos"
tdate=`expr substr "$fstr" 1 $dpos`
fstr=${fstr/$tdate/}
tdate=${tdate/$FS/}
out_text "NEW STR:" "$fstr"
dpos=`expr index "$fstr" $FS`
tmail=`expr substr "$fstr" 1 $dpos`
ttext=${fstr/$tmail/}
tmail=${tmail/$FS/}

dpos=`expr index "$tdate" $FS_d`
let "dpos-=1"
tdate=`expr substr "$tdate" 1 $dpos`

out_text "date: \"$tdate\""
out_text "email: $tmail"
out_text "ttext: $ttext"
out_text "ptext: $ptext"
out_text "------------------------------------------------------"
# ,
if [ "$ptext" == "$ttext" ]
then
#
# , . ( )
out_text "Text EQ found!"
if [ "$tdate" == "$curr_date" ]
then
out_text "Date EQ found!"
let "str_found+=1"
fi
fi
out_text "str_found end: $str_found"
out_text "------------------------------------------------------"

done < "$logfile" #
out_text "str_found: $str_found"
if [ 0 -lt "$str_found" ]
then
out_text " !"
exit
fi

#
echo "$ptext" | mutt $email -s "$theme"

# , ,
echo "$(date +%Y-%m-%d.%H-%M)|$email|$ptext" >> $logfile

I left all my comments in the script, as well as debug output. Maybe someone will fit. By default, message display is disabled. If someone has an idea how to improve the script, I will be happy with the amendments.
For example, there is such a construction:
while IFS=: read name passwd uid gid fullname ignore
do
echo "$name ($fullname)"
done </etc/passwd # .

and the cycle of reading the log file to view already sent messages seems to be altered as follows:
while FS=| read tdate tmail ttext
do
.....
done < "$logfile"
but for some reason it does not work as described: (