Install and configure Collectd to collect and send data to Graphite (Grafana)

I recently wrote an article about configuring and installing Graphite + Grafana , now I’ll tell you how to install and configure Collectd to collect server status data, send it to Grapfite and display in Grafana .

Foreword

Collectd is a system for collecting, storing and sending metrics about the state of the machine on which it is running.

It has many out-of-box plugins and is very flexible in customization.

And so, let's get started.

Install Collectd

Installation, as in most cases, is simple:

 sudo apt-get install collectd collectd-utils 

Basic setting Collectd

First we need to stop the collectd service.

 sudo service collectd stop 

I suggest making a backup of the standard configuration file and writing your own.

 sudo mv /etc/collectd/collectd.conf /etc/collectd/collectd.conf.bak sudo touch /etc/collectd/collectd.conf 

All settings are in the file /etc/collectd/collectd.conf , we will edit it.

 Hostname "graph_host" #    ( IP   Grafana #    ,    FQDNLookup true 

This completes the basic setup.

Plugin configuration

collectd lot of embedded plugins.

Consider the most necessary.

Mysql plugin

Add the line to the same configuration file

 LoadPlugin mysql 

And below we enter its settings:

 <Plugin mysql> <Database DataBaseName> Host "HOST.OR.IP" Port "3306" User "UserName" Password "PassWord" Database "databasename" MasterStats true </Database> <Database DataBaseName2> Host "HOST.OR.IP" Port "3306" User "UserName" Password "PassWord" Database "databasename2" MasterStats true </Database> #       </Plugin> 

Nginx plugin

It's all a little more interesting.

First you need to configure Nginx itself, namely to enable the nginx_status page.

We check the availability of the required module as part of Nginx :

 nginx -V 2>&1 | grep -o with-http_stub_status_module 

If the output of the command is:

 with-http_stub_status_module 

then everything is fine Otherwise, you need to rebuild Nginx with this module, or upgrade to the latest version of nginx/1.10.1 , which includes this module out of the box.

Next, configure Nginx , edit the file /etc/nginx/conf.d/your.domain.conf :

 server { listen 80; server_name your.domain.com; location / { #     ,    . } location /nginx_status { stub_status on; access_log off; allow IP.AD.DR.ES; # IP     Nginx deny all; } } 

Now, if we open http://your.domain.com/nginx_status in the browser, we will see a 403 error.

This means that left users will not be allowed here.

Now add the line to the collectd configuration file:

 LoadPlugin nginx 

And prescribe settings:

 <Plugin nginx> URL "http://your.domain.com/nginx_status" VerifyPeer false VerifyHost false </Plugin> 

Memory, uptime, users and cpu plugins

Add lines to the same configuration file

 LoadPlugin memory LoadPlugin uptime LoadPlugin users LoadPlugin cpu 

These plugins do not require configuration.

I think the names make it clear what these plugins are responsible for.

Openvpn plugin

If your server is an OpenVPN server, then this plugin will help display some data.

Add the line to the same configuration file

 LoadPlugin openvpn 

And prescribe settings:

 <Plugin openvpn> StatusFile "/etc/openvpn/openvpn-status.log" ImprovedNamingSchema false CollectCompression true CollectIndividualUsers true CollectUserCount false </Plugin> 

With the values ​​of the settings you can play at your discretion.

Swap plugin

If your system uses SWAP {target = "_ bank"}, then this plugin is for you.

Add the line to the same configuration file

 LoadPlugin swap 

And prescribe settings:

 <Plugin swap> ReportByDevice false ReportBytes true </Plugin> 

Vmem plugin

Add the line to the same configuration file

 LoadPlugin vmem 

And prescribe settings:

 <Plugin vmem> Verbose false </Plugin> 

Syslog plugin

Add the line to the same configuration file

 LoadPlugin syslog 

And prescribe settings:

 <Plugin syslog> LogLevel info </Plugin> 

Plugin aggregation

I haven’t figured out this plug-in yet, but judging by the content, it gives the CPU parameters and, possibly, has additional settings. I give the default settings.

Add the line to the same configuration file

 LoadPlugin aggregation 

And prescribe settings:

 <Plugin "aggregation"> <Aggregation> #Host "unspecified" Plugin "cpu" PluginInstance "/[0,2,4,6,8]$/" Type "cpu" #TypeInstance "unspecified" SetPlugin "cpu" SetPluginInstance "even-%{aggregation}" GroupBy "Host" GroupBy "TypeInstance" CalculateNum false CalculateSum false CalculateAverage true CalculateMinimum false CalculateMaximum false CalculateStddev false </Aggregation> </Plugin> 

Bind plugin

If bind9 DNS server is installed on your server, then enable this plugin.

Add the line to the same configuration file

 LoadPlugin bind 

And prescribe settings:

 <Plugin "bind"> URL "http://localhost:8053/" ParseTime false OpCodes true QTypes true ServerStats true ZoneMaintStats true ResolverStats false MemoryStats true <View "_default"> QTypes true ResolverStats true CacheRRSets true Zone "your.domain.com/IN" #   ,     </View> <View "_default2"> #      </View> </Plugin> 

Df plugin

df - display of free and occupied space on all mounted partitions.

Add the line to the same configuration file

 LoadPlugin df 

And prescribe settings:

 <Plugin df> Device "/dev/vda1" #       df -h MountPoint "/" #    FSType "ext4" #    FSType rootfs FSType sysfs FSType proc FSType devtmpfs FSType devpts FSType tmpfs FSType fusectl FSType cgroup IgnoreSelected true ReportByDevice false ReportReserved false ReportInodes false ValuesAbsolute true ValuesPercentage false </Plugin> 

Plugin disk

Works similar to the plugin above, but displays several other metrics.

Add the line to the same configuration file

 LoadPlugin disk 

And prescribe settings:

 <Plugin disk> Disk "vda" #   Disk "/vda[1,2,3,4,5,6]/" #     IgnoreSelected false </Plugin> 

Ping plugin

Checks availability and response time from the specified domain.

Add the line to the same configuration file

 LoadPlugin ping 

And prescribe settings:

 <Plugin ping> Host "www.google.com" Interval 1.0 Timeout 0.9 TTL 255 SourceAddress "8.8.8.8" Device "eth0" MaxMissed -1 </Plugin> 

Plugin write_graphite

The main plugin for which this article is written. Sending data to Graphite .

Add the line to the same configuration file

 LoadPlugin write_graphite 

And prescribe settings:

 <Plugin write_graphite> <Node "Graphite"> #   Host "127.0.0.1" Port "2003" Protocol "tcp" LogSendErrors true Prefix "collectd." Postfix "collectd." StoreRates true AlwaysAppendDS true EscapeCharacter "-" </Node> </Plugin> 

Additional settings

Add lines to the same configuration file

 <Include "/etc/collectd/collectd.conf.d"> Filter "*.conf" </Include> 

This is an import of settings from the /etc/collectd/collectd.conf.d directory with a .conf extension.

Run collectd

Run collectd

 sudo service collectd start 

And if everything went without errors - go to Grafana , add dashboards, and prescribe metrics.

All metrics from collectd are written as follows:

 collectd.MetrikaName.******** 

We build the necessary graphics and enjoy beautiful monitoring.

Full list of plugins

A complete list of collectd plugins collectd of the box can be found here .

Afterword

In the next article I will talk about plugins for Grafana .

Original article published on my blog.