Collect metrics from .NET applications using Telegraf

One of the most important tasks in the design of systems is the organization of monitoring the status of all nodes, including a large number of services. In conditions when additional forces and means are not allocated for this, it is necessary to make the most of ready-made solutions.

I think many people have a picture of this in the project:

Something is sent somewhere, somehow processed and kept on one nail. The task was to collect data processing statistics from all points and put them in one place in order to build graphs and write reports.

Thanks to the article, the choice fell on a bunch of Telegraf-Elasticsearch-Grafana. Telegraf fits perfectly into the project to organize monitoring of the state of both hardware and publicly available third-party software, but I will separately address the issue of measuring the load on my own services. In this case, we are talking about .NET services running in docker containers under Linux. All services form several stages of processing incoming information, and I needed to measure the number of successfully processed and rejected packets with additional marks of the processing stage, the source, and others to enable subsequent statistical processing.

I will skip the installation process and go straight to the configuration. So, Telegraf is able to receive messages with metrics via tcp, udp, and also through unixsocket:

[[inputs.socket_listener]] #service_address = "unixgram:///tmp/telegraf.sock" service_address = "udp4://:14230" data_format = "json" json_name_key = "name" namepass = ["query_pass"] tag_keys = ["appname","fromip"] 

Services will send messages every time the next packet is processed, therefore we will additionally configure aggregation. The interval of 10 seconds is quite enough, depending on the load of a particular system.

 [[aggregators.basicstats]] period = "10s" drop_original = true stats = ["sum"] namepass = ["query_pass"] fieldpass = ["pass","fail"] 

Go through the parameters: query_pass - the name of the metric that combines future measurements, pass - successful processing, fail - not. Additionally, metrics will be tagged with appname and fromip .

Now for some code. I like to send metrics via udp and unixsocket, although other options may be appropriate for you.

 UdpClient udpClient = new UdpClient("127.0.0.1", 14230); byte[] datagramBytes= Encoding.UTF8.GetBytes("{\"name\":\"query_pass\",\"pass\":1,\"fromip\":\"127.0.0.1\",\"appname\":\"application\"}"); udpClient.Send(datagramBytes, datagramBytes.Length); datagramBytes= Encoding.UTF8.GetBytes("{\"name\":\"query_pass\",\"fail\":1,\"fromip\":\"127.0.0.1\",\"appname\":\"application\"}"); udpClient.Send(datagramBytes, datagramBytes.Length); 

All this is perfectly summed up and folds into the base, in my case it is elasticsearch ( screenshot ).

 [[outputs.elasticsearch]] urls = [ "http://localhost:9200" ] # required. timeout = "5s" health_check_interval = "5s" index_name = "telegraf-%Y.%m.%d" # required. manage_template = true template_name = "tp_telegraf" overwrite_template = true 

All cats.

PS: here is the final project for sending metrics under net.core