Splunk 7.0. What's new?

A month ago, Splunk at its 8th annual conference Splunk Conf 2017 presented the release of a new major release Splunk 7.0. In this article we will talk about the main innovations and improvements of the platform, as well as show a couple of examples.

Metrics

In the new release, Splunk introduces a separate data type for indexing - metrics. These are mostly technological data describing a specific process or system, or rather their state. These are well-known indicators, such as: CPU utilization, free RAM or disk space, AWS (Amazon Web Service) indicators, IoT, and much more. A full list is available here . One thing to say, in order to load metrics into Splunk, they must have three required fields: time, metric name and value, the rest is optional, see the picture below.


Also, when creating a new index, now you need to specify that this is data with metrics and not just logs. Source types such as StatsD and collectD are available out of the box, for the rest you will need to configure transforms.conf and props.conf , but this is not difficult.
')

New SPL commands for metrics

To work with metrics, Splunk has made new mstats and mcatalog commands . True, mcatalog is not officially supported yet, so we will show only examples of queries with mstats, which, as you already understood, are very similar to stats , the main difference from stats is that the search starts right away with pipe line |, and not with filtering.

Calculate the average speed:

| mstats avg(_value) WHERE metric_name="car.speed" AND driver_id="*" span=1m 

Add a division by car_ip, for this we use an additional timechart:

 | mstats avg(_value) prestats=t WHERE metric_name="car.speed" AND driver_id="*" span=1m BY car_ip | timechart avg(_value) AS speed span=30m BY car_ip 

Why is all this necessary?

Good question, because before all the same could be done using normal indexing, timechart and stats, and everything worked. In fact, the answer is obvious - this is speed. For metrics, the speed of processing search queries, according to the developers, increases 20 times and at the same time reduces the overall load on the system. Impressive, isn't it?

Improved visualization

Now annotations can be added to the graphics, Splunk called this feature Event Annotation . True, it should be immediately noted that it only works with line charts, column charts, and area charts.



You can choose a color, the message in the annotation can be either static or formed from a specific field of the event, well, it is clear that the annotation is a separate request in the same time interval as the main request. All this is done very simply, one has only to add a couple of lines to the original xml dashboard. Below is an example, but in detail about it here .

Updated Machine Learning Toolkit

The main update of the ML Toolkit is the addition of a new algorithm, ARIMA , which brings it very close to the real Machine Learning. In addition, the role model was expanded, now each user can save their work separately, as well as the API is expanded, about it in detail here .

Total

It is clear that in this release there are a lot of different features, such as: updated Monitoring Console, Chart Enhancements, Report Action Enhancements , general optimization of the platform and search acceleration and much more, but we tried to tell about the most important and interesting innovations.

Additionally

For the most in-depth study of the issue is to install the application Splunk Enterprise 7.0 Overview , as well as watch the official release video .

Also, do not forget that for any question regarding Splunk: its implementation, the development of applications on it, the addition of new, complexly indexed events and everything else you can contact us through our website.

PS

On June 28, 2018, “ Splunk Getting Started ” will be taught in Moscow , where in 6 hours the participants will receive a theoretical base and practical skills for working in Splunk. Learn more about learning and register at this link .