Fast detection of MIB modules supported by an SNMP device
When implementing systems for monitoring and managing IT infrastructure, one often comes across “non-standard” devices. Often, only such a device is known for sure that it supports SNMP. Connecting it to the project will have to start with the answer to the question of what information about yourself it provides. Usually, a full device survey is conducted for this, and the data obtained is analyzed to reveal useful information ... But here, as they say, there are nuances. In this post I will talk about one such thing - about the algorithm we have developed for quickly determining the MIB modules “supported” by a device .
Data on the device can be quite a lot. For example, the Cisco 2600 router, with which I experimented a bit in the process of writing this article, produces more than 12 thousand values.
And I must say, this is far from the limit.
In this regard, a couple of problems arise: how to find useful / necessary data in this pile of informational wealth, and how to do it in a reasonable time?
')
The decision of the first lies on the surface: in accordance with the ancient strategy of “divide and conquer”, it is necessary to divide all collected values into a relatively small number of categories, each of which should be assessed for utility. The answer to the question “how and to which categories should the data be split?” Is also quite obvious in this case - all (well, or almost all) data were sorted before us into MIB modules, where descriptions of logically related data elements (variables) are usually added .
The standard way of splitting SNMP data in AggreGate Network Manager is by MIB modules.
Thus, the subtask of highlighting the necessary information is reduced to displaying the received data on MIB-modules (communication is carried out by the variable identifier - OID ).
There are many tools doing something like this. And all of them (at least, all that are known to us) perform a complete device survey.
So, for example, the MIB Walk utility works as part of the Engineer's Toolset from SolarWinds , which polls the same "pussy" from my computer for 3.5 - 4 minutes. It seems to be not so much. But we must bear in mind that this is not the “biggest” device, and that it is available to me on a weakly loaded local network. In the context of this “combat” project, where there is a serious traffic, and the device is on another network, the time of a full survey can grow by orders of magnitude. And while such a survey is going on, a specialist, whose duties include connecting a device, will somehow become distracted, which is called “lose context”, and here you will have to add time to “return to the task” (which often turns out to be a significant weight). It should also be noted that there are often many such devices in a serious project - in some cases we had to study between 2 and 3 dozens of devices. In the end, comes a significant amount.
One way or another, our own specialists in the implementation of monitoring systems, as well as users who independently set up the system, at some point often began to mention waiting for the completion of a complete survey of SNMP devices as one of the factors that significantly “slow down” their work. And I had to invent a way to reduce the time of useless waiting. As a result, we invented and implemented the following algorithm in our system.
Good problem formulation is half the solution. We can describe the task as follows.
The MIB module is a description of some set of SNMP variables. In light of this, the following answer to the question sounds logical: we will assume that the MIB module is supported if at least one variable described in it is present on the device .
Note : There is a small complexity: the same value can be described in different MIBs. Below we take this into account.
The idea of optimization directly follows from this definition: if we find one variable from a certain MIB module on the device, the remaining variables of this module can be excluded from the survey . Since the variables of the MIB module most often go in fairly large blocks, we can not only noticeably, but, as practice shows, radically reduce the amount of data we need to get from the device. Due to this, the polling time will also decrease.
We get this algorithm :
Thus, we do not "walk" (WALK) on the device, but literally rush along it in big jumps.
Bearing in mind the high “accuracy” of OIDs in the MIB module, the algorithm can be slightly improved by preliminarily “thinning out” the original OID list: if a certain sequence of OIDs belongs to the same MIB module (or, more generally, to one set of MIB modules), it makes no sense to check them all - the GET_NEXT request to the first one in any case will either give one of this group, or show that this data block is missing on the device.
The figures show the result of the discovery of MIB modules on the Cisco router mentioned above.
Top of the list of detected modules:
And this is his last page:
As you can see, found 64 MIB-module . By the way, the running time of the algorithm is 1–2 seconds.
The following screenshot shows the result of detection on a “non-standard” Hirschmann Railswitch RSB20 device.
The last two entries represent the “custom” MIB modules shipped with this device.
The “live” process of exposing MIB modules on Hirschmann can be seen in our video clip about connecting non-standard devices (gourmets may be interested in the English version ). True, all the magic with MIBs remains behind the scenes and fits into a two to three second segment, but our approach to working with SNMP devices will become clear.
The algorithm for fast detection of MIB modules supported by the device was implemented in the AggreGate SNMP driver. At the moment, he is well-established and has been working stably in various projects for monitoring IT infrastructure at various levels for several years now. Over the past year, no errors have been identified in it, which indicates, at least, that the idea is correct. Prior to this, from time to time there were inaccuracies, but 99% of them were associated with the agent-specific implementations of the agent implementations that were incorrect in terms of SNMP specifications. But the client is always right, had to make corrections to the driver, taking into account such "features"; it concerned the implementation of this algorithm.
Why is this necessary?
Data on the device can be quite a lot. For example, the Cisco 2600 router, with which I experimented a bit in the process of writing this article, produces more than 12 thousand values.
And I must say, this is far from the limit.
In this regard, a couple of problems arise: how to find useful / necessary data in this pile of informational wealth, and how to do it in a reasonable time?
')
The decision of the first lies on the surface: in accordance with the ancient strategy of “divide and conquer”, it is necessary to divide all collected values into a relatively small number of categories, each of which should be assessed for utility. The answer to the question “how and to which categories should the data be split?” Is also quite obvious in this case - all (well, or almost all) data were sorted before us into MIB modules, where descriptions of logically related data elements (variables) are usually added .
The standard way of splitting SNMP data in AggreGate Network Manager is by MIB modules.
Thus, the subtask of highlighting the necessary information is reduced to displaying the received data on MIB-modules (communication is carried out by the variable identifier - OID ).
There are many tools doing something like this. And all of them (at least, all that are known to us) perform a complete device survey.
So, for example, the MIB Walk utility works as part of the Engineer's Toolset from SolarWinds , which polls the same "pussy" from my computer for 3.5 - 4 minutes. It seems to be not so much. But we must bear in mind that this is not the “biggest” device, and that it is available to me on a weakly loaded local network. In the context of this “combat” project, where there is a serious traffic, and the device is on another network, the time of a full survey can grow by orders of magnitude. And while such a survey is going on, a specialist, whose duties include connecting a device, will somehow become distracted, which is called “lose context”, and here you will have to add time to “return to the task” (which often turns out to be a significant weight). It should also be noted that there are often many such devices in a serious project - in some cases we had to study between 2 and 3 dozens of devices. In the end, comes a significant amount.
One way or another, our own specialists in the implementation of monitoring systems, as well as users who independently set up the system, at some point often began to mention waiting for the completion of a complete survey of SNMP devices as one of the factors that significantly “slow down” their work. And I had to invent a way to reduce the time of useless waiting. As a result, we invented and implemented the following algorithm in our system.
Algorithm for quick discovery of MIB modules available on an SNMP device
Good problem formulation is half the solution. We can describe the task as follows.
A list of MIB modules and an SNMP device are given.Such a formulation immediately raises the question: What does "MIB-module supported by device" mean?
It is required to determine whether this device is “supported” by each of these MIB modules.
The MIB module is a description of some set of SNMP variables. In light of this, the following answer to the question sounds logical: we will assume that the MIB module is supported if at least one variable described in it is present on the device .
Note : There is a small complexity: the same value can be described in different MIBs. Below we take this into account.
The idea of optimization directly follows from this definition: if we find one variable from a certain MIB module on the device, the remaining variables of this module can be excluded from the survey . Since the variables of the MIB module most often go in fairly large blocks, we can not only noticeably, but, as practice shows, radically reduce the amount of data we need to get from the device. Due to this, the polling time will also decrease.
We get this algorithm :
- First, we will create lists of OIDs described in each MIB of our library. For each OID, remember which MIB it belongs to (there may be several, remember?), And merge these lists into a single set, sorting the OIDs in lexicographical order
- Now, having received the next variable from the device using GET_NEXT and defining the MIB-modules to which it belongs, we not only “include” these modules in the list of supported, but we can remove from the list all OID belonging to (only) these MIB-modules -s
- The next GET_NEXT is already done for the first variable in the list of polls.
Thus, we do not "walk" (WALK) on the device, but literally rush along it in big jumps.
Bearing in mind the high “accuracy” of OIDs in the MIB module, the algorithm can be slightly improved by preliminarily “thinning out” the original OID list: if a certain sequence of OIDs belongs to the same MIB module (or, more generally, to one set of MIB modules), it makes no sense to check them all - the GET_NEXT request to the first one in any case will either give one of this group, or show that this data block is missing on the device.
results
The figures show the result of the discovery of MIB modules on the Cisco router mentioned above.
Top of the list of detected modules:
And this is his last page:
As you can see, found 64 MIB-module . By the way, the running time of the algorithm is 1–2 seconds.
The following screenshot shows the result of detection on a “non-standard” Hirschmann Railswitch RSB20 device.
The last two entries represent the “custom” MIB modules shipped with this device.
The “live” process of exposing MIB modules on Hirschmann can be seen in our video clip about connecting non-standard devices (gourmets may be interested in the English version ). True, all the magic with MIBs remains behind the scenes and fits into a two to three second segment, but our approach to working with SNMP devices will become clear.
Conclusion
The algorithm for fast detection of MIB modules supported by the device was implemented in the AggreGate SNMP driver. At the moment, he is well-established and has been working stably in various projects for monitoring IT infrastructure at various levels for several years now. Over the past year, no errors have been identified in it, which indicates, at least, that the idea is correct. Prior to this, from time to time there were inaccuracies, but 99% of them were associated with the agent-specific implementations of the agent implementations that were incorrect in terms of SNMP specifications. But the client is always right, had to make corrections to the driver, taking into account such "features"; it concerned the implementation of this algorithm.
Source: https://habr.com/ru/post/255839/
All Articles