Architecture IoT-solutions on a real example

We continue to talk about solution development companies (ISVs). In this issue, the technical director of the company INPROSISTEM AlexandrSurkov tells about the experience of developing the architecture of security IoT-system SESMIK .

Many believe that the concept of “Internet of Things” is inextricably linked to the network that we use every day. One can imagine a picture where a multitude of devices, united in a single whole through a global network, exchange data between themselves and servers and create a digital picture of the world. In this article I will talk about how we made a system that combines hundreds of sensors.


The concept of "Internet" should be considered much broader. In this case, it is a common network for devices. It may contain 10 devices, and maybe 10,000. It can be wired, and maybe wireless. It may be located in the same room, and may cover several countries. It all depends on the tasks that are put in front of the system.
')
At the same time creating even a small network of devices is accompanied by many difficulties.

Formulation of the problem

We were assigned the task of developing a perimeter security system. The perimeter is a fence surrounding an object. Its length is unlimited.

The system was created from scratch. By the time the design began, there was a prototype of a sensor capable of collecting perimeter vibrations, by analyzing which, it was possible to clearly determine whether the fence was being overcome or destroyed. Experimentally, we determined that the sensors should be placed approximately every 10 meters.

In addition to sensors, control devices with relays and controlled devices with “dry contact” were planned. The system should work in outdoor conditions with a wide range of temperatures and weather conditions.

So, there is:

• 3 types of devices;
• At least 100 devices per kilometer;
• The number of kilometers is unlimited;
• The system must have street performance.

You can immediately identify the main issues of architecture:

• Organization of data transmission and power;
• Distribution of information flows: where and how to analyze data;
• Security solutions: what protocols to use;
• How to manage so many devices.

General solution scheme

Clemens Vasters has written an excellent article on what the developer of the Internet of Things is facing. The same problems had to be solved by us.

Many solutions were dictated by GOST and other requirements for such systems. But much had to come up with themselves.

First, it was necessary to understand how to distribute information flows in the system. For the analysis of oscillations, both the time and frequency components were used. The frequency range from 0 to 500 hertz. Means, measurements need to be done 1000 times a second. Each measurement is made in 3 axes of 10 bits each.
Total 3 * 10 * 1000 = 30 kilobits per second from only one sensor. Per kilometer (100 sensors) it is already 3 megabits.

You can transfer this data, but how to process it? A perimeter of 10 km would have given 30 megabits of data per second. It turns out that the server load increases with the number of devices.

It was decided that each sensor would process the data itself and give only alarm facts. This significantly reduced the amount of data and distributed the computational load.

Networking was chosen wired due to the fact that wireless requires a battery. Since the system is real-time, the battery life will be very small. In addition, negative temperatures greatly reduce the capacity of the power supply. Yes, and mute the wireless connection is relatively easy, and this is already a very significant drawback to the security system.

GOSTs prohibit laying 220V on the fence, so our devices will be “powered” from constant voltage.

We chose a bus as the network architecture. This made it possible not to pull the wire to each sensor.

However, the bus imposes a restriction on the length of the network and the number of devices. Therefore, a new device was introduced: the gateway. It has 2 buses, is engaged in device management and data broadcasting to and from the server. He also controls the parameters of nutrition and the environment.
Thus, another large load is removed from the server: sensor management.

In addition, such a modular approach with a distributed computational load makes it possible to integrate a very large number of sensors into a common system without a significant increase in server requirements.

The remaining actuators as well as the sensors are connected to the bus.

The result is such a scheme:



Selection of protocols and methods of interaction

Now it was necessary to determine how the data will be transmitted. Each device in the system participates in 2 types of exchange:

• command from the server - the answer to the server;
• asynchronous event from device to server.

Commands are used to change the status of a device and its settings. Events are generated if the device has information for the server (for example, an alarm from the sensor)

On the server side, the gateway is considered to be the same actuating device as everyone else, since, in addition to managing devices on the bus, it also takes care of monitoring power and temperature parameters.

As a bus, the choice was mainly between RS485 and CAN . Only these two standards made it possible to connect multiple devices over fairly large distances.

As a result of the analysis, we preferred CAN. The network parameters are as follows:

• Maximum 500 meters;
• Maximum 100 devices;
• The speed is 50 kbps.

In our opinion, these are the optimal balance of speed, device density and network length for our system.

The gateway uses a microcontroller with two CAN drivers on board, which allowed us to make 2 flank and close one gateway 1 km.

CAN also has other advantages: noise immunity, collision resolution and guaranteed delivery of packets to the addressee. In addition, it has a fairly strong model for diagnosing errors in the network.

However, CAN is only the link layer of the network. To work directly with it, there are many higher level protocols. The most famous of them is CANOpen .

After examining the various options, we concluded that none of the protocols does not suit us. Some are too complicated for our task, some require fees for use, and some do not support what we would like to implement.

And I wanted to implement the system on the principle of PlugAndPlay :

• Connecting and disconnecting devices without power off;
• Automatic detection of system configuration changes;
• The system should start working immediately after assembly.

.

As a result, we managed to do what was intended by writing our own simple but powerful enough protocol. Since we are talking about a small bus bandwidth and a small computing power of microcontrollers, the type of protocol was chosen byte. Due to the optimization of bandwidth, the protocol turned out to be rather strongly connected with CAN, but we managed to make it theoretically portable to other standards.

The protocol allows:

• Detect connected devices on the fly
• detect disconnection of devices;
• work in the mode of request-response;
• send asynchronous events;
• transfer streaming data from the device.

Having established the exchange between devices and the gateway, it remains to deal with the server.

Gateway has an Ethernet output. This is the most universal data transmission technology. The network can be organized as you like: optical channels, wireless channels, a regular twisted pair - at the same time we can always connect to this network using optical converters and access points. This allows the customer to design the network infrastructure of any complexity and length.

Data transfer was organized using TCP / IP based Socket Berkeley . This solution allows the server to receive information from any sensor guaranteed and does not depend on software platforms. We have developed our own protocol over TCP / IP. It is also byte, to optimize the work on the side of the microcontroller. Byte protocols have a big minus: complexity with the subsequent modification. However, the text protocol for the microcontroller device is too redundant.

The most difficult in terms of software development was the server. We implemented an asynchronous multi-threaded interaction model, which allowed us to obtain a “live” system that instantly reacts to any changes. Connecting a new device, loss of communication with the gateway, alarm from the sensor, opening the lid on the device - any event in the system is instantly recorded, even if they occur simultaneously.

As a result, we got a flexible modular system, managed through a single center - the server. It also has its own protocol, allowing you to connect to it and receive events in the system. This allows you to use our system as an integral part of a large complex and scale it almost to infinity.

Security questions

With the security of the system, everything turned out to be quite simple. The fact is that all networks that are located on protected objects are themselves protected objects. Thus, all networks with which the system works become “trusted”.

In addition, the “price” of hacking the security information system is much higher than other ways to overcome. In other words, an experienced intruder will find a simpler way to overcome the obstacle, and a less experienced one simply cannot crack the system.

Therefore, we did not use any special ways to protect information, limiting ourselves only to the basic principles.

What's next?

Despite the fact that the system has already been formed, we continue to actively develop it and look for new methods of application.

One of the directions of development of the system is machine learning. Using these algorithms, you can filter out regular interference, such as noise from a truck train and aircraft. In experiments for this direction Azure Machine Learning helps us a lot. It contains many ready-made solutions for machine learning, which allows you to quickly get results.
Analysis of fence oscillations is not the only way to use the technologies incorporated in our system. Controlling vibrations of high-rise buildings, pipelines and gas pipelines, fragile loads, vibration diagnostics of turbines and moving parts of various designs is not a complete list of possibilities.

The number of sensors in such systems will only increase and here we practically do not pass the transition to cloud systems at sites for which the use of the Internet is not prohibited.

The new IoT technologies from Microsoft seem very promising to us. A single Windows platform can theoretically save a lot of time, since you can write code that is common for different hardware platforms.

And for data processing use Azure IoT Suite. According to the developers, it contains tools that allow not only to combine and manage multiple IoT devices, but also to process large amounts of data from them. This is what we are going to check in the near future.

Conclusion

When we started developing the system, the concept of “Internet of Things” has not yet gained such popularity. There was little experience, we encountered many things for the first time. Now, when much is written and told about this concept, it has become clear that the right path has been chosen.

The work was difficult and long. Creating the first commercial version of the system took about 3 years. The first went to the development of engineering samples of individual devices. Another year was spent on developing the system as a whole. The third year was debugging and debugging.

During this time we have gained tremendous experience in solving various engineering problems. Moreover, the selection of buildings, cable products, organization of production and logistics took no less effort than the development of the system itself.

Now the system is mounted and works on many objects in Russia and abroad. The largest of them consists of several perimeters with a total length of more than 15 km. In the design are larger objects.

More information can be obtained on the system website .