What is the Tarantool IIoT platform?

Recently, in a press release, we talked about the launch of Tarantool IIoT , a platform for the industrial Internet of things. The news spread through many electronic publications. But what is Tarantool IIoT and how it works - the topic remained not fully disclosed. We decided to fix it. Details under the cut.

And what is IIoT in general?

First, I remind you that IIoT is the industrial internet of things. This is such a concept, when industry objects have access to the Internet both to the network and access to Internet services (which work in data centers - data centers). Let's look at the drawing:

What do we see here? The figure is divided into two parts - "center" and "on the ground."

A “center” is a data center or cloud, that is, any IaaS cloud service where IT infrastructure can be deployed: Amazon, Azure, etc.

"On the ground" - I honestly do not know which correct term to use here. I'm sure the readers will correct me and offer a beautiful definition. I'd rather tell you what I mean by "on the ground": in the field (in the literal sense of the word - in the agricultural field), at the production site (at the factory, factory, metalworking, mining and oil and gas facilities), at the repair site, at the transport (in a car, truck, train wagon, locomotive), at city infrastructure (at places where information on electricity and water consumption is collected, in sewers, along streets, at lighting objects), at long-distance infrastructure objects (along highways, railways possibly email lines electrotransmission). In general, “on the ground” is everywhere that the hand of the Internet is just beginning to get. You can still say "in the field", but then there is a risk of narrowing the circle of the industrial Internet of things only to the agricultural field.

And what is Tarantool IIoT? What problems is he meant to solve?

Tarantool IIoT is a distributed software platform for collecting information from sensors and sending it to the center and to local control systems. Tarantool IIoT works both "in the center" and "on the ground", connecting them into a single information network. In this network, the maximum of logic is carried away into software, so from the point of view of hardware, it can be assembled from commodity components, that is, it does not need special suppliers of proprietary hardware and software platforms. It became clearer? If not, read on :-)

Under the commodity-components refers to easily replaceable cheap components. For example, if we recall data centers, then from the point of view of their iron, commodity is cheap servers, for example, supermicro or even fully self-assembled servers. From the point of view of iron "on the ground", these may be the cheapest sensors and IoT hubs (i.e., ARM-based small computers)

Inside our data centers in Mail.Ru Group, we have long been almost completely switched to commodity-iron. Sometimes without a branded brand warranty and support, but inexpensive and interchangeable. At the same time, reliability and performance are provided at the software level. If something breaks, then it is not critical for services, because everything is duplicated and fixed at the software level - we just calmly send the hardware for repair. Timing and generally the possibility of repairing iron does not affect business continuity. In the same way, large American IT corporations have long gone: all the complexity is transferred to software, while hardware is standardized and cheapened to the maximum. And now we propose to extend this approach to the world of IoT.

The question is, why places and the center to connect into a single information network? The answer is:

1. On the ground there are sources of information (temperature, vibration, geo-location, humidity, voltage, pressure, etc.). It would be good to collect this information, aggregate and transfer to the center for analytics, in order to draw conclusions about the number and points of losses during production, consumption of raw materials and fuel, field performance, service periods, the probability of machine and instrument failures, etc.
2. On the ground there is infrastructure, which is sometimes convenient to manage from the center, including on the basis of previously obtained analytics. To control means to send commands (call API) from the center to the places that will activate (turn off, turn on, shrink, increase, slow down, speed up) local objects. In my opinion, it is too early to talk about the full and widespread production management from the center (no one wants a 15-ton press to drop due to a frozen network connection), but many non-critical things can be done. For example, if, based on the analysis of data from the vibration sensors of the equipment, it became clear that the equipment would soon fail, then a command could be sent from the center to the factory that would light a red flashing light (yes, that’s right: if you push this event onto a tablet employee, he simply will not notice him, because he will be busy). Seeing a blinking light bulb, the staff will stop the broken machine and order early maintenance.

On the ground, as you understand, has its own specifics, different from the data center. I’ll make a reservation at once that we at the Mail.Ru Group are only at the beginning of the development of our IIoT strategy and we have been testing the IIoT market not so long ago, so perhaps we don’t fully understand all the details. That's what we think about specificity. It:

• huge areas (take a medium-sized factory - it can surpass even a large data center by hundreds of times);
• harsh conditions (heat, cold, humidity);
• increased risk of damage to IT facilities;
• the high cost of replacing IT infrastructure facilities (getting into the field a thousand versts off-road and driving around a hundred square kilometers there is no way to hit the road in Moscow);
• information that is external to the IT infrastructure and should be collected using temperature, humidity, density, geo-location, weight, voltage, etc. sensors

What does all this specifics mean? What we can not build in factories, fields, ships, city streets and other objects is what we build in data centers: rows of identical racks with servers inside, connected via optical fiber. It will be insanely expensive.

And what we still can not? In the IoT world, the role of servers is played by IoT Hubs - microcomputers (industrial analogues of Raspberry PI), which are scattered over many kilometers of sites. Between them, fiber optic is expensive. And in racks it is also expensive to collect them, because due to the huge distances IoT Hubs are too many, and financially limited enterprises cannot form local clusters of a large number of IoT Hubs. That is, there are no localized racks. There is iron scattered around the area. Further, companies / facilities have millions of sensors scattered throughout (see above, which sensors are we talking about) from which IoT Hubs collect information. Again - how to collect it? If there are few sensors, then by wire. And if there are a million of them, then through radio channels (a million wires, along the wire from the sensor, can you imagine what a tangle is?).

As a result, in order for the solution to be more or less cost-effective, the whole structure may look like this:

Sensors collect information and send it to the hubs (these are not network hubs, they are simply called the same way: they are microcomputers with a small motherboard, process, memory and OS), hubs transmit information to the Internet (via mobile communication, Wi-Fi, satellite communication, Ethernet etc.), and it reaches the data center. The reverse flow travels exactly the same way - from the data center to hubs, from hubs to sensors (or to local programmable logic controllers).

We go further. Still remember the specifics of the world IIoT? So that all this construction does not cost cosmic money, I want to:

• IoT Hubs consisted of potentially replaceable iron, this will reduce the likelihood of the vendor lock-in and the price of iron, which you need oh how much because of the huge distances;
• on top was an open OS (for example, a kind of Linux - OpenWrt);
• all the logic of interaction with any types of sensors and analysis of their protocols went to the level of software on top of the operating system (so as not to depend on expensive black branded boxes);
• The entire logic of sending signals to the sensors and equipment also went to the level of local software on the IoT Hub, and was not hardcoded in Vendor iron boxes (meaning processing logic, and not physical data transfer);
• The software running on IoT Hubs was very fast. IoT Hubs are slow, and slow, because cheap, and cheap, because they need a lot, and a lot of it is necessary because of the specifics (see above). And remember about the specifics in the form of heat-cold-humidity-vibration, that is, these hubs need special enclosures. And the price cannot be increased. So, it is necessary to make the technical stuffing even cheaper, therefore, even more slowly. Can you imagine how fast the software should rustle in order for the debit with the loan to converge?

That is, it would be good, in other words, to have what we used to see in data centers: replaceable inexpensive hardware, all logic at the software level, and all fault-tolerance at the software level.

And now let's look at the software in the data center. There, besides OSes, almost always many other components are used - DBMS, application servers, monitoring systems, DevOps scripts and systems. And already on top of this infrastructure is written business logic. In a good way, all the described infrastructure is configured to automatically (well or semi-automatically) take on the main problems on the hardware side (switch the load to the database replica, if the master copy is not available, distribute the load among several nodes of the application servers, automatically pour) a clean machine that is connected to a cluster, etc.).

That is, there are several specializations. Business logic developers (product developers), in an amicable way, mostly think about business logic and focus on business problems. And there are engineers and developers of the rest, a huge underwater part of the iceberg - DevOps, system administrators, database developers, developers of application servers, monitoring systems and other (let's call them infrastructure) components.

That would be so on the ground. Cool? That would be the development of IoT Hubs was the same specialized.

Tarantool IIoT - under the hood

And here we smoothly approach the Tarantool IIoT. I will not say that it is intended to replace all of the above infrastructure with IoT Hubs, but it can partially do this. Look at the picture:

Tarantool IIoT is the usual Tarantool, but compiled and tested for ARM and for x86 (the MIPS version is currently being created). In addition, automatic support for the MQTT and MRAA protocols is built into it - these are the main protocols for working with sensors. Tarantool IIoT is installed directly on IoT Hubs. You can also add here that to transfer data between the sensors and the hub (in the “on-site” diagram) you need special LORA or 6LoWPAN equipment. Which would seem bad (remember, we don’t want unique vendor boxes). But the whole point is that this equipment does not have to understand the protocols of multiple sensors. Its task is only to wrap the protocol in a protocol understandable for the hub (usually this is our favorite old kind TCP / IP). The rest of the analysis can be done on Tarantool IIoT using scripts that work directly on the IoT Hub.

This is the advantage of the software platform over the iron - any equipment and any protocols can be supported independently, without a vendor. Technical details, how it is all arranged and how programmed using Tarantool, can be found in the article “ Master-master replication and scaling of applications between all IoT devices and the cloud ”.

In all other respects, Tarantool IIoT is a full-featured Tarantool, i.e., a DBMS with an embedded application server, a synchronous transaction log, replication between nodes, two engines (in-memory - memtx and on-disk - vinyl), stored procedures, asynchronous jobs, support queues and other pies.

What does Tarantool IIoT give a developer?

Tarantool IIoT is installed on IoT Hubs. What does this give?

1. On Tarantool IIoT, you can write scripts that receive information from sensors, parse it and save it to a local database right inside Tarantool IIoT (it persists in the IoT hub flash memory).
2. The information stored on the IoT Hub is replicated locally between several IoT Hubs for reliability (so that the whole system continues to work even after one or several hubs have failed). Replication at Tarantool is immediately out of the box, i.e. it is not necessary to write your own queue system. Here, by the way, an important point: Tarantool is not only an application server, but also a DBMS. This means that it is possible to persist data and replicate data between nodes. Very important properties in the light of the above. And it is very important that it works out of the box and has been tested for years (for the Tarantool core is exactly the same as for the usual non-IoT services).
3. The information stored on the IoT hubs is replicated to the data center using Tarantool embedded tools. And only that part or aggregation that you want to replicate (the setting is programmed and works directly on the IoT Hub).
4. You can automatically download the code and stored procedures in Tarantulas in the field, i.e., change the logic of local aggregation from the data center (i.e., from anywhere, from where there is access to the data center: from the office or from home via VPN) without going to the site.
5. It is possible to change the degree of detail of information delivery to the center, while locally, locally, receiving as much information from a single data center from the data center (and therefore from the web interface, which authorized people have access to from any device from anywhere in the world) How much is needed, and process it without transfer to the data center.

As a result, the product developer gets a step towards the degree of comfort that is present when creating software that works in the data center. There is no need to think that the channels between the hubs locally and between the hub and the data center are unreliable and break: replication will still deliver data in both directions without loss and without doubling. The developer has less of a headache about the client-server architecture, about files, about data loss, about hub failures - all this is handled by Tarantool IIoT exactly the same as it happens in the data center when using the usual Tarantool. A developer can concentrate more on business logic.

I’ll add here that the data replication mechanism between nodes has been engaged in data delivery, which has been tested in Tarantool for nine years now. That is, the reliability of the mechanism is quite high.

In fact, Tarantool IIoT is a fully programmable open platform that allows you not only to develop IIoT applications on local devices (without going to the field!), But also to transfer best practices from the data center to IoT devices in the field.

And this platform also gives you a free hand in the sense that equipment manufacturers previously tied them to you. For example, you already have a ready and working iron IoT infrastructure, which automatically delivers all of the sensors to the data center. You have purchased a new batch of cheaper sensors that are not supported by your current equipment. You will have to wallow in the legs of the equipment vendor so that he will make changes there for big money. Well, or just buy a new version of the equipment. Or give up cheap sensors. And all these changes are iron, not software, that is, with downtime, field visits, installation and other troubles.

With the help of Tarantool IIoT, you can easily receive data from these new cheap sensors, parse them and then send them to your equipment or local software in the right format. Without downtime and without big expenses.

It is clear that in itself Tarantool IIoT is small. For complete comfort, other infrastructure systems are also needed - monitoring, remote software upgrade (the same Tarantool needs to be upgraded), automatic Linux pouring and other joys. But, as it seems to us, this is already a step in the right direction.

Why generally Tarantool IIoT, but not $ any_other_technology IIoT?

In conclusion, I want to tell you why we took Tarantool as the basis for our IIoT product. Let's go back to the specifics of IoT Hubs for a second: these devices have slow processors with a small number of cores (usually one or two cores), little RAM, little flash memory, which is also slow and unreliable. Why? The devices should be very cheap (the price range is approximately: 30–100 dollars), because there are so many of them, because the infrastructure is scattered for many kilometers - and for some more reasons, see above. Remember, I was talking about specifics? From her not to leave. This is on the one hand. On the other hand, the sensors generate a huge amount of information (tens and hundreds of parameters per second can be received from one sensor, and the number of sensors per hub sometimes amounts to hundreds and thousands). All information must be managed to process, despite the harsh conditions and not very fast equipment.

Accordingly, for such devices, you need an oh-so-very fast software, a very fast DBMS and a very fast application server that can work on a single processor core, using the memory as efficiently as possible and not straining the disk.

And with all these limitations, Tarantool showed the best results on our tests. The closest competitor in speed can be Redis (with CouchBase and Aerospike, unfortunately, we did not succeed: they are very slow on single-core machines, but maybe we just do not know how to prepare them). The main disadvantages of Redis in terms of applicability in IoT are: the application server is underdeveloped, there are no stored procedures (and you cannot update the code inside the database in runtime), there are no jobs, there is no replication master, there are no transactions - there is a greater likelihood of data loss. Plus, it is unclear whether the authors of the product plan to develop at all in this direction. Here is an example of comparing two products: Tarantool vs Redis . A bunch of Python and Redis can in some sense replace Tarantool, but in terms of speed, it is likely to lose a lot. Immediately make a reservation - there are no comparative tests yet, plus we recommend the speed of work to be compared in each case, case by case.

Another important nuance: on IoT devices, when the DBMS and the application server are combined (and in Tarantool they are combined and work in the same process), the overhead of communicating with each other will obviously be much less (here, overhead, which are enormous even when interacting with fast DBMSs ( asynchronous processing ). In this case, any overhead costs for the transfer of information between different processes are very large in the light of the low performance of the devices and the huge amount of information that needs to be processed.

There is also such a good thing - SQLite, it is good and fast, but, unfortunately, it does not have replication and application server out of the box - that is, you should also dance on top of it with a tambourine. The rest of the traditional DBMS (I will not name it, there is a large list of reputable products, everyone knows them perfectly), we simply didn’t start up normally on the small IoT hardware or didn’t even cope with the load from the sensors. However, if you have a positive experience of using other DBMS on IoT devices, then we will be happy if you share it!

This is all I wanted to tell you about Tarantool IIoT. We are already in the process of pilots with various manufacturing, transport and other companies. As soon as there will be a production, we will definitely write about it in detail. Follow the news!