IoT architecture

Almost a year ago, I started publishing a series of articles on IoT solution architecture. (Link to the first article habr.com/ru/post/420173 ). And finally, the second article of the series is given to you.


IoT projects look similar to each other, and it is true that they have common components. But at the same time, IoT projects have fundamental differences, starting with the demand of the market and classes of tasks to be solved.

Obviously, developing every solution “from scratch” is extremely inefficient. Many components can be reused on the basis of programming patterns.

In this article, we will analyze the various classes of IoT solutions and their respective implementations. As a result, recommendations will be given on the steps of implementing the IoT architecture and we will try to highlight the general trends of IoT solutions.
')
We will also try to formulate the main problem areas, from consideration of which, it is worth starting to build architectures of specific IoT solutions and possible problem areas of architectures.

IoT application areas

Today, there is a division into three main geographical regions where IoT solutions are applied:

Europe

The European IoT market is considering projects related to the saving of natural resources. A typical example is day / night lighting, room heating automation, water management, and fire alarms.

Far Eastern region

In this market, the first priority of applying IoT solutions is security. Typical applications are cameras that detect unusual situations on the road, in transport, at home. The second major area is security solutions and disaster warnings, such as seismic activity, hurricanes, typhoons, tsunamis.

North American market

In this market, IoT projects are mainly represented, aimed at optimizing business processes, such as transport optimization, efficient delivery of goods, as well as elements of smart houses and cities.

Of course, IoT solutions penetrate from one market to another and are not exclusive prerogatives. In each region, we can find all classes of IoT solutions. It is important for us to classify these classes of IoT products that exist today in world markets. The following is a list of such IoT decision classes:

• Smart City. The main tasks are the management and regulation of traffic, night / day street lighting, notification of danger to pedestrians, the definition of unusual and dangerous situations in the city.
• Smart House. The main tasks are Security, intelligent doorbell, TV and kitchen control, automatic watering and lighting systems, fire, gas, water leakage, and house temperature alarms.
• Weather and natural disasters. Meteorological information, seismic activity, fire control. Weather forecasting.
• Optimization of use of resources in the house, city, country. Lighting, consumption of electricity, heating, optimization and prediction of use, such as fuel in power plants.
• Optimization of transportation, deliveries, storage and sorting. Companies such as DHL, FedEx uses a solution to build optimal transport routes. Storage and sorting terminals at major airports.
• Factory monitoring and control, conveyor line management. Control robots. Sort goods, raw materials and testing of finished products.
• Complicated mechanisms, high-tech devices, such as modern cars, airplanes, etc. Automated control system, anti-theft protection, control of system components. Recognizing the face and body of the driver to prevent sleep, loss of attention. Predicting maintenance and replacement of system components.

IoT architecture

General topology IoT solutions

The figure below shows the layered architecture of IoT solutions. The IoT topology is different from the conventional layer model, such as OSI. This is not a linear or more complex flow graph. Some components are optional and may not be available in a particular class of solutions. There can be two types of logic - M2M (from car to car) and M2P (from car to man), as well as more special cases such as C2C (from car to car, usually in one cell of LTE mobile communication).

IoT solution has two physical locations - the first is the end (peripheral) devices, and the second is in the Backend data center on servers or in the cloud. At the same time, this is not a classic client-server application architecture, as we will see later.



Below we analyze each level separately and compare its features with actual classes of IoT solutions.

Physical Layer - Physical Layer

This level represents two types of operations - the collection of information (Sensors) and the implementation of mechanical work (Executive mechanisms).

Sensors can be divided into the following categories:

• Sensors:
  
  
  • Illuminated: Photo diodes / transistors / resistors, PIR detectors
  • Sound: Microphones, ultrasonic sensors
  • Switches, in particular the limit switches registering the extreme points of the mechanical movement. Measuring angles of rotation or speed of rotation.
  • Electromagnetic sensors that measure changes in physical characteristics, such as capacitance, inductance, resistance.
• Complex or composite sensors. These include specialized sensors, such as gas, spectrum, etc., as well as a separate type of information gathering device that is becoming more widely used - Video cameras.

In IoT solutions, physical elements have certain general requirements:

• As low as possible due to the large amount in the IoT solution.
• Powered by batteries, which in turn requires low power consumption. Today's market demand - the operation of peripherals without service from 1 to 10 years.
• Often the location in remote and remote places with minimal installation and maintenance costs.
• In the case of video cameras, the primary image processing with the decision-making based on artificial intelligence

The actuators of the IoT solutions open the locks of the entrance doors, power the engines, selsyns, turn on / off the lights, heating, water, gas, etc. There have been no major changes in the implementation of the actuators. Therefore, this part of the IoT solution will not be covered in this article.

Below is a table of the physical layer in various classes of IoT solutions:

We can summarize two problems for the requirements of the physical layer:

• Low power consumption. Requires a high level of integration with the upper layers.
• The use of video cameras. It also requires a high degree of integration with the upper levels and the built-in AI / ML functions implemented in the peripheral device.

Edge Layer - Peripheral Computing Level

This level is usually connected to a single sensor or actuator. It provides minimal functionality for converting analog information to digital and / or vice versa. To connect sensors, there are the same requirements for price and power consumption. Many manufacturers that produce these types of devices do not have a single standard for the data model, configuration, and operation, which creates separate integration problems.

To reduce power consumption, peripherals typically have four modes of operation:

• Sleeping mode
• The mode of measurement and data collection from sensors
• Communication mode, transmission and reception of information
• Installation and Connection Mode

Below is a block diagram of the peripheral device.

A peripheral device usually combines three levels: physical, peripheral computing, and communication. The main functionality of the peripheral computing level is local ETL (Extract, Transformation and Load) - receiving, transforming and saving information from sensors. This level is responsible not only for collecting information from the sensor, but also for bringing it to a standard form, filtering interference, preliminary analysis and local storage.

Below is a table of the level of peripheral computing in various classes of IoT solutions:

So, the basic requirements of the level of peripheral computing:
Low power consumption. This can be achieved using low-power hardware and Sleep / WakeUp algorithms. Often the presence of a local element of artificial intelligence.

Local Network Layer - Peripheral Communication Layer

Data transmission is the most energy-intensive part of the peripheral device, since most peripheral devices are not connected to the mains and wired communications. In addition, peripherals can be located quite far from the Gateway (within a few kilometers). On the other hand, the amount of information transmitted is usually quite small. The following protocols are used at the level of peripheral communication:

• ZigBee / Zwave
• Ble
• Lora
• Proprietary low band

To increase distance and reliability, Ad Hoc and Mesh communications are widely used today at this level.

For configuration purposes, the NFC protocol can also be used. During the first installation and / or maintenance process, a service engineer with a mobile application can connect to a peripheral device through a peripheral communication layer. Sometimes the Q-code printed on the peripheral device is also used for authentication.

Below is a table of the level of peripheral communication in various classes of IoT solutions:

Gateway Layer - Gateway Level

There are several reasons for having a gateway level in an IoT solution:

• If the Backend receives raw information, it will increase its capacity and the costs will be very high.
• Backend operation cannot guarantee real-time response for a large number of peripheral devices.
• Due to security restrictions, some information cannot be sent to the Backend and cannot be constantly monitored by humans. Such information includes data from outdoor surveillance cameras, medical information, etc.

The gateway should provide the following basic functionality:

• Implement a second level of ETL from its peripheral devices.
• To fix a critical situation and issue a local reaction, even without communication with BackEnd. This can be compared with the signals of a person's heartbeat or breathing of the lungs without participation of the brain.
• Communicate with BackEnd. It sends processed information from the peripheral devices to the server and receives configuration data for the peripheral devices.
• Save information about the status of peripheral devices, and the data they collected.

In some cases, the AI ​​/ ML functionality (artificial intelligence / machine learning) must be present at the gateway level. The gateway device is mainly powered from the mains or has a large built-in battery, but some solutions also require low power consumption. In this situation, an additional problem arises - a synchronization protocol for communicating with a peripheral device. One of them (gateway or peripheral device) should transmit the message “Ready for communication” more often than the other device is ready to contact. The choice will depend on the total power consumption of each device and the required time without maintenance.

Today we have a growing number of applications with a source of information in the form of a video camera. In these particular solutions, Gateway and Edge can be integrated together. The functionality of AI / ML in such applications is becoming very popular. With new engine intelligence accelerators for embedded systems, this solution has become a reality.
Separately, it must be said about gateways for smart home solutions. A gateway in this class of solutions is often combined with STB devices — television adapters or with a home security control unit. For the first integration, there is already an open platform RDK-V. In the near future, we should expect further integration of all three components — the gateway + STB + security into one device. It will also probably have NAS (local file storage) and AI / ML services for machine video / audio recognition. Audio recognition devices such as Alexa are based on the Cloud infrastructure, but it is likely that the primary recognition will be carried to the peripheral level.

Below is a table of the gateway level in various classes of IoT solutions:

Wide Network Layer - External Link Level

This layer separates the peripheral and BackEnd parts of the overall solution. The gateway is mainly connected to BackEnd using a mobile wireless connection, such as 4G / 5G, but sometimes wired Internet access is used. The logical layer of external communication has a standardized protocol for IoT solutions, which is called LvM2M. The LvM2M protocol was designed to access each peripheral device, but since many peripheral vendors do not support LvM2M interfaces, the gateway device can solve this problem and create a wrapper for communicating with peripheral devices.

The external communication layer also contains communication services and ISO models within itself. It includes balancing and location-based services based on DNS service, COAP transport, DTLS encryption, and many other components that are beyond the scope of this article.

One important comment we should make here. LvM2M protocol uses the DTLS encryption protocol. The DTLS protocol is a protocol with security keys and a handshake session. It works on a point-to-point basis. To decrypt DTLS packets, we must use the same Back End instance that we had during the connection session. This creates a problem for the load balancer (Load Balancer), which is part of the security level on our circuit. The load balancer, in turn, is necessary for automatic scaling at high system load. To avoid this limitation, DNS is used as a load balancer. Every N DNS request gets a new IP address of the security level instance.

Below is a table of the level of external communication in various classes of IoT solutions:

Security Layer —

This layer provides AAA (Authentication, Authorization and Accounting - Authentication, Authorization and Accounting) functions and encryption / decryption along with other Internet-related services. All Cloud has its own security implementations, but functionally they are all built on the principle of roles and permissions. As noted in the paragraph above, this level also functions as a terminator of an encrypted DTLS connection.

Connecting an end user to Backend also has a security level component.
Below is a table of the security level in various classes of IoT solutions:

Middleware Layer —

This layer provides internal Cloud functionality for load balancing, message queuing, and streaming. The components of this layer should be duplicated and automatically scaled. The level is implemented mainly on the basis of microservices or PaaS from Cloud providers. Such a requirement arises from the paradigm of jumps and dips in the volume of data. Automatic scaling reduces the cost of backend implementation. The actual implementation of the service may be different, but the general principle remains the same - to provide asynchronous message transmission with buffering and load redistribution. Thus, the various components of Backend can do their work independently and scale horizontally depending on the load.

The figure shows a schematic block diagram of intra-server communication patterns. Load Balancer is designed to distribute the load between different services. Queue queues provide intermediate buffering for the implementation of asynchronous operation of sequential services. Subscribers are recipients who subscribe to the queues corresponding to their logic in order to receive successive messages after processing previous messages.

Below is a table of the level of intra-server communication in various classes of IoT solutions:

Etl layer - the level of data collection, processing and storage

The inner level ETL (extract, transform, and load) is the third ETL operation. The first was in the peripheral device, the second - in the gateway. Back End ETL collects data from all peripheral devices and gateways and is responsible for the following operations:

• Collection of information
• Bringing information to a standard form

The general scheme of the implementation of this layer is shown in the figure. The Extract operation involves reading information from relevant queues. The transformation operation can be performed by specialized Cloud services, such as Lambda, or by computing facilities inside containers and simply virtual machines. Each of the above methods has its own positive and negative properties. For example, the Lambda service is convenient for almost complete automation, but it has a significant creation time and therefore is not applicable if a quick reaction to the events that occur is required. Also, Lambda is poorly suited for continuous processing, since it is charged at the time of use. The most commonly used service is containerized computing. They are conveniently scaled and easily transferred to various BackEnd.The main objective of this operation is to bring data to a convenient form for storing, sorting and searching. For this, data is often combined from different messages and even queues.

Storage operations (Load) are intended for saving, sorting and subsequent retrieval of information. Depending on the type of information and its uses, different tools are used. If the data does not have a strict scheme (table columns), then they are stored in NoSQL databases. However, if the data can be systematized by a fixed scheme, then SQL database types are used. The latter, in turn, have 2 types - OLTP (Online Transactional Processing) and OLAP (Online Analytic Processing). As the name suggests, the first type is more suitable for the ETL process itself - writing new values ​​to the database, while the second one is more convenient for searching and analyzing data. Therefore, often after downloading to the OLTP database, in the background, the data is copied to the OLAP. There are situations when the data is not convenient or not possible to store in databases,for example, a record, This data is recorded in Bucket, and the metadata of records is stored in databases. To reduce storage costs, obsolete data is archived or deleted. And the last component of this level is internal notification of the availability of new stored data for presentation to customers and for analysis services.

Below is a table of the level of collection, processing and storage in various classes of IoT solutions:

Big Data and Analytic Layer - Analytics Level

Depends on the specific IoT application. Big data and analytic level will extract situational information from the entire set of peripheral devices. This part is less standardized, because it is very different from one application to another due to the various problems of solutions. Algorithms AI / ML are also widely used in this layer.
A separate category is the prediction of future events, such as the necessary parts in the warehouse, the consumption of future resources, the weather, etc.

The table below shows the level of analytics in various classes of IoT solutions:

Notification layer —

There may be several components at this level, but they all have a subscription notification algorithm. The client application subscribes to the necessary events and, when this happens, receives an information signal - a notification. These are mainly email and mobile clients, less phone calls (used for emergency alerts). The mobile application is forced to go into sleep mode for power consumption, but iOS and Android have a notification mechanism indicating the arrival of new data.

Below is a table of the notification level in different classes of IoT solutions:

Presentation Layer —

An IoT application can have two streams: M2M (from car to car) and M2P (from car to man). The presentation layer associated with the M2M stream, where the Back End processes the information and provides it to the client or the support engineer. Today there is no standardized UI / UX representation for this level, but I hope that it will appear in the near future.

The presentation layer is also responsible for maintaining, configuring, and changing the state of the system including peripherals and gateways. It also presents commands for controlling actuators of peripheral devices.

Below is a table of the presentation layer in various classes of IoT solutions:

Configuration Layer - configuration level

This level refers to both the M2M and M2P streams and works as a storage for three types of peripheral status:

• Current state of the peripheral device
• , .
• — . .

A peripheral device and even a gateway can only have a short connection time to the Backend. We discussed this earlier. Any status change from the client or system is stored at this level, and during the communication time is sent to the gateway or peripheral device.

In order for this logic to work, the following communication process is usually implemented:



If the gateway is present in the information transfer scheme, then most of the information from the peripheral devices is sent to the server part as data packets collected from several peripheral devices.

Below is a table of the configuration level in various classes of IoT solutions:

Results and how to build the architecture of IoT solutions.

Summing up the above. The following development trends are observed in IoT solutions:

• Sensors are divided into 2 groups:
  
  • Simple, cheap, with the lowest possible energy consumption. Low speed and high range information transfer. These are in fact one-off devices that are not subject to maintenance.
  • Based on the camcorder. The device is integrated with a peripheral computer. It has built-in mechanisms for pattern recognition and making basic decisions.
• ETL — , backend. , , — .
• , . . — LvM2M.
• Backend Cloud. AWS.
• , WEB . . .

How to start building an architectural IoT solution? There is no single approach to answering this question. And here I will give my personal opinion:

1. Determine the data model that we can get from the gateway, i.e. transmitted in the backend.
2. Check which peripheral devices can collect data and how it should be processed to bring it to the model transmitted by the gateway.
3. Check the requirements for peripheral devices - distances, amount of information, power consumption, etc.
4. Select the appropriate peripheral computing device, its location in relation to the sensors, the protocol of their work.
5. Solve the Cloud architecture parts, including:
   
   
   • Security
   • Load distribution
   • Asynchronous data transfer within the Cloud
   • Elements storage, form and data life cycle
   • Build a graph of information transfer system
   • Build analytical models, AI / ML component
   • Develop types and contents of notifications
   • Configure redundancy and auto scalability of services
   • Estimate the cost and optimize
6. UI / UX design for mobile clients
7. Build feedback to the peripheral device

I hope this article will be useful, at least for the first session with IoT projects. In the future, I will try to provide specific implementations of IoT solutions.