Interaction Templates for the Internet of Things
Before starting a new IoT project, it is worth considering what information exchange templates are best suited for it. In fact, this decision should be made as early as possible, even before the protocols, communication methods and supporting infrastructure of the system being developed are chosen. The basis of this recommendation is one simple reason: not having made such a decision at the very beginning, the developer, as the project progresses, risks driving himself into a corner, from which you can only seriously rework the code, architecture, solution security model, and how it interacts with the outside world.
Today we will look at eleven interaction patterns in IoT systems.
The request - response template (request - response) is probably the most widely known communication pattern. Its implementation provides for the presence of a client, or a caller, that makes requests to a certain service located on the server providing the services. The server is also called the respondent or the respondent.
Template "request - response"
')
It is this template that uses the HTTP protocol . It is also the basis of service-oriented architectures, web services and REST solutions. This is a practical template, in particular, if the project architecture provides for client and server parts or master and slave entities.
In addition to HTTP, the request-response pattern is supported by protocols such as the Constrained Application Protocol (CoAP) and the Extensible Messaging and Presence Protocol (XMPP).
The main disadvantage of this pattern is the inequality of the participants in the data exchange, which is quite evident in the topology of the Internet. Bidirectional data exchange, when both participants request data from each other, can be difficult to implement, especially if there are firewalls in the data path.
When planning to use the request-response pattern in a project, you need to decide which parts of the system will be clients and which parts will be servers. If, for example, a certain sensor is a client, and the IoT gateway is a server, the sensor itself will decide when to send its own readings to the server. The server, if it needs information from the sensor, will not be able to independently request it. If the sensor is made server and the gateway is a client, the sensor can be polled at any time. However, there is one problem: if the sensor is not sufficiently protected, anyone can connect to it. If a reliable security system is involved in such a solution, then, as a result, the way the client and server interact, and the system as a whole, will become more complicated. It may be necessary to add additional services to the project, sensors will have to be equipped with more powerful hardware. In addition, all this will be more difficult to manage.
This event subscription allows the client to subscribe to events of a specified type on the server. The server notifies the client whenever an event of interest occurs. As a result, there is no need to constantly poll the server.
Event subscription template
An advanced event subscription mechanism may include customer-dependent requirements regarding which events and under what conditions they are interested in. The advantages of using the event subscription over the earlier “request - response” template are that the data exchange between the client and the server requires approximately two times less messages. In addition, data is transmitted to the client when a certain event occurs, and not upon request, which minimizes the time between the occurrence of a certain situation that interests the client and the moment when he learns about it.
Protocols that support this pattern include CoAP, XMPP, and the General Event Notification Architecture, which is part of the Universal Plug and Play HTTP-based architecture.
Asynchronous messaging (asynchronous messaging) provides the ability to send messages between peer systems that are at the same level of hierarchy. This pattern implies bidirectional messaging.
Asynchronous Messaging Template
If the protocol used supports asynchronous messaging, any other data transfer patterns can be built on its basis.
Among the protocols that support this pattern are XMPP, Advanced Message Queuing Protocol (AMQP), and, at the IP level, the User Datagram Protocol (UDP). However, in the case of using UDP to implement this pattern, there may be problems with firewalls.
Applications that perform critical functions need to know that the message was delivered to the recipient at least once. As a matter of fact, using the asynchronous data exchange pattern, this requirement is fulfilled. A message may be lost on the way, but using the request-response template allows you to request that the message be sent again, until a confirmation (or response) is received from the party who should receive the message. Here you need to take into account that both the message and the answer about its receipt can be lost, so this template guarantees that the message will be delivered at least once. However, the fact that the message will be delivered to the recipient no more than once (or at least once) is not very suitable for some applications, for example, those that use the concept of transactions or count messages.
The use of a reliable messaging template ensures that the message will be delivered to the recipient exactly once. Among the protocols that support reliable message delivery are Message Queuing Telemetry Transport (MQTT), AMQP. In addition, thanks to open extensions, such functionality is supported by HTTP and XMPP.
The previous template is busy messaging between two objects. Sometimes, however, a more efficient approach is required if the same information needs to be sent to several recipients at the same time. The simplest of the templates that implement this functionality is “multicast messaging” (multicasting). Within this template, the sender sends one message through an intermediate link in the system (it can be a broker or a router), after which the message is sent to several recipients, each of whom registered to receive such messages.
Multicast Messaging Template
Through the use of this template, you can reduce the load on the network, since the sender does not need to send the same message to everyone who expects it. In fact, the sender does not even need to know who exactly will receive the message. This template can be quite useful in a variety of situations. For example, when synchronizing multiple devices or distributing the same data among multiple recipients. Multicast messaging is supported by the XMPP, AMQP and UDP protocols.
Here it is appropriate to make some warnings. They concern the use of multicast messaging to implement other communication schemes.
So, although this template can be used to reduce the load on the network, it is often referred to as a way to bypass the restrictions in the protocol used, as well as to implement the “event subscription” template on the basis of a certain protocol. If, for example, to use multicast messaging to reduce delays in networks where necessary, but it is impossible, to implement the “event subscription” template, this template will not increase, but increase the network load. In addition, multicast data systems are more difficult to secure.
With regard to improving the efficiency of network bandwidth usage with the use of multicast data transmission, resources can be saved only if the recipients consume most of the received data. If a significant proportion of the data transmitted in this way is not used by recipients, this is a reason to consider other interaction patterns.
The Publisher-Subscriber template (publish-subscribe) is an extension of the multicast message delivery template. The principal difference between the two is that the transmitted messages are stored on the intermediate node. These messages, or links to them, are then distributed to subscribers interested in them.
Features of the implementation of the template depends on the protocol used, depends on it, as well as on the settings of the intermediate node, and which messages are stored. This can only be the most recent message, or a specified number of messages, or all messages.
Here, in addition, the difference in the transmission of the message itself and links to it is important, since it affects the required network bandwidth and, as a result, the performance of the solution.
If subscribers use most of the messages, the transmission of the messages themselves is more efficient, as is the case with multicast data. If the actual data consumption by recipients depends on some additional factors, then it is more efficient to send links to messages. They are smaller than the messages themselves, and subscribers are likely to use only a small number of them in order to get those messages that are referenced. In this case, in order to receive a message by reference, it is required to make additional calls to the message storage node using the “request-response” model.
The Publisher / Subscriber template supports protocols such as MQTT, AMQP, XMPP.
Queues, specifically, FIFO queues is a data exchange pattern that allows one or more entities to send certain messages or tasks to be processed into a queue, after which one or more recipients receive these messages in the order in which they were queued .
Template "turn"
The queue is usually located at an intermediate node, or on a network to which all participants in the data exchange are connected. Queues are a great way to load balance. In turn, they collect tasks from various sources and distribute them among existing processors, possibly with different productivity.
Using a queue, you can avoid hard communication between the systems that transmit data and the systems that receive and process this data. As a result, depending on the actual workload on the system, it is possible to increase or decrease the number of data receivers and transmitters. Among the protocols we have already mentioned, only AMQP has built-in queue support.
Message brokers are usually standardized components of the supporting network infrastructure of IoT projects. They beautifully solve the problems caused by the limitations that firewalls impose on bidirectional data exchange between devices. A broker allows entities to connect to it by transmitting messages between clients connected to it. Since all connections are made through a broker, only the broker must be accessible from the Internet. The firewall does not need to accept or redirect incoming connections to devices, as it would be necessary if you use a protocol that provides peer-to-peer communication, strictly limited to a similar messaging model.
In addition to managing messages, brokers can provide additional services to connected clients. For example, a broker can act as an intermediary in the implementation of a multicast messaging template, publisher-subscriber and queue templates.
In addition, message brokers typically provide client authentication services. This makes it easier to work in distributed networks, where device authentication can be a daunting task. Thus, if a broker can report on the status of already authenticated participants of the system involved in the data exchange, other participants can use this information to make security decisions. This, moreover, eliminates the need to implement its own authentication scheme for each participant in the data exchange.
Although the exchange of messages between peer systems is only one of the options for organizing communications, such solutions should provide for client authentication. Otherwise, the security of the system will seriously suffer. If you use a protocol that includes message brokers, then you probably will not need to develop your own support services that allow the solution to work reliably and safely.
The XMPP, AMQP, and MQTT protocols, in one form or another, use this pattern.
“Federation” (federation) is an important template in which a certain global network is divided into logical parts. This allows for global scaling of the solution and provides everything needed for its natural growth.
Federation Template
The main idea here is to allow solutions to be increased without limiting the performance of the existing network infrastructure, using the divide and conquer approach.
When communicating without brokers, for example, as with HTTP and CoAP, a federated structure is available at the domain level. Each domain points to its own set of IP addresses, and its own web server is associated with it. You can add new web servers to the system, in new domains, without restricting access to existing systems. This approach is one of the main keys to the success of the World Wide Web.
When using protocols involving brokers and supporting federations, brokers connect to each other to route messages. Each broker manages authentication in their own domain and knows how to connect to other domains to redirect messages to them. In addition, federated broker networks provide a convenient solution to the problem of global identification of data exchange participants.
The most widely known protocol using brokers and federations is the Simple Mail Transfer Protocol (SMTP). Among the protocols we are talking about in this material, able to work with brokers, only XMPP supports the federation.
Consider a conditional example. Suppose we have some factory-made device that is planned to be used in the IoT system. If, for example, it is planned to be used in a system with multicast data transmission, we will immediately encounter some difficulties associated with the integration of devices into the system.
They consist in the fact that “things” are only aware of their own identification information (it could be something like a MAC address), but they know nothing about how they will be “visible” on the network to which they are going to be connected. , nor about a certain main network device with which they will have to interact.
After installation and configuration (the more automated the configuration is, the better), “things” will learn about their network identification information, but not about how to connect to the host device. In turn, the main device knows its own network address, as well as the factory data of “things” (which, for example, can be quickly entered into the system by scanning stickers on the boxes), but not the network data of other devices.
The discovery messaging template allows you to create a mechanism by which the network identification data of the slave devices is mapped to the network data of the host. This is done using the general knowledge of the initial identification parameters of the slave devices.
Pattern "detection"
This template is implemented using the "registry of things" (Thing Registry), accessible on the network as the "things", and the main device. Clients are registered in the registry, and the main device accesses them through the registry, using only their factory identifiers. If the request is successful, the network identity of each of the participants in the data exchange is sent to the other, and both thus know how to interact with each other. There is an XMPP extension that supports this pattern.
On the Internet, the ability to make informed security decisions is important. When using the delegation of trust pattern, end devices redirect requests to a more secure, trusted system in real time, and after receiving a response, they perform certain actions.
Template "delegation of trust"
The actions of a trusted entity, when new requests are received from client systems, can be based, for example, on machine learning, or on settings that are set by the administrator, possibly reacting to requests from the system when it receives new requests.
In order to be able to implement this pattern, you must use asynchronous bidirectional messaging. There is an XMPP extension that supports delegation of trust.
In conclusion, I would like to note that by adhering to standards and open specifications, it is possible to improve the compatibility of IoT development with existing systems. Similarly, by using open, standardized, interchangeable components, the need for expensive infrastructure can be avoided. And the correct choice of the data exchange pattern is a great way to protect yourself from serious reworking of the project in the later stages of its development.
Some of the templates that we talked about may increase the complexity of the project at the very beginning of working on it, but these additional costs are nothing compared to the cost of possible future discrepancies. For example, the difficulties associated with integration into the existing environment. Immediately it is almost impossible to foresee, however, the correct choice of the template will help to avoid such problems.
Today we will look at eleven interaction patterns in IoT systems.
Template "request - response"
The request - response template (request - response) is probably the most widely known communication pattern. Its implementation provides for the presence of a client, or a caller, that makes requests to a certain service located on the server providing the services. The server is also called the respondent or the respondent.
Template "request - response"
')
It is this template that uses the HTTP protocol . It is also the basis of service-oriented architectures, web services and REST solutions. This is a practical template, in particular, if the project architecture provides for client and server parts or master and slave entities.
In addition to HTTP, the request-response pattern is supported by protocols such as the Constrained Application Protocol (CoAP) and the Extensible Messaging and Presence Protocol (XMPP).
The main disadvantage of this pattern is the inequality of the participants in the data exchange, which is quite evident in the topology of the Internet. Bidirectional data exchange, when both participants request data from each other, can be difficult to implement, especially if there are firewalls in the data path.
When planning to use the request-response pattern in a project, you need to decide which parts of the system will be clients and which parts will be servers. If, for example, a certain sensor is a client, and the IoT gateway is a server, the sensor itself will decide when to send its own readings to the server. The server, if it needs information from the sensor, will not be able to independently request it. If the sensor is made server and the gateway is a client, the sensor can be polled at any time. However, there is one problem: if the sensor is not sufficiently protected, anyone can connect to it. If a reliable security system is involved in such a solution, then, as a result, the way the client and server interact, and the system as a whole, will become more complicated. It may be necessary to add additional services to the project, sensors will have to be equipped with more powerful hardware. In addition, all this will be more difficult to manage.
Event subscription template
This event subscription allows the client to subscribe to events of a specified type on the server. The server notifies the client whenever an event of interest occurs. As a result, there is no need to constantly poll the server.
Event subscription template
An advanced event subscription mechanism may include customer-dependent requirements regarding which events and under what conditions they are interested in. The advantages of using the event subscription over the earlier “request - response” template are that the data exchange between the client and the server requires approximately two times less messages. In addition, data is transmitted to the client when a certain event occurs, and not upon request, which minimizes the time between the occurrence of a certain situation that interests the client and the moment when he learns about it.
Protocols that support this pattern include CoAP, XMPP, and the General Event Notification Architecture, which is part of the Universal Plug and Play HTTP-based architecture.
Asynchronous Messaging Template
Asynchronous messaging (asynchronous messaging) provides the ability to send messages between peer systems that are at the same level of hierarchy. This pattern implies bidirectional messaging.
Asynchronous Messaging Template
If the protocol used supports asynchronous messaging, any other data transfer patterns can be built on its basis.
Among the protocols that support this pattern are XMPP, Advanced Message Queuing Protocol (AMQP), and, at the IP level, the User Datagram Protocol (UDP). However, in the case of using UDP to implement this pattern, there may be problems with firewalls.
Template "reliable delivery of messages"
Applications that perform critical functions need to know that the message was delivered to the recipient at least once. As a matter of fact, using the asynchronous data exchange pattern, this requirement is fulfilled. A message may be lost on the way, but using the request-response template allows you to request that the message be sent again, until a confirmation (or response) is received from the party who should receive the message. Here you need to take into account that both the message and the answer about its receipt can be lost, so this template guarantees that the message will be delivered at least once. However, the fact that the message will be delivered to the recipient no more than once (or at least once) is not very suitable for some applications, for example, those that use the concept of transactions or count messages.
The use of a reliable messaging template ensures that the message will be delivered to the recipient exactly once. Among the protocols that support reliable message delivery are Message Queuing Telemetry Transport (MQTT), AMQP. In addition, thanks to open extensions, such functionality is supported by HTTP and XMPP.
Multicast Messaging Template
The previous template is busy messaging between two objects. Sometimes, however, a more efficient approach is required if the same information needs to be sent to several recipients at the same time. The simplest of the templates that implement this functionality is “multicast messaging” (multicasting). Within this template, the sender sends one message through an intermediate link in the system (it can be a broker or a router), after which the message is sent to several recipients, each of whom registered to receive such messages.
Multicast Messaging Template
Through the use of this template, you can reduce the load on the network, since the sender does not need to send the same message to everyone who expects it. In fact, the sender does not even need to know who exactly will receive the message. This template can be quite useful in a variety of situations. For example, when synchronizing multiple devices or distributing the same data among multiple recipients. Multicast messaging is supported by the XMPP, AMQP and UDP protocols.
Here it is appropriate to make some warnings. They concern the use of multicast messaging to implement other communication schemes.
So, although this template can be used to reduce the load on the network, it is often referred to as a way to bypass the restrictions in the protocol used, as well as to implement the “event subscription” template on the basis of a certain protocol. If, for example, to use multicast messaging to reduce delays in networks where necessary, but it is impossible, to implement the “event subscription” template, this template will not increase, but increase the network load. In addition, multicast data systems are more difficult to secure.
With regard to improving the efficiency of network bandwidth usage with the use of multicast data transmission, resources can be saved only if the recipients consume most of the received data. If a significant proportion of the data transmitted in this way is not used by recipients, this is a reason to consider other interaction patterns.
Publisher - Subscriber Template
The Publisher-Subscriber template (publish-subscribe) is an extension of the multicast message delivery template. The principal difference between the two is that the transmitted messages are stored on the intermediate node. These messages, or links to them, are then distributed to subscribers interested in them.
Features of the implementation of the template depends on the protocol used, depends on it, as well as on the settings of the intermediate node, and which messages are stored. This can only be the most recent message, or a specified number of messages, or all messages.
Here, in addition, the difference in the transmission of the message itself and links to it is important, since it affects the required network bandwidth and, as a result, the performance of the solution.
If subscribers use most of the messages, the transmission of the messages themselves is more efficient, as is the case with multicast data. If the actual data consumption by recipients depends on some additional factors, then it is more efficient to send links to messages. They are smaller than the messages themselves, and subscribers are likely to use only a small number of them in order to get those messages that are referenced. In this case, in order to receive a message by reference, it is required to make additional calls to the message storage node using the “request-response” model.
The Publisher / Subscriber template supports protocols such as MQTT, AMQP, XMPP.
Template "turn"
Queues, specifically, FIFO queues is a data exchange pattern that allows one or more entities to send certain messages or tasks to be processed into a queue, after which one or more recipients receive these messages in the order in which they were queued .
Template "turn"
The queue is usually located at an intermediate node, or on a network to which all participants in the data exchange are connected. Queues are a great way to load balance. In turn, they collect tasks from various sources and distribute them among existing processors, possibly with different productivity.
Using a queue, you can avoid hard communication between the systems that transmit data and the systems that receive and process this data. As a result, depending on the actual workload on the system, it is possible to increase or decrease the number of data receivers and transmitters. Among the protocols we have already mentioned, only AMQP has built-in queue support.
Message Brokers Template
Message brokers are usually standardized components of the supporting network infrastructure of IoT projects. They beautifully solve the problems caused by the limitations that firewalls impose on bidirectional data exchange between devices. A broker allows entities to connect to it by transmitting messages between clients connected to it. Since all connections are made through a broker, only the broker must be accessible from the Internet. The firewall does not need to accept or redirect incoming connections to devices, as it would be necessary if you use a protocol that provides peer-to-peer communication, strictly limited to a similar messaging model.
In addition to managing messages, brokers can provide additional services to connected clients. For example, a broker can act as an intermediary in the implementation of a multicast messaging template, publisher-subscriber and queue templates.
In addition, message brokers typically provide client authentication services. This makes it easier to work in distributed networks, where device authentication can be a daunting task. Thus, if a broker can report on the status of already authenticated participants of the system involved in the data exchange, other participants can use this information to make security decisions. This, moreover, eliminates the need to implement its own authentication scheme for each participant in the data exchange.
Although the exchange of messages between peer systems is only one of the options for organizing communications, such solutions should provide for client authentication. Otherwise, the security of the system will seriously suffer. If you use a protocol that includes message brokers, then you probably will not need to develop your own support services that allow the solution to work reliably and safely.
The XMPP, AMQP, and MQTT protocols, in one form or another, use this pattern.
Federation Template
“Federation” (federation) is an important template in which a certain global network is divided into logical parts. This allows for global scaling of the solution and provides everything needed for its natural growth.
Federation Template
The main idea here is to allow solutions to be increased without limiting the performance of the existing network infrastructure, using the divide and conquer approach.
When communicating without brokers, for example, as with HTTP and CoAP, a federated structure is available at the domain level. Each domain points to its own set of IP addresses, and its own web server is associated with it. You can add new web servers to the system, in new domains, without restricting access to existing systems. This approach is one of the main keys to the success of the World Wide Web.
When using protocols involving brokers and supporting federations, brokers connect to each other to route messages. Each broker manages authentication in their own domain and knows how to connect to other domains to redirect messages to them. In addition, federated broker networks provide a convenient solution to the problem of global identification of data exchange participants.
The most widely known protocol using brokers and federations is the Simple Mail Transfer Protocol (SMTP). Among the protocols we are talking about in this material, able to work with brokers, only XMPP supports the federation.
Pattern "detection"
Consider a conditional example. Suppose we have some factory-made device that is planned to be used in the IoT system. If, for example, it is planned to be used in a system with multicast data transmission, we will immediately encounter some difficulties associated with the integration of devices into the system.
They consist in the fact that “things” are only aware of their own identification information (it could be something like a MAC address), but they know nothing about how they will be “visible” on the network to which they are going to be connected. , nor about a certain main network device with which they will have to interact.
After installation and configuration (the more automated the configuration is, the better), “things” will learn about their network identification information, but not about how to connect to the host device. In turn, the main device knows its own network address, as well as the factory data of “things” (which, for example, can be quickly entered into the system by scanning stickers on the boxes), but not the network data of other devices.
The discovery messaging template allows you to create a mechanism by which the network identification data of the slave devices is mapped to the network data of the host. This is done using the general knowledge of the initial identification parameters of the slave devices.
Pattern "detection"
This template is implemented using the "registry of things" (Thing Registry), accessible on the network as the "things", and the main device. Clients are registered in the registry, and the main device accesses them through the registry, using only their factory identifiers. If the request is successful, the network identity of each of the participants in the data exchange is sent to the other, and both thus know how to interact with each other. There is an XMPP extension that supports this pattern.
Template "delegation of trust"
On the Internet, the ability to make informed security decisions is important. When using the delegation of trust pattern, end devices redirect requests to a more secure, trusted system in real time, and after receiving a response, they perform certain actions.
Template "delegation of trust"
The actions of a trusted entity, when new requests are received from client systems, can be based, for example, on machine learning, or on settings that are set by the administrator, possibly reacting to requests from the system when it receives new requests.
In order to be able to implement this pattern, you must use asynchronous bidirectional messaging. There is an XMPP extension that supports delegation of trust.
Results
In conclusion, I would like to note that by adhering to standards and open specifications, it is possible to improve the compatibility of IoT development with existing systems. Similarly, by using open, standardized, interchangeable components, the need for expensive infrastructure can be avoided. And the correct choice of the data exchange pattern is a great way to protect yourself from serious reworking of the project in the later stages of its development.
Some of the templates that we talked about may increase the complexity of the project at the very beginning of working on it, but these additional costs are nothing compared to the cost of possible future discrepancies. For example, the difficulties associated with integration into the existing environment. Immediately it is almost impossible to foresee, however, the correct choice of the template will help to avoid such problems.
Source: https://habr.com/ru/post/397241/
All Articles