Intel Internet of Things Gateways: sending messages to a MQTT broker using Python

Today we set up a gateway for the Internet of Things to send messages to the MQTT broker (server) using a Python script. Messages will be received by the computer, signed on the relevant topic on the broker. After subscribing, messages posted with this topic will be sent to the computer. In the process of studying this material you follow these steps:

• Installing an MQTT client in Python on a computer.
• Configure MQTT broker.
• Installing an MQTT client on an Intel Core, Atom or Quark based gateway.
• Creating and running a script that sends MQTT messages from the gateway to the broker, which, ultimately, will go to the computer.

Here is the gateway, MQTT broker and computer interaction diagram.


Gateway, MQTT broker and computer interaction

MQTT

MQTT is a messaging protocol that can be used to transfer data from a gateway to a local or cloud server. The server redirects data to remote clients who have subscribed.
')
Here's what you can find out about MQTT from Wikipedia : “MQTT (formerly known as MQ Telemetry Transport) is a simplified network protocol that runs on top of TCP / IP. It is used for messaging between devices on a “publisher-subscriber” basis. The protocol is designed to communicate with remote devices, in cases where it is important that the code takes up as little space as possible, as well as to work in networks with low bandwidth. The publisher-subscriber interaction scheme requires a broker, an intermediary between the sender and the receiver. The broker is responsible for delivering messages to customers who are interested in these messages, based on the topics of messages and subscriptions issued by customers. ”

There are several software packages for creating MQTT brokers. They differ in their capabilities, in some of them, on top of the standard MQTT protocol, additional functions are implemented. Here is a comparison of several brokers.

Mosquitto

Mosquitto is an open source MQTT broker (BSD license) that implements the MQTT version 3.1 protocol. and 3.1.1. In this tutorial we will look at using Mosquitto to deploy an MQTT broker on a local network.

Python and MQTT

The Python programming language can be used to create MQTT messages. The choice can fall exactly in Python if, for example, code written in this language is easily embedded in the application, or if the developer is familiar with Python. Sometimes this language is convenient to use due to the fact that programs written on it do not need to be compiled. They are interpreted on the gateway at run time.
The Python interpreter is included in the standard set of pre-installed software on Intel Gateways for the Internet of Things running Wind River Linux. Thus, Python scripts can be executed on gateways without additional configuration of the working environment. So, the Python 2.7.2 interpreter is included in Wind River Linux 5 and in Intelligent Device Platform XT 2. In Wind River Linux 7 and in Intelligent Device Platform XT 3.1, there is Python 2.7.3 and 3.3.3.

Paho

Paho's Python client provides a client class that supports MQTT v3.1 and v3.1.1 on Python 2.7 and 3.1.1. In addition, it contains some helper functions that greatly simplify the publication of MQTT messages.

Computer settings

Consider setting up a computer that you plan to use to subscribe to messages with some kind of test subject on an MQTT broker on the Internet. After the computer subscribes to the topic, messages published with this topic will be delivered to it.

In order to set up a computer, you will need, first, to install Paho and related software, and second, to test the system by connecting to a broker.

Prerequisites

• Ubuntu Linux 12.04 or higher must be installed on the computer.
• The computer is connected to the Internet.
• The computer must have a connection with the MQTT broker.
• Broker can be located either in the local network or on the Internet.
• The firewall, if used, should not block the two-way exchange of information using the ports 1883, 8883, 8884, and 8885 that are open on the computer.

Paho installation

1. Download the tar.gz archive with the latest version of Paho from eclipse.org . In our case, it was the org.eclipse.paho.mqtt.python-1.1.tar.gz .

2. Place this file in the selected directory.

3. Unzip the archive. For example, with the following command:

 tar -xf org.eclipse.paho.mqtt.python-1.1.tar.gz 

4. The contents of the archive will be in a folder whose name matches its name. Navigate to this folder, and rename it if necessary.

 cd org.eclipse.paho.mqtt.python-1.1 

5. Run the following python commands:

 python setup.py install 

Paho is now installed on the computer.

Paho Testing

Test the Python script using Paho, and at the same time connect to the MQTT broker by subscribing to the active topic. For example, if you are interested in a broker located on the Internet, you can use the broker test.mosquitto.org for testing. This will allow you to check both the network connection and the operation of the software.

1. Go to the directory with examples for Paho:

 cd org.eclipse.paho.mqtt.python/examples 

2. Modify the sub.py script by editing the mqttc.connect and mqttc.subscribe :

 mqttc.connect("test.mosquitto.org", 1883, 60) mqttc.subscribe("#", 0) 

Consider these commands:

 Mqttc.connect — test.mosquitto.org:  MQTT-,     . — 1883: ,    MQTT-. — 60: -   

 Mqttc.subscribe — #:  ,    ,       . — 0:     (Quality of Service, QoS). 

3. Run the script:

 python sub.py 

If the network connection and the program are working properly, test.mosquitto.org will be received from the MQTT broker test.mosquitto.org . Their content does not interest us now. If nothing comes, check the connection to the Internet and the edited lines of the sub.py script.

4. To stop the script, use the keyboard Ctrl-C .

Set up MQTT broker and subscription

In the previous section, we installed Paho on a computer and tested it. Now consider the features of MQTT brokers. Our goal is a working broker who can receive messages from publishers and send them to subscribers. There are several options for organizing interaction with the broker, they are all suitable for, in order to reproduce what we describe below.

• Already configured broker located in the local network or on the Internet (in the cloud). This approach will allow the use of an existing broker. This broker should be ready to receive and send messages. In this case, you should be able to create on it the necessary topics, if they have not yet been created.
  
  
• New broker in the local network, created using Paho. This broker will work on the computer. At the same time, whether it will be the same computer that receives messages from the gateway, or another, does not play a special role. To create a broker, you can use the appropriate Paho libraries for Python.
  
  
• New broker in the local network, created using Mosquitto. Such a broker will also work on a computer, but to create it you will need to use Mosquitto.
  

Using an existing broker connected to a local network

Best of all, if you already have an MQTT broker, ready to work with the application. It must be located on the selected network, must be ready to send and receive messages for the application, you must be able to configure it. If you already have a ready-to-work broker, you can skip straight to the section on setting up the gateway.

If you have not yet set up a broker, there are several ways to get a simple MQTT broker to test the application. Now we will consider them. Please note that here we are not concerned with the configuration of servers or networks. In addition, among the options listed below for working with existing brokers or ways of setting up new ones, you will need only one. Therefore, by selecting it and configuring the system, you can also proceed to the section on configuring the gateway.

Paho: working with an existing broker, accessible via the Internet

On the Internet, you can find several MQTT brokers that are suitable for testing. Here we will use a broker available at http://test.mosquitto.org/ . We have already talked about it, testing the correctness of the system, not paying attention to any topics or messages. Now we use it for more meaningful work and with that, and with another.

The procedure for interacting with this broker is already familiar to you, under the Paho Testing section. However, here we, instead of subscribing to all topics with the mqttc.subscribe("#", 0) command, mqttc.subscribe("mytopic", 0) to a specific topic with the mqttc.subscribe("mytopic", 0) command. Here mytopic is the name of the MQTT theme. The broker will automatically create such a topic if it does not already exist.

Now, if the gateway (or any other computer) sends a message to the broker with the mytopic theme, the broker will send this message to the terminal window in which the sub.py script sub.py .

Installing an MQTT broker on a local network using Paho

Use this method if you want your computer to play the role of an MQTT broker located on a local network. Here, we will need the previously installed Paho Python libraries. The setup procedure is generally similar to the one already discussed. However, you need to pay attention to the fact that the computer and the gateway must be on the same local network, and that the beginning of interaction with the broker (in the sub.py script) looks like this:

 mqttc.connect("localhost", 1883, 60) mqttc.subscribe("mytopic", 0) 

That is, we are connecting not to the server on the Internet, but to the local host, to the network interface of the "internal loop" of the computer. Usually localhost is used as an alias for the IP address 127.0.0.1.

If you subscribe to the topic mytopic , referring to the local host, a local MQTT broker will be automatically launched and such a topic will be created on it. If the gateway now publishes a message with the mytopic theme, contacting the computer at its IP address, this message will be sent to the Linux terminal, in which the sub.py script that subscribed to this topic is executed.

Installing an MQTT broker on a local network using Mosquitto

This method, again, involves using the developer’s computer as a broker located on the local network. But this time we will use Mosquitto.

Details about Mosquitto can be found here . In addition, the construction of the form <mosquitto_command> --help allows you to get help on the commands of this package. In order to start the process of the MQTT broker on the computer, follow these steps.

1. Open a new Linux terminal window.
The broker will be executed in this window, while on the same computer you can receive MQTT messages, only they will arrive in another terminal window.

2. Install Mosquitto, if you have not already done so.

 sudo apt-get install mosquitto 

3. Create a theme and subscribe to it using the mosquitto_sub .

 mosquitto_sub -d -h localhost -t mytopic 

Consider this command:

 -d:    . -h localhost:  –   IP- 127.0.0.1. -t mytopic:  MQTT-.    ,    . 

4. Check if the computer is connected to the same local network that the gateway is connected to.
The local MQTT broker will start automatically after subscribing to a certain topic. The gateway, by contacting the broker at the IP address of the computer, will be able to publish messages with this topic, which will go to the Linux terminal, where mosquitto_sub is mosquitto_sub .

Gateway Setup

Let's do the configuration of the gateway. It involves two basic steps:

• Installing an MQTT client written in Python on the gateway.
  
  
• Connecting the gateway to the test MQTT broker to check for a network connection and correct script operation.

Prerequisites

• This example uses a gateway that runs Wind River Linux 7 with Wind River Intelligent Device Platform XT 3.1. If your gateway uses a different OS, its configuration process may differ from the one shown here.
  
  
• The gateway's operating system must be compiled using the - withtemplate=feature/mqtt in the settings. See here for more details.
  
  
• The gateway must be connected to the same network as the MQTT broker, provided that the broker is located on the local network.
  
  
• If you use an MQTT broker that is hosted on the Internet and use a firewall, it should not block ports 1883, 8883, 8884, and 8885.
  

Installing Paho on the gateway

Installing Paho on the gateway is the same as installing on a computer. You need to download the archive with the latest version of Paho, unpack it and execute the python setup.py install command.

Checking work with MQTT-broker, posted on the Internet

The procedure for checking the operability of the software installed on the gateway and the network connection repeats the procedure for checking the computer.

Sending and receiving MQTT messages

Now that we have a ready-to-work MQTT broker, a computer and a gateway are set up, everything is ready to create a Python script that will publish messages.

Here we will use the code samples from the Paho distribution to create a simple script that can be modified later to fit the needs of a specific project. The script can be created both directly on the gateway and on the computer. If the script is created on the computer, it will need to be copied to the gateway and run there.

Security

Consider the following considerations when creating and running Python scripts.

• If you use the built-in Python interpreter, IMA-security does not hurt, allowing the interpreter to execute new scripts.
  
  
• If you do not use the built-in Python interpreter, and, at the same time, IMA-security is enabled, each script must be signed before starting.
  
  
• If McAffee Embedded Control is enabled on the gateway, the script must be added to the whitelist.

In the example below, we assume that neither IMA nor the white lists on the gateway are included.

Creating and running a Python script to publish MQTT messages

1. Go to the examples folder, which contains code samples for Paho:

 cd org.eclipse.paho.mqtt.python-1.1/examples 

2. Copy and rename the pub-single.py script:

 cp pub-single.py my-pub.py 

3. Edit the my-pub.py file, changing its last line so that the message is sent to the existing MQTT broker and when sending it, the theme that was set when we set up the computer to receive MQTT messages was used (we called the "Mytopic"):

 publish.single("mytopic","Hello", hostname="<broker_location>") 

Consider the device of this command:

 — "mytopic":  ,   . — "Hello":   — "<broker_location>":  MQTT-. 

If a computer is used as a broker, you need to substitute its IP address in the local network here. For example, something like 192.168.1.5. If the broker is on the Internet, then there must be either its IP address or the domain name of the server. For example, test.mosquitto.org .

4. If you were preparing the my-pub.py script on a computer, copy it to the gateway.

5. Run my-pub.py on the gateway to send a message from it to the specified MQTT broker with the mytopic theme:

 python my-pub.py 

Receive MQTT messages on a computer

After the gateway publishes a message with a topic, and the broker receives the message, it will send a message to all MQTT clients that are subscribed to the topic.

To publish, given the settings of our script, just run the python my-pub.py command python my-pub.py on the gateway. When it is executed, the “Hello” message will be sent to the broker, and the computer will be sent from it. On the same computer, given that it has a terminal window open, waiting for messages from an MQTT broker, a message with the following content will be received:

 mytopic 0 Hello 

Consider its structure:

 mytopic:  . 0:   Hello:  

Results and ideas

You have just, using the Python programming language, sent an MQTT message “Hello” with the subject line “mytopic” to an MQTT broker located on the local network, and then received this message on a computer that is subscribed to this topic. What's next? Here are some ideas that relate to the development of an example from this material. We hope they will allow you to expand the use of the MQTT message exchange system in your projects.

• If there are sensors in the system, the gateway can read their readings and publish them as MQTT messages.
  
  
• MQTT messages are well suited for transmitting some information intended for storage in a cloud or local database. If this scenario corresponds to the system you are working on, consider it.
  
  
• Increase the payload carried by messages, perhaps by using a more complex structure of messages and topics.
  
  
• Experiment with Paho features that allow you to send multiple messages with one command.
  
  
• Increase the security of the MQTT messaging system by encrypting, authenticating clients, using whitelists or other means.