Web server - your first Arduino network program

Introduction

In my publications [1,2,3] the sensor chain is described in detail - Arduino-interface Python. In real conditions of industrial production, the sensors are located at a considerable distance not only from each other but also from the place where the centralized processing of the measuring information takes place. It is logical to transmit information from sensors over a computer LAN using well-developed network technologies. This publication is written in style exercises for assembling and configuring a network application with Arduino at home.

Formulation of the problem

1. Use the Arduino Ethernet library with the Arduino Ethernet Shield extension to create a web server.
2. Create remote access to Arduino using the network on your home computer.
3. Use the standard Arduino example to provide moisture and motion sensor values ​​using a web server.
4. Develop web applications using Python.

Stages of solving the problem

1. Design and build hardware for using Arduino and Ethernet Shield.
2. Running the default sample from the Arduino development environment as the starting point for creating a server.
3. Changing the example to place your equipment and redeploy code.
4. Web application development using Python.

Connect Ethernet Shield over Arduino Uno

Arduino Ethernet Shield is an officially supported open source network extension designed to work with Arduino Uno. Ethernet Shield is equipped with an RJ45 connector for connecting to an Ethernet network. Ethernet Shield is designed for installation on the Arduino Uno, and it expands the pins of the Arduino Uno at the top of the board. Ethernet Shield is also equipped with a slot for microSD memory cards for storing important files over the network. Like most extensions, Ethernet Shield is powered by the Arduino board to which it is attached.


')
Each Ethernet Shield has a unique hardware (MAC) address. You can see it on the back of the board. You can write down this hardware address, as it will often be required in upcoming exercises. Also make sure that you are familiar with installing the Arduino Ethernet Shield for the following exercises.

Below is a diagram of the fritzing scheme required for this exercise. The first thing you need to do is connect an Ethernet Shield on top of your Arduino Uno.

Arduino uno

When deploying Arduino hardware for a remote connection without USB, you will need to provide external power to the board, since you no longer have a USB connection to power the board.

Now connect the Arduino Uno to your computer using a USB cable. You also need to connect the Arduino to your local home network using an Ethernet cable. To do this, use a straight CAT5 or CAT6 cable and connect one end of the cable to your home router.

This router must be the same device that provides network access to the computer you are using. Connect the other end of the Ethernet cable to the Ethernet port of the Arduino Ethernet Shield. If the physical layer connection is established correctly, you should see a green light on the port.



Now it's time to start coding your first Ethernet example. In the code, you will need to change the MAC and IP addresses so that it works for your configuration. Although you can get the Ethernet Shield MAC address from the back of the board, you will have to select an IP address according to the configuration of your home network.

When you have already received the IP address of the computer you are working with, select another address in the range. Make sure no other network host uses this IP address. Use these MAC and IP addresses to update the following values ​​in the code.

byte mac [] = {0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02}; IPAddress ip (192, 168, 0, 102); //        File --> Examples --> Ethernet --> DhcpAddressPrinter 

In an IP network, the visible range of IP addresses for your network is a function of another address, called a subnetwork or subnet. Subnet your LAN IP network can help you choose the appropriate IP address for Ethernet Shield in the range of IP addresses of your computer. You can learn about the basics of the subnet at [4].

Before delving into the code, compile the code with these changes and upload it to Arduino. After successfully completing the download process, open a web browser and enter the IP address specified in the Arduino sketch. If all goes well, you should see text showing the values ​​of the analog pins.

To better understand what happened here, back to the code. As you can see, at the beginning of the code, we initialize the Ethernet server library on port 80 using the Ethernet Server method from the Ethernet library:

 EthernetServer server(80); 

At runtime setup (), the program initializes the Ethernet connection via Ethernet Shield using the Ethernet.being () method with variable mac and ip , which you defined earlier. The server.begin () method will start the server from here. Both of these steps are required to start the server if you use the Ethernet library for the server code:

 Ethernet.begin (mac, ip); server.begin (); 

In the loop () function, we initialize the client object to listen for incoming client requests using the EthernetClient method. This object will respond to any request coming from connected clients that are trying to access the Ethernet server through port 80:

 EthernetClient client = server.available (); 

After receiving the request, the program will wait for the download of the request. It will then respond to the client with formatted HTML data using the method:

 client.print (): while (client.connected()) { if (client.available()) { char c = client.read(); Serial.write(c); # Response code } 

If you try to access the Arduino server from a browser, you will see that the web server responds to clients with data on analog contacts. Now, to get the correct values ​​for the humidity and PIR sensors that we connected in hardware design, you will have to perform the following code modification.

Here you will notice that we respond to customers with calculated relative humidity values ​​instead of raw readings from all analog outputs. We also changed the text to be printed in the web browser to match the correct sensor name:

 if (c == '\n' && currentLineIsBlank) { // send a standard http response header client.println("HTTP/1.1 200 OK"); client.println("Content-Type: text/html"); client.println("Connection: close"); client.println("Refresh: 5"); client.println(); client.println("<!DOCTYPE HTML>"); client.println("<html>"); float sensorReading = getHumidity(analogChannel, temperature); client.print("Relative Humidity from HIH4030 is "); client.print(sensorReading); client.println(" % <br />"); client.println("</html>"); break; } 

In this process, we also added the Arduino function, getHumidity (), which will calculate the relative humidity from the values ​​observed from the analog outputs.

 float getHumidity(int analogChannel, float temperature){ float supplyVolt = 5.0; int HIH4030_Value = analogRead(analogChannel); float analogReading = HIH4030_Value/1023.0 * supplyVolt; float sensorReading = 161.0 * analogReading / supplyVolt - 25.8; float humidityReading = sensorReading / (1.0546 - 0.0026 * temperature); return humidityReading; } 

You can apply these changes to the WebServer Arduino example for the testing phase, or simply open the WebServer_Custom.ino sketch from the Exercise 1 - Web Server folder in your code directory. As you can see in the open sketch file, we have already modified the code to reflect the changes, but you still have to change the MAC and IP addresses to the corresponding addresses.

After you finish these minor changes, compile and load the sketch into Arduino. If everything goes according to plan, you should have access to the web server using a web browser. Open the IP address of the newly prepared Arduino in a web browser.

You should get the same answer as shown in the following screenshot. Although we only show moisture values ​​through this sketch, you can easily attach motion sensor values ​​using additional client.print () methods.



Like the mechanism we implemented in this exercise, the web server responds to the request made by the web browser and provides the web pages you are looking for. Although this method is very popular and universal for delivering web pages, the payload contains a lot of additional metadata (data about the data) compared to the actual size of the sensor information.

In addition, server implementation using the Ethernet server library takes up a lot of Arduino resources. Arduino, being a device with limited resources, is not suitable for running a server application, since Arduino resources must be prioritized for processing sensors, rather than communication.

A web server created using the Ethernet library supports a very limited number of connections , which makes it unsuitable for large-scale applications and multi-user systems.

The best approach to solving this problem is to use the Arduino as a client device or use light communication protocols designed to work with hardware devices with limited resources. In the next few sections, you will learn and implement these approaches for Arduino communication on Ethernet.

Web application development using Python

Having implemented the previous program, you have enabled networking on the Arduino. In the previous example, we created an HTTP web server using methods available from the Ethernet library. Having created the Arduino web server, we made Arduino resources available on the web. Similarly, Python also provides extensibility through various libraries to create web server interfaces.

By running a Python-based web server on your computer or other devices, such as Raspberry Pi, you can avoid using the Arduino to host a web server. Web applications created using high-level languages ​​such as Python can also provide additional features and extensibility compared to Arduino.

We will use the Python library, web.py, to create a Python web server. We will also use this library to create interactive web applications that allow you to transfer data between an Arduino client and a web browser.

After you learn the basics of web.py, we will interact with Arduino from web.py using serial ports to make the Arduino accessible via the Python web server. Then we will update the Arduino communication method from serial interface to HTTP-based messaging.

Python Web Framework - web.py

A web server can be developed in Python using various web frameworks such as Django, bottle, Pylon, and web.py. We have chosen web.py as the preferred web structure due to its simple but powerful features.

The web.py library was originally developed by the late Aaron Swartz to develop a simple approach for building web applications using Python. This library provides two basic methods, GET and POST , to support the HTTP Representation State Transfer (REST) architecture.

This architecture is designed to support the HTTP protocol by sending and receiving data between clients and the server. Today, the REST architecture is implemented by a huge number of websites for transmitting data over HTTP.

Installing web.py

To get started with web.py, you need to install the web.py library using Setuptools. On Linux and Mac OS X, run one of these commands on the terminal to install web.py:

 $ sudo easy_install web.py $ sudo pip install web.py 

On Windows, open a command prompt and run the following command:

 > easy_install.exe web.py 

For pythn 3.61, type in the command line:

 > easy_install-3.61.exe web.py 

If Setuptools is configured correctly, you can install the library without any difficulty. To verify the library installation, open the Python online help and run this command to make sure that you imported the library without any errors:

 >>> import web 

The first Python web application

Implementing a web server using web.py is a very simple and straightforward process. The web.py library requires the declaration of the required GET method to successfully launch the web server.

When a client tries to access the server using a web browser or another client, web.py receives a GET request and returns the data specified in this method. To create a simple web application using the web.py library , create a Python file using the following lines of code and execute the file using Python.

 import web urls = ( '/', 'index') class index: def GET(self): return "Hello, world!" if __name__ == "__main__": app = web.application(urls, globals()) app.run() 

When executed, you will see that the server is now running and accessible via the address 0.0.0.0 : 8080. Since the server program is running on an IP address of 0.0.0.0, you can access it using the same computer, local host, or any other computer on the same network.

To test the server, open a web browser and go to 0.0.0.0 : 8080. When you try to access the server from the same computer, you can also use 127.0.0.1 : 8080 or localhost : 8080. 127.0.0.1 IP address actually means localhost, that is, the network address of the same computer on which the program is running. You can see the response of the server displayed in the browser, as shown in the following screenshot:



To understand how this simple code works, check out the GET method in the previous code snippet. As you can see, when a web browser requests a URL, the GET method returns Hello, world! into the browser.

In the meantime, you can also observe two more required components of web.py in your code: the urls and web.application () methods. The web.py library requires initialization of the location of the response in the declaration of the urls variable.

Each web.py based web application requires that the application (urls, global ()) method be called to initialize the web server. By default, web.py applications run on port 8080 , which can be changed to a different port number, specifying it at run time. For example, if you want to start the web.py application on port 8888 , run the following command:

 $ python webPyBasicExample.py 8888 

Although it only returns plain text, you have successfully created your first web application using Python. We will move on from here and build more complex web applications in the following chapters using the web.py library.

To develop these complex applications, we need not only the GET method. Let's begin the study of preliminary concepts to further improve your familiarity with the web.py library.

Basic web.py concepts for developing complex web applications

The web.py library has been designed to provide convenient and simple methods for developing dynamic websites and web applications using Python. Using web.py , it’s easy to create complex websites using just a few additional Python concepts.

Thanks to this limited training course and easy-to-use methods, web.py is one of the fastest ways to create web applications from programming languages. Let's start by understanding these concepts of web.py in detail.

URL handling

You may have noticed that in our first web.py program we defined the variable urls, which points to the root folder (/) of the Index class:

 urls = ( '/', 'index' ) 

In the previous declaration, the first part, '/', is the usual expression used to match actual URL requests. You can use such expressions to process complex requests coming to your web.py server and point them to the corresponding class.

In web.py, you can associate different places on your landing pages with the appropriate classes. For example, if you want to redirect the data / data location to a data class in addition to the Index class, you can change the urls variable as follows:

 urls = ( '/', 'index', '/data', 'data', ) 

Under this condition, when the client sends a request for access to the address http: // <ip-address>: 8080 / data, the request will be sent to the data class, and then to the GET or POST method of this class.

GET and POST methods

We created an Arduino-based web server that works on port 80, we used a web browser to access the web server. Web browsers are one of the most popular types of web clients used to access a web server; Other types are c URL, wget and web crawlers.

The web browser uses HTTP to communicate with any web servers, including the Arduino web server we used. GET and POST are the two main methods supported by the HTTP protocol for addressing server requests from a web browser.

Whenever you try to open a website in your browser or any other HTTP client, you actually request the GET function from the web server; For example, when opening a website URL [5], you are requesting that the web server hosting this website submit you a GET request for the location '/'.

In the URL Handling section, we learned how to associate web.py classes with URL placements. Using the GET method provided by the web.py library, you can associate a GET request with individual classes. After you receive a GET request, you need to return the corresponding values ​​as a response to the client. The following code snippet shows how the GET () function will be called when someone makes a GET request to the location “/”:

 def GET(self): f = self.submit_form() f.validates() t = 75 return render.test(f,t); 

The HTTP POST function is mainly used to send a form or any other data to a web server. In most cases, POST is embedded in the web page, and a request to the server is generated when the user sends a component that carries the POST function. The web.py library also provides the POST () function, which is called when the web client tries to connect to the web.py server using the POST method. In most implementations of the POST () function, the request includes some data passed through forms. You can extract individual form elements using the f ['Celsius' ...] value, which will give you the value associated with the Celsius form element. When the POST () function has performed the provided actions, you can return the relevant information to the client in response to the POST request:

  def POST(self): f = self.submit_form() f.validates() c = f['Celsius'].value t = c*(9.0/5.0) + 32 return render.test(f,t) 

Templates

Now you know how to redirect an HTTP request to the appropriate URL, as well as how to implement methods to respond to these HTTP requests (that is, GET and POST). But what about the webpage that should be displayed after receiving the request? To understand the rendering process, let's start by creating a folder called templates in the same directory where our web.py program will be located. Templates that will be used to render web pages on demand will be stored in this folder. You must specify the location of this template folder in the program using the template.render () function, as shown in the following line of code:

 render = web.template.render ('templates') 

After you create an instance of the render folder, it's time to create template files for your program. According to the requirements of your program, you can create as many template files as you like. To create these template files in web.py, a language called Templetor is used. Read more about this at [5]. Each template file created using Templetor must be stored in HTML format with a .html extension.

Let's create the test.html file in the templates folder using a text editor and paste the following code snippet into the file:

 $def with(form, i) <form method="POST"> $:form.render() </form> <p>Value is: $:i </p> 

As you can see in the previous code snippet, the template file begins with $ def with () , where you need to specify the input arguments as variables in brackets.

After rendering the template, they will be the only variables that you can use for the web page; For example, in the previous code snippet, we passed two variables ( form and i ) as input variables. We used a form object using $: form.render () to display it inside a webpage. When you need to display a form object, you can directly pass another variable by simply declaring it (that is, $: i). Templetor will display the HTML file of the template as it is, using variables when they are used.

Now you have the test.html template file, ready for use in your web.py program. Whenever the GET () or POST () function is executed, you need to return the value to the requesting client. Although you can return any variable for these requests, including None, you will have to display a template file where the answer is related to loading the web page. You can return the template file using the render () function, followed by the template file name and input arguments:

 return render.test (f, i); 

As you can see from the previous line of code, we return the rendered test.html page by specifying the function render.test () , where test () is just the name of the file without the .html extension. The function also includes the form object, f, and the variable i, which will be passed as input arguments.

Forms

The web.py library provides simple ways to create form elements using the Form module. This module includes the ability to create HTML form elements, receive input from users, and validate these inputs before using them in a Python program. In the following code snippet, we create two form elements: Textbox and Button , using the Form library:

 submit_form = form.Form( form.Textbox('Celsius', description = 'Celsius'), form.Button('submit', type="submit", description='submit')) 

In addition to Textbox (which receives text input from users) and Button (which sends the form), the Form module also provides several other form elements, such as Password for receiving hidden text input, Dropbox for receiving a mutually exclusive drop-down list, Radioto receiving mutually exclusive inputs from several options and Checkbox selects a binary input from the specified options. Although all these elements are very simple to implement, you should only select form elements according to your program requirements.

In implementing a form via web.py, the webpage must perform a POST method every time the form is submitted. As you can see in the following form implementation in the template file, we explicitly declare the form submit method as POST:

 $def with(form, i) <form method="POST"> $:form.render() </form> 

Conclusion

The publication contains all the necessary data to build in the home network application for remote control of temperature, humidity and movement.

Links

1. Prototyping in the Python-Arduino environment.
2. Methods for developing a flow of motion sensor software that works with Arduino.
3. Using Python to process real-time information from sensors that work with Arduino.
4. Subnetwork.
5. web.py templating system.