Compact server for Django applications

Introduction

Many novice web developers are thinking about where to place their creation. Usually for these purposes are used machines running * NIX of similar systems. My choice was Raspberry PI, because the raspberry:

• running on full Linux
• for a long time lying on the table and gathering dust.

I want to talk about how to set up a server running on a network with a dynamic external IP address. Such a solution is not suitable for launching large projects, and is quite suitable for demonstrating your portfolio and personal use.

We will need

1. Raspberry PI models B, B + or Raspberry PI 2 (because these models have Ethernet on the boards) with Raspbian installed and an SSH server activated. You can read about the setting here , here or here . In addition to Raspian for Malinka there are a large number of alternative distributions . Here, as they say, "for every taste and color."
2. Working Django application.
3. Router with DDNS support. This item is optional, as DDNS can be configured on the malinka itself.

I will work with the model B + raspberry.

Training

On the Malinka installed Raspbian 7.8.
First you need to find the raspberry on the network to connect to it via ssh.
nmap -sP 192.168.1.1/24 | grep raspberry



In my case, there are two small rams in the network, one of which is mine with IP address 192.168.1.100. On some networks, nmap does not show network device names.

In this case, raspberry pi can be found by its MAC address, which has the prefix B8: 27: EB.
sudo nmap -sP -n 192.168.1.1/24 | grep -B 2 B8:27:EB


The -B option for grep determines how many preceding lines should be printed.
')
Connect to raspberry ssh.
ssh pi@192.168.1.100


First, let's ramp up the raspberry to 1 GHz using raspi-config .
Installing Python Package Manager
sudo apt-get install python-pip

We proceed to install the necessary packages. My web application uses MySQL. Nginx and gunicorn are used as Frontend and Backend, respectively.
sudo apt-get install nginx gunicorn mysql-client mysql-server python-mysqldb


During the installation of mysql, you must enter data for the root user DBMS. python-mysqldb is the driver needed when working with models in Django. Django install from Python repositories.
sudo pip install django

At the time of writing, the current versions of nginx and gunicorn in the repositories for Malinka 1.2.1 and 0.14.5, respectively. MySQL version for raspberry 5.5. Just to work with Django you need to install SciPy .
sudo apt-get install python-scipy

nginx 1.2.1 is out of date. Newer can be collected from source . Fresh gunicorn can be installed from the Python repositories.

Server Tuning

We place the web application on the malinka (for example, in / home / pi).
If you have working configs, then it is enough to copy them into the appropriate directories:

• for nginx / etc / nginx / sites-enabled /
• for gunicorn /etc/gunicorn.d/

C nginx nothing complicated. I would like to draw attention to the settings for gunicorn.

 CONFIG = { 'mode': 'wsgi', 'working_dir': '/home/pi/project', #'working_dir': '/home/pi/project/project', 'user': 'www-data', 'group': 'www-data', 'python': '/usr/bin/python', 'args': ( '--bind=127.0.0.1:8081', '--workers=5', # 5    '--graceful-timeout=60', '--timeout=60', #'--debug', #'wsgi:application', 'project.wsgi', ), } 

If working_dir (the path to the wsgy.py file) specify '/home/pi/project/project' and specify 'wsgi:application' in args, then on the malinka the workers start first, then die without specifying the reason (under Ubuntu, for example gunicorn works with both options).

MySQL transfer

You can dump the existing database using the mysqldump utility.
mysqldump -u root -p dbname > dbname.sql

The resulting file consists of a set of SQL statements that restore the structure, as well as information stored in the database.

On Malinka create a database. We start mysql shell.
mysql -u root -p

Add a new database.

 mysql> create database dbname character set utf8 collate utf8_general_ci; mysql> grant all privileges on dbname.* to someusr@localhost identified by 'somepassword'; 

Recover data from dump. With a dump size of 162 MB, the recovery time was about 10 minutes.
mysql -u root -p dbname < dbname.sql

It should be noted that the database is best stored on an external drive, otherwise the micro SD card can quickly become unusable due to frequent write operations. How to do this can be found here . The mysql config is located along the path /etc/mysql/my.cnf

Check

Restart nginx and gunicorn. If everything is configured correctly, you can open the main page.

Go to load testing. Install the apache benchmark.
sudo apt-get install apache2-utils

We test in 4 threads 1000 with requests of Raspberry PI model B +.
ab -c 4 -n 1000 http://192.168.1.100/


 vladislav@vladislav-N53SV:~$ ab -c 4 -n 1000 http://192.168.1.100/ This is ApacheBench, Version 2.3 <$Revision: 1528965 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.1.100 (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.8.0 Server Hostname: 192.168.1.100 Server Port: 80 Document Path: / Document Length: 24839 bytes Concurrency Level: 4 Time taken for tests: 1309.607 seconds Complete requests: 1000 Failed requests: 0 Total transferred: 25018000 bytes HTML transferred: 24839000 bytes Requests per second: 0.76 [#/sec] (mean) Time per request: 5238.429 [ms] (mean) Time per request: 1309.607 [ms] (mean, across all concurrent requests) Transfer rate: 18.66 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 0.1 1 1 Processing: 4924 5237 91.4 5227 6419 Waiting: 4919 5227 91.3 5217 6403 Total: 4925 5238 91.4 5228 6420 Percentage of the requests served within a certain time (ms) 50% 5228 66% 5245 75% 5255 80% 5265 90% 5296 95% 5335 98% 5382 99% 5667 100% 6420 (longest request) 

Queries are slow because most of the time the query takes work from the database. The Raspberry PI 2 model B came to me recently. Look at what it can do with the same settings and data.

 vladislav@vladislav-N53SV:~$ ab -c 4 -n 1000 http://192.168.1.14/ This is ApacheBench, Version 2.3 <$Revision: 1528965 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.1.14 (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.8.0 Server Hostname: 192.168.1.14 Server Port: 80 Document Path: / Document Length: 24838 bytes Concurrency Level: 4 Time taken for tests: 170.083 seconds Complete requests: 1000 Failed requests: 0 Total transferred: 25017000 bytes HTML transferred: 24838000 bytes Requests per second: 5.88 [#/sec] (mean) Time per request: 680.330 [ms] (mean) Time per request: 170.083 [ms] (mean, across all concurrent requests) Transfer rate: 143.64 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 0.1 1 1 Processing: 569 678 104.6 650 1338 Waiting: 567 676 104.1 647 1334 Total: 569 679 104.6 651 1338 Percentage of the requests served within a certain time (ms) 50% 651 66% 682 75% 708 80% 727 90% 796 95% 890 98% 1045 99% 1138 100% 1338 (longest request) 

Raspberry PI 2 processes requests on average 6.16 times faster. Malinki developers are not deceived.

DDNS setup

You can configure DDNS on the router or on the malinka itself. I choose No IP because I have been using it for several years. Consider free use.

Configure DDNS on the router

For example, I will configure DDNS on ASUS RT-N56U with firmware from padavan version 3.4.3.9-091. Open the router menu page
(e.g. 192.168.1.1). WAN-> DDNS.

We select the no-ip.com service, specify the registration data, as well as our added host (technopark-test.ddns.net).
The remaining parameters are set at will.

Now, when changing the external IP address, our application remains available on the network.

Configuring Port Forwarding

We need to give the web application when accessing the host. The router redirects incoming packets that come from outside port X to the internal port Y. In the router's menu, go to WAN-> Port Forwarding. It is necessary to redirect the external port 80 to port 80 Malinki. Add a new rule and apply the changes.

Now Malinka processes all incoming packets on port 80. Check by typing in the address bar of the browser host received in No IP.

Now our web application is available for Internet users.

Configure DDNS on Malinka

This option is not suitable if the Malinka has a private IP , as it will send its local IP address to the No IP service. This is another way to find out the IP address of Malinka LAN. Install the DDNS client.
sudo apt-get install ddclient

During installation, you must select a service. Select other and enter dynupdate.no-ip.com, dyndns2 protocol, username, password, interface - eth0, hostname.

To check, I set the IP update interval to 60 seconds. In the / etc / default / ddclient file you need to set the value daemon_interval = "60".

Dessert

My raspberry was lying on the table for a long time and gathering dust along with tm1638 and DHT11 , displaying temperature and humidity readings in the room and other information.
Still, I was curious to try managing the Raspberry PI GPIO from django. I developed a simple web application that brought together my early work. It allows you to see the temperature and humidity measured by the DHT11, some useful information, controls the 8 relay module (which can be used to control electrical appliances) and sends text to tm1638.




To manage GPIO, you need to run the server as root. This is a potential vulnerability.
Full-fledged use of a web application involves server operation without superuser rights, setting up https, adding account administration capabilities, maintaining logs, separating access to managed devices, electrical appliances on a schedule, and much more.
But this is a completely different story article.

Conclusion

Having a Raspberry PI model B, B + or Raspberry PI 2, a power bank , as well as an "open" Ethernet jack, we get a compact server that can be used to showcase our experiences. Setting up a server for Django applications on a Raspberry PI running Raspbian is not much different from any other Linux build. Packages in repositories may be outdated. To work with new versions, you can manually collect programs from sources.

Source code of the demo application.

PS I want to thank my colleagues in Technopark for their help in preparing the material.
PSS I am ready to listen to good advice and comments, and then correct the material.

The article was created in the framework of the article Technopark@Mail.Ru article contest (park.mail.ru).