Quick and automatic launch of Django on IIS 7.x in a production environment (+ performance tests)

Django is a popular framework for building Python web applications. Its popularity is constantly growing due to the availability of rapid development tools, integrated administrative interface and high speed of work. There is a simple and reliable way to deploy and run django applications on the IIS web server using the Web Platform Installer and the Helicon Zoo package repository.

Helicon Zoo is a repository of popular web-frameworks and applications for Microsoft IIS. It uses Microsoft Web Platform Installer (WebPI) technologies to deploy applications. With their help, various dependencies are processed and the installation process itself takes the necessary components, such as Python, Django, various database drivers and modules. Well, the module itself Helicon Zoo, which the whole thing sticks together with MS IIS 7.

How to use

In web development, there are two rather independent environments - development and production. Helicon Zoo can be used both in production and on the developer’s machine, or both. In any case, the sequence of actions is approximately as follows:
')
First you need to download and install the Web Platform Installer from Microsoft - ( http://www.microsoft.com/web/downloads/platform.aspx ). WebPI already contains a large number of such frameworks and applications for IIS as PHP, ASP.NET, Wordpress, Drupal, phpBB. To connect Helicon Zoo you need to add a new feed to WebPI:

• We launch WebPI and we press Option
  
  
• In the field “Display additional scenarios” we insert http://www.helicontech.com/zoo/feed and click Add feed
  
  
  

Now, if you select Applications, Tools in the WebPI interface, then at the end of the list you can see new applications: Blank Django Project, Blank Rails Project, Blank Perl Project, Blank Mojolicious Project:



The Blank Django Project is a simple “Hello, World!” Application that is used to install all the dependencies required to run a Django application and, in fact, an empty Django project. To install, click “Add” and “Install”, a form with a list of dependencies will appear:



To accept the license and start downloading and installing, click “I Accept”. After installing and downloading all dependencies, the installed project can be launched by clicking on “Launch application in browser”:



This window means that Python, Django, and everything you need to run the applications have just been installed on your computer. Now, if this is a development machine, you can start developing with this empty “Hello Worls” application. After you do something with this application, you can upload it to the production server with all the files, directories, necessarily with the web.config file, and if the server has previously installed the Blank Django Project (no matter what directory, only dependencies), the application should earn there. Every time you upload your changes to the server, you need to restart the Application Pool. This is a feature of Django (and Rails), for other frameworks, restarting may not be necessary. When restarting, data will be automatically migrated - manage.py syncdb . This allows you to use the solution on shared hosting, where the user does not have access to the console.

Under the hood

The core of Helicon Zoo is a native IIS module, in fact, playing the role of a bridge between the IIS web server and frameworks on Ruby, Python, Perl, etc. web server. The interaction between them occurs asynchronously, using technology I / O Completion Port. The transport uses either named pipes or tcp sockets. IIS 7, IIS 7.5, and IIS Express are supported.

The basic configuration of the Helicon Zoo module is located in the <heliconZooServer> section of the applicationHost.config file. This section describes all the FastCGI engines that will be worked through Zoo. Here is an example of the description of the engine for running python wsgi applications (Django applications in particular), zoofcgi.py - this is a python worker that implements transport on named pipes (named pipes):

<engine name="python.2.7.pipe" fullPath="c:\python27\python.exe" arguments="-O %SystemDrive%\Zoo\Workers\python\zoofcgi.py" transport="pipe" /> 

Web.config

Configuration of web applications that work through Zoo is done via the web.config file. Here is an example of such a file for configuring a Django application to work in a 32-bit pool:

 <?xml version="1.0" encoding="UTF-8"?> <configuration> <system.webServer> <heliconZoo> <application name="django.project.x86" > <environmentVariables> <add name="PYTHONPATH" value="%APPL_PHYSICAL_PATH%;%PYTHONPATH%" /> <add name="DJANGO_SETTINGS_MODULE" value="settings" /> <add name="DEPLOY_FILE" value="deploy.py" /> <add name="DEPLOY_LOG" value="log\deploy.log" /> </environmentVariables> </application> </heliconZoo> <handlers> <add name="django.project.x86" scriptProcessor="python.2.7.pipe" path="*" verb="*" modules="HeliconZoo_x86" preCondition="bitness32" resourceType="Unspecified" requireAccess="Script" /> </handlers> </system.webServer> </configuration> 

Here are the important points:

• scriptProcess = "python.2.7.pipe" - link to the FastCGI engine defined in applicationHost.config;
• the PYTHONPATH environment variable — the way in which python is looking for its modules; in our case, we add the path to our django application to PYTHONPATH;
• environment variable DJANGO_SETTINGS_MODULE - python path to the Django application settings.py configuration file, for example, 'mysite.settings'
• The environment variable 'DEPLOY_FILE' is the path to the file inside the application that will be launched before the first request to the application is executed after the restart of the IIS application. In this file you need to execute, for example, commands for database synchronization: 'manage.py syncdb –noinput';
• the environment variable 'DEPLOY_LOG' is the path inside the application to the file where the output of the DEPLOY_FILE command for debugging purposes will be recorded. The directory in which this log file is located must have write permissions for the user from whom the IIS application is running (usually the IIS_IUSRS group).

Static content

The Django framework is not suitable for fast and secure processing of static files (images, scripts, style files). Statics should be processed by the web server directly. To do this, you need to put all the static files in a directory and turn off the request handler in the django application in it:

 <?xml version="1.0" encoding="UTF-8"?> <configuration> <system.webServer> <handlers> <remove name="django.project.x86" /> </handlers> </system.webServer> </configuration> 

Sample Django Project on IIS 7

Suppose our django project consists of three applications:

• site - the site itself with templates, statics, urls.py and settings.py
• blog
• store

Static files located in site / media will be accessible via the / media / virtual directory (i.e. MEDIA_URL = '/ media /'). After installing the Blank Django Project to the root of the site and copying our django project, we get the following directory structure:



/ media / is the virtual directory that points to site / media; Zoo module is disabled in it as described above. In the root web.config, PYTHONPATH points to the site root and DJANGO_SETTINGS_MODULE is set to 'site.settings':

 ... <heliconZoo> <application name="django.project.x86" > <environmentVariables> <add name="PYTHONPATH" value="%APPL_PHYSICAL_PATH%" /> <add name="DJANGO_SETTINGS_MODULE" value="site.settings" /> </environmentVariables> 

Performance

And finally, the most interesting - performance testing. Deployed testing will be later, now while in a hurry, but it is quite possible to judge from these results. Test machine as a server - Core 2 Quad 2.4 Ghz, 8 Gb RAM, gigabit network. To generate the load, a more powerful computer with Apache Benchmark was used. Ubuntu 11.04 Server x64 was used to test Apache and Nginx. IIS 7 tests worked on Windows Server 2008 R2. Any virtualok - honest iron. The most advanced uwsgi as well as wsgi and fast_cgi were used as a transport for Nginx for comparison. More on IIS 7 compared with PyISAPIe.

Two small scripts were written as test pages. The first one displays the current time in high resolution - this is done so that there is confidence that the pages are not taken from the cache. The second does the same thing, but pre-stores the result in the database. All this through a template to use the real infrastructure of Django, the database is MySQL. The settings are everywhere by default, because The goal is to test the most typical configurations. So, the results (in queries per second):



Here everything is more or less clear and stable. Uwsgi is ahead, apparently due to closer integration with the Django code - you will need to smoke it ... PyISAPIe lags far behind, which is understandable.



But with the database, the result is not as stable. Why Nginx + fcgi + MySQL showed such strange 175 queries per second is not clear. The result of MySQL on Windows is also depressing, although on a shared hosting the problem may not be so critical. The fact is that the performance drops mainly due to some internal locks in MySQL, the server itself is almost not loaded, while it generates these 104 queries. It can be assumed that with an increase in the number of sites on the server and, accordingly, an increase in the number of databases, if they are not blocked among themselves, the total capacity of the server will be sufficient.

So we decided to add MS SQL Express to the tests. Its result is also understandable - it all comes down to Python itself and the database driver for it, but overall it looks pretty decent. PyISAPIe with MS SQL Express did not work.

I would like to separately note the ability of IIS 7 to hold a large number of connections. So IIS 7 + Helicon Zoo easily withstood thousands of simultaneous connections (more we just had nothing to generate), while Ubuntu, with the default settings, with the increase in the number of connections, quickly passed. And Apache also turned out to be a very voracious memory. So with the growing number of connections, Apache consumed about 3 GB of memory in 20 seconds of the test - they did not check further.

findings

The presented solution shows stable performance and decent performance. It is quite ready for use both as a development environment and in production. Especially important is the possibility of using Helicon Zoo by various Windows hosting services in order to provide Django services to its customers. I would like to hope that with the growing number of Django servers on Windows, its developers will pay more attention to debugging and optimizing their code under the Windows platform. Yes, and the army of existing Windows developers can be a good help for current open-source projects.