Our experience with Django, or 10 useful modules that make life easier

For the past 15 weeks, we have been actively working on the Stillion project, our first mainstream project written in Django. An interesting experience was gained that we would like to share with the community.
The article, above all, will be interesting for newcomers to Django.

Django Modules

south

South is a mechanism for migrating schemas and data to the database for django, then without which it is impossible to live with joint development. Unfortunately, in Jang there is no regular migration mechanism for tracking changes in the project database schema. To fill this gap, we used the south, which showed itself from the good side, the main thing is to understand how it works, everything is perfectly described in the documentation . Rake: migration can roll without fixtures, this should be used. Data migrations must be written by hand. If you use abnormal fields (for example, a WYSIWYG field) and then abandon them, the old migrations will not work (but you can rewrite them manually).

django-mptt

Implementation of nested sets for django models. Adds the appropriate fields, as well as methods of working with the tree. An indispensable thing for the implementation of hierarchical structures in a relational database. Fein-cms was used in the admin panel to represent the trees.

django-annoying

A set of useful utilities, such as the decorators of the types render_to and ajax_request or HttpResponseReload. Recommended to enhance the elegance of the code.
')

django-debug-toolbar

A module that displays a panel with debug information. We were primarily interested in the SQL query log, which is supplemented with information about the source of the query, time, etc.

pymorphy

Morphological analyzer integrated with Django. Used to induce words that are stored in normal form (that is, it saves us from storing various forms of a single word). He also knows how to put words in the plural:

morph.pluralize_inflected_ru(u'', 38) >>>  

Disadvantages - slow sqlite as backend (there are faster alternatives to Shelve, CDB and Tokyo Cabinet, but we don’t like them), not all words are correctly processed, but this happens rarely.

pytils

In order not to suffer from "March 18, 2011" we use pytils to work with Russian dates. The module, by the way, is able to transliterate the Russian text and choose the correct case depending on the number, just like pymorphy (although without a dictionary, you will have to store 3 forms of the word in the template).

sorl-thumbnail

An application for automatically creating thumbnail images. It works on the basis of the key-value of the repository (of course, the redis available from the box is used as a backend ). Not the miniatures themselves (they are stored in files) are stored in the backend, but only meta-information (a schematic diagram of the module operation ).

 {% thumbnail image "100x100" crop="top" as im %} <img src="{{ im.url }}"> {% endthumbnail %} 

django-compress

Gluing and minifying CSS and JS. In the configuration file sets are stored for gluing, in the template we indicate the name of the assembly and compress generates a minified version (filters YUI and CSSMin). You can extend the functionality with your own filters. It supports the versioned file name, which is very convenient when setting the Expires header for a long time to come (manage.py syncompress will update the names and contents of the minified files itself when laying out production).

django-sentry

A very useful application from the creators of DISQUS, allowing you to conveniently log errors into the database. Sentry intercepts exceptions (for example, Http404), saves them and provides a beautiful interface that shows the frequency of the error (including graphically). Possible to collect errors from multiple servers. Sentry Review on Habré

django-admin-tools

The application helps to make admin dashboards useful. There is a built-in set of widgets, but it is not difficult to write your own. You can also create a menu for your tasks, which will be available on all admin pages. Overview of Admin Tools on Habré

Comments

When we made comments, we decided that authorization through third-party services (vkontakte, facebook, openid and others like them) would be the best option. This is very convenient for the user, because he does not need to invent and remember a lot of passwords from different resources. In order to do this quickly and easily, there is a Loginza service. We didn’t like the interface very much and we implemented the integration with the main services ourselves.


The publicauth module was taken as a basis , but it had to be thoroughly finished. The matter was not limited to adding integration with mail.ru and other Russian services. We added the “correct” signature verification of the openId providers and made a rather friendly widget in “two clicks”.

Performance Optimization

In practice, one of the weaknesses of django was the ORM. He, like all ORM, contributes to the production of a large number of requests and consumes a lot of memory when extracting data. Without caching, the generation of the main page took a few seconds. Of the possible optimization strategies, caching turned out to be the cheapest and most efficient since the database schema did not allow the use of standard tools such as select_related.

Caching

Caching blocks by timeout, available out of the box, did not suit us, since it did not provide a convenient way to reset the cache. Therefore, the following strategy was chosen: the cache lives indefinitely (TIMEOUT = 0), and one of the parts of the cache entry key is the date of the last update of the main object. Dependent objects update the main using post_save signals. Thus, when the object is changed, the name of the key for caching is automatically changed, which allows you to see the most relevant data on the site. Redis with the allkeys-lru test algorithm is used as the cache backend (the cache keys that have not been used for a long time will be deleted when the maxmemory limit is reached). In the radish, by the way, we also store user sessions to unload the database a little more.

Order by RAND ()

In order not to load the database with a sample of random values ​​via .order_by ('?'), The following approach is used: the data that needs to be mixed is retrieved and put into the cache (especially if the logic of sampling this data is not trivial and requires a large number of queries) list of identifiers, and when issuing to this list applies random.shuffle ().
Summary: cache wisely :-))

As a conclusion

Django is a great framework with lots of nice additions; when you use it, you understand that it was created for our convenience. Enjoy your coding!

Shl. Do you have any familiar designers / front-end developers looking for work (permanent, in the office, Moscow)?