Caching in django

In continuation of my article about templates, I want to tell you about the implementation of caching in Django. The main focus will be on caching parts of the template - this question was raised here and was the reason for writing these two articles. In the previous article I was too carried away with the description of the patterns themselves, so I will try to correct this.



What is caching, and what benefits it gives under load, I think no need to explain to anyone. (You can read the introductory words at the beginning of the relevant section of the Django documentation, the links are at the end of the article.)

• whole site;
• specific page;
• plot template;
• some data.

Let us dwell on the implementation of this in Django.
')

Connect and configure

Setting up the caching mechanism in Django is very simple. In the project settings file, you need to specify the CACHE_BACKEND variable containing cache storage parameters: where, how long and how much. The following storage types are supported:

• Memcached (including on another machine);
• database;
• file system;
• local memory.

The following cache options are supported:

• timeout - cache update time, in seconds;
• max_entries - the maximum number of entries in the cache;
• cull_frequency is the percentage of old records that is deleted when max_entries .

The syntax is described in the documentation, and there is no point in repeating it.
For those who have their own servers or the opportunity to allocate a couple of gigabytes of memory for the cache, I think this article will be of little interest. Therefore, I will not describe storage in Memcached and local memory. The choice between the database and the FS will give the reader. How much faster one way or another I did not compare. For myself, I chose the FS because DB is already loaded.
In addition, for ease of development, a “dummy” repository is maintained, which actually does not store anything.

Retreat: production & development version

I think it would be appropriate to describe the method of painlessly combining these two versions.
The main Django project configuration file is settings.py . I added two more files: settings_dev.py and settings_pub.py , and in settings.py wrote:

 import platform
 DEV_MODE = (platform.node ()! = 'Dpp.su')

 if DEV_MODE:
	 DEBUG = True
	 from settings_dev import *
 else:
	 DEBUG = False
	 from settings_pub import *

The platform.node function returns the name of the computer, and I compare it with the name of my server. Perhaps you can come up with a more universal solution, but this is enough for me.
Thus, the different settings I put into these files.
In our case, the variable CACHE_BACKEND in settings_dev.py will be equal to 'dummy:///' , and in settings_pub.py - 'file:///path/to/cache/'

Caching the entire site

To cache the entire site, add 'django.middleware.cache.CacheMiddleware' to the list MIDDLEWARE_CLASSES and the variable CACHE_MIDDLEWARE_SECONDS .
All pages without parameters are cached.
Read more in the documentation.

Page caching (view)

To cache a page, use the cache_page function:

 from django.views.decorators.cache import cache_page
 def cache_this (request):
	 ...
 cache_this = cache_page (cache_this, 60 * 15)

You can also use it as a decorator:

 @cache_page (60 * 15)
 def cache_this (request):
	 ...

The parameter is the cache refresh time in seconds.

Data caching

For caching data, the Django cache provides the expected functions: set, get, delete and a couple extras. Cache settings are taken from the variable CACHE_BACKEND . Details in the documentation.

The previous part of the article mostly duplicated the Django documentation - specifically to show those unfamiliar with junga, but familiar with Habr that caching in Django does not require any effort from the programmer.

Template fragment caching

Caching parts of the template seems to me the most convenient to use. For dynamic sites (read “in general”) you cannot cache the entire page. And in the template we cache only the section we need.
First, in the template you need to connect the library of caching tags.

 {% load cache%}

The cache tag takes two required parameters: the cache refresh time (in seconds) and the name of the cache to be cached.

 {% cache 500 sidebar%}
	 .. sidebar ..
 {% endcache%}

In addition, you can pass any number of additional parameters to store multiple versions of the cache. Examples may be different versions of the cache depending on the user name or on the page number when paginated by the data inside the block.

 {% cache 500 sidebar request.user.username%}
	 .. sidebar for logged in user ..
 {% endcache%}
 {% cache 500 sidebar page%}
	 .. content varies by page parameter ..
 {% endcache%}

I will try to show clearly what happens at this and what advantages it gives us.
Take an example from the previous article. For the blog/tag_index.htm template, the following tree will be built (for color matching, see the previous article):

What can I cache? First of all, it is necessary to cache rarely changing data retrieved from the database.
If we give the user the opportunity to change the information in the "basement" of the site, it is worth caching it - it changes very rarely.

 {% block footer%}
	 {% cache 5000 foot%}
		 {% block copyright%}
		 {% endblock%}
	 {% endcache%}
 {% endblock%}

Looking closely at the examples of templates in the previous article, you can see that the second column (sidebar) does not change for all pages of the blog. Since quite a few requests are executed to fill it (I have a list of tags, the number of articles for each tag, a list of recent articles), it is worth caching.

 {% block sidebar%}
	 {% cache 500 blog_sidebar%}
		 {% block tags%}
		 {% endblock%}
		 {% block recent%}
		 {% endblock%}
	 {% endcache%}
 {% endblock%}

The list of articles also makes quite a lot of requests: you need to select articles depending on the page and for each article choose the corresponding tags.

 {% block content%}
	 {{tag.title}}
	 {{tag.text}}
	 {% cache 500 article_list tag.id page%}
		 {{block.super}}
	 {% endcache%}
 {% endblock%}

We get the following picture:

What happens when this page is requested?
When constructing a tree, for each tag, the functions that construct it are called, which provide data for drawing. It is at this point that requests are prepared in the database. Requests themselves directly occur when accessing data due to their lazy nature in Django. Then the template parser scans the tree and for each block calls the draw function. In the case of a caching tag, tags embedded in it will be constructed and drawn only if the cache has not yet been created or is outdated.
If you do not use tags (be lazy to “produce” tags for one request into the database), then all the data must be provided to the template itself. Yes, due to the lazy nature of requests in Django, the difference is not very significant. However, it turns out not very logical - the controller seems to have worked, but nobody used the results of its work. You can even write a bunch of Python code in the expr tags inside the template itself - the cache doesn't care. However, such “violations” of architecture are much more difficult to maintain. And if another person wrote it ... working in a web-studio and supporting a bunch of websites, I was convinced.

Now a comment on the article , which was the reason for writing all this.
dmmd , in fact, you offer a rudimentary version of what is implemented in Django: tags and lazy requests to the database. The essence remains the same, but the architecture of Django seems to me to be much more slender, obvious and, accordingly, easier to maintain.

Links to official documentation:

• djangodocs: Django's cache framework
• djangobook: Django's cache framework
• djangobook (translation): Caching

PS: this is, as always, a reprint from my blog.