Django models and data concurrency problem solving

Hello!

About Django models there are already a lot of articles on Habré, but I want to share with the public how to use them effectively and not step on the rake.

Starting data

• 2 Django servers running under uWSGI
• 1-2k requests per second
• The project with the movement of money inside

What's next?

Suppose we implement a method of updating the balance for the user. And this method looks like this:
')

class Profile(models.Model): …. def update_balance(self, balance): self.balance += balance self.save() 

In this case, if two simultaneous requests to update the balance come to us, then the balance will update only the second request, because the last request supplanted the first one and took the old data.

At this stage, the F method comes to help us in conjunction with .update ()
F () returns us the value from the database up to date. and the previous section can be written as

 class Profile(models.Model): …. def update_balance(self, balance): Profile.objects.\ filter(pk=self.pk)\ .update(balance=F('balance') + balance) 

In this case, we always get the actual value of the field and some will say that this method solves the problem for us, but it is not. In this case, although everything is implemented correctly, we believe, but this does not solve the problem.

In this case comes to the aid of transactions at the database level.

What is this transaction and how to use it?

To begin with, in Django 1.4.x and 1.5.x, you can enable Transaction Middleware. In Django 1.6+, it was replaced with the ATOMIC_REQUESTS constant, which can be included with each database used in the project.

They work as follows. When the request came to us and before sending this request for processing in view, Django opens a transaction. If the request was processed without exceptions, then commit to the database or rollback is done, if an exception is thrown.

The difference between ATOMIC_REQUESTS and Middleware is that Middleware is enabled for the entire project, and ATOMIC_REQUESTS can be used for one or several databases.

The downside to using this approach is that it creates an overhead on the database.
In this case, manual transaction management comes to the rescue.

Manual transaction management

Django provides many options for working with the django.db.transaction module.

Consider one of the possible methods of manual control - this is transaction.atomic

transaction.atomic is both a method and a decorator and is used only for view methods.

You can secure the purchase of goods by wrapping the view in the decorator. for example

 ... from django.db import transaction ... @transaction.atomic def buy_something(request): .... request.user.update_balance(money) return render(request, template, data) 

In this case, we included the transaction atomicity for the purchase of goods. All responsibility for data integrity was shifted to the database and atomicity solves our problem.

Even in conjunction with atomic transactions, you can use the select_for_update method.
In this case, the row being modified will be blocked for the change until update is called.
Our balance update method can now be written like this:

 class Profile(models.Model): …. def update_balance(self, balance): Profile.objects.select_for_update().\ filter(pk=self.pk)\ .update(balance=F('balance') + balance) 

Findings:

• Atomicity comes to the rescue
• Make atomic only critical sections of code
• Use select for update to lock data during change.
• If possible, try to make transactions as short as possible so as not to block work with data in the database.

Extras: MySQL was told about transaction levels in MySQL: transaction isolation levels .