Django ORM. Add sugar

The Django framework is perhaps the most popular for the Python language. However, for all its popularity, ORM is often criticized - namely, lookup syntax through underscores. In fact, this choice of syntax is quite reasonable - it is easy to understand, expand, and most importantly - simple, like a mop. However, you want beauty, or even straight grace. But beauty is a relative concept, so we will start from specific tasks. If intrigued - welcome under cat.

In fact, lookup through underscores has two major drawbacks:
1. Poor string readability, if it is long enough.
Example:

>>> query = SomeModel.objects.filter(user__savepoint__created_datetime__gte=last_week) 

The string is poorly readable, since at first glance created_datetime can be confused with created__datetime, or vice versa - you can not put a second underscore between user and savepoint. In addition, the lookup parameter can be confused with the model field. Of course, on a closer look, you can see what is meant “more or equal”, but when reading a code, this is the loss of precious seconds!
')
2. It is difficult to reuse the search string. Take as an example the query above and try to sort the results by the created_datetime field.

 >>> query.order_by('user__savepoint__created_datetime') 

As you can see, we had to type a long line again, and we cannot save it as a variable, since in one case we use a string, and the keyword argument is above.

A picky reader will notice that we could save the main part of the string in the variable query_param = 'user__savepoint_created_datetime' and make such a hack:

 >>> SomeModel.objects.filter(**{'{}_gte'.format(query_param): last_week}).order_by(query_param) 

But such a code is even more confusing, that is, the main task of refactoring is to simplify the code, it has not completed.
From the first paragraph, it follows that we need to somehow replace the underscore with a dot. We also know that we can override the behavior of comparison operations: __eq__, __gt__, __lt__, and others.
To use it in a similar way:

 >>> SomeModel.user.savepoint.created_datetime >= last_week 

However, the first solution is not so remarkable, since it is necessary to patch the Djang classes of models and fields, and this already imposes big restrictions on using the solution in the real world. In addition, the model name can be quite long, which means our solution will be too verbose when you have to combine several filters. We will come to the aid of the class Q - very short, does not contain anything extra. Do something like this - and call it S (from Sugar). We will use it to generate strings.

 >>> S.user.savepoint.created_datetime >= last_week {'user__savepoint__created_datetime__gte': last_week} 

However, it is still not very convenient to use:

 >>> SomeModel.objects.filter(**(S.user.savepoint.created_datetime >= last_week)) 

The Q class will come to our aid again - it can be directly passed to the filter, so we will return its finished copy!

 >>> S.user.savepoint.created_datetime >= last_week Q(user__savepoint__created_datetime__gte=last_week) >>> SomeModel.objects.filter(S.user.savepoint.created_datetime >= last_week) 

So, we have API of use, business remains for implementation. The impatient can immediately open the repository github.com/Nepherhotep/django-orm-sugar .

Task number 1. Redefining Comparison Operations

Open the documentation here docs.python.org/2/reference/datamodel.html#object.__lt__ and see what functions are available to us. This is __lt__, __le__, __gt__, __ge__, __eq__, __ne__. Override them so that they return the corresponding Q object:

 def __ge__(self, value): return Q(**{'{}__gte'.format(self.get_path()): value})  . . 

However, the is operation cannot be overridden, there will also be difficulties with checking for contains in the Python style:

 'substr' in S.user.username 

Therefore, for such operations we create functions of the same name:

 def contains(self, value): return Q(**{'{}__contains'.format(self.get_path()): value}) def in_list(self, value): return Q(**{'{}__value'.format(self.get_path()): value}) 

As a demonstration of convenience, we add a useful in_range method:

 def in_range(self, min_value, max_value): return (self <= min_value) & (self >= max_value) 

Its convenience is that you can pass two parameters at once, which was not possible using keyword arguments.

 >>> SomeModel.objects.filter(S.user.savepoint.created_datetime.in_range(month_ago, week_ago)) 

Task number 2. Creating child instances when accessing by point

 >>> S.user.savepoint.create_datetime 

First, we will still work with the attributes of the object, not the class. But since we used the class above without calling the constructor, we simply create an object at the module level. Secondly, the source class itself will be called more responsible - SugarQueryHelper.

 class SugarQueryHelper(object): pass S = SugarQueryHelper() 

To generate attributes on the fly, you need to override the __getattr__ method — it will be called last if the attribute is not found in other ways.

  def __getattr__(self, item): return SugarQueryHelper() 

But we also need to remember the name of the passed parameter in order to generate a Q object based on it, as well as a reference to the parent class.

 class SugarQueryHelper(object): def __init__(self, parent=None, name=''): self.__parent = parent self.__name = name def __getattr__(self, item): return SugarQueryHelper(self, item) 

Now it remains to add path generation, and the module is ready!

  def get_path(self): if self.__parent: parent_param = self.__parent.get_path() if parent_param: #  ,       return '__'.join([parent_param, self.__name]) #         return self.__name 

Now this method can be used not only inside SugarQueryHelper, but also for those cases when you need to pass a query string to order_by or select_related.
We show that this solves the problem voiced above - reusing the query string.

 >>> sdate = S.user.savepoint.created_datetime >>> SomeModel.filter(sdate >= last_week).order_by(sdate.get_path()) 

Further development

The module turned out quite good, despite the triviality of execution. But there are things that could be improved.

If you look closely, the S object does not allow access to fields that are named the same as auxiliary functions — contains, icontains, exact, etc. Of course, it is unlikely that anyone would think of calling fields that way, but by law Murphy such cases will ever occur.

What can be done here? Slightly change the scheme of work - override the __call__ method, and if it is called, then the name of the last object in the path can be skipped. At the same time, the auxiliary functions themselves should be started with an underscore, and registration (without an underscore) through the decorator:

 @register_hook('icontains') def _icontains(self, value): return Q(...) 

However, such a decision seemed to me less obvious in some situations. In the end, because the library only generates Q objects, and the usual way via keywords is still available, I decided not to refine this implementation (the version is available in the special_names branch).

The next thing you could do was implement its Q compliant. Those. instead of importing S and Q, only Q could be used in both cases. However, the Q object has a rather complicated implementation, besides there are public methods that will be incomprehensible to the library user. Solve the problem nahrap did not work, so left as is.

You can also do the same filtering directly from the customized query manager docs.djangoproject.com/en/1.8/topics/db/managers/#custom-managers . Then, by redefining it, you can make a query of the form:

 >>> SomeModel.s_objects.user.age >= 16 <QuerySet> 

However, it breaks the understanding of what is happening within the framework of the approach in jang. In addition, the benefit of this approach is completely lost if you have to combine two or more filters:

 >>> (SomeModel.s_objects.user.age >= 16) & (SomeModel.s_objects.user.is_active == True) vs >>> SomeModel.objects.filter((S.user.age >= 16) & (S.user.is_active == True)) 

Not so brief, is it? Not to mention possible problems, if you try to combine queries from different models - the syntax allows!

Afterword

As you can see, writing a useful thing is not so difficult. In this case, all heavy operations fall on the shoulders of standard jungle functions - after all, the query manager himself checks that he was given a number, date, or F-object, whether field names are correct, and so on. Also, the module will work in both the second and third versions of python.
If someone has any ideas, comments or suggestions - write in the comments or send a pool of requests.

Thanks for attention!