Pangurman Recipes
Recently launched a service for booking restaurants Pangurman . Inside is a more or less typical django site. I will try to tell how everything is arranged there (with pictures). The article will not be anything super-clever, but I hope someone’s a couple of tricks or ideas will seem useful and somehow simplify life.
Standard toolkit - PostgreSQL, Django 1.4 (+ GeoDjango), Sentry, Celery (+ redis), Fabric.
There is nothing unusual on the start page: registration form. As I already wrote , the self-view forms are self-made, django-registration sawed out with indignation after I realized that I didn’t use any views or forms from there, I tried to use the model and the bug tracker of the project was disabled :)
')
When registering, a greeting letter is sent, it is written as html. To send html letters, we use templated-emails : in templates there is an emails folder in which all letters are stored. Each letter is in its own folder containing 3 files: email.html (html version of the letter), email.txt (text version of the letter) and short.txt (subject).
Letters are sent like this:
You can send User-s or email addresses to the mailing list. Nonsense, but simple and convenient, on different projects we have been using this approach for quite some time.
There is another problem with html-letters - it is not very clear how to look at them when developing without sending them to myself all the time. To check locally, we use django-eml-email-backend (the dumbest application with email backend, which saves emails to eml-files instead of sending). Most email clients open eml files without problems (there is a preview on a poppy even with a space).
After registering and setting up an account, a person goes to the main page.
On it you can pay attention to the search form.
Elements were originally put together like this:
The guys try to stick with BEM, the input is often wrapped in a div and they often need to add some css-classes, data-attributes and so on.
To sew up html in the code - bad form, html should be in templates. There is no normal built-in tool to influence the rendering of form elements in a template in jang (and here we need to add data-behavior to the field). Because of these pieces, we are very actively using django-widget-tweaks ( bitbucket , github ) when screwing in the layout, this application allows the screwing process to be simplified and cleaned.
If you need some kind of standard binding for the field, do an html-snippet, which can then be connected via {% include%} (something like this):
The same for error output.
In this block we have 5 restaurants that are available for booking today. They turn out like this:
As you can see, a chain of own methods is built here (
To get around this limitation, use PassThroughManager from django-model-utils and write the QuerySet instead of the manager:
Such "managers on steroids" are obtained (I decided not to select a picture :)). They are used throughout the whole project instead of standard managers.
... and get to the page with a list of restaurants. On this page, a large search form, the results are updated Ajax.
I have a small personal “fad” about js: the site should be available without js. This gives some advantages when developing. In addition to the obvious ones (the error in js does not make the site inoperative) there are less obvious ones: for example, such a site is easier to test (you can test most of all without the help of selenium) + perhaps it is easier to write)
At the moment, we use the following approach to Ajax: Ajax and ordinary requests are processed by the same view, the only difference is in the template. Because Aayaksov template is part of a regular template, then we connect it in a regular template. Normal pattern:
Ajax template:
The template is selected based on request.is_ajax (); There is a declarative js on the client, which can turn any ordinary html form into an Ajax one:
In PanGurman decided to experiment a bit with this. At first it was the @ajax_template decorator, which added an AJAX view to the view:
Those. you wrap the usual view into the decorator, you prescribe the data attribute of the form and that's it: the form becomes an Ajax one.
But this option repeats
Therefore, from @ajax_template refused and while stopped on this option:
show is a successor from TemplateResponse, which for Ajax requests, first tries the template in the ajax subfolder. See https://gist.github.com/2416695
show has a couple more methods that I have always lacked - but, perhaps, wasn’t enough, because there are a couple of subtleties with it and not everything is fine (for example, .with_context will fall if the previous view returns the redirect), but it is quite possible to use it (and problems have little to do with AYAX).
In short, we make the site as if ajax does not exist, but add it later, where necessary, by placing the necessary part of the template in a subfolder.
We fall on the restaurant page.
It also has an Ajax (the available time is checked in the order form via ajax requests), but it is implemented in another way :)
The calendar is built on the client (without js there will be regular input fields) and therefore it is easier to get json from the server, and not html. Using django-tastypie quickly built an API, and through it json is given. It is not much longer than writing a view that gives json, by hand, but the API still comes in handy + different buns like throttling and freehand formats are obtained.
In another project used for the API django-piston; subjectively - django-tastypie is nicer.
A very primitive (and not quite complete) API reference is automatically generated by the following view:
Template:
You can see here: http://pangurman.ru/api-docs/
Payment is bolted through django-robokassa (then more options will be added), SMS notification of reservation - via imobis + celery.
In Pangurman, the user has a personal account from which to pay for purchases. You can replenish it with the help of robokassa, inviting friends, entering promotional codes. Then more ways will be added anyway.
Having read Fowler, for several years I have been using the following approach to the implementation of a “personal account”: the amount is not stored anywhere on the account (however, it can be cached sometimes), we keep transactions on a personal account in the database. This allows you to naturally get a history of operations + saves from errors.
For example, if a person through the payment system pays an amount insufficient to pay for the purchase, the amount will first be credited to a personal account, then they will try to write off, the write-off will not succeed and the person, as needed, will remain without a purchase, but with the amount on the account.
A similar model wanders from project to project with minor changes:
Here you can pay attention to the way choices. Write more, but there are no magic constants and you cannot mistakenly write to the base 'invite_bonus' instead of 'invite-bonus' - MoneyTransfer.INVITE_BONUS is always one and immediately falls with AttributeError, if you write it incorrectly (and the autocomplex appears).
Go to the help page . On this page, part of the text, obviously, it would be good to allow editors to edit without the participation of the programmer.
In django, there are flatpages for this, but we usually use a different approach: we make all the pages with regular views with normal templates, store everything in VCS, and mark the django-flatblocks with pieces of text that need to be edited:
For admins when you hover the mouse over the text shows a link to edit
(this is done by setting your flatblocks / flatblock.html template):
In blocks, markup is stored in markdown, on the edit page we show the widget from django-markitup (with preview):
All texts on the site, in which some markup is allowed, are edited using django-markitup (inspirational descriptions of restaurants with photos, for example). The markdown filter is configured in such a way that it does not touch the html-tags, so if necessary, any markup can be added to the text.
And, the extension is also screwed ( python-markdown-video ), which turns youtube or vimeo links into videos - in a good way it is done through oembed, you probably need it.
For restaurants there is a special panel through which they can now control the opening hours (when tables are available when not). The panel is implemented through another separate instance of AdminSite. There can be users who are appointed by "restaurant managers". The subtlety is that these users should not have the is_staff flag, otherwise they will be able to enter the regular admin panel, which they would not want. And so - the usual customized admin panel.
[rant on]
Often they write that, they say, Djang's admin panel is inflexible and so on. - I did not understand what people specifically mean :) The admin panel provides CRUD, layout and out-of-box design, and if you know what can be customized and what is difficult, then there are no problems. As an Orthodox alternative, some kind of framework for writing dashboards is usually suggested - this is a CRUD perpendicular thing, and there are such frameworks for django admins ( django-admin-tools , nexus ).
If any interaction does not fit into the CRUD - no problems, nothing prevents you from writing your view and putting a link to it from somewhere in the admin area (readonly_fields and list_display are often convenient here; you can also override the template). To throw out or rewrite the admin panel of desire for several years never occurred, use it in all projects, saves a lot of time. Perhaps the specific nature of such projects, or I am really missing something obvious, I don’t know.
[rant off]
Instead of raw_id_fields, we actively use the application with the django-salmonella name in the admin panel , which “infects” widgets to select related records with useful behavior (for FK and M2M:
In addition to the id of the object, its name becomes visible (updated by the Ajax); by clicking on the name, you can immediately get to the page of the object change, which is also convenient
For the tests began to use factory-boy (fork), about which it was already on the habr. Since that recording, one super-useful thing has appeared in the factory-boy: RelatedFactory. It's like a SubFactory, just the opposite (for example, you can make a factory for the user, who will immediately create a profile for him). Documentation can be read here . Example:
Now in the tests, you can write:
As a warm-up I did a factory-boy pull request with the addition of support for the 3rd python, but the 3rd python is a topic for a separate story)
There is one more trick in Django 1.4 with tests. By default, the PBKDF2 algorithm was used to hash passwords. In PBKDF2, one of the characteristics that provide security is low speed. But apart from being protected, a low speed of work still provides a low speed of work :) On the main site, this is not a problem at all (users don't log in so often and log in, and the delay is small), but in tests it turned out that this is a critical thing, especially if you use factory -boy and create many users by assigning them a password through a hash.
In the case of PanGurman, the installation of this line in test_settings.py accelerated the tests by 2 times:
Standard toolkit - PostgreSQL, Django 1.4 (+ GeoDjango), Sentry, Celery (+ redis), Fabric.
start page
There is nothing unusual on the start page: registration form. As I already wrote , the self-view forms are self-made, django-registration sawed out with indignation after I realized that I didn’t use any views or forms from there, I tried to use the model and the bug tracker of the project was disabled :)
')
When registering, a greeting letter is sent, it is written as html. To send html letters, we use templated-emails : in templates there is an emails folder in which all letters are stored. Each letter is in its own folder containing 3 files: email.html (html version of the letter), email.txt (text version of the letter) and short.txt (subject).
Letters are sent like this:
send_templated_email([user], 'emails/activation', context=context) You can send User-s or email addresses to the mailing list. Nonsense, but simple and convenient, on different projects we have been using this approach for quite some time.
There is another problem with html-letters - it is not very clear how to look at them when developing without sending them to myself all the time. To check locally, we use django-eml-email-backend (the dumbest application with email backend, which saves emails to eml-files instead of sending). Most email clients open eml files without problems (there is a preview on a poppy even with a space).
Ok
After registering and setting up an account, a person goes to the main page.
On it you can pay attention to the search form.
Elements were originally put together like this:
<div class="b-filter_selector time"> <select name="persons" data-behavior="Chosen"> <option value="1">1 </option> <option value="2" selected="selected>2 </option> <option value="3">3 </option> </select> </div> The guys try to stick with BEM, the input is often wrapped in a div and they often need to add some css-classes, data-attributes and so on.
To sew up html in the code - bad form, html should be in templates. There is no normal built-in tool to influence the rendering of form elements in a template in jang (and here we need to add data-behavior to the field). Because of these pieces, we are very actively using django-widget-tweaks ( bitbucket , github ) when screwing in the layout, this application allows the screwing process to be simplified and cleaned.
<div class="b-filter_selector time"> {% render_field search_form.persons data-behavior='Chosen' %} </div> If you need some kind of standard binding for the field, do an html-snippet, which can then be connected via {% include%} (something like this):
{% comment %} : {% include "inc/input.html" with field=form.query div_classes="search" no_label=0 %} {% endcomment %} {% load widget_tweaks %} {% if no_label != 1 %}{{ field.label_tag }}{% endif %} <div class="b-input {{ div_classes }} {% if field.errors %}error{% endif %}"> {{ field|add_class:'b-input_input' }} </div> The same for error output.
{% render_field %} from django-widget-tweaks can also add attributes (css-classes, for example) to existing ones: {% render_field form.emails rows="2" class+="b-input_textarea gray" autofocus="" %} {% include "inc/errors.html" with errors=form.emails.errors %} "Available today" block
In this block we have 5 restaurants that are available for booking today. They turn out like this:
restaurants = Restaurant.objects.enabled().free_today_in_city(city).prefetch_related('scenes')[:5] As you can see, a chain of own methods is built here (
.enabled().free_today_in_city(city) ). The built-in jung approach — putting such pieces into the manager — does not allow this, because manager methods return a QuerySet, which already has no custom methods — for filtering, it turns out, you can only use one of your methods.To get around this limitation, use PassThroughManager from django-model-utils and write the QuerySet instead of the manager:
class RestaurantQueryset(QuerySet): def enabled(self): return self.filter(enabled=True) def free_today_in_city(self, city): today = city.now().date() return self.filter(city=city).free(today) def free(self, date): # ... class Restaurant(models.Model): # ... objects = PassThroughManager.for_queryset_class(RestaurantQueryset)() Such "managers on steroids" are obtained (I decided not to select a picture :)). They are used throughout the whole project instead of standard managers.
Click on "View all"
... and get to the page with a list of restaurants. On this page, a large search form, the results are updated Ajax.
I have a small personal “fad” about js: the site should be available without js. This gives some advantages when developing. In addition to the obvious ones (the error in js does not make the site inoperative) there are less obvious ones: for example, such a site is easier to test (you can test most of all without the help of selenium) + perhaps it is easier to write)
At the moment, we use the following approach to Ajax: Ajax and ordinary requests are processed by the same view, the only difference is in the template. Because Aayaksov template is part of a regular template, then we connect it in a regular template. Normal pattern:
{% extends 'base.html' %} <!-- : , --> ... <div class="b-search-results_result"> {% include 'restaurants/ajax/list.html' %} </div> ... <!-- : , --> Ajax template:
<ul class="b-search-results_result_list"> {% for restaurant in restaurants %} <li>...</li> {% endfor %} </ul> The template is selected based on request.is_ajax (); There is a declarative js on the client, which can turn any ordinary html form into an Ajax one:
def restaurant_list(request, city_slug): # ... response = TemplateResponse(request, 'restaurants/list.html', {...}) if request.is_ajax(): response.template_name = 'restaurants/ajax/list.html' return response In PanGurman decided to experiment a bit with this. At first it was the @ajax_template decorator, which added an AJAX view to the view:
@ajax_template('restaurants/ajax/list.html') def restaurant_list(request, city_slug): # ... return TemplateResponse(request, 'restaurants/list.html', {...}) Those. you wrap the usual view into the decorator, you prescribe the data attribute of the form and that's it: the form becomes an Ajax one.
But this option repeats
'restaurants/(ajax/)?list.html' (which is annoying and is a source of errors), + a view can have several return points and not everything needs to be done ayaksovymi (and this means that the option is worse than just checking without decorator).Therefore, from @ajax_template refused and while stopped on this option:
def restaurant_list(request, city_slug): # ... return show(request, 'restaurants/list.html', {...}) show is a successor from TemplateResponse, which for Ajax requests, first tries the template in the ajax subfolder. See https://gist.github.com/2416695
show has a couple more methods that I have always lacked - but, perhaps, wasn’t enough, because there are a couple of subtleties with it and not everything is fine (for example, .with_context will fall if the previous view returns the redirect), but it is quite possible to use it (and problems have little to do with AYAX).
In short, we make the site as if ajax does not exist, but add it later, where necessary, by placing the necessary part of the template in a subfolder.
Found something, poke in a restaurant
We fall on the restaurant page.
It also has an Ajax (the available time is checked in the order form via ajax requests), but it is implemented in another way :)
The calendar is built on the client (without js there will be regular input fields) and therefore it is easier to get json from the server, and not html. Using django-tastypie quickly built an API, and through it json is given. It is not much longer than writing a view that gives json, by hand, but the API still comes in handy + different buns like throttling and freehand formats are obtained.
In another project used for the API django-piston; subjectively - django-tastypie is nicer.
A very primitive (and not quite complete) API reference is automatically generated by the following view:
def _api_resources(api): resources = {} api_name = api.api_name for name in sorted(api._registry.keys()): resource = api._registry[name] resources[name] = { 'list_endpoint': api._build_reverse_url("api_dispatch_list", kwargs={ 'api_name': api_name, 'resource_name': name, }), 'schema': api._build_reverse_url("api_get_schema", kwargs={ 'api_name': api_name, 'resource_name': name, }), 'doc': resource.__doc__, 'resource': resource } return resources def browse(request, api): resources = _api_resources(api) return TemplateResponse(request, 'tasty_browser/index.html', { 'api': api, 'resources': resources }) Template:
<h1>API {{ api.api_name }}</h1> <table class='tastypie-api'> <tr><th>Resource</th><th>url</th><th>Structure</th><th>Description</th></tr> {% for name, resource in resources.items %} <tr> <td>{{ name }}</td> <td> <a href='{{ resource.list_endpoint }}?format=json'> {{ resource.list_endpoint }}</a> </td> <td> <a href='{{ resource.schema }}?format=json'> {{ resource.schema }}</a> </td> <td> {{ resource.doc }} </td> </tr> {% endfor %} </table> You can see here: http://pangurman.ru/api-docs/
Payment is bolted through django-robokassa (then more options will be added), SMS notification of reservation - via imobis + celery.
Finance
In Pangurman, the user has a personal account from which to pay for purchases. You can replenish it with the help of robokassa, inviting friends, entering promotional codes. Then more ways will be added anyway.
Having read Fowler, for several years I have been using the following approach to the implementation of a “personal account”: the amount is not stored anywhere on the account (however, it can be cached sometimes), we keep transactions on a personal account in the database. This allows you to naturally get a history of operations + saves from errors.
For example, if a person through the payment system pays an amount insufficient to pay for the purchase, the amount will first be credited to a personal account, then they will try to write off, the write-off will not succeed and the person, as needed, will remain without a purchase, but with the amount on the account.
A similar model wanders from project to project with minor changes:
class MoneyTransfer(models.Model): PURCHASE = 'purchase' ROBOKASSA = 'robokassa' INVITE_BONUS = 'invite-bonus' REFUND = 'refund' PROMOCODE = 'promocode' TRANSFER_TYPE_CHOICES = ( (u' ', ( (PURCHASE, u' '), )), (u' ', ( (ROBOKASSA, u' '), (INVITE_BONUS, u' '), (PROMOCODE, u' '), (REFUND, u''), )), ) user = models.ForeignKey(User, verbose_name=u'') amount = models.DecimalField(u'', max_digits=10, decimal_places=2) created_at = models.DateTimeField(u'/ ', auto_now_add=True) comment = models.CharField(u'', max_length=255, blank=True, null=True) transfer_type = models.CharField(u'', max_length=20, null=True, blank=True, choices=TRANSFER_TYPE_CHOICES) content_type = models.ForeignKey(ContentType, null=True, blank=True) object_id = models.PositiveIntegerField(null=True, blank=True) reason = GenericForeignKey('content_type', 'object_id') Here you can pay attention to the way choices. Write more, but there are no magic constants and you cannot mistakenly write to the base 'invite_bonus' instead of 'invite-bonus' - MoneyTransfer.INVITE_BONUS is always one and immediately falls with AttributeError, if you write it incorrectly (and the autocomplex appears).
The table was booked, but stop, and how does it work?
Go to the help page . On this page, part of the text, obviously, it would be good to allow editors to edit without the participation of the programmer.
In django, there are flatpages for this, but we usually use a different approach: we make all the pages with regular views with normal templates, store everything in VCS, and mark the django-flatblocks with pieces of text that need to be edited:
<div class="b-text"> {% flatblock "how-it-works" 600 %} </div> For admins when you hover the mouse over the text shows a link to edit
(this is done by setting your flatblocks / flatblock.html template):
{% load url from future %} {% load markup %} {% if user.is_staff %} <div class="b-flatblock-edit"> <a class="flatblock-edit-link markitup-flatblocks-edit-icon" title = " " href='{% url "edit_flatblock" flatblock.pk %}?next={{request.path}}'> </a> {{ flatblock.content|markdown:"video" }} </div> {% else %} {{ flatblock.content|markdown:"video" }} {% endif %} In blocks, markup is stored in markdown, on the edit page we show the widget from django-markitup (with preview):
All texts on the site, in which some markup is allowed, are edited using django-markitup (inspirational descriptions of restaurants with photos, for example). The markdown filter is configured in such a way that it does not touch the html-tags, so if necessary, any markup can be added to the text.
And, the extension is also screwed ( python-markdown-video ), which turns youtube or vimeo links into videos - in a good way it is done through oembed, you probably need it.
Dashboard for restaurants
For restaurants there is a special panel through which they can now control the opening hours (when tables are available when not). The panel is implemented through another separate instance of AdminSite. There can be users who are appointed by "restaurant managers". The subtlety is that these users should not have the is_staff flag, otherwise they will be able to enter the regular admin panel, which they would not want. And so - the usual customized admin panel.
[rant on]
Often they write that, they say, Djang's admin panel is inflexible and so on. - I did not understand what people specifically mean :) The admin panel provides CRUD, layout and out-of-box design, and if you know what can be customized and what is difficult, then there are no problems. As an Orthodox alternative, some kind of framework for writing dashboards is usually suggested - this is a CRUD perpendicular thing, and there are such frameworks for django admins ( django-admin-tools , nexus ).
If any interaction does not fit into the CRUD - no problems, nothing prevents you from writing your view and putting a link to it from somewhere in the admin area (readonly_fields and list_display are often convenient here; you can also override the template). To throw out or rewrite the admin panel of desire for several years never occurred, use it in all projects, saves a lot of time. Perhaps the specific nature of such projects, or I am really missing something obvious, I don’t know.
[rant off]
Instead of raw_id_fields, we actively use the application with the django-salmonella name in the admin panel , which “infects” widgets to select related records with useful behavior (for FK and M2M:
In addition to the id of the object, its name becomes visible (updated by the Ajax); by clicking on the name, you can immediately get to the page of the object change, which is also convenient
Tests
For the tests began to use factory-boy (fork), about which it was already on the habr. Since that recording, one super-useful thing has appeared in the factory-boy: RelatedFactory. It's like a SubFactory, just the opposite (for example, you can make a factory for the user, who will immediately create a profile for him). Documentation can be read here . Example:
class ProfileF(factory.DjangoModelFactory): FACTORY_FOR = Profile city = factory.SubFactory(CityF) balance = 0 @classmethod def _prepare(cls, create, **kwargs): # 'balance=555' balance = kwargs.pop('balance') profile = super(ProfileF, cls)._prepare(create, **kwargs) if balance > 0: MoneyTransferF(user=profile.user, amount=balance) return profile class UserF(factory.DjangoModelFactory): FACTORY_FOR = User username = factory.Sequence(lambda n: "username%s" % n) first_name = factory.Sequence(lambda n: u" %s" % n) last_name = factory.Sequence(lambda n: u" %s" % n) email = factory.Sequence(lambda n: "email%s@example.com" % n) is_staff = False is_active = False is_superuser = False password = '123' profile = factory.RelatedFactory(ProfileF, 'user') @classmethod def _prepare(cls, create, **kwargs): kwargs['password'] = make_password(kwargs['password']) return super(UserF, cls)._prepare(create, **kwargs) Now in the tests, you can write:
user = UserF(profile__balance=100) As a warm-up I did a factory-boy pull request with the addition of support for the 3rd python, but the 3rd python is a topic for a separate story)
There is one more trick in Django 1.4 with tests. By default, the PBKDF2 algorithm was used to hash passwords. In PBKDF2, one of the characteristics that provide security is low speed. But apart from being protected, a low speed of work still provides a low speed of work :) On the main site, this is not a problem at all (users don't log in so often and log in, and the delay is small), but in tests it turned out that this is a critical thing, especially if you use factory -boy and create many users by assigning them a password through a hash.
In the case of PanGurman, the installation of this line in test_settings.py accelerated the tests by 2 times:
PASSWORD_HASHERS = ['django.contrib.auth.hashers.MD5PasswordHasher'] Source: https://habr.com/ru/post/142703/
All Articles