Monitoring the execution of management teams in Django

Often, and in many projects, background tasks are used. But overwhelmingly, for some reason, no monitoring is used. We are talking about such services as Sentry, NewRelic or native ErrorReporting. There are no reports on how long the command runs, with what error, and how the process was completed. As a result, no one knows how well the project is working and they are perplexed when there is not enough data in the reports or in crm, or the statistics are incomplete / incorrect. This is not always noticeable and not immediately. That is, it is discovered much later, after the delivery of the project.

Juniors are not always aware of the services and their existence. But despite this, the need for a tool for logging such things does not disappear. As well as the correct setup crontab. You can monitor errors with the help of cron-a, but these are just alerts, but what is happening does not always clarify. It also does not indicate the moment when there was an error for the first time (if the notification was set up later by someone else). In addition, it is important to know the progress of how the script worked at certain intervals. For example, you work with statistics, carry out a calculation and daily inserts into the database. Every day there is more data. Immediately start the brakes due to the expansion of the table. It is clear that you need to refactor the code, it is possible to create partitions, indexes, to carry out some optimization. And again, monitor the execution and behavior of scripts for some time. In past posts I mentioned PinbaEngine. But not every project has the ability to set up such a bundle. Even having put, it is necessary to modify the code and run over the interface for this creation. That does not always want. In this post I want to talk about a small battery called django-mmc and how to properly configure this good.

Creating a project

$ pip install virtualenvwrapper $ mkvirtualenv habr $ pip install Django $ django-admin.py startproject habr 

I also noticed that many, even experienced ones do not use such a tasty thing as virtualenvwrapper. A useful thing. You can read about it here .

Battery installation

 $ cd habr/ $ pip install django-mmc 

')
Add a mmc battery to INSTALLED_APPS and synchronize the database:

 $ ./manage.py syncdb --noinput 

Now we initialize the necessary components of the library somewhere in the init file, until the app is initialized.
Add import and call to the habr / __ init__.py file:

 from mmc.mixins import inject_management inject_management() 

This completes the configuration process. As you can see there is nothing difficult and you can easily connect the battery to any existing project.

Check

To check, you need to run several commands and run web north:

 $ ./manage.py createsuperuser $ ./manage.py syncdb $ ./manage.py runserver 

Now it only remains to check the result in the administrative part of the project.
You can do this by going to http://127.0.0.1:8000/admin/mmc/mmclog/

Result

And finally, the result we went to:

Cron setup

If you want to receive notifications from cron, add the following lines at the beginning of your crontab:

 SHELL=/bin/bash PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games MAILTO=habr@local.host 

The first two lines indicate walking and the path where to look for executable files. Mailto specifies the address where the output of background tasks should go.

The latest version adds the ability to ignore hosts or specific commands. The logs display the peak memory that the script has devoured.
Possibility of notifications to mail addresses + Sentry specified in the admin panel.

This battery helped me to identify problem areas in the project in a timely manner, find out which teams work out incorrectly or fall with some time period, and eliminate this problem. I hope it will serve both beginners and experienced users.