Renaming processes in Python

When you write a multiprocess application on python, you want to have some kind of simplest way to monitor the work of individual processes. Now I’m not talking about detailed analytical reports on 10 pages, but about getting minimal information about health and the work of each process: what it is doing now, how much the system loads, how much memory it has consumed, etc. About how nginx or postgres report about themselves: a glance at top or ps is enough to understand what each process is doing and how.
It remains to figure out how to do this in Python on Linux (for example, Debian and FC).


In search of a solution:
Alas, it was not possible to solve the problem with “one swoop” - there was no ready-made solution. It was strange for me: such a developed language, a huge community, the first reports about the successful launch into space of the process renaming were dated almost 2003. And now, after five years, there is still no good ready-made solution. There were references to dead projects, recipes, various pieces of code. In addition, it turned out that top and ps, which have always seemed to me twin brothers, have different views on where to get the name of the process for the show. So, in a good way, it is necessary for each of them to carry out a separate renaming.

Analysis of the source showed that in order to successfully rename a process, you need to do two actions - replace its name in argv [0], and also make a system call with a new name. To do this on C is like sending two bytes, but on python it becomes a puzzle with an asterisk. Fortunately, python integrates well with C and we need to make a library call. Alas, it didn’t work out with standard tools like dl - it turned out that this library is standard only for 32-bit debian, and in 64-bit it isn’t even in advanced packages. I found such a package for dusky (suse), but sorry, I don’t feel like boiling terrible porridge from heterogeneous packages on my system. Alternative lib ctypes (whose very name speaks for itself), alas, did not help either. From exotics, I can also mention the solution on PyInline - compilation of a piece of C-shn code in a Python module as the program progresses with the loading of this module. But from such an "interpreted C" hair stand on end! And, as it turned out, not in vain - the program fell ill. In about half of the cases, the launch ended up falling out not even in the traceback, but immediately into the cortex with the throwing out of the stack guts on 3 screens. Maybe it was not in the inline itself - it is just that there is always a cocktail of modules in the volume program with which it could conflict, but in life there are more interesting and important tasks than raking these conflicts.
')
Decision:
In the end, I was tired of these flirting with the S-code code. If it is already clear that it is necessary to write the main working part on Syah, then these are crutches and gaskets! It is better to immediately make an honest C-shny code and give it interfaces to python - that is, to make such a module.

At that moment I could not even imagine how it all was done. Fortunately, the excellent architecture of the language and the detailed documentation made this task quite simple.

Thank you, Guido! Where can I enroll in the army of your fans? :)

In the end, everything turned out simple and beautiful: the source code is exactly 1 page + a short Makefile to build. After compilation, a so-file of about 7K is obtained.

Using
Easy peasy:

>>> import procname
>>> procname.getprocname () # get the current process name
'python'
>>> procname.setprocname ('Bla-bla test') # give a new process name - check in top and ps!
# you can see the process name again
>>> procname.getprocname ()
'Bla-bla test'

Where to get and how to collect?
I posted the project on google code: code.google.com/p/procname
Download the source in a temporary folder, and then as usual:

$ tar -zxf procname-XX.tar.gz
$ cd procname-XX
$ make

After that, you should have procname.so file in the current folder - copy it to the folder of your program, or if you want it to be used in any program, then to the /usr/lib/python2.5/site-packages folder
If instead of a file any errors are issued, check if you have everything you need. The build will require the python-devel, make package (the one that is gnu) and, of course, the gcc compiler :)
Most likely the problem will be in the wrong indication of the location of the Python.h and sys / prctl.h header files. Find them by searching and specify the folder in the Makefile. If it does not help - write here we will understand together.

Denial of responsibility:
As usual in such cases, this library is provided “as is”. I do not guarantee anything: incl. that it will work for you, will work correctly, or at least will gather. And also the fact that your server does not turn into the Large Andron Collider and does not absorb everything within a radius of 100 miles.
I'd add that lib tested on Debian (32- and 64-bit), FedoraCore (all with kernel 2.6). I have been using Liebu for almost a month on a “combat” server.

How about others * nix?
In other unix-ah organized a little differently. In particular, FreeBSD has a special system call setproctitle , which allows you to set the name of the process to display in ps. In the same place, by the way, its non-standard is indicated. However, there is an analogue in AltLinux . In Debian, the situation is incomprehensible , although it seems like Dmitry from Alts ported his lib, but I could not find it in etch and lenny.

How do I use it in practice?
In my program, each process at one time handles one task. When launched, it is first renamed, putting something like a task number and status in its name, something like this: ' ICd: 7562 - starting '. Then, for each important event (eg, change of status, new task, receiving a signal from the OS, etc.), it is renamed again. As a result, I see the progress of processing each task, I can detect anomalies in the resource consumption, slow down processing, repetitions, etc. and time to respond to them.

Conclusion
Monitoring processes has become a pleasant experience - you start the top, recline in the chair and watch how your “pitochki” happily bite into the data :)

In general, in my opinion, it would be right to make such a universal library of Python, so that on any Linux and frishka it would be possible to work freely with the names of the processes, without being tied to how it is implemented. And maybe even on Windows!

I would welcome any comments, suggestions, ideas.

PS Many thanks to all who helped in the preparation of this article for publication on Habré. Without you she would be impossible!