Runscript - utility to run python scripts

I think many people know the following situation. There are various activities in your project that need to be done from time to time. For each action you create a separate python script. In order not to climb, put the script in the root of the project. After a while, the entire root directory of the project is littered with these scripts and you decide to put them in a separate directory. Now the problems begin. If you specify the path to the script to the python interpreter that includes this new directory, then within the script, the import of packages located in the root of the project will not work. project root will not be in sys.path. This problem can be solved in several ways. You can change the sys.path in each script by adding the project root there. You can write a utility to run your scripts, which will change the sys.path before running the script, or simply lie in the root of the project. You can even come up with something. I got tired of reinventing the wheel every time and I created a bicycle runscript on which I love to ride.

You can install the library using pip:

$ pip install runscript

After installing the runscript library, you get a new console command run on your system with which you can run scripts. By default, the run command searches for scripts in the sub-directory of the current directory's script.
')
Let's look at a simple example. Create a script directory. Create an empty script / __ init__.py file, turning this directory into a python package. Now create a script / preved.py file with the following contents:

def main(**kwargs): print('Preved, medved!') 

The script is ready. Now we can run it:

$ run preved
Preved, medved!

Hooray! The script works. That's all that the runscript library does. I'm serious :) The run command runs the main function from a file whose name you gave it on the command line. It turned out that even such a simple functional was very convenient. I was surprised to notice that I use the run utility in every project because Everywhere there are simple scripts that need to be run.

Over time, the run utility has acquired a number of useful utilities, which I will now discuss.

Getting parameters via the command line

To pass any parameters to your script via the command line, you need to describe these parameters in the setup_arg_parser function inside your script. This function takes as input the ArgumentParser object to which you can add the necessary options. Further, when the script is called, the command line parameters will be passed to the main functions. Sample script:

 def setup_arg_parser(parser): parser.add_argument('-w', '--who', default='medved') def main(who, **kwargs): print('Preved, {}'.format(who)) 

Run:

$ run preved
Preved medved
$ run preved -w anti-medved
Preved, anti-medved

Notice how the main function received command line parameters in the form of ordinary named parameters. You should always specify ** kwargs. in addition to the parameters you need, the values ​​of all global parameters for the run utility parameters are transferred (read about them below).

Django Activation

If you tried to use the Django framework in your console scripts, then you know that you need to do something, otherwise there will be nothing. Something is to create the environment variable DJANGO_SETTINGS_MODULE, containing the path to the module with the settings. Usually the following lines are added to the python script:

 import os os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' 

Starting with django 1.7, you must also run

 import django django.setup() 

In order to perform these actions automatically in scripts launched via run, you need to create in the project root a file called run.ini, containing the following settings:

 [global] django_settings_module = settings django_setup = yes 

Profiling

Adding the key --profile when calling the script, we get a file with the results of the profiling of our script, which can be viewed in kcachegrind. The result is saved in the var / <script_name> .prof.out directory, so be sure to create this directory. You also need to install the pyprof2calltree module, which is needed to save the profiling result in the kcachegrind format.

$ run preved --profile
Preved medved
$ ls var /
preved.prof.out

Setting up script search locations

By default, the run utility searches for a script in two packages: grab.script and script. The grab.script package has been added to this list, because in many site parsing projects I run the crawl command from the grab.script package. If you need to change places for searching scripts, create the following setting in the run.ini file:

 [global] search_path = package1.script,foo,bar 

Now, if we run the `run preved` command, the run utility will attempt to import the preved module in the following order:

• package1.script.preved
• foo.preved
• bar.preved

Using lock files

Sometimes it is necessary to prohibit the simultaneous operation of several instances of the script. For example, we call the script every minute using cron and want to prevent multiple copies of the script from working simultaneously, which can happen if the work of one of the copies takes longer than a minute. Using the --lock-key option, we can pass the name of the lock file to be created in the var / run directory. For example, --lock-key foo will create a var / run / foo.lock file.

Another way to set the name of the lock file is to create the get_lock_key function inside your script. The result of its work will be used by the run utility to form the name of the lock file. The function will be useful if you want to generate the name of the lock file, depending on the parameters passed to the script.

 import time def get_lock_key(who, **kwargs): return 'the-{}-lock'.format(who) def setup_arg_parser(parser): parser.add_argument('-w', '--who', default='medved') def main(who, **kwargs): print('Preved, {}'.format(who)) time.sleep(1) 

We start simultaneously two copies of the script and see:

$ run preved -w anti-medved & run preved -w anti-medved
[1] 25277
Trying to lock file: var / run / the-anti-medved-lock.lock
Preved, anti-medved
Trying to lock file: var / run / the-anti-medved-lock.lock
File var / run / the-anti-medved-lock.lock is already locked. Terminating
[1] + Done run preved -w anti-medved

I talked about the basic features of the library runscript . I hope it will be useful to you.

In case of questions about the work of the library, you can always look at the source code, which is currently rather small: github.com/lorien/runscript/blob/master/runscript/cli.py