Habra Illumination or Experiments in an Isolated Python Environment

Test the latest version of your favorite framework. Run an application with a specific set of libraries. Install the necessary libraries in the list of dependencies. How to solve all these tasks without affecting the system files? This article focuses on virutalenv and pip utilities.

The vitrualenv utility is designed to create isolated Python environments. An environment is the interpreter itself and a set of libraries for it. The pip utility organically complements easy_install, making installation of packages even easier. Detailed information about all the utilities that virutalenv and pip give a developer, you can find out by visiting the pages pypi.python.org/pypi/virtualenv and pypi.python.org/pypi/pip . I will show by example how all this can be used to experiment with projects in the Python language.

The essence of this project is very simple. This small console program named hl.py will color the source code of the scripts to be inserted into the habratopic. Through the command line, it takes the path to the file with the source code and the language, and in response prints the desired html code. For example, she paints herself like this:

 $ ./ht.py ./hl.py python Traceback (most recent call last): File "./hl.py", line 3, in <module> pkg_resources.require('lxml==2.2.2') File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 524, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: lxml==2.2.2 

Oops, does not work. The message indicates that the lxml distribution kit of version 2.2.2 is not installed. Let's see what is in Ubuntu:
$ apt-cache show python-lxml | grep -E ^Version
Version: 2.1.5-1ubuntu2

There really is something similar, but definitely not the version that this program requires. And how do you know what lxml is? We use the yolk utility (http://pypi.python.org/pypi/yolk).
$ yolk -f summary -M lxml
bash: yolk:
$ apt-cache search yolk
$
Why didn't the Ubuntu developers make a package for such a useful utility? ...

However, this is not a problem. We have the opportunity to make our own Python environment that does not depend on the quirks of the creators of the distribution of the operating system. Save the following script as vipsetup and make it executable:

 #!/bin/sh # Setup virtual environment for python and pip # Usage: # vipsetup [directory] ENV_DIR="$1" VIRTUALENV_BIN='virtualenv' VIRTUALENV_ARGS='--no-site-packages' if [ ! -f $ENV_DIR/bin/activate ]; then if ! type "$VIRTUALENV_BIN"; then if ! type "easy_install"; then echo "Error: easy_install executable not found." echo "To install easy_install type:" echo " sudo apt-get install python-setuptools" exit 1 fi TMPDIR=`mktemp -d -t ve.XXXXXXXXXX` || exit 1 PYTHONPATH=$TMPDIR easy_install --install-dir=$TMPDIR virtualenv || exit 1 PYTHONPATH=$TMPDIR $TMPDIR/virtualenv "$@" $VIRTUALENV_ARGS || exit 1 rm -rf $TMPDIR else $VIRTUALENV_BIN "$@" $VIRTUALENV_ARGS || exit 1 fi fi if [ ! -e $ENV_DIR/bin/pip ]; then $ENV_DIR/bin/easy_install pip || exit 1 fi echo "" echo "Installation complete" echo "* To activate virtual environment type:" echo " . $ENV_DIR/bin/activate" echo "* To install python package type:" echo " $ENV_DIR/bin/pip install <package>" if [ -z "$PIP_DOWNLOAD_CACHE" ]; then echo "* To enable cache for downloads" echo " set environment variable PIP_DOWNLOAD_CACHE with a path" fi 

Create a new isolated environment using the commands:
$ mkdir -p /tmp/testenv
$ cd /tmp/testenv
$ vipsetup .
$ . /bin/activate
(testenv) $

Now install yolk using the pip command:
(testenv) $ pip install yolk
...
(testenv) $ yolk -f summary -M lxml
summary: Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API.

Here is a description. Install the required lxml version:
(testenv) $ pip install lxml==2.2.2

Now, everything is ready to run our program:
(testenv) $ ./hl.py ./hl.py python
#!/usr/bin/env python <br> import pkg_resources<br>
pkg_resources. require ( 'lxml==2.2.2' ) <br>
<br> from lxml. html import parse, submit_form, tostring<br>
<br> def highlight ( code , language ) :<br>
SERVICE = 'http://highlight.hohli.com/' <br>
doc = parse ( SERVICE ) <br>
form = doc. getroot ( ) . forms [ 0 ] <br>
form. fields [ 'language' ] = language<br>
form. fields [ 'use_font' ] = 1 # for habrahabr <br>
form. fields [ 'code' ] = code <br>
form. action = form. action or form. base_url # hack <br>
ans = parse ( submit_form ( form ) ) <br>
highlited = ans. getroot ( ) . cssselect ( '.source .src' ) [ 0 ] <br>
return '' . join ( [ tostring ( x ) for x in highlited ] <br>
) . replace ( '0</font>' , '0 </font>' # fix habrahabr <br>
) . replace ( 'blockquote>' , 'code>' ) # fix format <br>
<br> if __name__ == "__main__" :<br>
import sys <br>
def main ( * args, ** kwargs ) :<br>
fh = sys . stdin if args [ 0 ] == '-' else file ( args [ 0 ] ) <br>
language = args [ 1 ] <br>
code = unicode ( fh. read ( ) ) <br>
print highlight ( code , language ) <br>
return 0 <br>
sys . exit ( main ( * sys . argv [ 1 : ] ) ) <br>

Here it is, all so colorful. Enjoy. :) And write down somewhere a list of dependencies, as a keepsake:
( testenv ) $ pip freeze | tee requirements.txt<br>
lxml==2.2.2
wsgiref==0.1.2
yolk==0.4.1

If necessary, quickly restore using the command:
(testenv) $ pip install -r requirements.txt

')
PS: if the practical utility of this experiment seemed doubtful to you, do:
(testenv) $ deactivate
$ rm -rf /tmp/testenv
and we will assume that there was nothing. ;)