Python packages and their use

Good day everyone!

On Habré, the topic of organizing work with external packages using submodules or trees in Git has already been raised. It seemed like a good decision, but in fact it turned into inconvenience and confusion. Then I decided to transfer everything to python packages. What I will share now on the example of the Django application.

Instead of the preface

When you do a lot of different projects, your modules, extensions, utilities accumulate. Dependencies are established between them. And the question arises in the convenient organization of infrastructure development and deployment of projects. To make it clearer, I’ll tell you about Django projects.

Create a Django application and select it into a python package. The application will display news. In the administrative part of the site, we will need our own usability skills. For this we will specify in the dependencies the app-admintools package .

Training

Create an app-news repository in Gitosis:

gitosis-admin / gitosis.conf

[group apps] writable = app-news members = @developers @hosts 

send changes:

 git pull 

we initiate:

 git clone git@git.home.com:app-news 

The structure of our package

 +-app-news/ +-home/ +-news/ +-locale/ +-static/ +-templates/ +-__init__.py +-models.py +-urls.py +-views.py +-__init__.py +-MANIFEST.in +-setup.py 

setup.py

 from setuptools import setup, find_packages setup( name='home.news', version='0.1', description='News for Django', author='Elias', namespace_packages=['home'], # line 8 packages=find_packages(), platforms='any', zip_safe=False, include_package_data=True, dependency_links=['git+ssh://git@git.home.com/app-admintools@v0.1#egg=admintools-0.1'], # line 13 install_requires=['admintools==0.1'], # line 14 classifiers=[ 'Development Status :: 5 - Production/Stable', 'Environment :: Web Environment', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Topic :: Internet :: WWW/HTTP :: Dynamic Content', 'Framework :: Django', ], ) 

Here we declared the use of the home namespace (p. 8) and specified the dependencies for the home.news package in our repository (p. 13,14).

MANIFEST.in

 recursive-include home/*/locale * recursive-include home/*/static * recursive-include home/*/templates * 

The manifest defines the additional data that the package needs.

home / init .py

 import pkg_resources pkg_resources.declare_namespace(__name__) 

Register the namespace.

Create a remote v0.1 branch from master:

 git push origin master:v0.1 

Everything! Package is ready to install:

 pip install git+ssh://git@git.home.com/app-news@v0.1 

or in edit mode:

 pip install -e ./app-news 

or through setuptools:

 python setup.py develop 

but in this case, it is impossible to download dependencies as links to repositories, since setuptools does not support this format

Package connection

One method I described above, in the form of setup.py dependencies.

It is also convenient to store dependencies in a separate file requirements.txt (for example, for a site):

 git+ssh://git@git.home.com/app-news@v0.1 

and install via pip:

 pip install -r requirements.txt 

Conclusion

The idea of ​​shifting dependency management to Git was inconvenient (I used submodules). First of all, this manifested itself when it was necessary to write code and debug it in two projects at once, with common components that needed to be synchronized. Usually through a commit. Another problem occurred when two developers simultaneously updated the package and a link to it. As a result, non-trivial migration. And so on ... But now you can put the package in edit mode and edit the code without synchronization. At the same time, all the delights of access to the packages via ssh-keys remained.
I hope the article will help you simplify development and become a little happier :)

Thank you for your attention, good luck!