Hoster, put me a module, eh?

Quite regularly on the hosting is not pre-installed library for a particular program. And not always the hoster satisfies the request to install the library of the required version. It happens that the library is, but outdated. Or the server broke down at the host, he threw the site onto another, and there is a newer version of the library that is not compatible with the one on which the site software is running (Django users will understand me perfectly). What to do? Looking for a new hosting again? Always be ready to rewrite your program for another version of the library? Our way is to put it yourself. At the very least, you just have to reinstall it. In addition to PHP binary modules, almost all libraries for popular web technologies can be placed in your home directory without touching the hoster or depending on it. By the way, this can be done not only on shared hosting, but also on virtual servers - projects become weakly dependent on who and how set up the virtual server. Complicated?

What is needed for a simple and successful installation of the library in your home directory on an average hosting UNIX server? SSH access is desirable. You can do without it, but then you have to look for some web shell. It is desirable that the ssh-console does NOT work in the chroot environment, because in most cases it has a stripped-down view, or that the ssh chroot environment coincides with that of apache. Does your host have no ssh? This is an occasion to think. Also needed skills to work with the console UNIX. This is no more difficult than sending sets of numbers to short numbers in mobile phones. Do not be afraid of this, if you already undertook the development of sites.

In the course of the narration, I will often mention the $ PREFIX macro. One of the most common user errors is that usually the prefix is ​​understood as the directory where the library files will be installed. The fact is that distributions of libraries and programs have manual files, documentation, support files, possibly binary executable files, and the library files themselves. All installation systems want to create a certain conditional directory hierarchy for the specified files, starting just from $ PREFIX. For example, Python modules will lay out the binary files in $ PREFIX / bin, the guide files in $ PREFIX / shared / man, and the library files themselves in $ PREFIX / lib / pythonX.X / site-packages (XX is the Python interpreter version that, By the way, allows you to put different libraries for different versions). I recommend not opposing the default directory hierarchy, and also assume that in the general case $ PREFIX (a macro I defined does not exist in the system) and $ HOME (the user's home directory, a predefined shell variable) are identical, although no one obliges to follow these conventions .

It is not enough just to put the library, you need to tell the site software where to find it. For this, almost all the technologies I describe are environment variables. For each technology, I will describe them in more detail. But the very use of them all the same.
')
In order to set the environment variable for the ssh command line (in fact, for any console during a login), you should set it in the .profile file for the sh and bash shells (on most hosting sites, they are the main shells for users) located in the root of the user's home directory, the following lines:
VARIABLE=VALUE
export VARIABLE
For example, for Python 2.5 modules, it will look like this:
PYTHONPATH=$HOME/lib/python2.5/site-packages
export PYTHONPATH
I draw attention to the fact that in this case, the predefined variable $ HOME was used, which is automatically expanded by the shell. For non-sh and bash shells, the strings will be different, refer the reader to the documentation for these shells. Environment variables for a shell are best to install before starting the installation of libraries. Do not forget to “re-login” after the addition, so that the settings are applied.

In order for the environment variable to be seen by cron jobs, you should set it at the beginning of the crontab file:
VARIABLE=VALUE
For example, for Python 2.5 modules installed in the home directory of the user vasya, it would look like this:
PYTHONPATH=/home/vasya/lib/python2.5/site-packages
The path to the home directory on the servers of different hosters may differ from the above. Particular attention is paid to the fact that cron does not know how to "disclose" variables and the path must be prescribed completely as it is.

With a web server a bit more complicated. You should read the documentation applicable to a particular case. For example, in order for Perl5 CGI scripts running apache to see the libraries installed in the home directory of the user vasya, you should write the following line to the .htaccess file in the site root or in the directory where the CGI scripts are located:
SetEnv PERL5LIB /home/vasya/lib/perl5
From this description it is clear how environment variables are set in the apache web server.
But for Python 2.5 applications running on a WSGI server (any), it’s better to put the following lines in the WSGI initialization file:
import sys
sys.path.insert(0,"/home/vasya/lib/python2.5/site-packages")
From this example it is also clear how to proceed with other technologies in such cases.

For experienced UNIX users, I’ll note that for convenience, you can extend the PATH variable by specifying the $ HOME / bin directory so that the utilities that come with the libraries work without entering the full path from the console, and set the MANPATH variable, usually $ HOME / share / man , in order to be able to use the pages of the manual of the installed modules as well as the usual ones - the man command.

Installing CPAN Libraries (Modules) to the Home Directory (Perl)

Let me remind you that CPAN (Comprehensive Perl Archive Network - Perl archive) is the central repository of everything related to Perl. One way to view CPAN modules is to visit search.cpan.org .

In order for scripts to work with installed modules, you must set the environment variable PERL5LIB. You can specify several paths to search for modules by listing them separated by a colon, for example:
PERL5LIB=/home/vasya/lib/perl5:/home/vasya/lib1/perl5:/home/vasya/lib2/perl5

CPAN modules can be downloaded and compiled manually. As a rule, the sequence of compilation commands and the required modules are listed in the README file, usually included in the module distribution kit. All CPAN modules can be installed in one of the following ways: ExtUtils :: MakeMaker and / or Module :: Build (for more detailed information just read the documentation for these modules). ExtUtils :: MakeMaker uses the Makefile.PL file. To install into the home directory, you must specify the INSTALL_BASE environment variable:
perl Makefile.PL INSTALL_BASE=~
Module :: Build uses the Build.PL file. To install to the home directory, you must specify the key --install_base:
perl Build.PL --install_base ~
A tilde is usually “revealed” by the shell to the value of the user's home directory — this is our $ PREFIX, which I mentioned at the beginning of the article. Instead of a tilde, you can specify any other full path by making appropriate corrections to environment variables. The remaining keys and the preferred installation method are usually described in the README and INSTALL files inside the distribution package of the module.

Also included with the Perl distribution is a module called CPAN. It allows you to automate the installation of the necessary modules, including the installation of dependencies. The module can work in manual and batch mode. Consider for simplicity, the manual mode. To work with the module interactively, type the following command:
perl -MCPAN -e shell
When you first start the program will try to create a configuration file and will ask questions. The problem is that with any responses, the resulting configuration file, usually created in the .cpan / CPAN / MyConfig.pm directory inside the user's home directory, is not ready to be used to install modules locally. In it, we are interested in the following lines, which should be filled in the same way:
'build_dir' => qq[$ENV{'HOME'}/.cpan/build],
'cpan_home' => qq[$ENV{'HOME'}/.cpan],
'histfile' => qq[$ENV{'HOME'}/.cpan/histfile],
'keep_source_where' => qq[$ENV{'HOME'}/.cpan/sources],
'makepl_arg' => qq[INSTALL_BASE=$ENV{'HOME'}],
'mbuildpl_arg' => qq[--install_base $ENV{'HOME'}],
'prefs_dir' => qq[$ENV{'HOME'}/.cpan/prefs],
Instead of $ ENV {'HOME'} there may be a full path to the user's home directory, or any other path, for example: / home / vasya

Making sure the configuration is correct, you can install modules. Team
perl -MCPAN -e shell
launches a special interactive shell. Installing modules in it is done for example with the following command:
force install _
The force keyword allows you to avoid rejecting the installation in case of negative test results, which for various reasons may actually be negative.

The CPAN module also provides the opportunity for batch (software) management of the CPAN modules. Worth for more advanced settings of the module or for programming batch work, familiarize yourself with the documentation on CPAN.pm.

Installing PEAR libraries (modules) in your home directory (PHP)

Let me remind you, PEAR (PHP Extension and Application Repository) is a framework and component distribution system in PHP. Detailed information can be found at pear.php.net

Pear package manager itself is not part of the PHP distribution, so it is necessary that it be pre-installed on the hoster.

The main problem is that the php script “sees” where to get this or that component from (PHP as always goes “in its own way”). To do this, the include_path setting must be defined, for example, for the user vasya, like this:
include_path=".:/home/vasya/pear"
The easiest way is to tell the php interpreter this variable:
php -d include_path=".:/home/vasya/pear"
Or, in the script code, insert the construct:
ini_set("include_path",".:/home/vasya/pear");
This also applies to running scripts via cron.
In order for scripts run from under the apache web server to include PEAR components, it is required or to register in the script:
ini_set("include_path",".:/home/vasya/pear");
or add a line to the .htaccess file:
php_value include_path ".:/home/vasya/pear"

Before installing PEAR modules, you should tell pear that we want to put the components in your home directory with the command:
pear create-config $HOME .pearrc
This will create a configuration file used by pear in the future. Of course, instead of $ HOME, you can choose any other place, not forgetting to reflect this in the value of include_path.

Now you can simply install the required packages. For example:
pear install -o PEAR
will install the base component of PEAR with dependencies.

Installing Python libraries (modules) to the home directory (Python)

In order for scripts to work with installed modules, you should set the environment variable PYTHONPATH. You can specify several paths to search for modules by listing them separated by a colon, for example:
PYTHONPATH=/home/vasya/lib/python2.5/site-packages:/home/vasya/lib2/python2.5/site-packages

Manual installation from the archive

Usually the keys and the preferred installation method are described in the README and INSTALL files inside the module distribution kit. Usually, this is the command:
python setup.py install --prefix=$HOME
Problems can arise if the developers of the module ignored the --prefix key.

Installing with easy_install

The easy_install utility itself is able to search for packages in the pypi.python.org/pypi repository, download and install them. Or install already downloaded. Does not exist for Python versions 3.x. The utility is not included in the Python distribution kit; pre-installation of the setuptools module pypi.python.org/pypi/setuptools is required . To install modules using easy_install, the following directory must exist:
$PREFIX/lib/pythonX.Y/site-packages
where XY is the Python version. For example, to install modules for Python 2.5 using easy_install, you need to create the appropriate directory hierarchy with the following command:
mkdir $HOME/lib $HOME/lib/python2.5 $HOME/lib/python2.5/site-packages
If there is no such directory, easy_install will generate an error.

After that you can install the package:
easy_install --prefix $HOME _

As in all other cases, you can use other prefixes in place of the $ HOME examples given in the examples by making appropriate corrections to the environment variables.

Installing RubyGems in your home directory (Ruby)

Let me remind you, RubyGems is a package manager for the Ruby programming language that provides a standard format for Ruby programs and libraries, tools designed for simple installation management, and a server for distributing them. Packages can be installed from any gems server. By default, gems searches for packages on the rubygems.org server. RubyGems are not included in the Ruby distribution and must be pre-installed by the hoster.

RubyGems is the smartest installation system described. Run the command:
gem environment
and see what paths it indicates after the keyword GEM PATHS. We are interested in the variables GEM_HOME and GEM_PATH. Although gems itself correctly defines all variables, just in case write them down explicitly. This also applies to the case when you want to install packages in any non-standard place.

Installing with a gem is very simple. It is enough to type the command:
gem install _

Installing non-gems Ruby Libraries

You may need to use Ruby scripts as a library, independent of your application scripts. The Ruby interpreter must "see" them. To do this, set the environment variable RUBYLIB, for example:
RUBYLIB=/home/vasya/mylib/render

Installing binary libraries

Installation of binary libraries is difficult to formalize. But most likely you will never have to face their installation.

For the vast majority of distributions of these libraries, the rules below are true.

The distribution includes a configure file that accepts the --prefix key, for example, the command
./configure --prefix=$HOME
inside the unpacked distribution is likely to be sufficient for the correct configuration of the installation process.

If the library is written with regard to --prefix and everything is done correctly, then the sequence of commands after configuration:
make
make install
inside the unpacked distribution should lead to the installation of the library in the home directory.

In order for other libraries to see the installed library, you will most likely need to register the path to it in the LIB environment variable, but this depends on many factors.

On the way to the stars

To raise a positive attitude, I suggest a link to a working installation instruction on the shared hosting of the xapian search engine: wiki.diphost.ru/XapianInstall
It has everything - installing a binary component, and installing modules for Perl, Python and Ruby. I hope my master class, assembled bit by bit and debugged, will help readers to simplify the scheme of interaction with hosters and feel more free to choose technologies.