Installing Perl Modules in Gentoo

The biggest drawback of the Perl language ecosystem is the management of modules (another candidate for this role is the long-term Perl6 , but not about it). What is curious, the biggest advantage of this same ecosystem is the presence of a single archive of CPAN modules. Amazingly, we were able to assemble and organize the modules, but we didn’t implement a convenient installation / update / removal.

A wealth of choice ... or another TIMTOWTDI

There are many alternative approaches to this problem (and their number also indirectly indicates that none of them solves the problem well enough): cpan , cpanplus , cpanminus , pip , cpansite , minicpan / mcpani , perlbrew , cpan-outdated , cpan -listchanges , local :: lib , ...

So we can have:

Several versions of perl itself (of course, each with its own global modules), including installed in the user's home directory (see perlbrew).
Global (available at start of perl) and local (connected from any directory / directories, usually located inside a separate project or in the user's home directory) modules.
Global modules come in three types: core (that comes with perl), site (manually set by the admin) and vendor (installed by your OS package manager).
All global modules are in the subdirectories "number.version.perl /", and no one automatically cleans these directories. And when you install the new version of perl, new similar directories are created. And perl loads modules from directories of all available previous versions. So multiply core + site + vendor by the number of perl updates — that’s the number of directories / options that your global modules are in.
Sources of modules are also different: CPAN, local mirrors-overlays of CPAN with private modules, just their own or downloaded from the Internet modules missing in the CPAN-compatible system.

And all this joy must be administered: install, update, ... In Gentoo, to simplify the administration of global Perl modules, there is a g-cpan utility , so I want to tell you a little about it.

Workspace

To begin with, we specify a little what conditions we usually have to work in:

We are not talking about installing our modules on hosting, we work with a whole server / servers.
Several of our Perl projects are running on this server.
All who actively write on Perl write their modules. And not all of its modules can and should be laid out on the CPAN. All those who work on several projects use the same modules in different projects. Therefore, this situation sooner or later leads to the need to raise the local CPAN mirror / overlay for its modules in order to be able to install and update them with “regular” means - this does not solve the problem as a whole, but life becomes easier. So support for a local CPAN mirror / overlay is needed .
The question always arises: to put the necessary modules for the project (and its own and CPAN) globally (remember, we have our own server), or locally. There can be no definite answer here, there are arguments both for and against both approaches. I decided for myself this way: since it is necessary to administer (install, update, track outdated / full of holes versions, delete) all installed modules, and since it is impossible to assemble all the modules in one place (it’s impossible to get rid of global modules, because they are used by other system-wide applications, and completely without local modules, can’t be managed in most projects either), then you need to at least minimize the number of such places and the total number of installed modules . What leads us to the fact that most modules should be installed globally , and locally you need to install modules only if the versions of global modules are not compatible with a specific project. Yes, the main disadvantage of this approach is that updating global modules can break some projects - but firstly, in my experience, this happens quite rarely, and secondly, use a test server and check all updates on it before installing them on working servers - this should be done anyway, right? :) As a last resort, the workstation of the main developer will be suitable for the role of the test server, but minimal testing of updates is necessary.

We have already made our life a bit easier by limiting the installation of modules from a CPAN-compatible local mirror / overlay (see cpansite) and installing most of the modules in a single copy globally. The next problem: there are three global installation sites for the modules - core, site and vendor . And because of this, conflicts often arise. Previously, Gentoo’s priority was vendor-site-core. Because of this, manually installed via cpan / cpanplus / etc. In the site modules were often not available, because vendor was an outdated version. And utilities like cpan could infinitely update modules, not realizing that this does not work. This was solved by removing all the modules from vendor, and manually typing them into /etc/portage/profile/package.provided so that the portage did not install them again (after which it was necessary to follow closely during system updates, so that the portage did not install the packages from dev- perl / * or perl-core / *, and, if necessary, install these packages via cpan and write them in package.provided). Now site-vendor-core priority, but this also doesn’t completely solve the problem: some (and maybe I didn’t check) core modules, when updated with a utility like cpan, are not installed in the site, but in the core, on top of those versions which came together with perl itself. This creates two unexpected effects: first, if the vendor has a different version of these core modules (from perl-core / * packages), then, despite the “update” of the module, the cpan utility will still use the older version from vendor; and secondly, reinstalling the dev-lang / perl package (for example, caused by revdep-rebuild) rolls back all updates of core modules installed manually via cpan.
')

g-cpan

Given the above problems, as well as the inconvenience of "regular" tools like cpan / cpanplus, we smoothly approach that it would be great for all global modules to install, update and delete portazh means, like all other packages in the system. If we completely abandon the manual installation of modules in the site, then all the problems of priorities / module overlapping / perl reinstallation will also be resolved. Unfortunately, portage has packages for only a small part of CPAN modules, and of course there are no packages for our own private modules. To solve this problem, the g-cpan utility was developed - it can generate packages for any CPAN modules (including private modules from a CPAN-compatible mirror / overlay) and either save them in a local overlay or simply install it on the fly. When I tried to use it a few years ago, it didn’t work very well and was practically useless. But now it is already fully functional - at least I managed to install ~ 300 modules, and only for one of them I had to manually edit the ebuild.

As a side effect of using g-cpan, we are able to build standard Gentoo-shny binary packages for Perl modules on one server, and install them on other servers very quickly (the usual installation of modules, especially with testing, takes hours, and some modules also require interaction during installation).

Installation and Setup

We put:
```
emerge g-cpan 
```

You need to set an overlay where g-cpan will create ebuilds:

 echo "GCPAN_OVERLAY=/usr/local/portage" >> /etc/make.conf

We set our own list of CPAN mirrors (which includes our local mirror / overlay as the first item). The easiest way to copy an already configured list is from cpan:
```
 cpanmirrors=$(cpan -J | perl -0777 -ne 'eval;print"@{$CPAN::Config->{urllist}}"') echo "cpan $cpanmirrors" >> /etc/portage/mirrors 
```

Whether it is necessary to test the modules during installation - everyone decides for himself, but I prefer to run the tests, especially for my own modules. To force testing of perl-modules during installation, you need to add the ability to set your own hooks to entire package categories. One possible approach is to create the /etc/portage/bashrc.d/ directory and the following file / etc / portage / bashrc:

 # Useful variables and functions: # # EBUILD_PHASE= # clean # setup # unpack # compile # test # install # preinst # prerm # postrm # cleanrm # postinst # clean # # CATEGORY = app-admin # PF = eselect-opengl-1.0.6-r1 # P = eselect-opengl-1.0.6 # PN = eselect-opengl # PVR = 1.0.6-r1 # PV = 1.0.6 # PR = r1 (will be "r0" for packages without -r) # # einfo () # elog () # ewarn () # eerror () if [ "x${EBUILD_PHASE}" != "x" ]; then if [ -x "/etc/portage/bashrc.d/${CATEGORY}.${EBUILD_PHASE}" ]; then source "/etc/portage/bashrc.d/${CATEGORY}.${EBUILD_PHASE}" fi if [ -x "/etc/portage/bashrc.d/${CATEGORY}/${PF}.${EBUILD_PHASE}" ]; then source "/etc/portage/bashrc.d/${CATEGORY}/${PF}.${EBUILD_PHASE}" elif [ -x "/etc/portage/bashrc.d/${CATEGORY}/${P}.${EBUILD_PHASE}" ]; then source "/etc/portage/bashrc.d/${CATEGORY}/${P}.${EBUILD_PHASE}" elif [ -x "/etc/portage/bashrc.d/${CATEGORY}/${PN}.${EBUILD_PHASE}" ]; then source "/etc/portage/bashrc.d/${CATEGORY}/${PN}.${EBUILD_PHASE}" fi fi

after which you can enable testing of all packages in the perl-core / *, dev-perl / * and perl-gcpan / * categories:

 cat <<'EOF' >/etc/portage/bashrc.d/perl_module_test FEATURES="$FEATURES test" SRC_TEST="do" EOF cat <<'EOF' >/etc/portage/bashrc.d/perl_module_notest SRC_TEST= EOF chmod +x /etc/portage/bashrc.d/perl_module_* ln -s perl_module_test /etc/portage/bashrc.d/perl-core.test ln -s perl_module_test /etc/portage/bashrc.d/dev-perl.test ln -s perl_module_test /etc/portage/bashrc.d/perl-gcpan.test

and, if necessary, disable testing of some modules or somehow influence it:

 mkdir /etc/portage/bashrc.d/dev-perl ln -s ../perl_module_notest /etc/portage/bashrc.d/dev-perl/Crypt-SSLeay.test ln -s ../perl_module_notest /etc/portage/bashrc.d/dev-perl/Shell-EnvImporter.test ln -s ../perl_module_notest /etc/portage/bashrc.d/dev-perl/Term-ReadLine-Perl.test cat <<'EOF' >/etc/portage/bashrc.d/dev-perl/Inline.test MAKEOPTS=-j1 EOF chmod +x /etc/portage/bashrc.d/dev-perl/Inline.test

Further, usually on the CPAN the most recent versions of the modules are the most stable. This is all the more true for our private modules. So it makes sense to force the installation of the latest versions:

 cat <<'EOF' >>/etc/portage/package.keywords perl-core/* dev-perl/* virtual/perl-* # avoid depending on ~x86 sys-libs/readline dev-perl/Term-ReadLine-Gnu -~x86 # avoid depending on ~x86 sci-mathematics/pari dev-perl/math-pari -~x86 EOF

Now we need to update the CPAN index for g-cpan. Theoretically, g-cpan has an option --cpan_reload for this, but in the current version (0.16.2) it does not work. And since g-cpan uses the cpan utility index, you can work around this problem by running cpan and executing “cpan> reload index”.
Prepare a list of modules used by all Perl scripts on the server. I have managed high-rise sh-pipelines, but probably you should write a Perl script for this, and post it in the comments to this article. :)
We generate ebuilds for them (this will automatically create ebuilds and for dependency modules):
```
 g-cpan -g Module::Name1 Module::Name2 … 
```
There is a small chance that for some modules you will have to correct with handles /usr/local/portage/perl-gcpan/Module-Name/Module-Name-version.ebuild. (I have so far only got one: Crypt :: MatrixSSL.)
Remove the directory / usr / lib / perl5 / site_perl / *. CAUTION , at this moment, most Perl scripts on this server will break. Unfortunately, if site modules are not removed before installing vendor modules (which have lower priority), then during installation and testing of vendor modules, problems may arise due to the fact that site versions of modules already installed in vendor will be used. However, problems are usually solved, so if you can’t afford to “break the system” for a while, then install vendor modules first, but don’t forget to delete the catalog with site modules after that.
Remove all perl modules from /etc/portage/profile/package.provided.
We first install the modules required portage (if package.provided was used and they are not installed now):
```
 emerge -uDNa @world 
```
and then you need (it makes sense to put the Test :: * modules first, then Math :: * - this will speed up the installation of the rest). We repeat installation attempt several times, since not all modules correctly spelled dependencies:
```
 until emerge -uDN --keep-going y \ Test-Deep Test-Differences Test-Distribution … \ Math-BigInt-GMP math-pari … \ … do read -p "Press <Enter> to try again" done 
```

Update

The g-cpan utility does not provide a complete equivalent of "cpan> upgrade"

 g-cpan -u

will update only modules for which packages created g-cpan (i.e. from the perl-gcpan / * category, and packages from perl-core / * and dev-perl / * will not be updated). But this may not be so bad - in the end, to update these modules, it is usually enough to copy and rename the ebuild from perl-core / * or dev-perl / * to / usr / local / portage. But there is a possibility of more subtle control over the installed versions of the modules and a convenient rollback to previous versions.

Summary

In my opinion, g-cpan really simplifies the management of globally installed Perl modules. And in spite of the huge “Installation” section of 11 points, in fact there is nothing really complicated there (well, apart from determining which modules you need to install :)).

Successes in establishing order on servers!

Source: https://habr.com/ru/post/129055/

All Articles