Package Caching for Composer

Using a modern approach to project development, you begin to use the charms of a package manager, in the case of PHP development this is Composer. In this article, we briefly look at Composer, and then we’ll talk about setting up a local package cache.

This description in no way claims to be complete, but only gives a brief idea about this tool.

Description Composer and Silex example.

Take the descriptions from the official documentation:
Composer - is a tool for managing dependencies in PHP. Allows you to declare dependent libraries needed for a project, and install them into your project. Composer works in conjunction with Packagist.
Packagist - Composer repository of packages. Allows you to find the right packages, and Composer to find out where to get the source packages.

In other respect, Composer is not limited to Packagist storage and allows you to configure dependent packages from svn, git, pear. Packages must contain a properly configured configuration file (composer.json). Although this is not necessary, if the configuration file is not in the package you need, then you yourself will have to register it in the dependency configuration file of your project. Example
')
The configuration of project dependencies occurs through the Json configuration file (composer.json). The bottom line is to find the right package on Packagist, add the appropriate entry to composer.json and launch Composer, which downloads and configures the package itself (sets it up as the developer writes). An example .

Let's see an example of the minimal installation of the Silex framework on (debian / ubuntu):

mkdir /path/to/your/webroot/silex; cd /path/to/your/webroot/silex sudo apt-get install git php5 curl; curl -sS https://getcomposer.org/installer | php

This command downloads Composer in PHP archive format - composer.phar. Next, create the minimum composer.json:

 echo '{"require": {"silex/silex": "~1.1"} }' > composer.json php composer.phar install

Next, create a web / index.php:

 <?php // web/index.php require_once __DIR__.'/../vendor/autoload.php'; $app = new Silex\Application(); // definitions $app->run();

Everything, the Silex framework is ready, we omit the installation and launch of the web server, the Internet is full of ready-made descriptions.

The problem with the package manager is that too often you have to install the same packages. Each new installation of the project is an appeal to packagist.org to find the sources of dependencies of your package, dependencies of dependencies, etc. and further downloading the necessary packages. For the sake of fairness, it is worth noting that Composer has a built-in cache, but it is stored under ~ / .composer / cache, and is therefore individual for each user, not to mention separate environments for developers, testers, QA, production. And the same packages are downloaded everywhere.

Most packages are on a githaba, from where they download quickly enough so as not to wrap themselves in local caches. But when github is once again unavailable / slows down due to DDos, and even the size of dependencies reaches hundreds of megabytes - this becomes a problem. Github is not available - the work is worth it. I offer and currently use the Satis utility.

Satis - Composer static package repository, ultra-light, static version of Packagist, can be used to host private packages of your company, or your own.

As can be seen from the official description - the main goal of Satis is the ability to connect private packages. But also Satis has the ability to download packages from the same Packagist, store downloaded packages in zip or tar, and also distribute them. This is what we need. And so proceed to the installation:

 cd /path/to/your/webroot sudo apt-get install php5 git curl; curl -sS https://getcomposer.org/installer | php php composer.phar create-project composer/satis --stability=dev; cd satis

Next, create a config satis.json:

 { "name":"Project name", "homepage":"http://packagist.example.com", "archive":{ "directory":"dist", "skip-dev":false }, "repositories":[ { "type":"composer", "url":"https://packagist.org" } ], "require-dependencies":true, "require":{ "silex/silex":">1" } }

name : the name of your repository,
homepage : a link that will be available in the "/ path / to / your / webroot / satis / web" directory;
archive.directory : the path where the package cache will be stored;
archive.skip-dev : Whether to skip dev packages. Devs often change, but if we want to completely get rid of the dependence on packagist.org, github, bitbucket, etc. then set false;
repositories : a list of repositories, there may be several. You can specify direct package links to github, etc;
{"Type": "composer", "url": " packagist.org "} : specify the original Composer package repository - Packagist;
require-dependencies : whether to download dependency dependencies. Again, for complete independence, we set true;
require : list of packages which cache we want to use,;
{"Silex / silex": "> 1"} : for this example, take "require" from the example with the installation of the Silex framework, indicating the versions above the first;

And run the assembly:

 php bin/satis build satis.json web/

I got this output:

Scanning packages
Creating local downloads in 'web // dist'
Dumping 'doctrine / annotations-1.0.0.0'.
Dumping 'doctrine / annotations-1.1.0.0'.
Dumping 'doctrine / annotations-1.1.1.0'.
Dumping 'doctrine / annotations-1.1.2.0'.
Dumping 'doctrine / annotations-9999999-dev'.
Dumping 'doctrine / cache-1.0.0.0'.
Dumping 'doctrine / cache-1.1.0.0'.
Dumping 'doctrine / cache-1.2.0.0'.
Dumping 'doctrine / cache-9999999-dev'.
Dumping 'doctrine / collections-1.0.0.0'.
Dumping 'doctrine / collections-1.1.0.0'.
Dumping 'doctrine / collections-9999999-dev'.
Dumping 'doctrine / common-2.2.0.0'.
Dumping 'doctrine / common-2.2.0.0-RC1'.
Dumping 'doctrine / common-2.2.0.0-RC3'.
Dumping 'doctrine / common-2.2.0.0-RC4'.
Dumping 'doctrine / common-2.2.0.0-RC5'.
Dumping 'doctrine / common-2.2.0.0-beta1'.
Dumping 'doctrine / common-2.2.0.0-beta2'.
Dumping 'doctrine / common-2.2.1.0'.
Dumping 'doctrine / common-2.2.2.0'.
Dumping 'doctrine / common-2.2.3.0'.
Dumping 'doctrine / common-2.2.9999999.9999999-dev'.
Dumping 'doctrine / common-2.3.0.0'.
Dumping 'doctrine / common-2.3.0.0-RC1'.
Dumping 'doctrine / common-2.3.0.0-RC2'.
Dumping 'doctrine / common-2.3.0.0-RC3'.
Dumping 'doctrine / common-2.3.0.0-beta1'.
Dumping 'doctrine / common-2.3.9999999.9999999-dev'.
Dumping 'doctrine / common-2.4.0.0'.
Dumping 'doctrine / common-2.4.0.0-RC1'.
Dumping 'doctrine / common-2.4.0.0-RC2'.
Dumping 'doctrine / common-2.4.0.0-RC3'.
Dumping 'doctrine / common-2.4.0.0-RC4'.
Dumping 'doctrine / common-2.4.1.0'.
Dumping 'doctrine / common-2.4.9999999.9999999-dev'.
Dumping 'doctrine / common-9999999-dev'.
Dumping 'doctrine / inflector-1.0.0.0'.
Dumping 'doctrine / inflector-9999999-dev'.
Dumping 'doctrine / lexer-1.0.0.0'.
Dumping 'doctrine / lexer-9999999-dev'.
Dumping 'pimple / pimple-1.0.0.0'.
Dumping 'pimple / pimple-1.0.1.0'.
Dumping 'pimple / pimple-1.0.2.0'.
Dumping 'pimple / pimple-9999999-dev'.
Dumping 'psr / log-1.0.0.0'.
Dumping 'silex / silex-1.0.1.0'.
Dumping 'silex / silex-1.0.9999999.9999999-dev'.
Dumping 'silex / silex-1.1.0.0'.
Dumping 'silex / silex-1.1.1.0'.
Dumping 'silex / silex-9999999-dev'.
Dumping 'symfony / debug-2.3.0.0'.
Dumping 'symfony / debug-2.3.1.0'.
Dumping 'symfony / debug-2.3.2.0'.
Dumping 'symfony / debug-2.3.3.0'.
Dumping 'symfony / debug-2.3.4.0'.
Dumping 'symfony / debug-2.3.5.0'.
Dumping 'symfony / debug-2.3.6.0'.
Dumping 'symfony / debug-2.3.9999999.9999999-dev'.
Dumping 'symfony / debug-2.4.0.0-beta1'.
Dumping 'symfony / debug-9999999-dev'.
Dumping 'symfony / event-dispatcher-2.1.0.0'.
Dumping 'symfony / event-dispatcher-2.1.1.0'.
Dumping 'symfony / event-dispatcher-2.1.10.0'.
Dumping 'symfony / event-dispatcher-2.1.11.0'.
Dumping 'symfony / event-dispatcher-2.1.12.0'.
Dumping 'symfony / event-dispatcher-2.1.13.0'.
Dumping 'symfony / event-dispatcher-2.1.2.0'.
Dumping 'symfony / event-dispatcher-2.1.3.0'.
Dumping 'symfony / event-dispatcher-2.1.4.0'.
Dumping 'symfony / event-dispatcher-2.1.5.0'.
Dumping 'symfony / event-dispatcher-2.1.6.0'.
Dumping 'symfony / event-dispatcher-2.1.7.0'.
Dumping 'symfony / event-dispatcher-2.1.8.0'.
Dumping 'symfony / event-dispatcher-2.1.9.0'.
Dumping 'symfony / event-dispatcher-2.1.9999999.9999999-dev'.
Dumping 'symfony / event-dispatcher-2.2.0.0'.
Dumping 'symfony / event-dispatcher-2.2.1.0'.
Dumping 'symfony / event-dispatcher-2.2.2.0'.
Dumping 'symfony / event-dispatcher-2.2.3.0'.
Dumping 'symfony / event-dispatcher-2.2.4.0'.
Dumping 'symfony / event-dispatcher-2.2.5.0'.
Dumping 'symfony / event-dispatcher-2.2.6.0'.
Dumping 'symfony / event-dispatcher-2.2.7.0'.
Dumping 'symfony / event-dispatcher-2.2.8.0'.
Dumping 'symfony / event-dispatcher-2.2.9.0'.
Dumping 'symfony / event-dispatcher-2.2.9999999.9999999-dev'.
Dumping 'symfony / event-dispatcher-2.3.0.0'.
Dumping 'symfony / event-dispatcher-2.3.1.0'.
Dumping 'symfony / event-dispatcher-2.3.2.0'.
Dumping 'symfony / event-dispatcher-2.3.3.0'.
Dumping 'symfony / event-dispatcher-2.3.4.0'.
Dumping 'symfony / event-dispatcher-2.3.5.0'.
Dumping 'symfony / event-dispatcher-2.3.6.0'.
Dumping 'symfony / event-dispatcher-2.3.9999999.9999999-dev'.
Dumping 'symfony / event-dispatcher-2.4.0.0-beta1'.
Dumping 'symfony / event-dispatcher-9999999-dev'.
Dumping 'symfony / http-foundation-2.1.0.0'.
Dumping 'symfony / http-foundation-2.1.1.0'.
Dumping 'symfony / http-foundation-2.1.10.0'.
Dumping 'symfony / http-foundation-2.1.11.0'.
Dumping 'symfony / http-foundation-2.1.12.0'.
Dumping 'symfony / http-foundation-2.1.13.0'.
Dumping 'symfony / http-foundation-2.1.2.0'.
Dumping 'symfony / http-foundation-2.1.3.0'.
Dumping 'symfony / http-foundation-2.1.4.0'.
Dumping 'symfony / http-foundation-2.1.5.0'.
Dumping 'symfony / http-foundation-2.1.6.0'.
Dumping 'symfony / http-foundation-2.1.7.0'.
Dumping 'symfony / http-foundation-2.1.8.0'.
Dumping 'symfony / http-foundation-2.1.9.0'.

Now we need to tweak the configuration of packages in the project that our Satis cache will use, again, let's use the Silex example:

 cd /path/to/your/webroot/silex echo '{"repositories": [{ "type": "composer", "url": "http://packagist.example.com" },{ "packagist": false } ], "require": {"silex/silex": "~1.1"}}' > composer.json

Formatted version of composer.json

 { "repositories":[ { "type":"composer", "url":"http://packagist.example.com" }, { "packagist":false } ], "require":{ "silex/silex":"~1.1" } }

repositories : list of package repositories;
{"Type": "composer", "url": " packagist.example.com "} : link to our newly created Satis repository with type composer.
{"Packagist": false} : by default, if the package is not found in the list of specified repositories, Composer will search for packages on Packagist. This is good and convenient. But, if we break this behavior, we cannot be sure that we really have all the packages in our local storage. I’ll add my observation: if you have a lot of dependencies, Composer eats a lot of memory , some report dimensions in 3GB ... which is sad. So, we observed a similar situation, until we left only one storage facility. That is, or packagist.org or your Satis;
require : list of dependencies;
{"Silex / silex": "~ 1.1"} : for now we only need Silex;

Now, to make sure that our structure works, we need to clear the Composer cache, otherwise Composer will take the package from its cache:

 cd /path/to/your/webroot/silex; rm composer.lock; rm -fr vendor; rm -fr ~/.composer/cache php composer.phar install

Everything. If the settings are correct, Composer will now take packages from our local cache (http://packagist.example.com).

The minus of the proposed solution is that the cached packets have to be registered in the Satis (satis.json) configuration file with your hands, that is, Satis will not work as a proxy with auto-caching, which, in my opinion, is an omission. You also need to configure the cron script that will pull Satis build to update dev packages and download new versions of packages:
0 */12 * * * cd /path/to/your/webroot/satis/; php bin/satis build satis.json ./web/

PS inaccuracies and errors please in PM.
PSS other solutions for caching Composer packages, I propose to discuss in the comments.

Source: https://habr.com/ru/post/197666/

All Articles

Package Caching for Composer

More articles: