Package Manager opkg. Offline installation of packages in the root file system image

The lightweight package manager opkg, widely known in narrow circles, has become popular on embedded Linux not by accident. Opkg is used in many embedded distributions and projects, for example, in OpenEmbedded , Yocto Project , OpenWRT , Ångström , Arago Project and some others. The manager is easy to use, in order to complete the work, the built-in help is quite enough, and on the World Wide Web there are many articles about the ipk package itself (opkg works with this format): how to create it, how to install, etc., etc. . However, the overwhelming majority of information is devoted to how to work on an already installed system on the target platform (target) in an online mode, but the specificity of Embedded implies that the root file system image and the kernel are prepared in advance on some instrumental platform (host) different from target. In other words, we assemble the kernel and file system on the working computer, package it into an image, replicate the image with iron. This article focuses on how to install packages into the prepared rootfs image using the opkg manager.

Road rake and bike

Many years ago, when I was an engineer at one small factory, when I launched Linux on the first shawl of my own production, using opkg I installed all the required packages from a remote repository, set up all applications, the laboratory head said: "Excellent! Now do the same on all devices in the party. " "Sure, not a problem!" - I replied. The system is there, it is running, it works. We copy all the files from the root to external media, then pack it into an image and enjoy life! At that time, I did not understand that when operating, the operating system performs a number of local settings, creates temporary files, configuration files, generates some keys, and when it first starts it also executes initialization scripts. Although transferring files from one working system to another using the blunt copying method from the media gave the result, but the effectiveness of this method very soon became doubtful for me. It is impossible to get a "clean" system in this way: the system remembers its previous life in another hardware body, and from time to time it is stifled by phantom pains.

The next step for me was to understand the structure of the package itself * .ipk . In fact, the ipk package is an archive, which can be easily unpacked using the command:
')

ar -x *.ipk 

As a result, we get:

  . ├── control.tar.gz ├── data.tar.gz └── debian-binary 

The data.tar.gz archive contains files that should be placed in the target’s root directory.
The archive control.tar.gz contains service files: a file with a description and scripts. The idea is simple: since ipk is just an archive with scripts, we can always unpack it into the directory with the file system, and then run (if necessary) the scripts. But we will have to install all dependencies of the package manually.
And if the dependencies have more dependencies? There is an idea, maybe write a script to automate the process? As it often happens in the linux world, if a task has arisen before you, then, most likely, such a task did not arise before you alone, and, most likely, you are not the first in this matter.
I didn’t have to go far, in fact, the opkg package manager itself has such a mode, when packages are installed in the inactive rootfs file system. At the same time, the architecture of the host machine (where the opkg utilities are launched) and the target machines can be excellent. This mode is called Offline mode . In this mode, opkg becomes the most powerful tool for cross-development.

We collect opkg for host

To work offline, opkg must run on the host. For a long time, Ubuntu (Ubuntu 14.04 LTS is now on my computer) settled on my computer, and we will build our toolkit on it. I did not manage to find a repository with opkg for Ubuntu, therefore we collect a set of utilities from source codes.
You can get the source code from the Yocto Project git repository:

  git clone git://git.yoctoproject.org/opkg.git cd opkg 

In fact, the configuration and compilation of the project is done in a fairly standard way, but there are some nuances, and therefore everything is in order.
Run:

 ./autogen.sh 

To the note: if you run ./autogen.sh with the --clean parameter, then all works on the project configuration will be deleted.
After executing ./autogen.sh , the configure script appears in the source directory, it will configure the package, determine and set system-dependent variables. As a result of the script, a Makefile is created. You can view all script options in the standard way:

 ./configure --help 

We will build the package for the current platform, so we’ll skip the options for configuring cross-compilation. We will take care of the installation. By default, running make install , the script scatters all useful files (binaries, scripts, documentation) in the root directory: /etc , /usr/local , which is completely useless to us. We're not going to use opkg to configure packages on the current system? In addition, having installed the manager in the system folders, to use the utilities, you will need superuser rights, in my opinion, this is unnecessary when setting up the embedded linux image. The configure.sh script allows you to specify a prefix for the package installation directory. Specifying any working directory as a prefix, we will inform the installer where to put the package. If necessary, you can separately specify the prefix for architecture-dependent (binaries and libraries) and architecture-independent (scripts and documentation) files.
With fantasy, I have always been weak, because for the installation in the home directory we will create the opkg_offline directory.

 mkdir ${HOME}/opkg_offline 

Run the configuration:

 ./configure --prefix=${HOME}/opkg_offline 

If necessary, deliver the required dependencies. So, on Ubuntu 14.04 I needed to deliver libarchive-dev , libcurl4-gnutls-dev , libssl-dev , libgpgme11-dev to build successfully.


Compile and install opkg:

 make make install 

Package Manager is built and installed. Executable files are located in the opkg_offline / bin directory. To work with them, you can set the path to the PATH variable, either export (call in) for each terminal session, or do as I do — go to the opkg_offline directory and run directly ./bin/opkg .

Anatomy Short Course

Let's take a quick look at how the package manager works in standard mode. After executing the opkg update command, the utility reads configuration files that are located by default in /etc/opkg and have the extension .conf. From these files, the system determines the type of architecture, for example, armv5hf-vfp or armv5tehf-vfp (there may be several supported architectures, you can set a priority for each), a list of repositories, and some settings of the program itself. Next, for each repository from the list, an archive of the type *_Packages.gz . Default archives are placed in the var/cache/opkg/ . After unpacking, the contents are placed in var/lib/opkg/lists . Each archive contains a text file with a list of packages in the repository. For each package, in addition to the name, there is a version, architecture, size, short description, license, and most importantly, dependencies. Based on these files, the package manager can provide information about the required package upon request, and when installing it, determine all dependencies and resolve them.
The opkg list command opkg list all packages available for installation; The opkg list-installed command opkg list-installed will show only installed packages, the opkg info command will show information about the specified package, and if it is installed, then the installation time.
To install a package, you must run opkg install packname . As a result, the required package from the repository will be downloaded to the temporary directory and distributed. All files from the data.tar.gz archive will go to rootfs in their places, and based on the contents of control.tar.gz, service files will be created in the var/lib/opkg/info packname.control : packname.control - complete information about the package, packname.list - the list of directories in which the files from data.tar.gz went (this list will go through opkg when the package is deleted), and script files such as packname.postinst , packname.preinst , packname.prerm , packname.postrm , whose purposes are clear from the title. Information about the installed package will be added to the var/lib/opkg/status file as (example for the popular minicom ):

 Package: minicom Version: 2.6.2-r0.2 Depends: libtinfo5 (>= 5.9), libc6 (>= 2.17) Status: install ok installed Architecture: armv7ahf-vfp-neon Installed-Time: 1454529423 

It is important to pay attention to Status . If the package was installed according to all the rules: all files are copied to their place, all scripts are executed, then the status will be Status: install ok installed . When working offline, all files will be copied, but the scripts will not be executed, such packages will be marked as Status: install ok unpacked .
In this case, opkg has a special post configuration mechanism for packages. It is launched by the opkg configure <packname> . If you specify the name of a specific package, the scripts from var / lib / opkg / info will be executed for this package; if the name is omitted, the manager will configure for all packages whose status is Status: install ok unpacked . Thus, when installing packages to the host in offline mode, when you first load the operating system on the target, you should run opkg configure . You can entrust this either to a special script or, if systemd is used, to a special service.

Work with target rootfs

It is time to try the system in action. For example, install the minicom serial port terminal emulator.
To install packages, we need an unpacked image of the root file system of the target rootfs platform. Suppose that the opkg manager is installed in rootfs, and there are configuration files in the etc/opkg directory * .conf. If it is not there, or for some reason we do not want to use the configuration from rootfs, we can use the parameter to specify which settings file to use: -f etc/opkg/opkg.conf . The path to the target file system is passed through the parameter --offline-root /path/to/rootfs .
Updating package lists:

 bin/opkg update --offline-root /path/to/rootfs 

We look through the list of available packages, we look for minicom.

 bin/opkg list --offline-root ~/board/rootfs/angstrom/rootfs-v2015.10 | grep minicom minicom - 2.7-r0.0 - Text-based modem control and terminal emulation program Minicom is a minicom-dbg - 2.7-r0.0 - Text-based modem control and terminal emulation program - Debugging files minicom-dev - 2.7-r0.0 - Text-based modem control and terminal emulation program - Development minicom-doc - 2.7-r0.0 - Text-based modem control and terminal emulation program - Documentation 

See information about the package:

 bin/opkg info minicom --offline-root ~/board/rootfs/angstrom/rootfs-v2015.10 

  Package: minicom Version: 2.7-r0.0 Depends: libtinfo5 (>= 5.9), libc6 (>= linaro-2.20) Status: unknown ok not-installed Section: console/network Architecture: armv7at2hf-vfp-neon Maintainer: Angstrom Developers <angstrom-distro-devel@linuxtogo.org> MD5Sum: e4d11b7277fbc1c7db6bbd97ac52ca2c Size: 79354 Filename: minicom_2.7-r0.0_armv7at2hf-vfp-neon.ipk Description: Text-based modem control and terminal emulation program Minicom is a text-based modem control and terminal emulation program for Unix-like operating systems 

Install the package:

 bin/opkg install minicom --offline-root ~/board/rootfs/angstrom/rootfs-v2015.10 

An var/lib/opkg appeared in the var/lib/opkg file:

  Package: minicom Version: 2.7-r0.0 Depends: libtinfo5 (>= 5.9), libc6 (>= linaro-2.20) Status: install user unpacked Architecture: armv7at2hf-vfp-neon Installed-Time: 1454594718 

After the system was launched from the created image and the opkg configure command opkg configure , the entry in the file changed:

  Package: minicom Version: 2.7-r0.0 Depends: libtinfo5 (>= 5.9), libc6 (>= linaro-2.20) Status: install user installed Architecture: armv7at2hf-vfp-neon Installed-Time: 1454594718 

Since custom rootfs is designed for an embedded computer, the final image size matters. Therefore, I recommend that after all the necessary packages have been installed, delete the downloaded lists and clean the cache:

 rm -rvf ~/board/rootfs/angstrom/rootfs-v2015.10/var/cache/opkg/* rm -rvf ~/board/rootfs/angstrom/rootfs-v2015.10/var/lib/opkg/lists/* 

To the note: the option --volatile-cache will allow clearing the cache automatically upon completion of work.

Instead of conclusion

Despite the performance, Offline mode has some drawbacks. The fact is that the opkg configure command runs only \*.postinst opkg configure for execution, but the question remains with the execution of \*.preinst . Due to the fact that \*.preinst is quite rare in packages, for me it is acceptable to view the scripts manually, and, if necessary, to work them out when you first start the target system (special services for systemd). I would be grateful for the advice.

Read on:

• IPK-package or what it eats
• OpenWrt.OPKG Package Manager . The most intelligible help.
• Yocto Project . Repository Sources