TeamCity as a Debian repository

... or using TeamCity to build *.deb packs and not only.

Write an article prompted me to get acquainted with the module tcDebRepository . I naively believed that "now I will connect it, and everything will magically work." As usual, it did not work, and in the end a certain experience was accumulated, which wanted to be systematized.

The article is by no means an introduction to the basics of TeamCity and assumes that the reader is already familiar with TeamCity itself and with the Debian GNU / Linux infrastructure. If you already have an idea of what is integration integration, but you have never once held TeamCity in your hands - you should probably come here . You can read about building packages in Debian in the Debian New Maintainers' Guide .

For games (in case someone wants to reproduce the results), a TeamCity 10 and 3 agent server running Debian 8.0 (Jessie) was used . 3 agents is the limit for TeamCity Professional . Everything I wrote down, I think, is easily transferred to any other distribution based on Debian GNU / Linux , for example, Astra Linux .

Plans

Rather arbitrarily, I chose 4 packages for experiments:

Taking into account the limitations of the Professional type license on the number of build configurations, it was possible to “dial” up to 20 packages.

Training

TeamCity is downloaded from the official site. In addition to TeamCity itself, we will need to install the build-essential package on each of the agent machines, as well as the dependencies necessary for the assembly for all four packages (from the categories build-depends and build-depends-indep ). This will minimize (but not necessarily eliminate) problems with dependencies during assembly.

Kinds of packages in Debian GNU / Linux

Packages, among other features, are divided into "native" ( native ) and external ( non-native ) ( more ). Native packages ( autotools-dev , debhelper , dpkg ) are usually developed as part of the Debian project, and the source code already contains the meta information needed for building (the debian/ directory in the root of the source code tree).

The difference between external packages ( bash ) is that the source code is in no way tied to Debian, and maintenance engineers (in the Russian documentation this is called the “Debian developer”, in the English language it is simply the “maintainer”), you must maintain a parallel source tree with meta information and patches (this is the very contents of the debian/ directory).

General settings

The binary packages we will build are, in TeamCity terminology, “ artifacts ”. Accordingly, you need to specify that we expect to have a dry residue at the end of the next assembly, specifying the artifact paths :

For native packages, artifacts pkgname.orig.tar.{gz,bz2,xz} and pkgname.debian.tar.{gz,bz2,xz} not created.

Connecting the source code to TeamCity

Most often, just with this step there is nothing difficult: just go to the build configuration settings (build configuration) and add a new version control system root (VCS root). For native packages, this operation needs to be performed once, for external packages - as a rule, twice (but exceptions are possible when both developers (outside the Debian project) and maintenance engineers use the same DVCS ( Git , Bazaar ), and changes to the code constantly "wander" from one repository to another, at the same time without causing merge-conflicts for meta-information and patches).

The only feature is that in our case the artifacts will be collected outside the source tree (one directory above), so we need to configure checkout rules so that, say, the source code of the dpkg package is not downloaded to the current working directory, but in the package of the same name subdirectory, i.e. dpkg/ . This is achieved by adding a single line to checkout rules :

 +:.=>dpkg

and ultimately it looks like this:

Now you can add a VCS trigger and do not return to the version control setting:

Integration with Bazaar

"But bash requires integration with Bazaar to build bash , and the full-time delivery of TeamCity does not support this system!" - the attentive reader will say, and he will be right. In addition, TeamCity , alas, will not allow us to add a Git URL of the form bzr::http://bazaar.launchpad.net/~doko/+junk/pkg-bash-debian - JGit has too many restrictions.

There is an external module to support Bazaar , but it has at least two serious drawbacks:

Agent-side checkout is not supported, and
Bazaar’s standard delivery is not enough, since the module relies on the functionality of bzr xmlls and bzr xmllog , which is revealed only during the build process:

Since the TeamCity server worked on Windows for me, I refused to have a fun adventure in the form of installing Bazaar on the server side, and instead, in the case of the bash package, I just added another build step ( Bazaar integration for the poor) using the Command Line Runner and the following script shell:

 #!/bin/bash # # vim:ft=sh: # export LANG=C export LC_ALL=C set -e rm -rf bash/debian bzr branch http://bazaar.launchpad.net/~doko/+junk/pkg-bash-debian bash/debian major_minor=$(head -n1 bash/debian/changelog | awk '{print $2}' | tr -d '[()]' | cut -d- -f1) echo "Package version from debian/changelog: ${major_minor}" tar_archive=bash_${major_minor}.orig.tar rm -f ${tar_archive} ${tar_archive}.bz2 # +:.=>bash checkout rule should be set for the main VCS root in TeamCity tar cf ${tar_archive} bash tar --delete -f ${tar_archive} bash/debian # Required by dpkg-buildpackage bzip2 -9 ${tar_archive}

This approach does not allow us to "see" the changes in one source tree (out of two) and automatically start the assembly when they (changes) appear, but for the first experience it is quite sufficient.

NB! Since the Command Line Runner cannot highlight the syntax of the script code, I would recommend It's All Text for Mozilla Firefox and SeaMkey users . , allowing you to edit the contents of text fields in an external editor. You can connect Vim or Emacs and enjoy syntax highlighting, autocompletion, ~~chess and poetess~~ .

Build: customization

To build, we just need to use the already familiar Command Line Runner , which calls dpkg-buildpackage . The -uc and -us -uc mean that we don’t want to create digital signatures for our packages. If we still want to, we’ll have to upload the appropriate pair of GnuPG keys to each of the agents.

Also note that dpkg-buildpackage should not be executed in the current working directory, but in a subdirectory of the same name package (where the source code tree will be uploaded). If the version control setting is done, the "Working directory" field can be filled in with one mouse click, without manually entering the directory name:

Build: troubleshooting

Quality code

Oddly enough, but the quality of the code (or, more precisely, the style of development) can be a serious problem in the way of implementing continuous integration. It was empirically found out that, in the case of bash , the versions in two code trees were out of sync: the latest commits in the main tree correspond to version 4.4 , although the debian/changelog file almost already two years ago stopped at version 4.3 , and the code of one version with the metainformation of another version not going together. Ok, so I need a bash-4.3 branch in the main tree.

And now you can look at the commits tree and enjoy.

Here comes the bash-4.3-testing branch with the bash-4.3-rc2 and (not visible below) bash-4.3-rc1 - and then it suddenly ends. If you believe the history of versions, the release of bash 4.3 did not take place.
At the same time, a few days later, a commit appears on the master branch with the bash-4.3 tag, which is not preceded by any operation like merge or cherry-pick .
A quick look at the history and content of commits leads to the feeling that all development is conducted in the local branch of one person, and git push on savannah.gnu.org occurs at regular intervals, and through git merge --squash -s ours (for each commit incredibly long and difficult to read diff ).
The " Bash-4.3 patch XY " commits (46 patches for version 4.3 in total) are put into the master (they are not on bash-4.3-testing ), and after 3 weeks the label bash-4.4-beta2 appears on the master branch. This means that the last stable state " bash 4.3 plus patches", alas, is nowhere to take. Thank God, TeamCity allows you to build by tag (flag " Enable to use tags in the branch specification "), which was finally done.

Summary:

What I saw is not similar to the traditional branch creation scheme, nor to git-flow .
Yes, I am aware of the fact that the assembly according to the tag nullifies the whole point of continuous integration, but we will communicate with the developer bash another time.

Dependencies

When you run the first build, we will see that dpkg-buildpackage completed work with return code 3 :

As a result of viewing the assembly protocol, it turns out that some dependencies are still missing:

But here we installed everything that was required (on all agents), and dpkg-buildpackage completed with the same code. What's the matter? There are several nuances here.

Most likely, you will build software from a Debian unstable or Debian experimental . In this case, to satisfy the dependencies required for the build, the TeamCity agents must also be running under Debian unstable or Debian experimental (otherwise dpkg-buildpackage will "swear" that your stable versions of the dependencies are "outdated"). To suppress the error, sometimes it is enough to add the -d switch:
```
 dpkg-buildpackage -uc -us -d 
```
A special case of obsolete dependencies is the configure script created by a newer version of GNU Autotools than is currently installed on the system. dpkg-buildpackage not able to diagnose this situation - instead, we see cryptic messages about missing m4 macros in the build protocol. The solution is to re-create the configure script using the current version of GNU Autotools . Just add the following command in the first step of the build:
```
 autoreconf -i 
```

"Broken" unit tests

If we still want to deceive ourselves and build our package, it will be enough to run dpkg-buildpackage in a modified environment:

 DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -uc -us

About other methods of self-deception can be read here .

Finish line

After all the circles of hell are completed, we will see that the next build ended with the creation of artifacts:

If you do not want the number of single-unit artifacts to increase with each increment of the package (ultimately offering a wide range of dpkg_1.18.16_i386.deb , dpkg_1.18.17_i386.deb and dpkg_1.18.18_i386.deb ), the contents of the working directory is selectively clean before each build. This could be done manually, immediately before dpkg-buildpackage calling rm -rf with the notorious artifact paths as arguments, but there is a better way - a regular TeamCity module with the affectionate name "Mop". This is what its settings look like (the key here is "Before next build start"):

But the corresponding fragment of the assembly protocol will look like this if the Swabra module is correctly configured:

Now is the time to set up our Debian repository. This is achieved by adding artifact filters in the tcDebRepository module settings . Some inconvenience is that for each configuration (read: software package) you have to add a new filter that is actually identical to the previous one:

Already existing artifacts will not be indexed, so after the final configuration of the Debian repository, at least one build must pass in each configuration. After this comes anticipation:

Package Catalogs by Architecture

and list of available packages

When you add a repository in /etc/apt/sources.list you can observe all the same packages already from the client:

It is noticeable that the packages do not have a digital signature

NB! If you build for several architectures ( i386 , x32 , amd64 , arm ), you should either have several separate build configurations corresponding to the same package and differing in agent requirements, or, in addition to VCS Trigger , add a Schedule Trigger with the "Trigger build on" flag all enabled and compatible agents ":