Geekly Articles each Day
📜 ⬆️ ⬇️

Become a maintainer - update the repository

This article is not aimed at solving any fundamental problems, but it helps, with a minimum of effort, to automate the building of packages for your public repository.
Let's say you build ten packages for your favorite users, each for two distributions (say, Debian unstable and Ubuntu jaunty), and each for two architectures (amd64 and i386). Remember, we learned about the wonderful tool for creating repositories - reprepro. So, since then, he never learned to include packs in packs in the repository - only one at a time. This means that you have to enter your gpg-signature 10 * 2 * 2 = 40 times. And how to effectively collect these 10 packages, especially if they are updated daily?

In fact, my situation was even worse. In total, when updating the qutIM repository, I had to enter my not short gpg-key as many as 176 (one hundred seventy-six) times. I must say that by the time when I finally got tired of it and overcame my laziness, I carried out this procedure in just 3 minutes - one second to enter. But we will not torture you, we will immediately correct the situation. So, install the gnupg-agent package and edit your ~ / .bashrc file , adding the following lines at the end:
  1. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  2. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  3. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  4. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  5. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  6. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  7. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  8. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  9. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi
  10. # # GnuPG # gpg-agent --daemon --enable-ssh-support --write-env-file " ${HOME} /.gpg-agent-info" if [ -f " ${HOME} /.gpg-agent-info" ] ; then . " ${HOME} /.gpg-agent-info" export GPG_AGENT_INFO export SSH_AUTH_SOCK export SSH_AGENT_PID fi

Now when you start the local console, you will run a gpg-agent that “remembers” the gpg-key you entered for a while. Voila, we got rid of one big problem.

What's next? Next build packages. I will tell you about it using the example of the same my favorite qutIM: among the packages I collect there are those that are almost daily updated from svn, and there are those that were downloaded once, and since then they are just rebuilt when updating kernel qutIM.
For a start, I’ll tell you a little about how the directory and package naming structure is organized in me, to make it clearer.
At the root of the build directory, I initially only have a build script and directories of those packages that are not updated from svn, for example, qutim-plugin-floaties. At the root of the build directory, all source directories are siphoned from svn and the collected packages are put. Even in this directory, I have a wonderful subdirectory of controls, in which are the debian-folders for packages. For example, to find the debian-folder for the qutim-protocol-jabber package, I call controls / qutim-protocol-jabber / debian /
Well, in the rep / directory I have repository directories for different distros - rep / jaunty /, rep / unstable /, etc.
From this we will make a start.
')
So, let's start with the caps of our shell script, which will build:
  1. #! / bin / bash
  2. BUILDERRORS = "" # variable where build errors are written
  3. MAJORVERSION = "0.2a" # main version of each package
  4. DISTRIBUTIONS = "unstable testing stable jaunty" # repository names

Now let's write two small simple functions, one of which will simply build a package from the current directory for all distributions and architectures, and the second will update package versions in all our pbuilder shells.
  1. build_it ( ) {
  2. OLDDIR = ` pwd`
  3. cd $ 1
  4. for i in $ DISTRIBUTIONS ; do
  5. for j in i386 amd64; do
  6. DIST = $ i ARCH = $ j pdebuild - --basetgz / var / cache / pbuilder / $ i - $ j -base.tgz
  7. if [ [ $? ! = "0" ] ]
  8. then
  9. BUILDERRORS = " $ BUILDERRORS
  10. $ 1 - $ i - $ j "
  11. fi
  12. done
  13. done
  14. cd $ OLDDIR
  15. }
  17. update_it ( ) {
  18. echo "Updating pbuilder environments ..."
  19. for i in $ DISTRIBUTIONS ; do
  20. for j in i386 amd64; do
  21. DIST = $ i ARCH = $ j pbuilder --update --basetgz / var / cache / pbuilder / $ i - $ j -base.tgz
  22. if [ [ $? ! = "0" ] ]
  23. then
  24. BUILDERRORS = " $ BUILDERRORS
  25. Updating - $ i - $ j "
  26. fi
  27. done
  28. done
  29. }


It's simple, isn't it? The first function will receive the name of the directory as input, go there and try to build a package for all architectures. The second one doesn't need anything at all; it just updates all the pbuilder shells. Note that both commands in the case of errors do nothing, just append the message to the error log.
Now we need to make a universal function that will collect our bags. All packages differ from each other by name, version and place from which to get them. This will use. First of all, in those debian / changelog files that I have stored in the controls folder, I have instead written the version “--VERSION--” everywhere. And secondly, I wrote such a wonderful little function:
  1. make_package ( ) {
  2. _REV = $ 1
  3. _URL = $ 2
  4. _NAME = $ 3
  5. _DIR = $ 4
  6. _BREV = $ 5
  7. DIRNAME = ""
  8. PACKAGENAME = ""
  9. if [ [ $ _URL ! = "" ] ]
  10. then
  11. rm -rf " $ _DIR "
  12. export $ _REV = ` LANG = C svn export $ _URL $ _DIR | awk '$ 1 == "Exported" && $ 2 == "revision" {print $ 3}' | sed 's /.$ //' `
  13. if [ [ $ {! _ REV} = "" ] ]
  14. then
  15. echo "Error! Can't get $ _NAME revision!" && exit 1
  16. fi
  17. echo "Received $ _NAME revision: $ {! _ REV} "
  18. else
  19. export $ _REV = "1"
  20. fi
  21. export REVISION = " $ {! _ REV} "
  22. DIRNAME = " $ {_ DIR} - $ {_ BREV} . $ {! _ REV} "
  23. PACKAGENAME = " $ {_ DIR} _ $ {_ BREV} . $ {! _ REV} "
  24. echo " $ _NAME directory is: $ DIRNAME "
  25. rm -rf " $ {PACKAGENAME} .orig.tar.gz" " $ DIRNAME "
  26. cp -r " $ _DIR " " $ DIRNAME "
  27. tar czf " $ {PACKAGENAME} .orig.tar.gz" " $ DIRNAME "
  28. echo " $ _NAME source archive is: $ {PACKAGENAME} .orig.tar.gz"
  29. cp -r "controls / $ {_ DIR} / debian" " $ DIRNAME /"
  30. sed -i "s # - VERSION - # $ {_ BREV} . $ {! _ REV} -1 #" " $ DIRNAME / debian / changelog"
  31. echo "Building $ _NAME ..."
  32. build_it " $ DIRNAME "
  33. }


Let's take a closer look at what this function does. At the input, we pass in succession five parameters:

The function is universal - if the URL is passed, the source files are downloaded from it and the .orig.tar.gz archive is automatically created. If not transmitted, it is created from an existing directory.
Now the package can be assembled in one simple line:
  1. make_package "QREV" "qutim.org/svn/qutim" "qutIM" "qutim" " $ MAJORVERSION "

What is the very first QREV parameter passed for? When assembling the package, we give qutIM the “older” version “0.2a” and get the version of the package itself, for example, “0.2a.294-1”. If I build a protocol package, I want its version to include the kernel version:
  1. make_package "MRIM_REV" "qutim.org/svn/mrim" "MRIM" "qutim-protocol-mrim" " $ MAJORVERSION . $ QREV "

At the output we get the version "0.2a.294.357-1". Isn't it convenient? Thus, we always know which version of the source we received.
If we have a package locally and I simply reassemble it, without pumping out svn - it doesn’t matter, just pass an empty parameter as a URL:
  1. make_package "FLOATIES_REV" "" "Floaties" "qutim-plugin-floaties" " $ MAJORVERSION . $ QREV "

We have only one thing left - to update the packages in the repository, put it from the build machine to the server and let us know when assembling which packages we had errors:
  1. echo "Copying packages to local repository ..."
  2. for j in $ DISTRIBUTIONS ; do
  3. for i in ` ls / var / cache / pbuilder / $ j / result / * amd64.changes`; do
  4. reprepro -b rep / $ j / --ignore = wrongdistribution -C main include $ j $ i
  5. done
  6. for i in ` ls / var / cache / pbuilder / $ j / result / * _i386.deb`; do
  7. reprepro -b rep / $ j / --ignore = wrongdistribution -C main includedeb $ j $ i
  8. done
  9. done
  11. echo "Updating main repository ..."
  12. scp -r rep / * remotehost: / path / to / repo /
  14. echo "Done."
  15. if [ " $ BUILDERRORS " ] ;
  16. then
  17. echo "There were errors in:"
  18. echo " $ BUILDERRORS "
  19. fi

Notice, since we “screwed” the gpg-agent to our console, we no longer need the --ask-passphrase key for reprepro.

All these wonderful scripts not only allow me to add new packages to the repository now by simply preparing the debian / directory for the package and one line in the script, but they also do not require my attention. To build, I now run my script, and I'm going about my business. Returning a few hours later I enter the gpg key once and the password from the remote repository server once. Simple and convenient.

Ps Information for those who like to experiment: besides the convenient reprepro, there is also the official dak utility, which is used for repositories in at least Debian. I never used it, because for work it uses PostgreSQL and an incorrect python language (I won’t give a holivarit about it, and don’t ask for it). But anyone can try.

Pps For those who want to use scripts in their own, I suggest that I use the full script I use. Edit for yourself on health: pastebin.com/f3289286a

Source: https://habr.com/ru/post/61644/

More articles:


All Articles