1. Hi.

On Habré suspiciously little information about the Open Build Service (hereinafter OBS) and other platforms.
And I don’t want to talk about the freshness of the existing one.
Recently there was a release of version 2.7 , came the long-awaited changes.
But, for the story, I want to tell a little about one use case 2.6 (release - February 2015).

The example for the material is inspired by recent need.
So, let's compile php-ffmpeg based on ffmpeg 3.0 for Centos (7, and even 6 *!).

2. Scheme

For a long time I performed all the necessary commands using the copy-paste method from my wiki.
Absolutely everything is done in the console, incl. through the OBS API.
Now the whole process is divided into 4 stages:

cd ~ && git clone https://github.com/BOPOHA/obs-fast-c7ffmpeg && cd obs-fast-c7ffmpeg bash -x obs-fast-c7ffmpeg-n1-prepare.sh #      cd ~/obs-fast-c7ffmpeg bash -x obs-fast-c7ffmpeg-n2-mkvps.sh bash -x obs-fast-c7ffmpeg-n3-buildtargets.sh bash -x obs-fast-c7ffmpeg-n4-buildpkgs.sh 

• n1 - under the host we prepare for work, install the means of virtualization, reboot.
• n2 - set up a VPS based on a prepared .qcow2 image
• n3 - we configure OBS: we add our tsentosa, we create the target project
• n4 - we send tarballs and specs to the assembly

Everything you need for sbroki available in github .
Used hardware: Xeon E3-1230 3.2 GHz / 8GB / 2x500GB / 100Mbs / CentOS 7.x
The elapsed time is ~ 2 hours (40 minutes per rsync rpm-ok, and a little over an hour on the build process itself).
Further, for each of the scripts, I will note the most important, in my opinion, moments.

n1-prepare.sh

Here we add some options to GRUB_CMDLINE_LINUX to increase performance.
Separately, it costs kvm-intel.nested = 1, this allows you to use virtualization capabilities inside the OBS container, thereby speeding up the assembly process and reducing the load on the disks.
Here, in general, nothing more interesting, install a third-party openSUSE repository: Tools to work through the API, and libvirt / kvm packages.

n2-mkvps.sh

All the necessary data for setting up the environment was taken from here: openSUSE: Build_Service_private_installation .

 wget http://download.opensuse.org/repositories/OBS:/Server:/2.6/images/obs-server.x86_64.qcow2 qemu-img create -f raw -o size=128G obs-server_os.raw virt-resize --expand /dev/sda1 obs-server.x86_64.qcow2 obs-server_os.raw losetup /dev/loop0 obs-server_os.raw kpartx -a /dev/loop0 mount /dev/mapper/loop0p1 /mnt echo "rcsshd start swapon /dev/vdb1 sleep 360 && screen -dmS upgraderepos bash -x /root/upgrade.repos.bash & # " >> /mnt/etc/init.d/boot.local mkdir /mnt/root/.ssh ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub > /mnt/root/.ssh/authorized_keys cat ~/$REPONAME/___env/upgrade.repos.bash >> /mnt/root/upgrade.repos.bash chmod -R 700 /mnt/root/.ssh umount /mnt/ kpartx -d /dev/loop0 losetup -d /dev/loop0 

Here we take the prepared image. By default, it does not have enough free space for normal operation, so we make changes in size, and a small change in FS:

• add multiple commands to autoload
• generating ssh-key on the host and copying the pub-key to the OBS.
• Copy script to update rpm-ok.

 qemu-img create -f raw -o size=128G obs-server_lvm.raw losetup /dev/loop0 obs-server_lvm.raw parted -s /dev/loop0 mklabel msdos parted -s /dev/loop0 mkpart primary 0.00GB 8GB parted -s /dev/loop0 mkpart primary 8GB 128GB parted -s /dev/loop0 set 2 lvm on kpartx -a /dev/loop0 sleep 2 mkswap /dev/mapper/loop0p1 pvcreate /dev/mapper/loop0p2 -ff -y vgcreate "OBS" /dev/mapper/loop0p2 lvcreate -L 32G -n "server" /dev/OBS vgscan mkfs.ext4 /dev/OBS/server vgchange -an OBS kpartx -d /dev/loop0 losetup -d /dev/loop0 

As described in Build_Service_private_installation, this piece of code creates a second disk, with an LVM partition and the necessary markup.
When loading, the system will recognize it and continue to configure itself.

 virt-install \ --name OBS_server \ --ram 7168 --vcpus 4 \ --disk path=/home/libvirt_images/obs-server_os.raw \ --disk path=/home/libvirt_images/obs-server_lvm.raw \ --graphics spice \ --os-variant sles11 \ --boot=hd & sleep 180 && virsh reboot OBS_server sleep 60 

sleep 180 - waiting for the first OBS boot and initialization of LVM, after which the OBS reboot.
sleep 60 - after that we think that OBS is ready for work

 VPS_MAC=`virsh dumpxml OBS_server | grep -o '..:..:..:..:..:..'` VPS_IP=`arp -an | grep $VPS_MAC | sed 's|^.*(\(.*\)).*$|\1|'` touch /etc/hosts /root/.ssh/known_hosts sed -i "/linux/d; /$VPS_IP/d;" /etc/hosts /root/.ssh/known_hosts echo "$VPS_IP linux" >> /etc/hosts ssh linux -oStrictHostKeyChecking=no uptime 

Here on the mac-address we are looking for IP OBS, and register its hosts.

It should be noted that the parameters of the size of the disk 128G, and the number of vcpus - are interrelated.
By default, the number of Werkers is equal to the number of vcpu. If you need to build a large project, then 5GB may not be enough, you will need to either increase the partition size, or reduce the number of cores, or change the acc. parameter in / etc / sysconfig / obs-server. Below is an example:

lsblk obs

n3-buildtargets.sh

 #!/bin/bash set -e LANG=en_US.UTF-8 REPONAME='obs-fast-c7ffmpeg' export ACC=Admin export REPO_NAME="home:$ACC:$REPONAME" export TMP_DIR="/home/tmp" export PKG_DIR="$HOME/$REPONAME" mkdir -p $TMP_DIR rm -rf /root/.config/osc/trusted-certs/linux_443.pem yes 2 |osc list osc meta prj -F - centos7 << EOF <project name="centos7"> <title>centos7</title> <description></description> <person role="maintainer" userid="Admin"/> <person role="bugowner" userid="Admin"/> <repository name="epel"> <arch>x86_64</arch> </repository> <repository name="updates"> <arch>x86_64</arch> </repository> <repository name="os"> <arch>x86_64</arch> </repository> </project> EOF cd $PKG_DIR osc meta prjconf -F ___env/centos7.conf centos7 osc meta prj home:$ACC -F - << EOF <project name="home:$ACC"> <title>$ACC's Home Project</title> <description></description> <person userid="$ACC" role="maintainer"/> <person userid="$ACC" role="bugowner"/> </project> EOF osc meta prj $REPO_NAME -F - << EOF <project name="$REPO_NAME"> <title>$REPO_NAME</title> <description>$REPO_NAME</description> <person userid="$ACC" role="maintainer"/> <repository name="CentOS_7"> <path project="centos7" repository="epel"/> <path project="centos7" repository="updates"/> <path project="centos7" repository="os"/> <arch>x86_64</arch> </repository> <repository name="CentOS_6"> <path project="centos6" repository="epel"/> <path project="centos6" repository="updates"/> <path project="centos6" repository="os"/> <arch>i586</arch> <arch>x86_64</arch> </repository> <build> <disable repository="CentOS_6"/> </build> </project> EOF 

At first glance, everything is simple here, but in fact, such manipulations are not clearly highlighted anywhere in the documentation.

 osc meta prj -F - centos7 << EOF 

Register the centos7 plugin repository.

 osc meta prj home:$ACC -F - << EOF 

Create a "Home Project" home: Admin.

 osc meta prj $REPO_NAME -F - << EOF 

Create a sub-project home: Admin: obs-fast-c7ffmpeg.

And the hardest part was finding the command osc meta prjconf -F ___ env / centos7.conf centos7 .
This is the build config of the corresponding OS.
The centos7.conf file can be smoked from the project community in one of the following ways:

 osc -A https://api.opensuse.org meta prjconf CentOS:CentOS-7 > centos7.conf curl https://api.opensuse.org/public/source/CentOS:CentOS-7/_config > centos7.conf 

n4-buildpkgs.sh

The last stage, the creation of packages, is implemented by the function:

 function push { cd $TMP_DIR PKG_NAME=$1 osc meta pkg $REPO_NAME $PKG_NAME -F - << EOF <package name="$PKG_NAME"> <title>$PKG_NAME</title> <description></description> </package> EOF cd $TMP_DIR/$REPO_NAME osc up cd $TMP_DIR/$REPO_NAME/$PKG_NAME rsync -av --delete --exclude=".osc/" $PKG_DIR/$PKG_NAME/ . osc addremove osc ci -m "init" } 

The function registers in the OBS package
goes to the local folder with the project (/ home / tmp / home: admin: obs-fast-c7ffmpeg /),
updates the package folder
synchronizes data with OBS and makes commit.
Immediately after this, the package begins to be assembled by one of the free Werkers.
To build php-ffmpeg in Centos7 you will need very little:

 push lame push ocl-icd push x265 push x264 push xvidcore push opencl-headers push ffmpeg push php-ffmpeg 

assembly process can be controlled differently

Where to get the material for assembly?

The easiest way is to use ready-made from src.rpm fedora (search here https://www.rpmfind.net/ ).

 ftp://195.220.108.108/linux/fedora-secondary/development/rawhide/source/SRPMS/o/ocl-icd-2.2.8-2.git20151217.0122332.fc24.src.rpm ftp://195.220.108.108/linux/fedora-secondary/development/rawhide/source/SRPMS/o/opencl-headers-1.2-9.fc24.src.rpm (centos 6) ftp://195.220.108.108/linux/sourceforge/u/un/unitedrpms/repository/srpm/lame-3.99.5-5.fc24.src.rpm ftp://195.220.108.108/linux/sourceforge/u/un/unitedrpms/repository/srpm/x264-0.148-2.20160420git3b70645.fc24.src.rpm ftp://195.220.108.108/linux/sourceforge/u/un/unitedrpms/repository/srpm/x265-1.9-1.20160221git40ba1eb.fc24.src.rpm ftp://195.220.108.108/linux/sourceforge/u/un/unitedrpms/repository/srpm/xvidcore-1.3.4-1.fc24.src.rpm ftp://195.220.108.108/linux/sourceforge/u/un/unitedrpms/repository/srpm/ffmpeg-3.0.1-2.fc24.src.rpm 

In some cases, minor modifications of specs were required, in which case the original file has the extension .orig.

* Centos 6

The system is good, reliable) but, the trouble is, the package base is outdated for building many packages of current versions.
To successfully build ffmpeg, you will need to additionally build ocl-icd-devel, soxr-devel, and their dependencies.
But the most difficult thing is to update several important system packages: autoconf, automake, m4, ... and they are interdependent.
Since this is a one-time task and we are lazy, we will use https://pkgs.org/ to search for the necessary packages.
Add the OBS found in the turnips and restart the service:

 cd /srv/obs/build/centos6/epel/x86_64/:full/ wget http://springdale.math.ias.edu/data/puias/computational/6/x86_64/ocl-icd-devel-2.2.8-1.git20151217.0122332.sdl6.x86_64.rpm wget http://springdale.math.ias.edu/data/puias/computational/6/x86_64/ocl-icd-2.2.8-1.git20151217.0122332.sdl6.x86_64.rpm wget http://springdale.math.ias.edu/data/puias/6/x86_64/os/Addons/soxr-devel-0.1.1-3.sdl6.x86_64.rpm wget http://springdale.math.ias.edu/data/puias/6/x86_64/os/Addons/soxr-0.1.1-3.sdl6.x86_64.rpm rcobsscheduler restart obs_admin --deep-check-project centos6 x86_64 

How to use

For starters, you need to forward ports. On 443 hangs a web-muzzle, on 82 - a repository.
The most simple and flexible option is to install nginx and add two lines to the default config:

 # grep location\ / -A3 /etc/nginx/nginx.conf location / { proxy_pass http://192.168.122.218:82/; proxy_redirect off; } # 

Next, you need to create the file /etc/yum.repos.d/obs-fast-c7ffmpeg.repo with the following content:

 [obs-ffmpeg-c7] name = obs-ffmpeg-c7 baseurl = http://123.123.123.123/home:/Admin:/obs-fast-c7ffmpeg/CentOS_7/ gpgkey = http://123.123.123.123/home:/Admin:/obs-fast-c7ffmpeg/CentOS_7/repodata/repomd.xml.key gpgcheck = 1 

Lastly.

For those who need to build packages at no cost, you may like this resource https://build.opensuse.org/ .
Among the minuses:

• outdated OS package base. For example, with the current Centos 6.6, the build was based on Centos 6.3 (and there openssl is very old, etc.)
• there is no possibility of connecting third-party repositories (the same epel-release)
• multiple package restrictions https://en.opensuse.org/openSUSE:Build_Service_application_blacklist

Useful links:
https://ru.opensuse.org/openSUSE:OSC
https://en.opensuse.org/openSUSE:Build_Service_private_installation
https://en.opensuse.org/openSUSE:Build_Service_private_instance_software_live_cycle
https://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks
https://en.opensuse.org/openSUSE:Build_Service_adding_build_targets
https://en.opensuse.org/openSUSE:Build_Service_Concept_CrossDevelopment