Continuous Integration in 10 lines of code or why BuildBot, Jenkins, TeamCity and the like are needed
The note is designed for those who already know what Continuous Integration is, but have not yet chosen which system to implement in themselves.
You can read what CI is and why to use it, on Wikipedia and here on Habré: article 1 , article 2 , tag CI .
And I will tell you what you should pay attention to when choosing a CI for your project, why you should use a ready-made third-party system and you should not get involved in writing your own “bicycle”.
')
It began with the fact that in one IT-company there was a conversation between colleagues from neighboring departments:
K1: Do you have continuous integration?
K2: Yes, tests are run for every commit on the trunk.
K1: What do they work on?
K2: Own script. Now go to Buildbot.
K1: Maybe I don’t understand something, but what’s so complicated? Up, run the tests, send the result, why some Buildbot, is it easier to write yourself?
Similar reasoning - “yes, why is there some kind of continuous integration, what’s complicated there, now we’ve gotten the script” –– I’ve met quite often, so I want to show with an example what most likely will be missing in a simple “knee-length” version.
So, we write "your little scriptic". I managed to keep within 10 lines, including shebang, a task in crontab and setting up sending letters.
Script micro_ci.sh, do not forget to make the file executable:
Create a directory / tmp / micro_ci, in / tmp / micro_ci / wc check out the project.
File /etc/cron.d/micro-ci:
Done! Despite the toy size, this "MicroCI" can work and be useful.
And of course, it lacks a lot in comparison with “older” systems. Let's look in more detail.
MicroCI behavior (line 6 in the script): exactly one kind of “build” -
Desired behavior: the ability to customize different types of builds that would be launched independently or sequentially one after another (Smoke tests, unit tests, building artifacts for display, etc.)
MicroCI behavior (lines 3-5 in the script): work with exactly one svn repository - the one from which the working copy
Desirable behavior: the ability to easily set up builds for several repositories, including in different VCS (svn, git, mercurial, etc.)
MicroCI behavior (line 4 in the script): work with exactly one branch - the one that has been cast into the working copy / tmp / micro_ci / wc.
Desired behavior: the ability to run tests and builds on some or all of the branches in the repository.
MicroCI behavior (line 2 in crontab): runs every 10 minutes and tries to get updates from the repository.
Desirable behavior - a variety of schedule options, including different for different builds:
MicroCI behavior (line 2 in the script): runs exactly on one machine, in the same working copy, no parallel work is provided.
Desirable behavior: the possibility of parallel builds in different builders, running the workers on different servers (for load distribution and for testing work on different architectures / OS).
MicroCI behavior (line 1 in the crontab): the results of all launches are sent via e-mail to one address.
The desired behavior: the delivery of notifications to different email-addresses, as well as via jabber, irc, sms, rss - everything that is convenient.
MicroCI behavior (line 1 in crontab): the entire output of the assembly gets into the letters, and nothing else.
Desired behavior: adaptive notifications (different for successful and unsuccessful builds) with comments about the test run system itself and links to the web interface.
MicroCI Behavior (line 1 in crontab, line 6 of script): emails are sent after each test run.
The desired behavior is the ability to flexibly configure the conditions for sending notifications:
The MicroCI is missing.
If an assembly generates new files in a working copy, they should be automatically deleted before the next run.
The MicroCI is missing.
The desired behavior: the ability to restart the tests on one of the past revisions - in case it seems that the assembly fell due to temporary environmental features.
The MicroCI is missing.
The desired behavior is the ability to easily integrate with other tools used in the team: monitoring, messaging, wiki, internal blog, etc.
The MicroCI is missing.
The desired behavior: you can download the patch to the CI system before the commit, the system will itself impose it on the project code and run all the necessary assemblies and checks. This is especially useful if tests are performed under different architectures or OS versions, and their launch by the developer is difficult.
TeamCity has this mode called Remote Run, Buildbot has trial builds, Jenkins has Patch Parameter Plugin.
The MicroCI is missing.
Desirable behavior: the CI system has its own web interface where you can see the build history, statistics, etc.
It is convenient when the CI system is distributed under a free license, and if necessary, you can make your own changes to its code.
MicroCI is written in the shell, it is very compact, but very inconvenient for the further development of the program.
If it’s likely that you will be editing the CI code, writing plugins and modules for it, then all other things being equal, you should choose a convenient and powerful programming language in which you already have good expertise.
Of course, our MicroCI has some advantages: it is very, very simple, the “installation” is trivial, no third-party software is required. But if you try to use it or a similar independent development in a real project, sooner or later you will have to implement all or almost all the features listed above. Are you ready for this? If you are not sure, take a third-party CI and do not add yourself work on the development of a new auxiliary project in addition to the main one.
And finally, some popular CI systems: Atlassian Bamboo , Buildbot ,CruiseControl (crossed out as a development stop), Jenkins , TeamCity , Travis CI (for github public repositories).
A large list is on Wikipedia .
If some important and useful CI features are not included in the review - write in the comments.
Have a good choice!
You can read what CI is and why to use it, on Wikipedia and here on Habré: article 1 , article 2 , tag CI .
And I will tell you what you should pay attention to when choosing a CI for your project, why you should use a ready-made third-party system and you should not get involved in writing your own “bicycle”.
')
It began with the fact that in one IT-company there was a conversation between colleagues from neighboring departments:
K1: Do you have continuous integration?
K2: Yes, tests are run for every commit on the trunk.
K1: What do they work on?
K2: Own script. Now go to Buildbot.
K1: Maybe I don’t understand something, but what’s so complicated? Up, run the tests, send the result, why some Buildbot, is it easier to write yourself?
Similar reasoning - “yes, why is there some kind of continuous integration, what’s complicated there, now we’ve gotten the script” –– I’ve met quite often, so I want to show with an example what most likely will be missing in a simple “knee-length” version.
So, we write "your little scriptic". I managed to keep within 10 lines, including shebang, a task in crontab and setting up sending letters.
Script micro_ci.sh, do not forget to make the file executable:
#!/bin/sh cd /tmp/micro_ci/wc && mkdir /tmp/micro_ci/lock || exit 2 rev=`svn info |grep 'Last Changed Rev' |sed 's/.*: *//'` svn up if test `svn info |grep 'Last Changed Rev' |sed 's/.*: *//'` = $rev ; then rmdir /tmp/micro_ci/lock ; exit 0 ; fi make test || status=$? rmdir /tmp/micro_ci/lock exit $status Create a directory / tmp / micro_ci, in / tmp / micro_ci / wc check out the project.
File /etc/cron.d/micro-ci:
MAILTO=project-tests@company.ru */10 * * * * tests /path/to/micro_ci.sh Done! Despite the toy size, this "MicroCI" can work and be useful.
And of course, it lacks a lot in comparison with “older” systems. Let's look in more detail.
Different builds
MicroCI behavior (line 6 in the script): exactly one kind of “build” -
make test (or another that is hard-coded in the script).Desired behavior: the ability to customize different types of builds that would be launched independently or sequentially one after another (Smoke tests, unit tests, building artifacts for display, etc.)
Work with repositories
MicroCI behavior (lines 3-5 in the script): work with exactly one svn repository - the one from which the working copy
/tmp/micro_ci/wc .Desirable behavior: the ability to easily set up builds for several repositories, including in different VCS (svn, git, mercurial, etc.)
Work with branches (brunches)
MicroCI behavior (line 4 in the script): work with exactly one branch - the one that has been cast into the working copy / tmp / micro_ci / wc.
Desired behavior: the ability to run tests and builds on some or all of the branches in the repository.
Assembly schedule
MicroCI behavior (line 2 in crontab): runs every 10 minutes and tries to get updates from the repository.
Desirable behavior - a variety of schedule options, including different for different builds:
- for each revision;
- by timer;
- weekly;
- when changing certain "interesting" files;
- etc.
Distributed work
MicroCI behavior (line 2 in the script): runs exactly on one machine, in the same working copy, no parallel work is provided.
Desirable behavior: the possibility of parallel builds in different builders, running the workers on different servers (for load distribution and for testing work on different architectures / OS).
Notification delivery method
MicroCI behavior (line 1 in the crontab): the results of all launches are sent via e-mail to one address.
The desired behavior: the delivery of notifications to different email-addresses, as well as via jabber, irc, sms, rss - everything that is convenient.
Notification Format
MicroCI behavior (line 1 in crontab): the entire output of the assembly gets into the letters, and nothing else.
Desired behavior: adaptive notifications (different for successful and unsuccessful builds) with comments about the test run system itself and links to the web interface.
Terms of sending notifications
MicroCI Behavior (line 1 in crontab, line 6 of script): emails are sent after each test run.
The desired behavior is the ability to flexibly configure the conditions for sending notifications:
- at every launch;
- failure;
- at a share of the failed tests above a certain threshold;
- when changing the state of the assembly: at the first failure after a succession of success and at the first success after a strip of failures;
- about the success of sending a notification only to the general mailing, about the failure - in addition to the author of the problem commit.
Clearing a working copy between runs
The MicroCI is missing.
If an assembly generates new files in a working copy, they should be automatically deleted before the next run.
Forced restart of tests on a specific revision
The MicroCI is missing.
The desired behavior: the ability to restart the tests on one of the past revisions - in case it seems that the assembly fell due to temporary environmental features.
Integrability with other intranet services
The MicroCI is missing.
The desired behavior is the ability to easily integrate with other tools used in the team: monitoring, messaging, wiki, internal blog, etc.
"Preliminary" assembly
The MicroCI is missing.
The desired behavior: you can download the patch to the CI system before the commit, the system will itself impose it on the project code and run all the necessary assemblies and checks. This is especially useful if tests are performed under different architectures or OS versions, and their launch by the developer is difficult.
TeamCity has this mode called Remote Run, Buildbot has trial builds, Jenkins has Patch Parameter Plugin.
Own web interface
The MicroCI is missing.
Desirable behavior: the CI system has its own web interface where you can see the build history, statistics, etc.
License opensource
It is convenient when the CI system is distributed under a free license, and if necessary, you can make your own changes to its code.
CI language
MicroCI is written in the shell, it is very compact, but very inconvenient for the further development of the program.
If it’s likely that you will be editing the CI code, writing plugins and modules for it, then all other things being equal, you should choose a convenient and powerful programming language in which you already have good expertise.
Conclusion
Of course, our MicroCI has some advantages: it is very, very simple, the “installation” is trivial, no third-party software is required. But if you try to use it or a similar independent development in a real project, sooner or later you will have to implement all or almost all the features listed above. Are you ready for this? If you are not sure, take a third-party CI and do not add yourself work on the development of a new auxiliary project in addition to the main one.
And finally, some popular CI systems: Atlassian Bamboo , Buildbot ,
A large list is on Wikipedia .
If some important and useful CI features are not included in the review - write in the comments.
Have a good choice!
Source: https://habr.com/ru/post/229873/
All Articles