In a section: the news aggregator on Android with backend. Sequential integration system

Introduction (with links to all articles)

“The system of sequential integration” is not sure that the translation is correct - it is better to call the system of continuous integration (continuous integration).
I encountered them for the first time at the beginning of my work, when 5-7 programmers intensively wrote code and tests, changed configuration files and poured all their results into trunk / master. As a result, quite often (so often that already enraged) something non-working appeared in the main branch. Moreover, this was revealed when it was necessary to deploy a test bench, which greatly slowed down the work of the testing team (they waited for the fix) and the developers (corrected accordingly). Those. Code performance was not monitored after it was placed in the repository.


Then Hudson (http://hudson-ci.org/) came to the rescue (now better known as Jenkins (https://jenkins.io/) , although formally Hudson himself remained, but not so popular and not so actively developing ) - he did a lot of things:

• performed the assembly and running of unit tests after each commit;
• after committing to the release branch, he killed the old one and launched a new test bench for the team of testers;
• generated reports on source codes (tesami coverage, test compliance standards);
• generated wiki documentation.
• besides, through its interface it was possible to track who made what commits, how the test coverage changed over time, how long the tests were built and run, and most importantly, the build was broken.

This was a real step forward in terms of improving the quality and transparency of the work of the development team (I'm not talking about minimizing the likelihood of “porridge” in master / trunk):

• it was possible to find out what typical mistakes were made when working with the version control system (someone was punished, someone was trained);
• To prevent the joint editing of one large file with configurations, it was divided and the build logic was changed;
• the rules for working with branches were developed and fixed (previously they worked according to the principle “after all, everyone understands ...”, ie there were no official rules).

')
But it was all before. Now the question is: what is the point of using a system of continuous integration when working alone?
Until a certain point there was really no meaning. Until the project grew (even with the participation of a single developer) and then the presence of the integration server showed its real value:

• Due to the fact that the stand for launching integration tests is formed using configuration files that contain versions of other components, which are then pulled up by the storage systems of ready-made artifacts (based on Nexus Sonatype), it is quite important to correctly specify the versions of the components and check them together performance. However, this check is a time-consuming thing (time to download artifacts, build a stand, launch, run integration tests). Therefore, on the integration server, I configured the task to my work branch, into which I sometimes merge working options and continue working. After some time I can check whether my assumption about working capacity is true or not;
• running unit tests for a local cluster with Apache Storm , even with the substitution of all components via Dependency Injection on Stub / Mock, takes a fair amount of time and I sometimes forget to run them all entirely, which leads (just sometimes) to idle code in the master - when the presence of CI is detected immediately after “git push”;
• (not systematically, but a couple of times), taking into account the presence of not only code in git, but also configuration files and data files, when adjusting “.gitignore” there were situations when the necessary files were excluded from the repository along with the unnecessary ones. At the same time, they were saved on my machine and the tests worked, but on the CI server, this fact was instantly tracked.

Additional interesting features of Jenkins are its interface, allowing you to monitor:

• runtime pipeline and its stages
  ;
• graphical reflection of the number of fulfilled / missed / fulfilled tests
  ;
• tabular reflection of the number of fulfilled / missed / fulfilled tests
  ;
• detailed data on the latest commits that caused the build
  ;
• detailed data on the commit that caused the specific assembly with the ability to go to the same GitHub for a detailed study of the changes
  .

Pipeline

The most significant functional improvement of Jenkins after some time of not using it was for me the appearance of the Pipeline plugin, which allows either through the interface of Jenkins itself or through a separate configuration file of the project itself ( Jenkinsfile file) to determine what will be executed / started / checked CI server when getting the code of your project / module.

This approach allows separating the repository polling configuration steps (performed in the Jenkins interface) and configuring the testing / integration procedures themselves (the Jenkinsfile file), allowing the developer or person responsible for the deployment to configure / modify / add the necessary testing / integration stages without access to Jenkins.

For example, in one of my sub-projects, the Jenkinsfile file has the following content:

 #!groovy node { def projectName = 'glr_parser' def gradleHome = tool name: 'gradle', type: 'gradle' stage('Checkout') { // for display purposes // Get some code from a GitHub repository // parent directory checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'CleanBeforeCheckout']], submoduleCfg: [], userRemoteConfigs: [[url: 'story_line2_build.github.com:fedor-malyshkin/story_line2_build.git']]] // project itself checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: "${projectName}"]], submoduleCfg: [], userRemoteConfigs: [[url: "story_line2_${projectName}.github.com:fedor-malyshkin/story_line2_${projectName}.git"]]] // deployment // checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'deployment']], submoduleCfg: [], userRemoteConfigs: [[url: 'story_line2_deployment.github.com:fedor-malyshkin/story_line2_deployment.git']]] } stage('Assemble') { dir(projectName) { // Run the maven build sh "'${gradleHome}/bin/gradle' -Pstand_type=test assemble" } } stage('Test') { dir(projectName) { try { sh "'${gradleHome}/bin/gradle' -Pstand_type=test test" } finally { junit "build/test-results/test/TEST-*.xml" } } } } 

As you can see, this is actually a slightly upgraded groovy (DSL for Jenkins). Pilpeline consists of stages, steps and nodes. At the same time, the stages are reflected in the Stage View Plugin, which everyone loves to show in presentations and other performances.

At the same time, the developer has the right to determine for himself whether he plans to limit himself to the verification stage: launching unit tests, verification of coding standards, launching integration tests, or placing tested artifacts into a production unit (thus implementing the principle of Continuous Delivery).

Tips

• The problem with setting up Jenkins tasks for me was the need to provide read access to the closed GitHub repository (I didn’t have the slightest desire to fill passwords with accounts into the settings), this was solved using GitHub Deploy Keys ( see my earlier article )
• To track the status of tasks on a mobile device (Android) I use a surprisingly good client with the obvious name Jenkins .
  A couple of screenshots:
  
  
  
  
• There is also an interesting thick client for the PC to track the status of the assembly of CatLight (but there is no experience of use).

At the moment, from the Jenkins minuses, we can note only insufficient documentation on the product: often in the documentation section on the developer’s website you can see the inscription: “This is still very much a work in progress” . However, this should not repel from a truly flexible and powerful product.

Thanks for attention!