Issues of release management of maven projects with CI approach to software development
Having read the article
“Simple Release Management with Git”
I wanted to share a couple of my thoughts about the release management of maven projects.
In this article, I’m not so much proposing a solution to the problems, but I want to formulate them more correctly.
In short: the main problem with the release management of maven projects is the maven itself, which was not designed to meet the requirements of the CI (Continuous Integration) approach to software development.
The SNAPSHOT version paradigm is outdated since CI became the standard approach to software development.
')
More details below.
A modern view of CI, described for example in the book “Continuous Delivery”, assumes that any branch is evil.
Ideally, development should be carried out in a single trunk branch. You should avoid creating any new development branch as much as possible.
Particularly harmful are the so-called " feature branches " branches, the only purpose of which is to postpone the difficulties of integrating new features until the last moment, which is completely contrary to the principle of Continuous Integration ("continuous integration").
The only place where, from my point of view, branches can be used is the case of the " release branch ", i.e. branches created to correct errors in the production version at a time when development is running parallel to the trunk.
And even they should not be in a hurry to create. In case of emergency, a release branch can always be created later, when needed, based on the tag or revision of the version control system.
A branch should be viewed as a “parallel development line”.
If you need to conduct parallel development of several lines of code at the same time, then you need to create a new branch. In practice, this occurs [should be] extremely rare. The only case is bug fixes and service releases of previous versions, for which release branches are used. I repeat: the comfort of the developer who receives the “seeming calm” when working in the feature branch is not a sufficient argument for creating a new branch.
Therefore, from my point of view, the presence of a single trunk mentioned in the article in many open source projects is a good example to follow.
Another principle within the Continuous Delivery methodology is that each build artifact generated by the CI system must receive a unique version number. This is necessary in order not to build a project in a new way, wasting time and risking making unintended changes every time you need to transfer a certain version of code to the next stage (stage) of the development pipeline. For example, on Staging after passing UAT (User Acceptance Test) tests of a specific version in the Test environment. Instead, an artifact with a unique version should be stored in the repository of artifacts (for example, Nexus, and in no case in VCS (version control system) with the code).
If your project uses maven to manage dependencies, then the real problem with release management is that maven is not designed for the fact that each new build receives a new unique version.
The classic sequence of actions to release a new version using maven is to manually create a new release branch, update pom.xml, build a project and, if successful, commit changes to VCS.
The version confusion occurs already at this stage, since two revisions in VCS now correspond to the same artifact. The topic of the transactionality of these changes will not be raised here.
To automate these actions, you can use the maven release plugin, which nevertheless does not solve the problem in essence and does not make the maven project more suitable for CI and Continuous Delivery / Deployment.
Using SNAPSHOT versions, at first glance, can make CI builds possible. In fact, the SNAPSHOT versions further exacerbate the confusion and require additional labor for updating the pom-files, since with the release you now have to remember not to manually update not only the project version, but also all the SNAPSHOT dependencies of this project, as well as are referenced. And if you add the version ranges from Maven 3 here, then the complexity of matching all parts of the puzzle quickly exceeds the reasonable.
Ideally, the CI build system should generate the version number of the artifact itself and transfer it to all builds of the Continuous Delivery pipeline chain.
Hudson / Jenkins, for example, do not know how to do this and have two fundamental flaws that exclude them from serious consideration when choosing a CI system:
A decent CI build system easily handles the task of generating and transmitting build versions. For example, my favorite Quickbuild , which in terms of price-quality ratio, is perhaps the best choice today (I made a comparison, however, a couple of years ago; I wonder where Thoughtworks Go has moved during this time).
Unfortunately, it is impossible to transfer the version to the maven build without side effects (was it?).
The maximum that can be done:
But in this case, in pom.xml, saved in the maven repository after deployment will store
no change, which will be a problem when building projects that depend on this.
So, the problems in the way of an effective CI maven project are roughly two:
The first problem is solved by the right choice of the CI system or additional labor costs for customizing the existing one.
The second problem can be solved for example by switching to using Ant + Ivy. Ivy initially does not suffer from maven childhood diseases, however, it requires significantly longer preparation and is also more difficult to set up (although this will have to be done, perhaps, only once).
As a compromise, we now use the following approach.
In reality, the configuration of our CI jobs is somewhat more complicated, since for important interfaces that are shared by our subsystems, our own versioning and branching is maintained. Accordingly, for each such sub-project, their CI and release jobs are configured. If the CI-build system well supports the hierarchical structure of jobs, then managing multiple configurations is not a problem, since all shared settings follow the DRY principle and are stored only in one place.
Returning to the article “Simple release management with Git tools”, I want to note that the rejection of relase branches for many will become an insurmountable obstacle to the implementation of the described approach.
Every new branch is evil. The intention to minimize the number of branches once and for all is highly commendable, but when trying to support more complex use cases (such as the independent release of hotfixes), the number of non-standard live branches being used grows as well as the complexity of their use.
And the real problem, the solution of which is proposed in the article, is not the problem of organizing branches / workflow with VCS, but the problem of maven unsuitability for the Continuous Integration approach to software development.
As a result of the comments, having thought a bit about what can be offered instead of SNAPSHOT, I came to the conclusion that I would welcome the development of the maven (or another build of the system and the dependency control system) in the following directions:
“Simple Release Management with Git”
I wanted to share a couple of my thoughts about the release management of maven projects.
In this article, I’m not so much proposing a solution to the problems, but I want to formulate them more correctly.
In short: the main problem with the release management of maven projects is the maven itself, which was not designed to meet the requirements of the CI (Continuous Integration) approach to software development.
The SNAPSHOT version paradigm is outdated since CI became the standard approach to software development.
')
More details below.
A modern view of CI, described for example in the book “Continuous Delivery”, assumes that any branch is evil.
Ideally, development should be carried out in a single trunk branch. You should avoid creating any new development branch as much as possible.
Particularly harmful are the so-called " feature branches " branches, the only purpose of which is to postpone the difficulties of integrating new features until the last moment, which is completely contrary to the principle of Continuous Integration ("continuous integration").
The only place where, from my point of view, branches can be used is the case of the " release branch ", i.e. branches created to correct errors in the production version at a time when development is running parallel to the trunk.
And even they should not be in a hurry to create. In case of emergency, a release branch can always be created later, when needed, based on the tag or revision of the version control system.
A branch should be viewed as a “parallel development line”.
If you need to conduct parallel development of several lines of code at the same time, then you need to create a new branch. In practice, this occurs [should be] extremely rare. The only case is bug fixes and service releases of previous versions, for which release branches are used. I repeat: the comfort of the developer who receives the “seeming calm” when working in the feature branch is not a sufficient argument for creating a new branch.
Therefore, from my point of view, the presence of a single trunk mentioned in the article in many open source projects is a good example to follow.
Another principle within the Continuous Delivery methodology is that each build artifact generated by the CI system must receive a unique version number. This is necessary in order not to build a project in a new way, wasting time and risking making unintended changes every time you need to transfer a certain version of code to the next stage (stage) of the development pipeline. For example, on Staging after passing UAT (User Acceptance Test) tests of a specific version in the Test environment. Instead, an artifact with a unique version should be stored in the repository of artifacts (for example, Nexus, and in no case in VCS (version control system) with the code).
If your project uses maven to manage dependencies, then the real problem with release management is that maven is not designed for the fact that each new build receives a new unique version.
The classic sequence of actions to release a new version using maven is to manually create a new release branch, update pom.xml, build a project and, if successful, commit changes to VCS.
The version confusion occurs already at this stage, since two revisions in VCS now correspond to the same artifact. The topic of the transactionality of these changes will not be raised here.
To automate these actions, you can use the maven release plugin, which nevertheless does not solve the problem in essence and does not make the maven project more suitable for CI and Continuous Delivery / Deployment.
Using SNAPSHOT versions, at first glance, can make CI builds possible. In fact, the SNAPSHOT versions further exacerbate the confusion and require additional labor for updating the pom-files, since with the release you now have to remember not to manually update not only the project version, but also all the SNAPSHOT dependencies of this project, as well as are referenced. And if you add the version ranges from Maven 3 here, then the complexity of matching all parts of the puzzle quickly exceeds the reasonable.
Ideally, the CI build system should generate the version number of the artifact itself and transfer it to all builds of the Continuous Delivery pipeline chain.
Hudson / Jenkins, for example, do not know how to do this and have two fundamental flaws that exclude them from serious consideration when choosing a CI system:
- Lack of Expression Language support in Job settings
- The absence of the possibility of hierarchical building of configurations with the inheritance of plug-in settings (existing plug-ins that solve a similar problem have very little practical benefit).
A decent CI build system easily handles the task of generating and transmitting build versions. For example, my favorite Quickbuild , which in terms of price-quality ratio, is perhaps the best choice today (I made a comparison, however, a couple of years ago; I wonder where Thoughtworks Go has moved during this time).
Unfortunately, it is impossible to transfer the version to the maven build without side effects (was it?).
The maximum that can be done:
<project xmlns="..." xmlns:xsi="..." xsi:schemaLocation="..."> <modelVersion>4.0.0</modelVersion> <groupId>org.xxx</groupId> <artifactId>xxx</artifactId> <version>${ciVersion}</version> <packaging>jar</packaging> <properties> <ciVersion>0.0.0</ciVersion> </properties> mvn -DciVersion=1.1.1 deploy But in this case, in pom.xml, saved in the maven repository after deployment will store
${ciVersion}
no change, which will be a problem when building projects that depend on this.
So, the problems in the way of an effective CI maven project are roughly two:
- The difficulty of generating and transferring a chain of jobs to a unique version using the CI system.
- Impossibility to transfer the generated version in maven bild.
The first problem is solved by the right choice of the CI system or additional labor costs for customizing the existing one.
The second problem can be solved for example by switching to using Ant + Ivy. Ivy initially does not suffer from maven childhood diseases, however, it requires significantly longer preparation and is also more difficult to set up (although this will have to be done, perhaps, only once).
As a compromise, we now use the following approach.
- All development is carried out in the trunk, which is configured on the SNAPSHOT version
- CI builds the trunk as soon as it sees a new check in VCS.
- When the trunk is ready for UAT, another configuration of the / job CI system is started manually, which:
- generates a unique version of the build (for example, based on the VCS revision)
- shakes a branch under construction from VCS
- uses the maven versions plugin to update the version of the project and all sub-projects to the relase-version generated in step 1 (optionally you can update the SNAPSHOT dependencies on the release version in this step)
- builds a project
- Deploit built artifacts with the release version of Nexus
(I note that after this change in pom files will not be committed to VCS) - VCS revision is remembered or created by tag
- If the previous step is completed successfully, the associated job configuration automatically starts up and is responsible for deploying the application to the target environment (for example, the test environment), which, using EL expressions, among other parameters, transfers the version of the artifact built in the previous step. This work is launched directly on the target machine via the remote CI agent of the system, or the entire deployment can be done via an ssh connection. The deployment script is also stored in the VCS.
- If necessary, issue a hotfix or prepare a service pack, we branch the desired revision (or tag) and change the release parameters of the job CI system so that it uses the new branch and generates the appropriate major / minor versions of the built artifacts. In the case of using Quickbuild, this procedure literally boils down to correcting two or three values of the parameters of the existing configuration without the need to create a new job from the template.
In reality, the configuration of our CI jobs is somewhat more complicated, since for important interfaces that are shared by our subsystems, our own versioning and branching is maintained. Accordingly, for each such sub-project, their CI and release jobs are configured. If the CI-build system well supports the hierarchical structure of jobs, then managing multiple configurations is not a problem, since all shared settings follow the DRY principle and are stored only in one place.
Returning to the article “Simple release management with Git tools”, I want to note that the rejection of relase branches for many will become an insurmountable obstacle to the implementation of the described approach.
Every new branch is evil. The intention to minimize the number of branches once and for all is highly commendable, but when trying to support more complex use cases (such as the independent release of hotfixes), the number of non-standard live branches being used grows as well as the complexity of their use.
And the real problem, the solution of which is proposed in the article, is not the problem of organizing branches / workflow with VCS, but the problem of maven unsuitability for the Continuous Integration approach to software development.
As a result of the comments, having thought a bit about what can be offered instead of SNAPSHOT, I came to the conclusion that I would welcome the development of the maven (or another build of the system and the dependency control system) in the following directions:
- Support for "strategies" of automatic version generation. Something like: “when building, assign an artifact version of this kind, using a random number generator, VCS revision, or whatever you like” (that's where the flexible Quickbuild EL expressions come to the rescue today)
- Instead of SNAPSHOT, introduce the concept of a release build so that maven knows that it’s built and referential in this build that artifacts can no longer be removed from the repository. But others, whose version numbers do not differ in general from release versions of artifacts, can be removed over time without serious consequences. Unlike the existing system, by default all artifacts receive a unique version and are candidates for removal from the repository. If suddenly any of them was referenced from the “release” -bild, then it automatically changes to the status of a release-artifact
Source: https://habr.com/ru/post/160145/
All Articles