In this article I will tell you about how I tried to create a beta stand and embed it in a regular gitflow. Together with the readers, we will go from the problems associated with this to the new scheme of working with git.
In our company, we used the well-known gitflow. Those who know what it is can go straight to the next section. For those who do not know, I will tell.
The main work is carried out in the development branch. For each new feature, a separate feature-branch is created. When you merge a feature branch in development, the application is assembled and laid out on a test bench, where QA specialists check its work.
Under each found bug from development, a hotfix branch is created in which it is eliminated. Then the hotfix branch merges back into development - and everything is new: the test bench is updated and the QA checks again.
When the development branch is otdebazhena and it has accumulated a sufficient number of features for the release release is created. It always contains the code, which at any time can be merged into the master and thereby update the production stand.
Due to the presence of a loop in the described scheme: rolled out on a test stand, checked, corrected, rolled out again, a huge number of errors disappear. But, alas, not all.
Of course, our QA does its job well, the number of bugs decreases as we get closer to the master, but itβs impossible to eliminate them for the following reasons:
The real user behavior is much more complicated and unpredictable synthesized test cases.
The features of real users (device, OS, browser, personal settings, etc.) are not taken into account.
We decided to eliminate the indicated problems with testing with the help of the beta stand - i.e. give our internal users of the system, customers, trusted customers and other persons early access.
Now after the development of the booth, where they are checked by a QA specialist, new features fall into the beta booth, where real users work with them. They are notified of the start of beta testing immediately after updating the beta stand. Errors in the beta version of the application are displayed in the logging system. Periodically, they are fixed and the beta stand is updated. When errors no longer occur, a release is created. Thus, a wide audience of users gets a stable version of the application.
At the beta testing stage, it is possible to get feedback from users, find out whether the introduced functionality will be convenient for a wide audience and what needs to be changed. Beta is a kind of pilot version of the application.
This scheme corresponds to the main stages of development: alpha, beta and release.
We discussed the aspect of Continuous Delivery, i.e. at what point to create pre-releases and releases. It's time to switch to Continuous Integration, i.e. develop the very scheme of working with git taking into account beta.
The first thing that comes to mind is to use the release branch to deploy to beta. The release branch is then considered to be the intention of the release, i.e. in other words, it is a pre-release (almost beta). And what harmoniously turns out, and nothing needs to be changed in gitflow. You just need to wind up a new rule in the CD to lay out the build to create / change the release branch and that's it.
Such a scheme would approximately look like this:
Note: BASE commits are indicated by dashed lines on the graph.
What happens on the graph?
At first glance, the working scheme. Now let's look at its pros and cons.
Pros:
A small number of main branches that need to be kept up to date. Most of the time they will be only the 2nd: dev and master.
Minuses:
The first problem arises when you need to add a hotfix to the master, without waiting for the new release approach. Well, you can hold the hotfix in the master and then in release and dev, in general, this is not a problem (unless you can get confused with the merge). The problem appears when there is no release branch. And how to update the beta environment without a release branch, and in any way, unless manually. It is not right somehow, we have a CD scheme set up, you will say, and I will agree.
There is a time lag when adding a hotfix to the main branches. In the flow diagram, the introduction of hotfix into the master looks like this: hotfix β master β dev β release. It should be like this: hotfix β master β release β dev, since release is more important than dev and changes should appear in it earlier. Merge dev, when you make a hotfix into it, it may not immediately succeed in the release branch - dev may contain changes that should not fall into the current (open) release branch. Thus, you need to wait for the next release before the hotfix appears on the beta stand. Or, for example, if a beta requires hotfix to be such a flow: hotfix β beta β master β dev, but should be like this: hotfix β beta β dev β master. In this scheme, the principle of causality is violated.
Regression update of the main branches (release and dev) occurs through the merge commit, which complicates the overlaying of the CD scheme on the CI. Also in this scheme it is easy to get confused - a large number of degrees of freedom. For example, you can merge hotfix into master, then merge the branch into release, but forget about dev.
Automatic version assignment is not possible due to the merzh-commit commits "left". You have to manually put down the version numbers in which you can get confused. You will need to make a versioned commit manually in the correct branch. There is also a possibility that the maintainer will forget to set the tag and then the application will be downloaded with the old version, which will lead to incorrect error logging.
Let's try to get rid of these problems.
Since the beta booth is always there, you need to make sure that the release branch always exists. Let's call it beta. Then it will be possible to deposit hotfix on the beta stand in the absence of a release branch. But in this case, the problem with the excess amount of merge commit will increase as the number of branches has increased. To solve this problem, it is enough to refuse to update the main branches with the merge strategy.
Graphically display the resulting flow can be as follows.
Note: BASE commits are indicated by dashed lines on the graph.
What happens on the graph?
Note: the main branches at all stages (pre-alpha, alpha, beta ...) are updated by the person responsible for the release (project maintainer). Team members work only with the c dev branch.
The circuit looks pretty cool, doesn't it? Let's look at its pros and cons and compare it with the previous one.
Pros:
A clearer beta role in CI / CD. There will be no problems with hotfixes and pushing changes through it. Those. the principle of causality is not violated, the flow will be as follows: hotfix β master β beta β dev.
Perhaps automated versioning and collecting the changelog, which is especially important for libraries. There is no possibility of a mistake.
Tests do not run again, which speeds up the process of making changes.
No extra merge commits.
Minuses:
A large number of branches. But it's not scary because they are responsible for the release. Also, each branch has its own role in automated testing, I will write a few words about it below.
The proposed scheme fully meets all stages of software development.
pre-alpha features are made, but not yet tested by a QA specialist. They may contain a large number of bugs.
alpha - at this stage, the application is assembled and rolls out to the development stand, where the QA manual operator checks his work.
Beta - conditionally stable application build, tested by a QA specialist. The assembly at this stage is laid out on a beta stand for testing by real users.
Release Candidate - an assembly that has passed all the stages of testing and is now awaiting its release.
Our projects have a large number of automated tests. The execution of all tests takes approximately 1 hour. To speed up the adoption of PR in the branches at each stage of the implementation of the feature, we perform only the tests that are important for this stage. For example, to accept the code in pre-alpha, we run the simplest tests: lint and unit. At the beta adoption stage, integration tests are also performed. At the release-candidate stage, in addition to the voiced tests, acceptance tests are also launched. Moreover, at this stage, tests are run on runners with different operating systems and under different browsers. After the release is created (the post-release stage is not indicated in the diagram) smoke tests are run.
Ask questions in the comments if something interesting is left behind the scenes.
References:
Source: https://habr.com/ru/post/317200/