Aida. Automation of work with Git, JIRA and TeamCity

When developing and testing a product, a lot of routine work appears. To avoid human error, we use AIDA.

AIDA (English Automated Interactive Deploy Assistant) is an account that greatly facilitates work with Git, TeamCity and JIRA.
Today we will talk about how with its help we managed to automate many working processes.

First of all, we’ll recall the version control system used in Badoo, then we’ll tell you how the release branches were automated and the branches were automatically merged into Git, we’ll talk about AIDA’s substantial assistance in working with JIRA (monitoring and changing the status of tasks, filling ) and TeamCity (continuous integration and deployment to a test environment).

Git workflow

Badoo uses Git as its version control system. The peculiarity of our model is that each task is developed and tested in a separate branch. Its name consists of a ticket number in JIRA and a free description of the task.
We collect and test the release from a separate branch into which we merge the completed tasks. Since the code on the combat servers is laid out twice a day, two new branches are created for release.
The name of the release branch is simple: build_ {release date} _ {time}
')
From this name, the whole team knows the date and time of release. Also at this time oriented hooks prohibiting changes to the release branch. For example, developers are not allowed to add a task to a release branch two hours prior to laying out on combat servers. Without such a restriction, the QA team would not have time to check all the tasks in the release branch.

The master branch for us is a copy of production. As soon as build updates the code on the servers, it merges into the master branch. Also from this branch are fixed urgent fixes on the site.

This scheme is shown in the picture:



Testing goes in several stages:

• Devel is the first stage of testing, each task is checked on the development environment, it is looked at on test bases and unit tests are run.
• Shot - a task is checked in a combat environment. Physically, this is the folder on the server where the repository branch is being downloaded and nginx is configured; has its first level domain - .shot. At this stage, translations into the main languages ​​are generated.
• Staging - the release is tested in a combat environment, translated into all languages, fully monitored. All tasks are checked again.

If there is something wrong with the release task, then we roll back the branch with it using git rebase. This function is not used by chance: revert does not suit us, since after a rollback, the release branch will merge into master, and the branches with tasks are constantly updated from it. As a result, when the developer of the problem task updates his branch, his code disappears, and you have to revert to the commit revert .

AIDA and Git

Due to the large number of branches in the model described above, many tasks arise on merging branches, forming a release and monitoring the code. Consider their automatic solution:

• First of all, AIDA creates a build branch. It “listens” to the master branch, and if the previous release branch merges into master, then a new branch is created.
  
• Then, in a new release branch, every minute the branches merge with the tasks already completed, tested and corresponding to the pattern in JIRA. If a conflict occurs during the merge, the developer and the release engineer receive a notification about it, and the task is redirected to the developer.
  
• Since the master branch is a copy of the code laid out on the combat servers, and changes are added to it, it must be constantly merged with the release branch. AIDA performs this merge if there is a change in the master branch. If there is a conflict, a corresponding message appears.
  
• If, after merging with the release branch, the developer adds a change to the branch with the task, then this commit will be caught and AIDA will report this.
  

AIDA and JIRA

We use JIRA to control development, release, and testing. Workflow in all projects is fully developed and formalized. In the process of developing and testing, part of the work in the bug tracker is performed by AIDA.
Basically, it helps us move tasks or displays one or another information in them. Here are some examples:

• developer fixes code changes in the central repository, ticket status automatically changes from Open to In Progress;
• if Shot is created for the ticket (the code is laid out in a separate combat environment), then the task status automatically changes to In Shot;
• The ticket is automatically reopened when the task is rolled back from the release branch;
• if the task has passed the code verification procedure and then a change in the branch follows, then the ticket is returned to the check.

It also helps to change the status of the task if there are certain resolutions:

• Fixed & Deploy on Production - the ticket automatically changes its status to On Production

AIDA informs us about all its actions with tasks.
This automation greatly simplifies the work with workflow and eliminates the routine actions.

AIDA and TeamCity

When building and automating on a test environment, we wanted to completely get rid of routine actions, but we had to manually write the changing names of the release branches in the CI server projects. At the moment, TeamCity catches changes in all branches according to a given mask (in our case it is the build_ * mask) and starts the build. Unfortunately, there is one problem: we cannot remove the default branch from the project configuration, and if changes come in there, we simply ignore it. In this case, the branch is pulled out, and we lose resources and time. We really hope that the developers of the CI server will soon fix this problem.
Links to tickets:
Support branch specification in the VCS trigger rule
Ignore default VCS branch

Let's see what happens with the build and automatic deployment:

1. The project in TeamCity is configured for branches with a build_ * mask.
2. After the changes in the release branch, the automatic build starts.
3. If successful, the deployment script starts on test servers.
4. With the help of quick smoke tests (using a simple curl), the test environment is checked.
5. If the tests do not pass, the release version is marked as bad and rolls back to the previous (good) release version.

The whole procedure takes three minutes. Tests only reveal fatal errors. In this case, all unit-, auto- and functional tests are run in parallel.
This is done so that the tester sees the task as quickly as possible in the test environment.

Results

Let's see what actions AIDA automates with us:

1. It works with Git, creates branches, merges them, warns us if something goes wrong.
2. Interacts with JIRA, automatically changes statuses and updates information in tasks.
3. Helps TeamCity run scripts, tests and deploy to a test environment.

AIDA can do a lot, but we don’t stop there and want to teach our assistant the following actions:

• automatic application of patches for Git from a special interface for collecting, controlling, recording and formalizing patches, with a full list of information and automatic notification;
• fully automatic rebase of tasks from the build branch by changing the ticket status in JIRA;
• automatic start of unit-tests for each branch of the task at the moment of its completion and after changing the status in the bug tracker with the subsequent report in JIRA.

If you are interested in a more detailed story about each type of automation, write about it in the comments and we will be happy to continue the series of articles on this topic.

PS Create yourself good helpers who won't let you down in difficult times!