How to help manual tester. Automators rush to the rescue

In any team that devotes due time to testing, the time comes when a question is asked about how to automate this process. How does this happen? There are several ways for development: either the testers themselves begin to automate, or a specially trained person is hired, who, as a panacea, has to solve all the problems. Regardless of how this happens, in the end we are all confronted with the need to somehow show what is happening, what is the reality - what has been done. As one colleague said, “automation for the sake of automation is like the cult of cargo,” as it happens that the automation department exists, but there is no result.

So, the main task of an automation engineer is to make life easier. This time, we are going to simplify our life to the manual testing department (if there is one) or to ourselves if the whole testing process is placed on our shoulders.

Little structural data

In our company there is a department of manual testing and automation. As a system for organizing tests, a fairly popular tool is used - TestRail. From my point of view, comfortable and quite functional.
Automation is also built on a fairly standard set of Ruby + Cucumber + Watir / Selenium (you can mention the Page Object pattern) + TeamCity.

What happens when a new build is given for testing, (in our case, every time we deal with regression)? The tester creates a new test of wounds, which includes all test cases from a specific test suite and the fun has gone. I am sure that everybody is familiar with the feeling when you for the 4th time manually drive out a registration or a smoke on the machine, clicking / tapping on all the elements and putting down the status for the next test. At this moment, before the red eyes, surely everything floats and the picture in the head repeats the famous:
')


And sometimes:



Right now we come to the rescue. It so happened that an idea arose. If we have automation, then why are we still not using the results of our work to make life easier? The idea is to use a report with TestRail instead of bulky and incomprehensible reports with Cucumber. Quite an interesting task - to make the tests in TestRail themselves change their status, depending on how the autotest passed.

Using the search for the omnipotent Google, TestRail API documentation was found to implement this good purpose. We decided to start to make sure that by launching our autotests all the information about the current status of the tests was displayed in TestRail. Actually, the goal was achieved - by clicking on the test run button in RubyMine, we automatically created a new testran and sent the results to the server. Quite simply, given the existing information on the site TestRail.

As it turned out, this was just the beginning.

In the end, we managed to make quite a decent functionality, namely, we set up the TestRail + TeamCity + Automation Framework integration.

Now the details, ladies and gentlemen.

Testray

Our first stop will be TestRail.

TestRail is a software for managing the data obtained as a result of testing. This tool helps you monitor processes, manage software, and organize your team.

With TestRail, you can create test cases, manage test suites, and coordinate the entire software testing process. TestRail provides an opportunity to improve performance and get a complete overview of the testing process.

We start with the fact that the first thing that our user / manual tester should do is create the actual testrane, which later we will use for updates.

Requirements:
Feature: Running AutoTests.
To use automation in real life.
As a user, I want the magic button “run wounds with auto-tests”.



No sooner said than done. The benefit of the TestRail functionality allows us to integrate our own code.

As a result, we have a button like this:



Yes, yes - Start Tests.


Since we have test cases for the web and mobile for one suite, then at the first stage we check, by the name of the test, which framework is used. Depending on what the test is intended for, we run the build on TeamCity.

In order for the user not to click too often on the button, we have organized fool protection - after clicking on the button, we add the key “in progress” to the name of the test, it blocks the magic button until our auto tests finish their work.

Automation Framework

At the final stage, the gem / library was created, which, when deployed, gives us integration with TestRail on any of our sub-projects.

Creating a heme is a different story, worthy of a separate article.

In short, our library test_rail_integration has a small functionality that we use in ourselves, but it seems to me that someone can also be useful.

First you need to install it:

 gem install test_rail_integration 

Next, add:

 TestRail::Hook.update_test_rail(scenario) 

In the after | scenario | hooks. And in the env.rb file:

 if ENV['TESTRAIL'] == '1' puts 'Option : TestRail [ON]' require "test_rail_integration" require 'test_rail_integration/generator/test_rail_hooks' end 

Here are the commands to run:

 test_rail_integration check_test_run_and_update 

In our case, the tests were run for 6 different localizations and all the results were displayed in 1 test country. Sometimes there was a situation when two updates came at the same time. It turned out that they did not see each other and after the feil came pass status. This option changed the overall picture of the status of the test. In general, this team passes all the tests and checks that it complies that there are no red results in the green tests. If there is, it changes the status of the test to red.

 test_rail_integration create_test_run 

Everything is simple here, with this command we create a test with the specified name in the suite project, which we wrote down in the config file. The command returns the number of the created tesan:

 test_rail_integration perform 

You need to write when you first start, since this command generates a configuration file to which you need to specify the necessary information about TestRail:

 test_rail_integration shoot 

This is the core funtional with flags:

 --test_run_id 

Here and everything is clear, you need to specify the number of test wound:

 --venture 

and

 --env 

Specific attributes that will be used when creating a command to run (if, for example, you need to run your tests through a specific command, then you need to write it to the config file using these variables, then we need to specify them at startup):

 --showroom 

This flag is also specific and, I think, no one else will come in handy:

 --command 

Here you can specify a new command for the current launch:

 --auto 

Since we use all the integration for 6 localizations, then by the number of the test, we look for the name and parse it on the parameters so that the test should have the name, say, DT SG staging. Here we find the necessary information to run.

 --simple 

Running without searching and checking for any parameters is the most useful command for you, dear reader.

 --type 

You can also determine the type number of tests that will be run.

The full command to run tests with an existing test will look like this:

 test_rail_integration shoot --test_run_id 1 --simple 

Or else:

 test_rail_integration shoot --test_run_id 1 --simple --command <cucumber -t > --type 3 

TeamCity

Here, strictly speaking, everything is simple. We just write the command in the same way as we would launch it locally, unless we can add several variables for convenience and combine the order of launching commands (in case we start the build automatically, first you need to specify the command to create a test, then start the tests with the number which came to us earlier).

That's all. Since this is my first experience with writing an article, criticism is welcome. I hope that I wrote all this quite understandable and this manual will be useful to anyone.


Repository link: github.com/Kirikami/test_rail_integration
Library link: rubygems.org/gems/test_rail_integration