TestRail: how to think first and then test

I hope many of you know that there is documentation in testing too. And you can conduct it in an old school, in xls-files and in daddies, or, like real Jedi, use a specialized tool.
This is one of such tools called TestRail , which we recently implemented in TestLab² , I want to tell you today. The tool was so successful that there was no silence and I decided to finally do something useful for society.



Brief official product summary:

• Official site: http://www.gurock.com/testrail/
• Purpose: maintenance of test documentation
• License: paid product from € 159 *
• Status: release 1.0
• Platform: LAMP

')
For all projects that we test, test documentation is always necessarily written, from a small checklist to a spreading test suite with cross-references and hundreds of cases. The idea to keep the documentation in local files was refused immediately and completely, it is inconvenient and simply unprofessional. We tried a “universal” solution, wiki (DokuWiki), but it did not catch on. It’s hard to say why, it’s probably a matter of habit, and it would be possible to “plant good with sticks”, but then TestRail turned up and rushed.

Short disclaimer in terminology:

• Test case - an atomic element of the test plan, consisting of the following components:
  
  • Description of the environment (environment) in which the case is executed (can be moved up a level to the test plan)
  • Execution steps
  • Expected outcome
• Test suite - a list of case-s, united by some common factor (the whole product, specific feature, etc.)
• Test plan - a list of test suites selected for the test
• Test run - pass (“run”) test plan

Installation and system requirements

Two options: typical LAMP or "IIS + SQLServer + FastCGI / PHP". IonCube and cURL are also required, but this is not uncommon. It works quite quickly, and after turning on gzip via mod_deflate, it became generally a bullet to fly.

Main features

A multi-user multi-project system for maintaining test documentation and recording test results. The product provides a place for everyone: for test managers who are more interested in "when there will be" and "how many bugs", for simple testers who can easily report on the completed cases, for analysts who will be able to bite into the wilds and find out what it is everything is so bad what the salt .

How does the workflow

1. A project is being created, the “Defect View Url” and “Defect Add Url” are specified for it. Integration with the bug tracker is simple and, at the same time, brilliant: your bug tracker.com / page-view /% id% , and TestRail itself substitutes this% id% of the bug. Fits most online bug trackers.
2. Milestone is created: in fact, the version of the product being tested consists of a name and a deadline.
3. Test Suite is created: a set of test cases, for example, UI Testing.
4. Inside the Test Suite, cases and, if desired, case sections are created: for example, in the “UI Testing” there are two sections “Client UI” and “Admin UI”. Sections support unlimited nesting.
5. Each specific test case consists of Preconditions, Steps and Expected Result. Here, in my opinion, everything is obvious. You can attach pictures to them and cross-link between them.
6. After you have described the system under test with the help of test cases, it's time to start the tests themselves. To do this, you create a Test Run, which includes the necessary Test Suite or specific test cases. He is assigned a performer (from active users of the system) and at this stage the work of the test designer ends.
7. The tester, having received a notification that a test run has been placed on it, begins its execution. Each test case is given Passed, Failed or Retest status. Comments and IDs of found bugs are also indicated there.
   After passing all the tests, the test run closes, the results are left to the task designer, and all the data on how the testing passed is left in the system.
8. At the request of the customer / user, all results can be exported to digestible formats, prepared for printing, or even copied to xml for some particularly perverse use. The system builds a small number of graphs that allow a quick assessment of the dynamics of the test run results and understand where everything is heading.

Comparison with competitors

Of the alternatives, I used to come across two systems: Rational Quality Manager and TestLink . Forgive me open-source advocates, TestLink never again want to see. Of course, it is free, yes, this is a weighty argument, but this is where its merits end. The interface is old and inconvenient, an ill-conceived system of references to specific cases or suites, the very structure of the suites ... in general, in my (and all-team) opinion, “unsuitable for service in wartime”.
The whole Rational Suite is good, but it is too massive for small projects and very much imposes its own methodology. Not to mention the fact that it stands as a wholesale batch of carpet-planes.
Of those I just read about , I’ll mention a module to Bugzilla called Testopia , but unfortunately I didn’t see it live or hear about someone using it in production. If your project uses BugZilla - I think it is worth looking at Testopia, a fairly high level of integration is promised.
Also, there are still a number of similar solutions embedded in integrated development systems (the same Mercury) or desktop standalone products ( TestLog , QaTraq ). They do not suit us due to the lack of normal web-access. If someone can tell in the comments more decent web-based systems of a similar direction - I will be glad.
UPD1 Extras found (thanks astenix and Khizhnyak ):

• testplanmanagement.com
• RTH

Personal impressions

For us, the system turned out to be almost perfect in terms of the minimum required set of features, and in terms of ease of daily use. We are engaged in outsourcing and conducting a lot of small projects, which perfectly fits into the vendor’s licensing policy: the limit limits only the number of active users of the system, so we deactivate client accounts at the end of projects.
Two drawbacks are worth noting: in the current version, TestRail does not support Unicode, which means that it is impossible to maintain documentation in Russian / Ukrainian in the current version without hacks. But the corresponding feature request is already announced, must be added soon. For our team, this is unprincipled because so far all projects worthy of getting into TestRail are in English.
The second problem is the lack of any wysiwyg editor in all text edit fields. It pleases engineers, but regularly confuses customers.

---

Summary

If you keep test documentation for a large project, or are already lost in tons of doc- and xls-files with cases - you should pay attention to TestRail. This is a product that does not need everyone and everyone, but if you are testing professionally, remember this name, perhaps TestRail will be useful to you in the next project.

If it is interesting, I am ready to develop the topic of various topics for testing and, for example, carefully study the same Testopia to tell you about it. Write in the comments.

* Those who know how to google can find discount coupons on the Internet.