Automation of acceptance testing or FitNesse to improve the quality of a software product

The quality of the software product, not least depends on the actual documentation and thorough testing. I would like to highlight the issue of developing and testing software in general and using the FitNesse environment in particular.

Intro

When they talk about software testing, they most often imply testing done after a fair amount of code has been written and there is a need to check "if they wrote what they wanted."
It is clear that the test coverage of the code, the types and duration of testing depend on many factors, but in this case it is necessary to mention the unit tests and the acceptance tests.
If unit testing is usually performed by the person who writes one or another piece of code, then the customer usually performs acceptance testing. And then it all depends on how high the requirements of the customer (and, importantly, how much he is able to qualitatively test the finished product he takes).
And so, unit tests are usually automated on performance (they are written once and run many times in automatic mode).
And acceptance tests are usually slowly run in manual mode and are constantly changing and are usually quite rarely recorded on paper.
Why all this talk about testing? Properly organized process of testing the created software product will ultimately save money and time to eliminate errors, and moreover, increase profits from the company's good reputation.

Problem definition

Usually, when explaining the importance of testing, they like to give a graph of the exponential increase in the cost of correcting errors in a software product, depending on the stage of its detection.
')


But also we must not forget that the cost of testing, especially manual, is too high. So if we have more than 4500 tests for the run (which is normal for the acceptance testing of an average project), then we will need more than 40 man-days to perform such testing once. Now imagine that we have found an error and after correcting this error, it will be necessary to re-run 4500 tests manually.



In addition to the problems with manual testing, there is a problem with keeping the documentation up to date.
It is necessary to ensure the synchronization of the following documents: requirements, user interface specifications, test specifications and their implementation.
For the specification of user interfaces, there is a definite tendency that the documentation is eventually freed from screen shots and design diagrams in order to make it more resilient to possible errors. But at the same time, such documentation becomes unreadable and it becomes difficult to understand. It displays too many low-level scenarios (Use Cases) and it becomes difficult to maintain. All this, ultimately, results in the fact that the developers have no motivation to keep the documentation up to date.



Test specifications typically completely duplicate user interface specifications with specific names, numbers, and strings in Use Cases (scripts). They are more specific and specific than the specification of user interfaces. But still leave the opportunity for free interpretation by people of what is reflected in them.
It is difficult to keep the motivation of developers and managers to keep documentation up to date.
The ideal option would, of course, be to use the robot to manually run tests and display the results in the documentation ...



But the era of such clever gizmos has not yet come, so you have to use other methods ...

FitNesse introduction

I propose to consider FitNesse

FitNesse is primarily a collaborative software development tool.
FitNesse allows customers, testers, and programmers to learn what their software should do and automatically compare it with what the software actually does. FitNesse allows you to compare customer expectations with the result.
FitNesse is a software testing tool.
Co-define AcceptanceTests — web pages that contain simple tables of inputs and expected outputs. Run these tests and see the results.
FitNesse is a wiki.
You can easily create and edit pages.
FIT (“Framework for Integrated Testing”) is the kernel that actually processes each FitNesse table using the Fixture Code associated with this table. Designed by Ward Cunningham as an extension of the xUnit environment. Supports most modern programming languages ​​(.Net, Java, Python, Ruby, C ++, ...).

FIT + Wiki + Web Server = FitNesse



In addition to FIT, there is currently support for SLIM technology, which you can read more about on the product website.



You can give an example of what the fixture code looks like, a table for the test and the result of the environment.

Example

Here is an example of fixture code for testing a certain C # application:

using System;
using System.Collections. Generic ;
using System.Text;
using Ranorex;
namespace NetFit
{
public class AddVIPTest : fit.ColumnFixture
{
///
/// UI Repository instance for VIP Application
///
private VIPRepo repo = VIPRepo.Instance;
private string gender;
private string lastName;
private string firstName;
///
/// Property for FirstName parameter.
/// By setting the property Ranorex directly clicks
/// the text box and simulates the keyboard events
/// Returns the current text value of the text box.
///
public string FirstName
{
set
{
this .firstName = value ;
repo.VIPApplication.FirstName.Click();
Ranorex.Keyboard.Press(firstName);
}
get
{
return repo.VIPApplication.FirstName.TextValue;
}
}
///
/// Property for FirstName parameter.
/// By setting the property Ranorex directly clicks
/// the text box and simulates the keyboard events
/// Returns the current text value of the text box.
///
public string LastName
{
set
{
this .lastName = value ;
repo.VIPApplication.LastName.Click();
Ranorex.Keyboard.Press(lastName);
}
get
{
return repo.VIPApplication.LastName.TextValue;
}
}
///
/// Property for Gender parameter.
/// Depending on the given value Ranorex selects
/// the right radio button.
/// Returns the currently selected gender
///
public string Gender
{
set
{
gender = value ;
if (gender.Equals( "Female" ))
repo.VIPApplication.Gender.Female.Click();
else if (gender.Equals( "Male" ))
repo.VIPApplication.Gender.Male.Click();
}
get
{
if (repo.VIPApplication.Gender.Female.Checked)
return "Female" ;
else
return "Male" ;
}
}
/// Method is used to simulate a click on
/// specified button.
///
///
/// Specifies the label of the button to press
///
public void Action( string button)
{
repo.VIPApplication.Self.FindChild(button).Click();
}
///
///
/// Returns the current text value of
/// the status bar.
///
public string ValidateStatusBox()
{
return repo.VIPApplication.StatusBar.TextValue;
}
}

* This source code was highlighted with Source Code Highlighter .

This is the table for the Test Case at FitNesse:

! | NetFit.AddVIPTest |
| FirstName | LastName | Gender | Action | ValidateStatusBox? |
| Marylin | Monroe | Female | Add | VIP count: 1 |
| Bill | Gates | Male | Add | VIP count: 2 |
| Hillary | Clinton | Female | Add | VIP count: 3 |

And this is the result of this test:


Using FitNesse, you can create test kits, which greatly simplifies the acceptance testing of software. In addition, since FitNesse is a WiKi, in one environment it is possible to store and maintain up to date all project documentation with reference to completed tests, both modular and acceptance tests.

In addition, you can watch a video from the automated testing conference where Uffe Overgaard Koch talks about Automated Testing of
Mobile Handsets.



I hope that this material will be useful.