How to conduct Testing Dojo

There is such a thing - Testing Dojo. These are competitions where participants look for bugs in applications. The one who finds the most is the one who won. Usually compete with teams. If you have to search for bugs manually, only testers are involved. If autotests go into battle, the developers are connected.

In 2GIS Testing Dojo has long become a good tradition: we spend it for the third year in a row. During this time we understood a lot about how to do better. We will share our experience under the cut: suddenly you will want to do your Testing Dojo.

rules

There are N teams of 2-3 people. Each team sits at the computer: one kodit, while others help with tips. All participants test the same application using autotests.
')
Competitions are divided into six sessions of thirty minutes. Each "session" simulates the release of the application. In each release there are new features and new and / or old bugs: everything is like in a real product.

Command tests are performed at the end of the session on the Continous integration server. According to the test results, the jury calculates points. For each bug that the team finds, the jury will award points. For each missed - deducts. The winner is selected on the basis of points for all six sessions.

According to these rules, we play. For your Testing Dojo, change the rules as you like. The main thing is that everyone understands them and, preferably, agrees with them.

What to prepare

1. Test application
2. Workplaces of participants with software on which they write and run tests.
3. Command repositories
4. CI server for automated builds.

Now let's talk about everything in order and give examples of Testing Dojo, which we conducted on June 6th .

Test application

When we selected a test application, we were guided by several principles.

1. The application is simple.
   Excessive complexity greatly slows down the development of tests: participants spend a lot of time to figure out the functionality. Therefore, the option “to take some 2GIS product and add bugs to it” was not suitable for our Testing Dojo.
2. The functionality of the application is well divided into individual features.
   We describe each feature in the form of a very short specification.
3. Some bugs are found only in one session, while others are repeated.
   So already ready tests bring points in subsequent sessions.
4. An application with the necessary functionality is quickly assembled and updated.
   The functionality corresponds to the release number.

For the last Testing Dojo, we wrote a fairly primitive catalog of products. In its first version there was only a rigorous search by part of the name. In the latter - the management of goods.

The current version of the application was collected and deployed to the test nodes of the Jenkins server at the end of each session. At the beginning of the next session, participants updated the application to the latest release version.

Script to update:

$appLink = 'http://opensource-ci.2gis.ru/view/Testing%20Dojo/job/td-tested-application/lastSuccessfulBuild/artifact/TestingDojo2015.exe' $outFolder = $env:UITestApps if ($outFolder -eq $null) { $outFolder = 'C:\app' } $outFile = (Join-Path $outFolder 'TestingDojo2015.exe') if (Test-Path $outFolder) { Remove-Item $outFile -Force } else { New-Item -ItemType directory -Path $outFolder | Out-Null } try { Invoke-WebRequest $appLink -OutFile $outFile Write-Host 'Update successful!' } catch { Write-Error $_.Exception.ToString() Write-Host 'Update failed!' } Write-Host "Press any key to exit.." $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyUp") > $null 

Information about the features of the current version (posted it on the event page):

Version 5. Features
Added the ability to simultaneously delete multiple records. You can select multiple entries at the same time by holding down the Ctrl or Shift key. Deletion takes place by pressing the “Delete Multiple” button, which appears when more than one element is selected at the same time.
You can sort the records by identifier or by name. The desired sort field can be selected from the drop-down list. The sorting order is changed using the switch.

Information for the jury about new and fixed bugs:

Version 5. Bugs
Fixed a bug with sorting
Repaired search
If Ctrl select several records, then all but the last one are deleted.

Choose for competitions a simple application with clear functionality. Make sure that most of the time, participants look for bugs and write autotests, and not read the documentation.

Workplaces

If you have properly equipped workplaces, consider that you have already made a good event in half. Any problems with hardware or software shift the start of the competition and get on the nerves of the participants.

For the last Testing Dojo, we prepared an audience for 10 jobs. They counted on 20 testers and developers from different companies in Novosibirsk.

We dedicated the competition to testing desktop applications for Windows, so we asked participants to write on the Winner.Desktop Selenium-based driver. This is a wrapper over our opensource-tool Cruciatus , we will tell about it in a separate article.

Thanks to Winium.Desktop, we write tests in different programming languages. For Testing Dojo, we chose C # and Python 3.x and prepared the environment for them. Visual Studio (+ Resharper) and PyCharm were installed on each computer with Windows 8.

With particular attention to set up a user profile system. The IDE, the application under test, the script for automatically updating the application and the tools for analyzing the user interface of desktop applications were placed on the desktop and the start page. In the browser, drove the start page to the most necessary resources.

Users created a script:

 # Set-ExecutionPolicy -ExecutionPolicy Unrestricted $hostName = [System.Net.Dns]::GetHostName() $cultureLCID = (Get-Culture).LCID $userName = "testingdojo" $userPassword = "2gisTD2015" # 1033 - English, 1049 - Russian # https://msdn.microsoft.com/en-us/goglobal/bb964664.aspx?f=255&MSPPError=-2147217396 if ($cultureLCID -eq 1049) { $rdpGroupName = "   " } elseif ($cultureLCID -eq 1033) { $rdpGroupName = "Remote Desktop Users" } else { Write-Host "Unknown LCID:"$cultureLCID; return } # add user # http://blogs.technet.com/b/heyscriptingguy/archive/2014/10/01/use-powershell-to-create-local-users.aspx # https://msdn.microsoft.com/en-us/library/aa772300%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396 $comp = [ADSI] "WinNT://$hostName" $user = $comp.Create("user", $userName) $user.SetPassword($userPassword) $user.UserFlags = 64 + 65536 # ADS_UF_PASSWD_CANT_CHANGE + ADS_UF_DONT_EXPIRE_PASSWD $user.SetInfo() # add user to Remote Desktop Users group # http://blogs.technet.com/b/heyscriptingguy/archive/2014/10/03/adding-local-users-to-local-groups.aspx $group = [ADSI] "WinNT://$hostName/$rdpGroupName,group" $group.Add("WinNT://$hostName/$userName, user") 

For each workplace secured the name of the team. In order to quickly prepare the working environment on computers and make life easier for participants, the names of the teams came up with themselves.

To do this, looked at the shelves of IKEA and found funny meaningless (hopefully) words: “duken”, “kokbanan”, “risdal”, “morvik” ...

So, follow the workplaces of the participants. Be sure to install the necessary software, enter the Wi-Fi password and install the test application.

Command repositories

For team repositories, we have prepared project templates and powershell script that push them into all repositories automatically. Both templates came to the repository, since we did not know which language the team would choose.

The repositories were cloned onto the computers of the respective commands, set up a global config and checked that there are rights to push.

If interested, take a look at the team repositories on Github .

Automated builds

For each team, we set up an assembly on Jenkins .

The assembly of the application under test was also executed. If necessary, teams drove tests and find out what condition they are in. But regardless of the state of the tests, we ran the assembly at the end of the session and counted the points.

What is the result

If you plan to conduct a Testing Dojo, decide on the theme of the event. After that, decide which application and which tool the participants will test. Choose a simple application with clear functionality.

Prepare workplaces for participants: set access rights for users on computers, install the necessary software, place shortcuts on the desktop, give access to the version control system.

To eliminate the human factor, automate manual operations: preparing repositories for tests, building and deploying applications to test servers, updating applications in each session.

These rules are a good base for a good event. Use them to do your Testing Dojo. I'm sure you will succeed!