Creating a tool for quick and efficient writing autotests on Selenium

Fundamental building block automation - testing
Rod Johnson

I’m not an ambassador for automating web interface testing, but this essay is more likely to be useful to comrades who already have experience in this field.

For completely newcomers it will also be useful, because I provide the source code where you can see how the interaction with the selenium is organized in the final product.
')
I will talk about how from scratch, having a little development experience, wrote a platform for running tests, and about the platform itself. I myself believe that my product has turned out to be very effective, which means it will be useful to many and has a place to consider.

Concepts

The testing process depends on the information system.

In order to remember my concept, it is necessary to understand which systems I focus on in the first place - those systems where there are usually specific linear business processes that are put in the key when conducting regression tests.

So, a system like srm. The key business entity is the supplier’s business proposals. A key consideration when conducting regression testing is the integrity of the business process.
The business process starts from registering a supplier in the system, then comes the creation of a quotation - goes to the review stage, which is performed by various types of internal users (and for each user a unique interface) to return the decision to consider the offer to the supplier.

It turns out that we go through a number of different interfaces, and almost always work with different ones. In fact, if you just look at it directly, it’s like watching a videotape — that is, This is a process that has a beginning and an end, and it is absolutely linear - no branches, when writing a test, we always know the expected result. Those. I want to say that already looking at this picture, we can conclude that it is unlikely to make tests polymorphic. In view of this, when creating a platform for running tests, a key factor was the speed at which tests were written.

Concepts that I set myself:

1. Autotest should be created as fast as possible. If we achieve this qualitatively, then other aspects, such as reliability and usability, should come by themselves.
2. Tests must be declared declaratively and live separately from the code. I did not even see another option. This increases the speed of writing, because if there is a ready interpreter - our platform, then you don’t need to add anything, you don’t have to go into the code once more - you can forget about the IDE platform after all. So the tests are easier to maintain. In this form, they are easier to learn to write, because no development skills needed, but only an understanding of the markup language. In this form, they are understandable to all participants in the process.

What I decided to give up at the start:

1. Do NOT wrap your system in a test framework. You can start the execution of a process without a test framework. “You want to invent a bicycle!”, Many will say. I think differently. The popular test frameworks used were created primarily to test the code from the inside, and we are going to test the external part of the system from the outside. It's as if I have a road bike, and I need to go down the mountain on the road (roughly, but reflects the train of thought). In general, we will write the framework ourselves - with blackjack and ... (although I know that, for example, JUnit 5 is already much more adapted to this kind of tasks).
2. Refusal to use wrappers for selenium. Actually, the key library is small in itself. To understand that you need to use 5 percent of its functionality, but it will take several hours to rebuild it completely. Enough everywhere to look for a way to write less code and train yourself to the pot. In the modern world, this desire often leads to absurdity and almost always causes damage to flexibility (I mean it is the approaches to “write less code” and not the cases of architectural frameworks).
3. Beautiful design results are not needed. Introduced this item, because more than once I come across this. When the autotest is completed, I need to know 2 things: the overall result (positive / negative), and if there was an error - where exactly. Perhaps still need to keep statistics. Everything else in terms of results is ABSOLUTELY unimportant. To consider a beautiful design as a significant plus, or to spend time on this beautiful design at the initial stages is an extra show off.

I will talk a little more about the level of development in the company and the conditions for creating a tool in order to clarify some details to the end.

Due to some confidential circumstances, I do not disclose the company where I work.

In our company, development has been far from the first year, and therefore all processes have long been established. However, they are far behind the current trends.
All representatives of IT understand that it is necessary to cover the code with tests, write autotest scripts at the time of harmonization of requirements for future functionality, flexible technologies significantly save time and resources, and CI that simply takes and simplifies life. But all this is only slowly reaching us ...

Similarly, the software quality control service - all tests are performed manually, if you look at the process “from above” - then this is the “bottleneck” of the whole development process.

Assembly Description

The platform is written in Java using JDK 12

Basic infrastructure tools - Selenium Web Driver, OJDBC

For the application to work on your PC must be installed FireFox browser versions above 52

The composition of the assembly application

With the application without fail there are 3 folders and 2 files:

• BuildKit folder - contains:

• jdk12, with which the application is launched (JVM);
• geckodriver.exe - to run Selenium Web Driver with the FireFox browser;
• SprintAutoTest.jar - the application instance itself

• Reports folder - reports are saved to it after the application completes the test case. It should also contain the ErrorScreens folder where the screenshot is saved in case of an error.

• TestSuite folder - web packages, javascripts, set of test cases (the filling in this folder will be described in detail separately)

• file config.properties - contains the config for connecting to the Oracle database and the values ​​of explicit expectations for WebDriverWait

• starter.bat - file to start the application (it is possible to automatically start the application without manually specifying TestCase if at the end enter the name TestCase as a parameter).

Brief description of the application

The application can be started with a parameter (the name TestCase) or without it - in this case, you must enter the name of the test case in the console yourself.

An example of the total content of a bat file, to run without a parameter : start "AutoTest launcher"% cd% \ BuildKit \ jdk-12 \ bin \ java.exe -Xmx768M -jar --enable-preview% cd% \ BuildKit \ SprintAutoTest.jar

When the application is launched in general, it scans xml files located in the "\ TestSuite \ TestCase" directory (without viewing the contents of the subfolders). In this case, there is a primary validation of xml files for the structure correctness (i.e., all tags from the point of view of xml markup are specified correctly), and the names specified in the “testCaseName” tag are taken, after which the user is prompted to enter one of the possible names of the existing test case studies In case of a wrong input, the system will ask you to enter the name again.

After the TestCase name is obtained, the internal model is built, which is a combination of TestCase (test script) - WebPackage (element store) as java objects. After building the model, TestCase is built directly (executable object of the program). During the construction of TestCase, secondary validation also occurs - it is checked that all specified forms in TestCase are in the associated WebPackage and that all elements specified in the action are in WebPackage within the specified pages. (The structure of TestCase and WebPackage is described in detail below)

After the TestCase is built, the script runs directly.

The algorithm of the script (key logic)

TestCase is a set of Action entities, which in turn is an Event entity set.

Testcase
-> List {Action}
-> List {Event}

When you start TestCase, the Action is started sequentially (each Action returns a logical result)

When the Action is started, the Event is started (each Event returns a logical result)

The result of the execution of each Event saves the result.

Accordingly, the test is completed either when all actions were completed successfully, or if the Action returned false.

* Failure mechanism

Because My test system is ancient and has errors / glitches being caught that are not errors, and some events do not work the first time, the platform implements a mechanism that can deviate from the rigid linear test concept described above (however, it is strictly typed). When catching such errors, it is possible to repeat the cases first and perform additional actions to be able to repeat the actions.

At the end of the application, a report is generated, which is saved to the "\ Reports" directory. In case of an error, a screenshot is taken, which is saved in "\ Reports \ ErrorScreens"

Filling TestSuite

So, the description of the test. As already mentioned, the main parameter required to run is the name of the test that should be run. This name is stored in the xml file in the “/ TestSuite / TestCase” directory. All test scripts are stored in this directory. They can be here any number. The name of the test case is not taken from the file name, but from the “testCaseName” tag inside the file.

In TestCase it is set what exactly will be done - i.e. actions. In the “/ TestSuite / WebPackage” directory, all locators are stored in xml files. Those. all in the best traditions - actions are stored separately, locators of web forms separately.

TestCase also stores the name of the WebPackage in the “webPackageName” tag.

Total picture is already there. To start, you must have 2 xml files: TestCase and WebPackage. They make up a bunch. WebPackage is independent - the name is specified as an identifier in the “webPackageName” tag. Accordingly, the first rule is that TestCase and WebPackage names must be unique. Those. Once again - essentially our test is a bundle of TestCase and WepPackage files, which are linked by the name WebPackage, which is specified in TestCase. In practice, I automate one system and I tie all my test cases to one WebPackage in which I have a description of all the forms.

The next layer of logical decomposition is based on a pattern like Page Object.


Next, I will try not to detail the structure of WebPackage and TestCase. For them, I created a DTD schema for WebPackage and XSD 1.1 for TestCase.

! IMPORTANT

By maintaining DTD and XSD circuits, the concept of quickly writing a test is implemented.

When writing directly to WebPackage and TestCase, you need to use xml Editor with built-in DTD and XSD validation functions in real time with autogeneration of tags, which will make the process of writing an auto test largely automated (all required tags will be substituted automatically, drop-down lists will be displayed for attribute values possible values, under the type of event will be generated corresponding tags) .

When these schemes are “screwed” to the xml file itself, you can forget about the correctness of the xml file structure, if you use a special environment. My choice fell on oXygen XLM Editor. Once again - without using such a program, you will not understand the essence of the speed of writing. Idea is not very suitable for this. it does not handle the XSD 1.1 “alternative” construct, which is key to TestCase.

Webpackage

WebPackaege - an xml file that describes the elements of web forms, located in the directory "\ TestSuite \ WebPackage". (there may be as many files as you like. The file name can be any - only the content has a value).



As already mentioned, the elements were not in the heap - everything is decomposed into web forms.

Key entity is

 <element> 

The element tag has 2 attributes:

• type
• alwaysVisible

The type attribute is required and specifies the type of the element. In the platform is set by type

At the current moment specifically for myself in the platform implemented the following types:

• 0 - has no functional meaning, usually some kind of an inscription
• 1 - button
• 2 - input field
• 3 - checkbox
• 4 - drop-down list (select) - not actually implemented, but left space for it
• 5 - for the srm drop-down list: write the name, wait for the value to appear - select by specific xpath pattern - type specifically for my system
• 6 - srm select - used on typical search type functions, etc. - type specifically for my system

The alwaysVisible attribute — optional — indicates whether an element is always present on the form, can be used during the initial / final validation of an Action (i.e. in automatic mode, it can be checked that when opening a form, all elements that are always on it are present when closing forms, all these elements have disappeared)

Possible values:

• 0 - by default (or if the attribute is not specified) - the element may be absent on the page (not validate)
• 1 - the item is always on the page

Optional optional attribute type is implemented for the locator tag.

Possible values:

• 1 - search for an item by id (respectively, in the locator specify only id)
• 2 - by default (or if the attribute is not specified) - xpath search - it is recommended to use only xpath search, since This method combines almost all the advantages of the others and is universal.

Testcase

TestCase - an xml file describing the test script itself, located in the "\ TestSuite \ TestCase" directory (there can be as many files as you like. The file name can be any - only the content has a value).



Here in this line you can see how to screw the xsd scheme so that the XML Editor can see it:

 <testCase xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="testShema.xsd"> 

In TestCase, I also use DTD entities that are stored separately in the same directory - a file with the .dtd extension. In it, I store almost all the data - constants. I also built the logic in such a way that in order to launch a new test, and in the course of the whole test, new unique entities were created, a new KA was registered, it is enough to change 1 digit in this file.


These constants are inserted into the tag value like this:

 <url>&srmURL;</url> 

- can be combined.

! Recommendation - when writing testCase, you should specify these DTD entities inside the document, and after everything is working steadily, transfer to a separate file. My xml editor has difficulties with this - it cannot find a DTD and doesn’t take XSD into account, therefore I recommend it

testCase

testCase - the parent tag contains:

• testCaseName - the name of our test case, this parameter is passed to the input of the application
• webPackageName is the name of WebPackage, which is specified in webPackageName (see paragraph above on WebPackage)
• actions - entity container action

action

Contains:

• name - name - it is recommended to specify the name of the form and key actions - what and why
• orderNumber - serial number - the parameter necessary for sorting action (entered due to the fact that when parsing xml in java when using certain tools, parsing can be carried out in a multi-threaded environment, and therefore the order can go) - when specifying the next action you can jump - i.e. when sorting, only “more / less” matters, and so it is possible to deliver action between those already described without the need to change the whole numbering
• runConfiguration - the actual description of what will happen within the bounds of action

runConfiguration

Contains:

• The openValidation attribute is optional, the default is “0”
  
  • 0 - do not conduct initial form validation
  • 1 - perform initial form validation
• the closeValidation attribute is optional, the default is “0”
  
  • 0 - do not perform final form validation
  • 1 - perform final form validation
• formName - the name of the form within which actions will be performed - the formName value from WebPackage
• repeatsOnError - optional, indicates how many repeats need to be made in case of failure
• events - event entity container
• exceptionBlock - optional - a container of event entities that are executed in case of an error

event

Minimum structural unit - this entity shows what actions are performed.

Each event is special, can have unique tags and attributes.

Base type contains:

• type attribute - indicates the type of element
• the hasExceptionBlock attribute is an optional attribute, the default is “0” —is required to implement the failure mechanism — the attribute indicates that we can expect an error at this event
  
  • 0 - no error expected
  • 1 - possible error is expected on the action
• attribute invertResult is an optional attribute, the default is “0” - the attribute indicates that the result of the event must be changed
  
  • 0 - leave the result of the performance of the event after the fact
  • 1 - change the result of the event to the opposite

Event types

Here I will give a minimum set of my events, such as a scout set, with which I can test almost everything with my system.

And now smoothly to the code to understand what the event is and how it is built. The code essentially implements the framework. I have 2 classes - DataBaseWrapper and SeleniumWrapper. These classes describe the interaction with infrastructure components, and also reflect the features of the platform. I will give the interface which implements SeleniumWrapper

 package logic.selenium; import models.ElementWithStringValue; import models.webpackage.Element; import org.openqa.selenium.WebElement; public interface SeleniumService { void initialization(boolean webDriverWait); void nacigateTo(String url); void refreshPage(); boolean checkElementNotPresent(Element element); WebElement findSingleVisibleElement(Element element); WebElement findSingleElementInDOM(Element element); void enterSingleValuesToWebField(ElementWithStringValue element); void click(Element element); String getInputValue(Element element); Object jsReturnsValue(String jsFunction); //Actions actions void doubleClick(Element element); void moveMouseToElement(Element element); void pressKey(CharSequence charSequence); void getScreenShot(String storage); } 

It describes all the features of Selenium and superimposes the platform chips - well, the main feature itself is the “enterSingleValuesToWebField” method. Remember that we in WebPackage specify type of an element. So, how to react to this type when filling in the fields is written here. We write 1 time and forget. The above method is worth fixing for themselves in the first place. For example, types 5 and 6, now operating - are only suitable for my system. And if you have such a thing as a filter and you need to filter a lot, and it is typical (in your web application), but in order to use it you must first move the mouse over the field, wait for some fields to appear, switch to some, wait There is something there, then go there and enter ... Stupidly prescribe the mechanism of action 1 time, give a unique type of this to everything in the switch construction - then do not bother - you get a polymorphic method for all similar application filters.

So, in the package “package logic.testcase.events” there is an abstract class that describes the general actions of the event. In order to create your own unique event, you need to create a new class, inherit from this abstract class, and you already have both dataBaseService and seleniumService in the bundle - and then you determine what data you need and what to do with them. Something like this. Well, accordingly, after creating a new event, you need to finish the class factory TestCaseActionFactory and, if possible, the XSD scheme. Well, if a new attribute is added, modify the model itself. In fact, it is very easy and fast.

So, scout kit.

goToURL - usually the first action - go to the specified link


fillingFields - Filling specified items

Special tags:

• fields - fiel entity container
  
  • field - contains an element tag
    
    • element - indicates the name of the element from the webPackage
    • value - what value to specify, has an optional attribute type (if the element is a checkbox, then one of the values ​​is specified: “check” or “uncheck”)

• type attribute - specifies how to take the value, optional, the default value is "1"
  • 1 - the specified value is taken
  • 2 - in this case, the specified JS function is executed from the "\ TestSuite \ JS" directory! IMPORTANT - the name of the txt file is indicated, without the ".txt" (and so far I have found applications for the js functions so far only in this form - I use it in one place to generate a random inn, but the range of possible applications is wide)
  • 3 - in this case, the query is specified as a value in the database, and the program substitutes the 1st result of the query

checkElementsVisibility - checks that the specified elements are present on the form (just visible, and not just in the DOM). In the field attribute, either an element from the WebPackage or xpath itself can be specified.


checkElementsInVisibility is similar to checkElementsVisibility, but vice versa

clickElement - click on the specified element


checkInputValues - check entered values


dbUpdate - perform an update in the database ( oXygen strangely reacts to 1 event type dbUpdate - I don’t know what to do with it and don’t understand why )


CheckQueryResultWithUtilityValue - checking the value entered by the user with the value from the database


checkFieldsPresenceByQueryResult - check for the presence of elements on the form by xpath using a pattern. If the desired pattern is not specified, the search will occur by the pattern .//* [text () [contains (normalize-space (.), "$")]], Where instead of "$" there will be a value from the database. When describing your own pattern, in the place where you should put the value from the database - you must specify "$". In my system there are so-called grids in which there are values ​​that are usually formed from some kind of view. This event for checking such grid


Wait - everything is simple - waiting for the specified number of milliseconds. Unfortunately, although it is considered to be a crutch, I will say for sure - sometimes it is impossible to do without it.


scrollDown - scroll down from the specified element. It is done this way: it clicks on the specified element and the “PgDn” key is pressed. In my cases, where I had to scroll down, it works perfectly:


userInput - input value in the specified element. The only semi-automatic in my automation, is used only for captcha. Specifies the element in which to enter a value. The value is entered in the pop-up dialog box.

About code

So, I tried to make the platform in accordance with the principles of Uncle Bob's Pure Architecture.

Packages:

application - initialization and launch + configs and report (do not scold me for the Report class - this is what is most possible in a hurry, then as quickly as possible)

logic is the key logic + services of Selenium and DB. Immediately events.

models - POJO on XML and all auxiliary object classes

utils - singletons for selenium and db

To run the code, you need to download jdk 12 and indicate everywhere that its chips are included. In Idea, this is done through Project Structure -> Modules and Project. Also do not forget about the Maven runner.And when running in a baht file, add --enable-preview. An example was.

Well, in order to still start, you need to download the ojdbc driver and by passing the jar to the “SprintAutoTest \ src \ lib” directory. I do not provide it, because at orakla there all is serious now - to download it is necessary to register, but I am sure that everyone will cope one way or another (well, check that all folders have been created, and that the report will not be saved)

Summary

So, we have a test run, the writing of tests on which is done really quickly. During the work week, I managed to automate 1.5 hours of manual work, which is performed by the robot in 5–6 minutes. This is approximately 3,700 lines of a concatenated test case and 830 described elements (more than 4,800 lines). The numbers are rough, and so it is not measured, but whoever is engaged in automation must understand that this is a very high figure, especially for systems unfriendly to robots. At the same time, I test everything — business logic, along the way I perform some negative tests for correctness of the filled attributes, and as a bonus I check completely every web form that I don’t lazy and describe all the functional and key elements that they need me or not (A small digression - closeValidation I use mostly only when writing a test.When it is stable, and it is clear that the locators do not intersect, I turn it off for all actions, so that the process goes faster).

At first glance, it seems that there are many xml lines, but in practice they are generated semi-automatically, and the error can be made only in the directly entered parameters (since in fact we have 2 levels of validation - the 1st is xml schemes, the 2nd check the availability of the specified forms and elements at the launch of the TestCase start).

Of the minuses - no clear boundaries of tests. As such, they are absent and you can blame me that this is just a macro runner, not a test. I stipulate it like this:

In the platform, tests are conceptually from my point of view divided into several levels of abstraction:

• + — « » + — ( – .. -)
• ( action — event c + )
• , , – -. , – . , . , – ,
• each action on the page is a unit test (which returns a single result true or false)

If you compare my approach with something, then the disadvantages of, for example, the popular Cucumber and the very concept of BDD are more significant for me (exactly when we test similar to my system):

• We cannot guarantee the performance of tests when testing an unstable web application. Those. in my case, for most tests, we cannot guarantee their performance, and they will fall for "When", which in my opinion is not at all acceptable if we describe testing with a set of tests.
• Well, everywhere is given this already vanishing example with login. Yes, for all my practice, it was in the login that there were never any errors (although it should of course be covered, that’s for sure). And the example is good, but for the rest of the tests, you need to sculpt endless crutches from Given and When - in the system I have cited for real tests you will need 99% on the description of intermediate conditions, until we get to this test itself - a lot of hassle, a little, in fact in the code to climb.

From what I would like to do, something to think about, but my hands have not yet reached:

• running not one but several tests in succession in one run
• I think that the entities should be pumped so that they can generate values ​​on the fact of a new test, and they were saved during the execution
• make centralized version control. Not just git, but to indicate which version of the test to run, or, again, a smart module that understands which version is relevant, which is not yet
• as I said before, to launch a new test with new values, I need to change 1 digit, automate it, as it were, and this is to make a smart module for it
• although it doesn’t bother me too much that I have all the locators stored in one place, nevertheless, it would be good to make an even more user friendly structure for their storage
• did not indulge with selenium server. I think it is worth thinking about this, as well as the possibility of further adaptation to CI, Team City, and so on.

Well, in general, that's all. I attach a reference to github: source .

I would be very happy constructive criticism, I hope this project will really be useful.