Benefit from Selenium and Jenkins CI

And so, suppose an abstract situation: you have a test suite written using Selenium WebDriver. It is constantly updated, overgrown with new details and you are really unbearable to understand logs, heaps of XML, look and think what flew and why. A little bit below, I will try to give a number of simple tips to stabilize my Selenium tests, as well as talk about ways to make friends with Selenium with Jenkins CI and the consequences of this friendship. For those who are interested - I ask under the cat.

1. Selenium WebDriver

Before you integrate Selenium with third-party things you need to achieve high stability. What do I mean by these words? Everything is very simple: regardless of the number of test runs, the load of your tested web application and the size of US external debt, your tests should show the same result. If you think your tests are perfect, then try running several hundred at the same time (substitute the number you need) of monotonous tests, cut the maximum allowable amount of RAM for your web application or in any other way get the tested product to work with creepy brakes and you get a lot of nasty bugs in your tests.

If the item on the page is not loaded, then wait a second.

A rather common approach to solving all problems for the Indian coder is:

Thread.sleep(1000); 

Usually such lines of code are marked with a comment in the spirit of “wait for element with id = myUnicID1 present” ... Why it is necessary to wait for 1000 milliseconds is not clear. As a result, at some point it happens that the element we need appears for 1003rd millisecond and our test falls with an error in the spirit of NoSuchElementException .
Naturally, our coder very quickly understands what the problem is and finds a fairly simple solution:

 Thread.sleep(5000); // 5 -   

300 tests, where everyone waits for 5 seconds empty = 25 minutes spent on incomprehensible things, but surely the error shown above will not appear. (Of course, everything depends on the project and the conditions in which it is tested, situations are possible where 5 seconds is not limit.) And if there are many more such tests, as well as the number of “doubtful” elements?
Thinking a little more, a slightly more rational solution of the problem comes to mind:

 while(!driver.isElementPresent(By.id("myUnicID1") ) ){ Thread.sleep(500); } 

This solution in certain known circumstances is equivalent to the code:

 while(true){ Thread.sleep(500); } 

But the funny thing is that this code gives us an ingenious, but very simple idea - writing a smart shell to the driver. Thus, we need to wait for the drawing of all the necessary elements, as well as the completion of all AJAX requests. How this is implemented is pretty well described here . If you need more complex implementations, there are plenty of ready-made solutions on the Internet.

Anything that is not in the documentation is impossible.

Indeed, the documentation provides a fairly rich functionality for testing, but it covers far from everything. For example: sometimes you want to take text from textarea, and when using the getText () method, an empty string is returned - mysterious js and other dodgers of html code become a nuisance. And if you need to know the color of a particular element, or get some information about the attributes? Do not be afraid to use executeScript () , the possibilities of js and jQuery should not disturb you, they should help in covering the tests. The main thing to remember is that the script must return some value (which will be the result of its execution). If your script does not return any useful information, but produces some magic actions, then let it return true if the magic is completed successfully. With Selenium, you can confidently test mouse actions, keystrokes, check pop-up windows, drag active elements, and more.
')

The element is not located, but I can see it!

I checked 100 times that my xpath points to an element correctly and I tried to determine it in all other ways, and all the errors in the trace fall to me, saying that I can't find it. Such situations most of all loosen the psyche and arise because of a misunderstanding of what is happening (or inattention). Sometimes you may even get the feeling that it is your version of SeleniumDriver that does not support this or that functionality and by finding here a version that does not support the functionality you need, you can give up. But, as a rule, the problem is in your code. If you can’t find it, but I’m sure that it is there, then, most likely, you are just not looking there. Am I in that frame? Do I switch between them correctly? In answers to these two questions, as a rule lies the solution.

Abstracted!

Many Selenium tests look like a sequential click on elements with 30-meter xpath, obviously stealthy from firefox. Often, tests largely duplicate clicking on the same elements or even some basic sets of actions. And if in your application a new functionality was added so that pressing the 'A' button (which is present in every second test) stops working? Is it really inconvenient to uproot and refactor a project? It is much more convenient to create a class (or several) that contains all these unique identifiers of these elements. It is easier to change one line than to search and correct a multitude.

2. TestNG

It so happened that the selenium tests are carried out not on the traditional jUnit, but on TestNG. Of course, it all depends on your choice, but the ability to set pre / post conditions for groups of tests and parameterized assemblies is lured. If anyone is interested, it’s written here quite well. I think it makes no sense to go into the implementation of the tests, I’ll just describe the concept on my fingers (if someone wants, I’ll attach explanatory code examples). Each group of tests is implemented in its class, which is inherited from the abstract class. The condition for a successful test is that the received XML file corresponds to a local (reference) one. Test requirements: speed of execution, test run time should not take into account pre / post conditions, tests should be easy to understand, random execution errors should not occur, the cause of the error should be clear from the trace.
Thus, I have created a lot of additional Exceptions, explaining the essence of a particular problem. There is some kind of “repository” of identifiers of the elements I need, and the logic of switching between frames is protected into relatively complex methods (which combine a set of simple actions).

3. Jenkins CI

Finally, you need to upload all these delights of life to some server, which would constantly spin around somewhere and, if anything, would report an error. I use Jenkins as CI. To successfully run Selenium on a server without a graphical shell, we need to install xvfb:

 yum install Xvfb 

Next you need to try to run xvfb and if you do not have enough fonts or other trifles, you should deliver everything you need. You can use this instruction. Now that we still have stable xvfb work, we need to install the xvfb plugin for jenkins (of course, you can do without it, but I try not to use self-written scripts once again). In the jenkins settings, specify the address to xvfb, and enable it in the project configuration After, you need to run the tests, and publish the results of their execution using this plugin . If you add e-mail notifications to all of this, you’ll have an autonomous testing system that worries you if something breaks. That's all, not so difficult, right?