About Selenium and one "bike"
Hello!
I would like to talk about the problems that I encountered in the process of mastering Selenium WebDriver, with their solution and how these solutions, in principle, can be used. All this is presented in the form of a prototype framework, the link to which will be at the end of the article.
In this post I want to share my ideas for the implementation of the Page Object template, how to handle errors that occur during the execution of tests, tell a little about logging. And also to share information about some of the tools that are implemented using Selenium WebDriver, and their own work.
')
The plan of my article is as follows:
1. Captain obvious, instead of entry.
2. A little about yourself, you must introduce yourself ...
3. Why Selenium?
4. About Page Object ...
5. Not a bug, but a feature!
6. And again about logging and reporting.
7. Are there no analogues?
8. Promised links.
9. In conclusion.
So let's go!
I don’t think that it makes sense to explain what software testing is, what strategies and models exist, what is its role in software development and quality assurance, and what projects and users are at risk if this process is absent. Besides, now programming is not an art, but a craft for many millions of specialists. Projects are implemented in a short time and with limited budgets, and often they can be monsters with simply immense functionality. Moreover, products can vary intensively from version to version, let's say improve. And each such "improvement" can be fatal.
In such conditions, automated testing plays a key role. What models, strategies and methods are better to find in other sources. My article is not about that. It is dedicated to automation tools for testing client-side web applications.
I have been doing regression automated testing for four years now. We test the desktop client of the product and the classes that describe its business logic. About the tool that is used on our project can be read here . During this time, many UI tests were automated, unit tests, a lot of things ... It was here that I watched the evolution of test automation, and myself participated in it.
At first, these were scripts written directly on the knee using automatic recording tools, in which the logic of the script was inseparable from the ways of interaction with the objects under test. As the number of tests increased, it became obvious to everyone that some kind of framework should be developed and used for automated scripts. Any sneeze in the application under test had very unfortunate consequences for the entire test run.
Then it became necessary to use software interfaces for interacting with the product and databases, because what was shown by the dialog forms was not enough ...
However, something I got carried away. But it was here that I realized how important it is to think from the very beginning of the strategy and tactics of interacting with the product of developers for its effective automated verification.
The tendency of applications to switch from their desktop clients to web clients cannot be noticed either by the deaf-blind or the person born and living somewhere in Antarctica. And development for web is now mainstream, it is not a secret to anyone.
I decided not to stand aside and try in my spare time to master something that is used to automate the functional testing of the frontend - part and to check its compatibility with various client platforms. And besides, there is some probability that the web client on our project to be ...
I chose Selenium Webdriver.
Why is Selenium?
Once upon a time it was a small tool that was used only by desperate enthusiasts from the world of web development. Now this is a formidable weapon that not only many thousands of automators around the world, but even the frontend - the developers have adopted to combat defects.
You can read about the product line of the Selenium project here or here .
But I must say that this is not a tool for automated testing directly . For example, Selenium Webdriver provides a variety of software tools and interfaces for emulating user interaction with the browser and content loaded in its window. These funds cover almost 100% of the possible variants of such interaction. All this may find wider application. But, for example, he cannot, by default, generate a report on the passage of a particular test.
Next, there are some other problems. Below are some of them:
The WebDriver and WebElement interfaces provide very limited functionality . It is mainly aimed at finding the elements of the active page and performing the simplest actions on the elements (entering values, clicking, etc.). Many other useful features are provided by interfaces such as HasInputDevices, Locatable, Options, Alert , etc. And using only WebDriver and WebElement will severely limit the capabilities of automated tests.
Every browser has its own driver . Now it is:
ChromeDriver;
FirefoxDrive;
HtmlUnitDriver (for Unit - tests at the level of the html-document);
InternetExplorerDrive ;
OperaDriver ;
RemoteWebDriver (my favorite, good for running tests on remote machines);
SafariDriver ;
Iphonedriver ;
AndroidDriver ;
PhantomJSDriver (some kind of exotic browser, I did not have enough patience to build it on my machine).
I think that the Yandex team will also release its driver soon.
A wide choice, for every taste and for any needs. But if in the test project some or some of these are clearly used, this will create problems when you need to quickly switch from browser to browser or use several at once.
Therefore, I would like to have some single way or interface to create the desired instance. It is desirable that he took some kind of setting that can always be changed as a parameter. This setting should include the browser type, features , timeouts . Well, the opportunity to have a blessing, too.
Writing classes that would be responsible for interactive work with pages. PageFactory is a great tool. However, its use becomes inconvenient due to the loss of clarity of code classes that describe very large pages, and duplicate elements are fraught with copy-paste. Output - select duplicate elements in separate blocks (classes) and use blocks already. But this does not solve the problem to the end. What if your service can work by opening pages on separate tabs or browser windows, do you have duplicate blocks on all these windows? And some of them also lie on the frames! I experimented on this and that , and immediately ran into the problem described. And the situation is aggravated by the fact that the instance implementing WedDriver sees only the active page (window / tab) or frame.
I would like to be able to split the page into such blocks that a working test could memorize and work with them directly, without calling driver.switchTo (). Window (somehandle) or driver.switchTo (). Frame (frameIdentity) .
Therefore, I set myself the task not only to master the above tool, but to make a project, or even better, a prototype framework based on Selenium Webdriver. And in addition - try to find analogues and make it so that something can easily integrate with them. I just think that if the task is this way, then the submitted material is absorbed much easier.
Java was selected as the language. Development environment - Eclipse.
I will not talk about how this path was gradually overcome. It is better to continue with the description of the problems and their solutions. Maybe I will not open America, but I will be glad if someone finds something useful for themselves ...
I think there are few such automators who would not know about the design - Page Object pattern. By the way, this pattern is also applicable for writing autotests UI of some desktop application. But now we will talk about its implementation in Java for the web interface.
About the reception you can read here .
Now I will show you at the moment the working code of some not very successful test.
Code
I think everyone understands what is good when you want to write a script where you need to make one input somewhere and 2-3 clicks on something. Such a test you quickly fix, if something is correct (!!!) has changed.
If the test should test more than the steps described above, it will result in an inconvenient to read sheet of code. And now we multiply all this by a huge test project. Have you imagined how a poor tester cries, what supports him after every Google change? And so that he does not cry like that, we need classes through which the test performs its actions on the page. And if anything has changed (again, this change is a feature, not a bug introduced), but the edits will be local, and not in the whole project.
There are many implementations of Page Object, including, I think, on your projects. Below I will demonstrate my implementation and the same test:
public interface IPerformsClickOnALink
public interface IPerformsSearch
public class SearchBar
public class LinksAreFound
public class AnyPage
public class google
test
And if you work hard and do something like this setup , the test can work like this .
Many experienced automators, probably, now thought: “Well, yes, Page Factory uses ... The author for some reason has complicated the description of the google homepage - you can also describe it as one class! Well, except that some kind of incomprehensible annotation appeared ... @PageMethod is called. The descendants of the classes Page and Entity ... We saw a similar here . The author, we all have already seen! ”.
Well yes. This was a simple example. I intend to complicate the description of the main page to show that it is possible to break the description of the same page into two independent classes. I will tell about this annotation below ...
And now the example is more complicated ...
The picture shows some situation.
What the test does:
1 goes to docs.google.com
2 opens an accessible Google document.
3 on the main page of the service causes the sharing dialog;
4 from the open document the same dialogue is called.
For clarity, the entire sequence of actions is shown in the form of screenshots. Active elements are highlighted:
Necessary: somehow handle both instances of this dialog in different windows. And that's what happened with me. This is the described precondition.
Now I will click on the field in which they are asked to enter names, email addresses, etc., in the pop-up item that appears, click "Cancel", after which the "Finish" button will be clicked. Actions will be performed in both windows. For clarity, the following sequence in the form of pictures.
That's how we coped with the task effortlessly. . This is the getCustomizingAccessDialog () method code for the document and home page.
Those. there is a pop-up block of elements located inside the frame, which we used as an object that exists independently and is picked up when needed ... We know that it is on the page of any document and on the main page of the service, and we get it from them.
In fact, for a long time I tried to find the optimal way to organize and describe one or another test object in different sources. Although it may be looking bad ...
Obviously, you need to group page elements into blocks for reuse. These blocks can even be transferred outside under the guise of independent objects and work directly with them. If this is not done, then the class will grow in the number of attributes and methods. Need decomposition. Otherwise, you will receive such objects:
If you look at the example described above, then methods of work with each type of pop-up dialog are added to the methods of working with the main content - access settings, modal forms of renaming, choice of options, comments in documents.
The above solution can be implemented painlessly when testing what is loaded in a single window (single tab) of the browser.
But in the case of the described rather complex behavior, you will have to somehow switch the webdriver from page to page, not forgetting to switch to the required frame.
And there was an idea! And what if you teach each such object to remember the path to itself and automatically switch the webdriver when calling the method of interactive interaction with the page.
This, it seemed to me, perhaps, if we deviate somewhat from the concept of “page”. Those. under it, I will mean not downloaded content, but a browser window with content loaded in it. This window has a unique string identifier by which you can easily identify it among others. OK! That is, before calling the method, it will switch to the desired window.
So. And what to do with frames? You can also make it so that each block of elements knows, in addition, which frame to switch to.
So I build a path.
As can be seen from the above list of class constructors, which I called Page (the name is very conditional, perhaps the class should be renamed), you can write a description of the loaded page as a whole, or in parts, in addition to this, indicating another object like it as a parent ) when it is necessary. Thus it is possible to build a connected graph of blocks of elements. Something similar to hierarchy can turn out.
OK! Well, the paths indicated, the hierarchy was built. What's next? How to implement automatic switching when you call, such as a click or a string in the field? Switching itself is implemented in this way.
But! Many may ask the question: “Do we need to call it back in each method?” No. Below I will try to explain how it works.
In my decision, I propose to use not the objects created by the designer, but their deputies (proxy, in other words). For this, the cglib library was used. And to be more precise, the MethodInterceptor to intercept callable methods, the Enhancer to create the proxy objects themselves and the Callback . Using the above types eliminates the need, in the future, to create a set of interfaces for each Page heir (see java.lang.reflect.Proxy ) and makes the implementation more flexible. However - there is a restriction - the heir of the Page class should not be final !!!
Those. when we try to call a particular method of some Page - object (heir of the Page class), this method intercepts and “switches” the object to itself. And after that the execution of the method itself.
Perfectly! But is this kind of interception occurring when each method is called? The question, I note fair. We must not forget that the Page-object is also an object of the java language. Such an object can have public or protected methods that are not related to browser interactivity. How to be? Obviously, you need to think of a way to divide methods into methods that are responsible for interactive interaction and all the others.
The @PageMethod annotation serves as such a “delimiter”. Ie, if the method is marked with such a marker, then a preliminary switch to the page or its fragment will be performed ...
In general, the solution looks like this.
Next, a little about the possibilities of creating Page - objects.
Any block of elements or page view can be created in three main ways:
- from the heir to the class Entity - this is something like a service model, open to the main or any other page, as a whole. In the examples described, this is Google (search engine), GoogleDocsApplication (Google Drive). By the way, the example with Google demonstrates that with the “application”, in principle, we can also work as with the Page object. I kind of included elements with a panel and search results in it.
“Entity” can generate those objects that are in the first (main) open window / tab of the service (example with Google Drive, “ docs.getContent (); ” - wrapping around one of these methods, example with Google - “ searchBar = get (SearchBar.class); ”- call one of these methods) or those that exist in newly appearing windows / tabs ( AnyGoogleDocument anyDocument = docs.get (AnyGoogleDocument.class, 1) ;).
The picture shows the entire list of methods of the Entity class that are already ready for use or may be blocked in the heirs:
- from another heir to the Page class. Ie ... We have a list of available Google Drive documents, on which we perform various actions - open, set labels, work with the pop-up menu and action bar buttons. We describe all this in a separate class (in the examples above, GoogleDocsMainPageContent).
The elements appearing as a result of these actions (the access settings dialog, various modal forms, etc.) are parts of this page. But they can functionally be considered as if in isolation from it.
Such elements can be described in a separate hereditary class Page.
In the same way, for convenience, larger, but more static blocks can be described (for example, a table or document editor).
In all the situations described, objects can be created with an indication of the parent.
The picture shows the entire list of Page class methods that are already ready for use or may be blocked in the heirs:
All the above methods for creating objects for testing are wrapped around class methods in the screenshot:
- using TestObjectFactory . The way so far is good when there are such components that can work on their own, without using the service. For example, a document, a table, a presentation, etc.can be opened without the help of the service, provided that there is an access right and a link to the document. If you need to check just such a situation, you can do so . Further the second use case repeats. But this option I have so far the most undeveloped.
I could talk about the underlying mechanisms and the jokes of the work of this system. But the chapter will be very big. This can be a topic for a separate note.
Any exception that occurs during the execution of an automated test signals problems.
This may be a signal of a bug introduced by the developer. The test itself can be poorly written. And should already fix this screamer. No exception and the tools themselves. The list of Selenium defects can be found here .
But I'm not talking about this.
How do you like this situation? For example, at the manual testing stage, a defect was identified. It is included in the bugtracker, it can be “Minor”. This bug is not fatal to the user, he can easily get around it. At the moment, developers have more important tasks, and the fix of this defect can be left until better times. And now we need a regression test, the path of which runs through the defect described. The test was written, and ... Oh, my God! It falls with some terrible exception that caused our bug.
The situation, I will tell you, is not a pleasant one. The best way out is to make some crutch that would allow catching a mistake and passing the test to the end. Otherwise, a terrible thing can happen - the test will only reach this noncritical error, it will fall and the rest will not be checked. Namely, some grief climbed there - the developer and made chaos there. But writing "crutches" can complicate the test script itself or the classes that describe what is being tested. The defect has already been fixed, and the “crutch” can be forgotten. Over time, the whole project may become like this.
There are even worse situations ...
A vivid example is the occurrence of a StaleElementReferenceException . You can read more here.. This is an example of when, during normal operation of the application under test, situations arise for the processing of which the tool used is simply not designed!
For example, I ran into this when I tried to write “tests” for a list of Google Drive documents and a spreadsheet document. The picture shows, after which action such an exception occurred most often (click on the highlighted green element).
After changing the value of the "asterisk", the further passage of the "test" was like a hike through a minefield. The error occurred when trying to return the "asterisk" to its original state, or to work with the checkbox located to the left, and sometimes when trying to open a document.
Ways to combat this error offer different -from repeating the same actions (searching for an element and performing actions) to increasing the virtual memory of the machine. OK!Suppose the second I can not do. Then the first thing remains for me. (+ ):
try
{
//some actions
}
catch (StaleElementReferenceException e)
{
//same actions again
}
, . , ? ?
, «» Google Drive.
ITestObjectExceptionHandler . . throwableList . , – - - , .
Further.
Page Entity . – . Page Entity . , Page , /**/ . Entity.
.., . , , . , , .
Google Drive , , , StaleElementReferenceException. , Google Drive, .
: StaleElementReferenceException . (!). , Google .
, - , .
For example. :
1. Google Drive
2.
3.
4. «»
5.
, , :
.
, - , , . , . : UnhandledAlertException. - , . , . , , , .
, :
1. ( ) ;
2. .
, , .
:
1. .
2. ( ) .
: . : FAILED TESTS BUT WITHIN SUCCESS PERCENTAGE. :
(!!!). . , , .
I will briefly turn on the cap. Probably, everyone understands that in addition to directly passing the tests, there must be some kind of exhaust, thanks to which one can judge the current state of the product. Well, or we need some kind of test passing test ... These tasks solve logging and reporting.
Since I used java for experiments, my head ached badly when choosing the right logging framework. And to choose was from what! The author of this article very well described the logging situation in java . And at first, I myself did not know what I wanted to get. In addition, it is necessary that a report could be generated from this log.
Eventually:
- I need a simple logger, which could register messages, their level, output data to the console - at least;
- it is very good that it was possible to “cling” to it objects of exceptions, if they arise;
- and quite well - the ability to attach a link to a file to a log entry that turned out during the test;
- Write video with special effects.
I think everything is clear with the first two points. Third. A file can be anything, an xml file, typed text or even a screenshot taken from a browser window. This can be useful when, for example, processing log entries.
As for the reports:
- I need a report that would be formed both during the run of a single test, and after running the entire ate;
- In this report, I would like to see a description of the steps performed on the selected test;
- It is desirable that such detailing could display screenshots or any other visual information.
I will not forget to mention that I chose TestNG as a framework for conducting the tests themselves. Why? First, because it has the ability to generate reports on the passage of tests. But when I studied in more detail, I realized that he had many other advantages, such as integrability with maven and ant , the ability to be used by such continuous integration systems as jenkins and hudson , multi-threaded work ... Very well this wonderful framework is described in this article . Who cares, you can see the documentation.
But, something I get distracted ...
As a logger, I decided to use java.util.logging.Logger . Why?
- it is in any java starting from 1.4, there is no need for extra dependencies on the project;
- it is easy to integrate with browser logs (Mozilla Firefox, for other bowsers, log extraction has not been implemented yet);
- in the case of using other logging libraries ( log4j , slf4j or logback , for example), it is always possible to send messages to their loggers.
But:
- I would like to get rid of the explicit initialization of the logger;
- There are advanced use cases - building a hierarchy of messages. A large number of levels that, in principle, should correspond to debugging information - FINE, FINER, FINEST ... Something extra is needed to be cut off.
- the selected logger, as well as those listed above, cannot attach files to their messages. This is not about FileHandler or something similar.
Now the solution.
In my project, I created a class that I called shortly and clearly - Log. Its main methods are:
- debug (String) - generates messages with the FINE level; I suggest using them for debugging purposes.
- error (String) - generates messages with the SEVERE level, signals of serious errors;
- message (String) - generates messages with the INFO level, signals about the normal state;
- warning (String) - generates messages with the Warning level, signals about possible problems;
- log (Level, String) - for those who still want to generate a message with any of these levels .
The same methods with an additional parameter in the form of an object caught an exception:
- debug (String, Throwable);
- error (String, Throwable);
- log (Level, String, Throwable);
- message (String, Throwable);
- warning (String, Throwable).
Next (note!), Methods with the ability to link a file and write a log:
- debug (String, File);
- error (String, File);
- log (Level, String, File);
- message (String, File);
- warning (String, File).
In fact, all this is a simple wrapping around java.utils.logging.Log and java.utils.logging.LogRecord . These are all static public methods that can be accessed anywhere in the project.
An example of use . You can use instead of comments. However, this is unlikely to surprise anyone.
So, how do I “cling” a file to a log? The fact is that this logger does not create LogRecord objects, but its successor, which I called LogRecWithAttach . It's simple! Who will find 10 differences from the "ancestor", that is 5 points and an increase in karma.
OK! But someone may already have a question: “Author, but why do you need such an excess?” I will try to answer ...
Well, first of all, one of the important functions that I wanted to implement was taking screenshots from the page. I believe that screenshots could be a good addition to the log and reports. And for convenience, as it seemed to me, it would be better if the log entry kept in itself a link to the photo page.
The functions of taking screenshots are performed by a class called Photographer. A list of his methods can be found here:
Its functions are as follows:
1. Removal of a screenshot from the active page, saving it to a file;
2. Creating a log entry with the desired level of importance, adding a link to the saved screenshot to it;
3. You probably noticed that some methods have WebElement as parameters. These methods select an element on the page, take a photo of it and return the element to its original state. Examples of such screenshots were shown earlier. Moreover, the highlight color depends on the “correctness” of the state of the selected page fragment. So, in the screenshot, an incorrect (well, like) state was photographed:
The list of documents with incorrect content (by the “test” condition, a text document and MS Access database should not be loaded) is highlighted in bright orange. This photo was taken by one of the calls to the Photographer.takeAPictureOfAWarning (WebDriver, WebElement, String) method.
True, there is one BUT (!!!) - the elements are allocated using javaScript'a. There was an idea to make projections on screenshots on the coordinates of elements. It even happened. However, for elements that sit inside frames or are on pages with scrolls, such projections (circled areas) were obtained with an offset. From the idea had to be abandoned. But on the other hand - now you can record video with special effects! ..
In general - my photographer, as it were, complements the log.
The second, not bad, opportunity is to attach any file, in principle. For example, is it really bad to find a link to a file in a report that displays a drilldown based on accumulated messages? For example, the one that was loaded into Google Drive from the example above. And this link to open it to view.
So, I smoothly turn to the construction of operational reports on the passage of one or more tests.
To begin with, for those cases when you need to convert the log with the above features, I came up with an interface that I called IlogConverter .
Here is an example of use. For example, we additionally use the log4j library. We need to have this file data logger. You can create a class whose object would check if the incoming message has an attachment. And if it is, an additional message log4j is generated. And so that this object could "listen" to the log, I suggest this method - Log. addConverter (ILogConverter). This mechanism works thanks to this code .
And this is what happens when a new message appears:
converting.convert (rec);
I made for my framework something like a “standard” implementation of the named interface, which is responsible for the formation of the TestNG report details. I called it ConverterToTestNGReport. What she does? It intercepts messages from the log, according to a certain html template (which I am currently hard-coding in the form of a final attribute) forms a report line:
Reporter.setEscapeHtml (false);
Reporter.log (htmlInjection);
Additionally - if the message has an attached screenshot - then we turn it into "<img src = \", if any other file is "<a href = \". True, there are only links to files. While the files themselves on these links for some reason do not open. But now it is enough for me (!!!).
The result is the following report fragments. I want to draw attention to the fact that these are not customized reports.
Of course, it may be rude, but for the first time it will do. I think it's worth figuring out whether it is possible to customize the granularity of the TestNG report. If so, then you can make such a caste and use it instead of the hard-coded template.
And so far everything seems to be good. But this is not enough. I will explain ...
OK! We have an easy to use logger. We even have a tool that allows you to build a report. But you also need to ensure that the status of the passed test is synchronized with the maximum level of the log messages.
I explain. If nothing is done, the test will have the status of SUCCESS if it passed to the end and did not fall. If in the process of its execution there was an uncaught exception, then the test will have the status FAILURE . If the test is dependent, and the defining test did not pass - SKIP . And now let us ask ourselves the question: “Is it possible to consider the test as fully passed if something happened during the course of its execution, because of which records with a level of WARNING or even SEVERE fell into the log?”. Ie, the passed test has the status of SUCCESS, however, looking at its detailing, we see yellow and red lines. I consider it, at least, misinformation.
My solution is below.
I implemented a class that links the thread (and TestNG supports multithreading) with the test itself . Further, storage of log messages with reference to a specific test (an instance of a class that implements ITestResult ) is implemented. At this moment I will not stop. I will demonstrate how the log message is bound to the tests. It is produced by the object of the mentioned ConverterToTestNGReport . Finally, I implemented the ITestListener interface as a class that:
- before starting the execution of tests (test methods), add an instance of ConverterToTestNGReport to the log as a listener.
- after executing each test method, synchronizes the status of its passing with that recorded the log:
1. if the test has passed to the end, but there are messages in the log with the SEVERE level - the status of its passing changes to FAILURE.
2. if the test passed to the end and no messages with the SEVERE or WARNING level were recorded - its status remains unchanged - SUCCESS .
3. If messages with the WARNING level were fixed, then I suggest changing its status to SUCCESS_PERCENTAGE_FAILURE . Or allow the "user" to choose the most appropriate status .
And now the final!
To build the reports described above, you need to perform the following action:
@Listeners ({ReportBuildingTestListener.class})
public class NewTest {
In conclusion to this chapter, I would like to say that logging is already largely automated. Those. graphically, any interactive action is recorded — the input of values ​​(before input and after input), clicks and submit (before an action is taken), the appearance of new tabs / browser windows, the results of clicking on links. For all the rest, certain "standard" text messages are formed.
Some of these functions are assumed by the classes, which are a layer between the tools that make up Selenium and the classes of a higher level.
This ExtendedEventFiringWebDriver is the heir to EventFiringWebDriver , which actually does all the dirty work. It is “dried” by the implementation of the IExtendedWebDriverEventListener interface (extended WebDriverEventListener ), which is part of the WebDriverEncapsulation as a nested class. Some of the logging functions are taken by WindowSwitcher and SingleWindow , the browser window / tab manager and the window / tab view itself.
Self logging can be configured for the project as a whole. However, I would like to talk about configuring and named classes in another article that may be ...
So! One of the tasks was to try to find analogues and try to work with them. It broadens the mind. In addition, many useful features are already implemented. And then why write a bike that does the same thing? Better anyway anyway! It is much more useful to see how the necessary functions are implemented, and learn how to use them.
And actually, what is the analogy?
The functionality that I completely refused to implement is working with objects whose classes implement WebElement in one form or another. Those. I decided to do nothing at all in this direction, because came across a video - a report about this wonderful framework. And rather there was an interest to try it in practice. This I fully managed during experiments with Google Drive. Moreover, it was possible to establish interaction with yandex-qatools htmlelements without any efforts. For example, here are the constructors of an abstract class, from which all Page - objects on this demo project are inherited. But I got such a set of attributes for a class that describes the general properties of all Google documents. Do you see the WebElement here? Everywhere typed attributes describe the page fields, and some of them are included in the packages of the named library. Some of the elements that were needed were not there. But with this tool they were easily described .
And all this was possible without sacrificing the functionality on which I was laying. Quite the contrary. The decomposition of the page descriptions, which I mentioned earlier, has become even more visual. This is a win-win situation!
Another interesting method of working with fields on the page is presented on the example of the heir of the PageObject class, which is implemented in the thucydides framework (it will be discussed later).
So, there are more than enough arguments in favor of not doing useless work in this direction.
In the course of my "research" I came across another tool - the thucydides already mentioned. Reading articles about it, looking at the repository and trying to write something, I came to the conclusion that it is best to use its capabilities and somehow make friends with it. And believe me, there are many reasons for this:
- the step-by-step organization of automated tests;
- the ability to describe scripts using the JBehave framework , and not just in the form of working code;
- A set of convenient methods and tools for working with pages;
- integration with JIRA and other issue trackers.
- reports. About coverage. Story. Etc.
one.
2
3
To tell the truth, I had an idea to make a second demo project in conjunction with thucydides. Not enough time either, or enthusiasm. But maybe I will come back to him. I just want to share an idea and find out how feasible it is. And now I’ll tell you what I managed to realize:
- implemented "cross-driver" .
- The ability to work with several open browsers in one test.
- implemented configurability.
You can make one setting for the whole project with the indication of the browser used, Capabilities parameters, timeouts (wait timeouts for Webdriver, your own - related to the waiting times for new windows / tabs, switching between them).
You can make a set of additional / individual settings that can partially overlap the total.
Those. - in general, the browser, its capabilities / settings and a set of timeouts are indicated. In additional - only the browser and features. In the case of creating a new instance of Webdriver for this setting, the specified browser will open, configured as necessary, but its timeouts will be taken from the general setting.
This can be used, for example, to parameterize tests. Example parameterization for TestNG . My example .
- mechanisms for controlling windows / tabs of the browser (including loading the necessary pages) without reference to the content.
- a set of tools that hide an instance of webdriver at the highest levels . All work happens through them. What is the gain - all the black work associated with the use of interfaces such as HasInputDevices , Options , JavascriptExecutor , etc. already done. Examples
If there is a sense, I will tell about the above in another article.
- described in 4 and 5, the method of organizing objects that implement interactive interaction.
- described in 6 logging and method of constructing reports on the example of the TestNG report.
As you can see from the above, I tried to focus on working with WebDriver and related components.
What I do not have in thucydides (an impression with a still distant sign) in this regard:
- the possibility of tracking and building a map of the opened pages. Classes Pages and PageUrls . I can only go directly to the link, go back and forth;
- tools for working with elements, an additional set of expectations . PageObject .
And now the idea in general:
1. I want to use my mechanism to open the browser.
2. The object of the successor of the Entity class described earlier is a wrapper for the Pages object. So I will try to combine the functionality of both classes.
3. You can somehow combine the functionality of the heirs of the Page (Chapter 4) and PageObject classes (thucydides).
4. Let's return to the report with number 3. As you can see, the steps can be very large (the very first one passed in a minute, there are 30 or more seconds). Surely these are not atomic steps. The screenshots presented in this report are the states at the end of the step. What if the step failed? For example, I would like to see illustrations of what caused the product to be in an incorrect state.
So, I think, if we turn the focus described in Chapter 6, provided that there is an opportunity to create a detail of a step in a report that can be called from the outside during its execution, that would be great!
5. Everything else does thucydides.
Like that!
At the moment, I think the ideal option is not to develop some kind of monstrous system, the functionality of which cannot be expanded through the use of third-party tools. , :
— ;
— , , Selenium Webdriver .
.
.
Google Drive. Yandex QA Tools Html Elements. , - . . FireFox, Chrome, Safari RemoteWebDriver, Chrome.
*.jar . - , :
1. jdk7.
2. maven project.
3. . selenium ( 2.35) TestNG!
4. pom.xml .
, . . , - . , . . - . . . , , , ?
, . …
, , , . , – – , - .
, . , - . , , . - , , .
. , . - . In this case:
— . :
FireFox ( 21.0);
Chrome (26-30);
Internet Explorer ( 9.0.8112);
Safari (5.1.7 Windows);
Opera (12.16)
HTMLUnitDriver – ;
RemoteWebDriver c .
— iPhone Android, PhantomJS.
— TestNG. .
— Junit .
— .
, - . ( ), , .
See you!
I would like to talk about the problems that I encountered in the process of mastering Selenium WebDriver, with their solution and how these solutions, in principle, can be used. All this is presented in the form of a prototype framework, the link to which will be at the end of the article.
In this post I want to share my ideas for the implementation of the Page Object template, how to handle errors that occur during the execution of tests, tell a little about logging. And also to share information about some of the tools that are implemented using Selenium WebDriver, and their own work.
')
The plan of my article is as follows:
1. Captain obvious, instead of entry.
2. A little about yourself, you must introduce yourself ...
3. Why Selenium?
4. About Page Object ...
5. Not a bug, but a feature!
6. And again about logging and reporting.
7. Are there no analogues?
8. Promised links.
9. In conclusion.
So let's go!
1. Captain obvious, instead of entry.
I don’t think that it makes sense to explain what software testing is, what strategies and models exist, what is its role in software development and quality assurance, and what projects and users are at risk if this process is absent. Besides, now programming is not an art, but a craft for many millions of specialists. Projects are implemented in a short time and with limited budgets, and often they can be monsters with simply immense functionality. Moreover, products can vary intensively from version to version, let's say improve. And each such "improvement" can be fatal.
In such conditions, automated testing plays a key role. What models, strategies and methods are better to find in other sources. My article is not about that. It is dedicated to automation tools for testing client-side web applications.
2. A little about yourself, you must introduce yourself ...
I have been doing regression automated testing for four years now. We test the desktop client of the product and the classes that describe its business logic. About the tool that is used on our project can be read here . During this time, many UI tests were automated, unit tests, a lot of things ... It was here that I watched the evolution of test automation, and myself participated in it.
At first, these were scripts written directly on the knee using automatic recording tools, in which the logic of the script was inseparable from the ways of interaction with the objects under test. As the number of tests increased, it became obvious to everyone that some kind of framework should be developed and used for automated scripts. Any sneeze in the application under test had very unfortunate consequences for the entire test run.
Then it became necessary to use software interfaces for interacting with the product and databases, because what was shown by the dialog forms was not enough ...
However, something I got carried away. But it was here that I realized how important it is to think from the very beginning of the strategy and tactics of interacting with the product of developers for its effective automated verification.
The tendency of applications to switch from their desktop clients to web clients cannot be noticed either by the deaf-blind or the person born and living somewhere in Antarctica. And development for web is now mainstream, it is not a secret to anyone.
I decided not to stand aside and try in my spare time to master something that is used to automate the functional testing of the frontend - part and to check its compatibility with various client platforms. And besides, there is some probability that the web client on our project to be ...
I chose Selenium Webdriver.
3. Why Selenium?
Why is Selenium?
Once upon a time it was a small tool that was used only by desperate enthusiasts from the world of web development. Now this is a formidable weapon that not only many thousands of automators around the world, but even the frontend - the developers have adopted to combat defects.
You can read about the product line of the Selenium project here or here .
But I must say that this is not a tool for automated testing directly . For example, Selenium Webdriver provides a variety of software tools and interfaces for emulating user interaction with the browser and content loaded in its window. These funds cover almost 100% of the possible variants of such interaction. All this may find wider application. But, for example, he cannot, by default, generate a report on the passage of a particular test.
Next, there are some other problems. Below are some of them:
The WebDriver and WebElement interfaces provide very limited functionality . It is mainly aimed at finding the elements of the active page and performing the simplest actions on the elements (entering values, clicking, etc.). Many other useful features are provided by interfaces such as HasInputDevices, Locatable, Options, Alert , etc. And using only WebDriver and WebElement will severely limit the capabilities of automated tests.
Every browser has its own driver . Now it is:
ChromeDriver;
FirefoxDrive;
HtmlUnitDriver (for Unit - tests at the level of the html-document);
InternetExplorerDrive ;
OperaDriver ;
RemoteWebDriver (my favorite, good for running tests on remote machines);
SafariDriver ;
Iphonedriver ;
AndroidDriver ;
PhantomJSDriver (some kind of exotic browser, I did not have enough patience to build it on my machine).
I think that the Yandex team will also release its driver soon.
A wide choice, for every taste and for any needs. But if in the test project some or some of these are clearly used, this will create problems when you need to quickly switch from browser to browser or use several at once.
Therefore, I would like to have some single way or interface to create the desired instance. It is desirable that he took some kind of setting that can always be changed as a parameter. This setting should include the browser type, features , timeouts . Well, the opportunity to have a blessing, too.
Writing classes that would be responsible for interactive work with pages. PageFactory is a great tool. However, its use becomes inconvenient due to the loss of clarity of code classes that describe very large pages, and duplicate elements are fraught with copy-paste. Output - select duplicate elements in separate blocks (classes) and use blocks already. But this does not solve the problem to the end. What if your service can work by opening pages on separate tabs or browser windows, do you have duplicate blocks on all these windows? And some of them also lie on the frames! I experimented on this and that , and immediately ran into the problem described. And the situation is aggravated by the fact that the instance implementing WedDriver sees only the active page (window / tab) or frame.
I would like to be able to split the page into such blocks that a working test could memorize and work with them directly, without calling driver.switchTo (). Window (somehandle) or driver.switchTo (). Frame (frameIdentity) .
Therefore, I set myself the task not only to master the above tool, but to make a project, or even better, a prototype framework based on Selenium Webdriver. And in addition - try to find analogues and make it so that something can easily integrate with them. I just think that if the task is this way, then the submitted material is absorbed much easier.
Java was selected as the language. Development environment - Eclipse.
I will not talk about how this path was gradually overcome. It is better to continue with the description of the problems and their solutions. Maybe I will not open America, but I will be glad if someone finds something useful for themselves ...
4. About Page Object ...
I think there are few such automators who would not know about the design - Page Object pattern. By the way, this pattern is also applicable for writing autotests UI of some desktop application. But now we will talk about its implementation in Java for the web interface.
About the reception you can read here .
Now I will show you at the moment the working code of some not very successful test.
Code
I think everyone understands what is good when you want to write a script where you need to make one input somewhere and 2-3 clicks on something. Such a test you quickly fix, if something is correct (!!!) has changed.
If the test should test more than the steps described above, it will result in an inconvenient to read sheet of code. And now we multiply all this by a huge test project. Have you imagined how a poor tester cries, what supports him after every Google change? And so that he does not cry like that, we need classes through which the test performs its actions on the page. And if anything has changed (again, this change is a feature, not a bug introduced), but the edits will be local, and not in the whole project.
There are many implementations of Page Object, including, I think, on your projects. Below I will demonstrate my implementation and the same test:
public interface IPerformsClickOnALink
public interface IPerformsSearch
public class SearchBar
public class LinksAreFound
public class AnyPage
public class google
test
And if you work hard and do something like this setup , the test can work like this .
Many experienced automators, probably, now thought: “Well, yes, Page Factory uses ... The author for some reason has complicated the description of the google homepage - you can also describe it as one class! Well, except that some kind of incomprehensible annotation appeared ... @PageMethod is called. The descendants of the classes Page and Entity ... We saw a similar here . The author, we all have already seen! ”.
Well yes. This was a simple example. I intend to complicate the description of the main page to show that it is possible to break the description of the same page into two independent classes. I will tell about this annotation below ...
And now the example is more complicated ...
The picture shows some situation.
What the test does:
1 goes to docs.google.com
2 opens an accessible Google document.
3 on the main page of the service causes the sharing dialog;
4 from the open document the same dialogue is called.
For clarity, the entire sequence of actions is shown in the form of screenshots. Active elements are highlighted:
Necessary: somehow handle both instances of this dialog in different windows. And that's what happened with me. This is the described precondition.
Now I will click on the field in which they are asked to enter names, email addresses, etc., in the pop-up item that appears, click "Cancel", after which the "Finish" button will be clicked. Actions will be performed in both windows. For clarity, the following sequence in the form of pictures.
That's how we coped with the task effortlessly. . This is the getCustomizingAccessDialog () method code for the document and home page.
Those. there is a pop-up block of elements located inside the frame, which we used as an object that exists independently and is picked up when needed ... We know that it is on the page of any document and on the main page of the service, and we get it from them.
In fact, for a long time I tried to find the optimal way to organize and describe one or another test object in different sources. Although it may be looking bad ...
Obviously, you need to group page elements into blocks for reuse. These blocks can even be transferred outside under the guise of independent objects and work directly with them. If this is not done, then the class will grow in the number of attributes and methods. Need decomposition. Otherwise, you will receive such objects:
If you look at the example described above, then methods of work with each type of pop-up dialog are added to the methods of working with the main content - access settings, modal forms of renaming, choice of options, comments in documents.
The above solution can be implemented painlessly when testing what is loaded in a single window (single tab) of the browser.
But in the case of the described rather complex behavior, you will have to somehow switch the webdriver from page to page, not forgetting to switch to the required frame.
And there was an idea! And what if you teach each such object to remember the path to itself and automatically switch the webdriver when calling the method of interactive interaction with the page.
This, it seemed to me, perhaps, if we deviate somewhat from the concept of “page”. Those. under it, I will mean not downloaded content, but a browser window with content loaded in it. This window has a unique string identifier by which you can easily identify it among others. OK! That is, before calling the method, it will switch to the desired window.
So. And what to do with frames? You can also make it so that each block of elements knows, in addition, which frame to switch to.
So I build a path.
As can be seen from the above list of class constructors, which I called Page (the name is very conditional, perhaps the class should be renamed), you can write a description of the loaded page as a whole, or in parts, in addition to this, indicating another object like it as a parent ) when it is necessary. Thus it is possible to build a connected graph of blocks of elements. Something similar to hierarchy can turn out.
OK! Well, the paths indicated, the hierarchy was built. What's next? How to implement automatic switching when you call, such as a click or a string in the field? Switching itself is implemented in this way.
But! Many may ask the question: “Do we need to call it back in each method?” No. Below I will try to explain how it works.
In my decision, I propose to use not the objects created by the designer, but their deputies (proxy, in other words). For this, the cglib library was used. And to be more precise, the MethodInterceptor to intercept callable methods, the Enhancer to create the proxy objects themselves and the Callback . Using the above types eliminates the need, in the future, to create a set of interfaces for each Page heir (see java.lang.reflect.Proxy ) and makes the implementation more flexible. However - there is a restriction - the heir of the Page class should not be final !!!
Those. when we try to call a particular method of some Page - object (heir of the Page class), this method intercepts and “switches” the object to itself. And after that the execution of the method itself.
Perfectly! But is this kind of interception occurring when each method is called? The question, I note fair. We must not forget that the Page-object is also an object of the java language. Such an object can have public or protected methods that are not related to browser interactivity. How to be? Obviously, you need to think of a way to divide methods into methods that are responsible for interactive interaction and all the others.
The @PageMethod annotation serves as such a “delimiter”. Ie, if the method is marked with such a marker, then a preliminary switch to the page or its fragment will be performed ...
In general, the solution looks like this.
Next, a little about the possibilities of creating Page - objects.
Any block of elements or page view can be created in three main ways:
- from the heir to the class Entity - this is something like a service model, open to the main or any other page, as a whole. In the examples described, this is Google (search engine), GoogleDocsApplication (Google Drive). By the way, the example with Google demonstrates that with the “application”, in principle, we can also work as with the Page object. I kind of included elements with a panel and search results in it.
“Entity” can generate those objects that are in the first (main) open window / tab of the service (example with Google Drive, “ docs.getContent (); ” - wrapping around one of these methods, example with Google - “ searchBar = get (SearchBar.class); ”- call one of these methods) or those that exist in newly appearing windows / tabs ( AnyGoogleDocument anyDocument = docs.get (AnyGoogleDocument.class, 1) ;).
The picture shows the entire list of methods of the Entity class that are already ready for use or may be blocked in the heirs:
- from another heir to the Page class. Ie ... We have a list of available Google Drive documents, on which we perform various actions - open, set labels, work with the pop-up menu and action bar buttons. We describe all this in a separate class (in the examples above, GoogleDocsMainPageContent).
The elements appearing as a result of these actions (the access settings dialog, various modal forms, etc.) are parts of this page. But they can functionally be considered as if in isolation from it.
Such elements can be described in a separate hereditary class Page.
In the same way, for convenience, larger, but more static blocks can be described (for example, a table or document editor).
In all the situations described, objects can be created with an indication of the parent.
The picture shows the entire list of Page class methods that are already ready for use or may be blocked in the heirs:
All the above methods for creating objects for testing are wrapped around class methods in the screenshot:
- using TestObjectFactory . The way so far is good when there are such components that can work on their own, without using the service. For example, a document, a table, a presentation, etc.can be opened without the help of the service, provided that there is an access right and a link to the document. If you need to check just such a situation, you can do so . Further the second use case repeats. But this option I have so far the most undeveloped.
I could talk about the underlying mechanisms and the jokes of the work of this system. But the chapter will be very big. This can be a topic for a separate note.
5. Not a bug, but a feature!
Any exception that occurs during the execution of an automated test signals problems.
This may be a signal of a bug introduced by the developer. The test itself can be poorly written. And should already fix this screamer. No exception and the tools themselves. The list of Selenium defects can be found here .
But I'm not talking about this.
How do you like this situation? For example, at the manual testing stage, a defect was identified. It is included in the bugtracker, it can be “Minor”. This bug is not fatal to the user, he can easily get around it. At the moment, developers have more important tasks, and the fix of this defect can be left until better times. And now we need a regression test, the path of which runs through the defect described. The test was written, and ... Oh, my God! It falls with some terrible exception that caused our bug.
The situation, I will tell you, is not a pleasant one. The best way out is to make some crutch that would allow catching a mistake and passing the test to the end. Otherwise, a terrible thing can happen - the test will only reach this noncritical error, it will fall and the rest will not be checked. Namely, some grief climbed there - the developer and made chaos there. But writing "crutches" can complicate the test script itself or the classes that describe what is being tested. The defect has already been fixed, and the “crutch” can be forgotten. Over time, the whole project may become like this.
There are even worse situations ...
A vivid example is the occurrence of a StaleElementReferenceException . You can read more here.. This is an example of when, during normal operation of the application under test, situations arise for the processing of which the tool used is simply not designed!
For example, I ran into this when I tried to write “tests” for a list of Google Drive documents and a spreadsheet document. The picture shows, after which action such an exception occurred most often (click on the highlighted green element).
After changing the value of the "asterisk", the further passage of the "test" was like a hike through a minefield. The error occurred when trying to return the "asterisk" to its original state, or to work with the checkbox located to the left, and sometimes when trying to open a document.
Ways to combat this error offer different -from repeating the same actions (searching for an element and performing actions) to increasing the virtual memory of the machine. OK!Suppose the second I can not do. Then the first thing remains for me.
try
{
//some actions
}
catch (StaleElementReferenceException e)
{
//same actions again
}
, . , ? ?
, «» Google Drive.
ITestObjectExceptionHandler . . throwableList . , – - - , .
Further.
Page Entity . – . Page Entity . , Page , /**/ . Entity.
.., . , , . , , .
Google Drive , , , StaleElementReferenceException. , Google Drive, .
: StaleElementReferenceException . (!). , Google .
, - , .
For example. :
1. Google Drive
2.
3.
4. «»
5.
, , :
.
, - , , . , . : UnhandledAlertException. - , . , . , , , .
, :
1. ( ) ;
2. .
, , .
:
1. .
2. ( ) .
: . : FAILED TESTS BUT WITHIN SUCCESS PERCENTAGE. :
(!!!). . , , .
6. .
I will briefly turn on the cap. Probably, everyone understands that in addition to directly passing the tests, there must be some kind of exhaust, thanks to which one can judge the current state of the product. Well, or we need some kind of test passing test ... These tasks solve logging and reporting.
Since I used java for experiments, my head ached badly when choosing the right logging framework. And to choose was from what! The author of this article very well described the logging situation in java . And at first, I myself did not know what I wanted to get. In addition, it is necessary that a report could be generated from this log.
Eventually:
- I need a simple logger, which could register messages, their level, output data to the console - at least;
- it is very good that it was possible to “cling” to it objects of exceptions, if they arise;
- and quite well - the ability to attach a link to a file to a log entry that turned out during the test;
I think everything is clear with the first two points. Third. A file can be anything, an xml file, typed text or even a screenshot taken from a browser window. This can be useful when, for example, processing log entries.
As for the reports:
- I need a report that would be formed both during the run of a single test, and after running the entire ate;
- In this report, I would like to see a description of the steps performed on the selected test;
- It is desirable that such detailing could display screenshots or any other visual information.
I will not forget to mention that I chose TestNG as a framework for conducting the tests themselves. Why? First, because it has the ability to generate reports on the passage of tests. But when I studied in more detail, I realized that he had many other advantages, such as integrability with maven and ant , the ability to be used by such continuous integration systems as jenkins and hudson , multi-threaded work ... Very well this wonderful framework is described in this article . Who cares, you can see the documentation.
But, something I get distracted ...
As a logger, I decided to use java.util.logging.Logger . Why?
- it is in any java starting from 1.4, there is no need for extra dependencies on the project;
- it is easy to integrate with browser logs (Mozilla Firefox, for other bowsers, log extraction has not been implemented yet);
- in the case of using other logging libraries ( log4j , slf4j or logback , for example), it is always possible to send messages to their loggers.
But:
- I would like to get rid of the explicit initialization of the logger;
- There are advanced use cases - building a hierarchy of messages. A large number of levels that, in principle, should correspond to debugging information - FINE, FINER, FINEST ... Something extra is needed to be cut off.
- the selected logger, as well as those listed above, cannot attach files to their messages. This is not about FileHandler or something similar.
Now the solution.
In my project, I created a class that I called shortly and clearly - Log. Its main methods are:
- debug (String) - generates messages with the FINE level; I suggest using them for debugging purposes.
- error (String) - generates messages with the SEVERE level, signals of serious errors;
- message (String) - generates messages with the INFO level, signals about the normal state;
- warning (String) - generates messages with the Warning level, signals about possible problems;
- log (Level, String) - for those who still want to generate a message with any of these levels .
The same methods with an additional parameter in the form of an object caught an exception:
- debug (String, Throwable);
- error (String, Throwable);
- log (Level, String, Throwable);
- message (String, Throwable);
- warning (String, Throwable).
Next (note!), Methods with the ability to link a file and write a log:
- debug (String, File);
- error (String, File);
- log (Level, String, File);
- message (String, File);
- warning (String, File).
In fact, all this is a simple wrapping around java.utils.logging.Log and java.utils.logging.LogRecord . These are all static public methods that can be accessed anywhere in the project.
An example of use . You can use instead of comments. However, this is unlikely to surprise anyone.
So, how do I “cling” a file to a log? The fact is that this logger does not create LogRecord objects, but its successor, which I called LogRecWithAttach . It's simple! Who will find 10 differences from the "ancestor", that is 5 points and an increase in karma.
OK! But someone may already have a question: “Author, but why do you need such an excess?” I will try to answer ...
Well, first of all, one of the important functions that I wanted to implement was taking screenshots from the page. I believe that screenshots could be a good addition to the log and reports. And for convenience, as it seemed to me, it would be better if the log entry kept in itself a link to the photo page.
The functions of taking screenshots are performed by a class called Photographer. A list of his methods can be found here:
Its functions are as follows:
1. Removal of a screenshot from the active page, saving it to a file;
2. Creating a log entry with the desired level of importance, adding a link to the saved screenshot to it;
3. You probably noticed that some methods have WebElement as parameters. These methods select an element on the page, take a photo of it and return the element to its original state. Examples of such screenshots were shown earlier. Moreover, the highlight color depends on the “correctness” of the state of the selected page fragment. So, in the screenshot, an incorrect (well, like) state was photographed:
The list of documents with incorrect content (by the “test” condition, a text document and MS Access database should not be loaded) is highlighted in bright orange. This photo was taken by one of the calls to the Photographer.takeAPictureOfAWarning (WebDriver, WebElement, String) method.
True, there is one BUT (!!!) - the elements are allocated using javaScript'a. There was an idea to make projections on screenshots on the coordinates of elements. It even happened. However, for elements that sit inside frames or are on pages with scrolls, such projections (circled areas) were obtained with an offset. From the idea had to be abandoned. But on the other hand - now you can record video with special effects! ..
In general - my photographer, as it were, complements the log.
The second, not bad, opportunity is to attach any file, in principle. For example, is it really bad to find a link to a file in a report that displays a drilldown based on accumulated messages? For example, the one that was loaded into Google Drive from the example above. And this link to open it to view.
So, I smoothly turn to the construction of operational reports on the passage of one or more tests.
To begin with, for those cases when you need to convert the log with the above features, I came up with an interface that I called IlogConverter .
Here is an example of use. For example, we additionally use the log4j library. We need to have this file data logger. You can create a class whose object would check if the incoming message has an attachment. And if it is, an additional message log4j is generated. And so that this object could "listen" to the log, I suggest this method - Log. addConverter (ILogConverter). This mechanism works thanks to this code .
And this is what happens when a new message appears:
converting.convert (rec);
I made for my framework something like a “standard” implementation of the named interface, which is responsible for the formation of the TestNG report details. I called it ConverterToTestNGReport. What she does? It intercepts messages from the log, according to a certain html template (which I am currently hard-coding in the form of a final attribute) forms a report line:
Reporter.setEscapeHtml (false);
Reporter.log (htmlInjection);
Additionally - if the message has an attached screenshot - then we turn it into "<img src = \", if any other file is "<a href = \". True, there are only links to files. While the files themselves on these links for some reason do not open. But now it is enough for me (!!!).
The result is the following report fragments. I want to draw attention to the fact that these are not customized reports.
Of course, it may be rude, but for the first time it will do. I think it's worth figuring out whether it is possible to customize the granularity of the TestNG report. If so, then you can make such a caste and use it instead of the hard-coded template.
And so far everything seems to be good. But this is not enough. I will explain ...
OK! We have an easy to use logger. We even have a tool that allows you to build a report. But you also need to ensure that the status of the passed test is synchronized with the maximum level of the log messages.
I explain. If nothing is done, the test will have the status of SUCCESS if it passed to the end and did not fall. If in the process of its execution there was an uncaught exception, then the test will have the status FAILURE . If the test is dependent, and the defining test did not pass - SKIP . And now let us ask ourselves the question: “Is it possible to consider the test as fully passed if something happened during the course of its execution, because of which records with a level of WARNING or even SEVERE fell into the log?”. Ie, the passed test has the status of SUCCESS, however, looking at its detailing, we see yellow and red lines. I consider it, at least, misinformation.
My solution is below.
I implemented a class that links the thread (and TestNG supports multithreading) with the test itself . Further, storage of log messages with reference to a specific test (an instance of a class that implements ITestResult ) is implemented. At this moment I will not stop. I will demonstrate how the log message is bound to the tests. It is produced by the object of the mentioned ConverterToTestNGReport . Finally, I implemented the ITestListener interface as a class that:
- before starting the execution of tests (test methods), add an instance of ConverterToTestNGReport to the log as a listener.
- after executing each test method, synchronizes the status of its passing with that recorded the log:
1. if the test has passed to the end, but there are messages in the log with the SEVERE level - the status of its passing changes to FAILURE.
2. if the test passed to the end and no messages with the SEVERE or WARNING level were recorded - its status remains unchanged - SUCCESS .
3. If messages with the WARNING level were fixed, then I suggest changing its status to SUCCESS_PERCENTAGE_FAILURE . Or allow the "user" to choose the most appropriate status .
And now the final!
To build the reports described above, you need to perform the following action:
@Listeners ({ReportBuildingTestListener.class})
public class NewTest {
In conclusion to this chapter, I would like to say that logging is already largely automated. Those. graphically, any interactive action is recorded — the input of values ​​(before input and after input), clicks and submit (before an action is taken), the appearance of new tabs / browser windows, the results of clicking on links. For all the rest, certain "standard" text messages are formed.
Some of these functions are assumed by the classes, which are a layer between the tools that make up Selenium and the classes of a higher level.
This ExtendedEventFiringWebDriver is the heir to EventFiringWebDriver , which actually does all the dirty work. It is “dried” by the implementation of the IExtendedWebDriverEventListener interface (extended WebDriverEventListener ), which is part of the WebDriverEncapsulation as a nested class. Some of the logging functions are taken by WindowSwitcher and SingleWindow , the browser window / tab manager and the window / tab view itself.
Self logging can be configured for the project as a whole. However, I would like to talk about configuring and named classes in another article that may be ...
7. Are there no analogues?
So! One of the tasks was to try to find analogues and try to work with them. It broadens the mind. In addition, many useful features are already implemented. And then why write a bike that does the same thing? Better anyway anyway! It is much more useful to see how the necessary functions are implemented, and learn how to use them.
And actually, what is the analogy?
The functionality that I completely refused to implement is working with objects whose classes implement WebElement in one form or another. Those. I decided to do nothing at all in this direction, because came across a video - a report about this wonderful framework. And rather there was an interest to try it in practice. This I fully managed during experiments with Google Drive. Moreover, it was possible to establish interaction with yandex-qatools htmlelements without any efforts. For example, here are the constructors of an abstract class, from which all Page - objects on this demo project are inherited. But I got such a set of attributes for a class that describes the general properties of all Google documents. Do you see the WebElement here? Everywhere typed attributes describe the page fields, and some of them are included in the packages of the named library. Some of the elements that were needed were not there. But with this tool they were easily described .
And all this was possible without sacrificing the functionality on which I was laying. Quite the contrary. The decomposition of the page descriptions, which I mentioned earlier, has become even more visual. This is a win-win situation!
Another interesting method of working with fields on the page is presented on the example of the heir of the PageObject class, which is implemented in the thucydides framework (it will be discussed later).
So, there are more than enough arguments in favor of not doing useless work in this direction.
In the course of my "research" I came across another tool - the thucydides already mentioned. Reading articles about it, looking at the repository and trying to write something, I came to the conclusion that it is best to use its capabilities and somehow make friends with it. And believe me, there are many reasons for this:
- the step-by-step organization of automated tests;
- the ability to describe scripts using the JBehave framework , and not just in the form of working code;
- A set of convenient methods and tools for working with pages;
- integration with JIRA and other issue trackers.
- reports. About coverage. Story. Etc.
one.
2
3
To tell the truth, I had an idea to make a second demo project in conjunction with thucydides. Not enough time either, or enthusiasm. But maybe I will come back to him. I just want to share an idea and find out how feasible it is. And now I’ll tell you what I managed to realize:
- implemented "cross-driver" .
- The ability to work with several open browsers in one test.
- implemented configurability.
You can make one setting for the whole project with the indication of the browser used, Capabilities parameters, timeouts (wait timeouts for Webdriver, your own - related to the waiting times for new windows / tabs, switching between them).
You can make a set of additional / individual settings that can partially overlap the total.
Those. - in general, the browser, its capabilities / settings and a set of timeouts are indicated. In additional - only the browser and features. In the case of creating a new instance of Webdriver for this setting, the specified browser will open, configured as necessary, but its timeouts will be taken from the general setting.
This can be used, for example, to parameterize tests. Example parameterization for TestNG . My example .
- mechanisms for controlling windows / tabs of the browser (including loading the necessary pages) without reference to the content.
- a set of tools that hide an instance of webdriver at the highest levels . All work happens through them. What is the gain - all the black work associated with the use of interfaces such as HasInputDevices , Options , JavascriptExecutor , etc. already done. Examples
If there is a sense, I will tell about the above in another article.
- described in 4 and 5, the method of organizing objects that implement interactive interaction.
- described in 6 logging and method of constructing reports on the example of the TestNG report.
As you can see from the above, I tried to focus on working with WebDriver and related components.
What I do not have in thucydides (an impression with a still distant sign) in this regard:
- the possibility of tracking and building a map of the opened pages. Classes Pages and PageUrls . I can only go directly to the link, go back and forth;
- tools for working with elements, an additional set of expectations . PageObject .
And now the idea in general:
1. I want to use my mechanism to open the browser.
2. The object of the successor of the Entity class described earlier is a wrapper for the Pages object. So I will try to combine the functionality of both classes.
3. You can somehow combine the functionality of the heirs of the Page (Chapter 4) and PageObject classes (thucydides).
4. Let's return to the report with number 3. As you can see, the steps can be very large (the very first one passed in a minute, there are 30 or more seconds). Surely these are not atomic steps. The screenshots presented in this report are the states at the end of the step. What if the step failed? For example, I would like to see illustrations of what caused the product to be in an incorrect state.
So, I think, if we turn the focus described in Chapter 6, provided that there is an opportunity to create a detail of a step in a report that can be called from the outside during its execution, that would be great!
5. Everything else does thucydides.
Like that!
At the moment, I think the ideal option is not to develop some kind of monstrous system, the functionality of which cannot be expanded through the use of third-party tools. , :
— ;
— , , Selenium Webdriver .
.
8. .
.
Google Drive. Yandex QA Tools Html Elements. , - . . FireFox, Chrome, Safari RemoteWebDriver, Chrome.
*.jar . - , :
1. jdk7.
2. maven project.
3. . selenium ( 2.35) TestNG!
4. pom.xml .
, . . , - . , . . - . . . , , , ?
9. .
, . …
, , , . , – – , - .
, . , - . , , . - , , .
. , . - . In this case:
— . :
FireFox ( 21.0);
Chrome (26-30);
Internet Explorer ( 9.0.8112);
Safari (5.1.7 Windows);
Opera (12.16)
HTMLUnitDriver – ;
RemoteWebDriver c .
— iPhone Android, PhantomJS.
— TestNG. .
— Junit .
— .
, - . ( ), , .
See you!
Source: https://habr.com/ru/post/192742/
All Articles