TDD for Responsive Design. Or how to automate site display testing for different devices using the Galen Framework
It is difficult to formulate one title, what is the Galen Framework. It all started with the fact that I had a need to test sites in different browsers and check: did the markup go, for example, in the same Internet Explore or Chrome. Then there was a fashion for Responsive Web-Design, and I had to manually change the width of the browser and check how sites are displayed. And, although all this time WebDriver and Selenium Grid were at hand, it was not possible to properly test the layout of the site in Java code. One of the ideas was: to take screenshots in different browsers in Selenium Grid and then collect them all into one big report, according to which one of the testers is obliged to run over and, in case of discrepancies, report on the defect. Unfortunately, this whole undertaking did not last long. Testers were too lazy to thumb through a huge report and compare screenshots, and they still missed minor defects. And then they went to the requirements of implementation in all sites Responsive Design. And here came the Galen Framework. The solution turned out to be simple: check the size and location of elements relative to each other. This required a special language Galen Specs, which would be easy to read and understand.
In short, the Galen Framework is a special language and a tool for testing the display of a site in a browser. It allows you to test responsive design, as well as conduct cross-browser testing of the site.
')
Galen Framework (or simply Galen) uses Selenium to open a page in the desired browser, resize the browser window and get information about the elements on the web page (absolute position and size).
As another idea, Galen proposes to develop a frontend with a TDD (Test Driven Development) approach. It's simple: first write the test using the Galen syntax, then implement the frontend, run the tests and refactor. You don’t have to follow this methodology as long as it’s just a concept. Unfortunately, the Galen Framework has not yet been thoroughly tested in combat, and now only begin to attempt to implement it in our company. However, I think it will be interesting to take a look at the very idea of ​​developing a frontend with a TDD approach using Galen tests. So let's get started. Imagine that we are thinking about how we can optimize the site for different devices.
Let's try to make a simple sketch for three different devices:
As can be seen from the figure, we identified 3 types of devices: mobile, tablet, desktop. And, judging by our outline, some elements of the site will be displayed differently on different devices.
For simplicity, we will do everything in one file: homepage.spec
We define the names of all elements on the page with which we will work. I will explain the Galen syntax later, and now I will try to use the most simplified test constructions. In the following posts, if users find this tool interesting, I will try to present a more sophisticated approach to testing. You can also see the entire syntax and examples on the official website galenframework.com
Since I am about to imagine in advance how I will build the markup in html, I can specify the "locators" for the test items. In general, in Galen you can use three types of locators: id, css, xpath. I advise you to avoid xpath, use it only in rare cases when id and css do not allow you to select the desired item from the page.
As you can see, some objects have * in their name. This is necessary when there are many similar elements on the page. Galen in this case will first find all the elements on the given locator, and then create objects with the specified name, only instead of * it will substitute a sequence number starting from 1. Take for example menu-item- *. If our site has 4 menu items, then Galen will create objects accordingly: menu-item-1, menu-item-2, menu-item-3, menu-item-4.
I advise you not to write tests for menu items. It is better to start with a simple one: testing the framework of our site. Let's try to explain to Galen how he should test our objects, depending on the size of the browser. We will use four tags: mobile, tablet, desktop, all. We will introduce the latter for our own convenience so that we don’t have to list all the devices for routine tests.
We will add the next piece of code to our homepage.spec file after the objects are declared, although, in fact, it does not matter where we write the checks ourselves.
As you can see, for desktop and tablet tags, the content object is the only one that has no width defined. This is because in Galen it is not yet possible to use arithmetic operations (for example, 100% of screen / width - 300px). But, on the other hand, it would be superfluous. In our test, there is a side-panel check that it is located next to the content object on the left side. Also indicated that the content is attached to the left side of the screen, and the side-panel - to the right. Thus, if the content width is too small or too large, it will be displayed in violation when checking “near: content 0px right”, and we will not miss this defect in any case.
The code above can be optimized, but I do not do this for greater clarity. While this is enough to proceed with the layout of our frame. Here's how to optimize it:
Let's now write our main page. Then I sketched a simple page where you can test our tests samples.galenframework.com/tutorial1/tutorial1.html
Our main page is ready. We can run tests in galen for different devices. By default, the Firefox browser will be used.
Run three commands from the command line for a separate check in the desktop, tablet and mobile version.
As a result, Galen will create an html report in which you can find all the perfect checks, as well as see error messages.
If you click on the check highlighted in red, you will get a screenshot with highlighted blocks that violated this very check.
As you can see, the message says “side-panel” is 0px below “content” instead of 5px. Those. in our version of the site there is no specified indent of 5 pixels between the “content” and “side-panel” objects for the mobile version. We need to do something about it. If this is a bug, then we will fix our layout and add this indent. If we do not care how big is the indent, then we can indicate that it should be within reason in this way: below: content 0 to 10 px. Those. the indent can be any in the range from 0 to 10.
After we correct our layout or fix the checks, we can run the tests again. Only a way to start some not very convenient. You have to run different commands separately and read reports in different folders. Let's use another feature of Galen - the so-called Galen Test Suites. Create a file galen-tutorial.test, write the following in it:
Is done. Now we can run our tests and merge all the reports into one.
The resulting report can be viewed at this link: samples.galenframework.com/tutorial1/reports/report.html
Our framework is ready and tested, we can proceed to more complex checks. I will try to use more features of the Galen language for demonstration. Imagine that we have added a site logo and text in the title
As you can see, new checks have appeared: “centered vertically inside” and “text is”. The first one checks that our logo is vertically in the middle and inside the header object, and the second one just checks the text.
You also probably noticed the new "~" symbol, which means that we want to make an approximate comparison within 2 pixels of error (in Galen, you can adjust the error for approximate comparisons).
Now even more difficult check. Let's verify that on the desktop and tablet the menu is laid out horizontally, and on the mobile version by a table in two columns, as we, in fact, conceived in our sketch at the beginning of the article. I will try to explain the syntax of each check in the code comments. If you remember, then at the top we defined menu objects as follows: “menu-item- * css #menu li a”. Those. Galen will find all the menu items; we have only 4 of them, we will end up with menu-item-1, menu-item-2, menu-item-3, menu-item-4 objects.
The construction [1 - 9] instructs Galen to parameterize the next object with all its checks and, where the “@” symbol is encountered, to substitute the current value of the parameter. You can specify a range, or you can specify specific values ​​separated by commas [1, 3, 6, 10]. Also, as seen in the upper example, we used the @ {+ 1} construct. These are the simplest arithmetic operations that allow us to point to the next object in the list.
And so you can continue indefinitely. As you can see, you can check almost any arrangement of elements on the page. The main thing is just to choose the method of checks correctly, so that later you do not create problems for yourself with test support.
If you are interested in finding out what other possibilities there are in the Galen Framework, you can visit the official website galenframework.com . Later I will try to post a few more articles (for example, how to automate testing in different browsers using Selenium Grid or how to use component testing correctly).
Briefly list the advantages and disadvantages (shortcomings) of Galen.
What can galen:
What still can not Galen:
The project is posted on GitHub github.com/ishubin/galen and is distributed under the Apache License, Version 2.0.
Galen is supported by only one person and is not common anywhere. We are trying to implement it in our company, but so far it’s too early to talk about any successes. If you are interested in the project, I will be glad to any feedback, especially constructive criticism, because The project is in beta and needs testing. You can submit defects here at github.com/ishubin/galen/issues . You can download the project at this link galenframework.com/download
In short, the Galen Framework is a special language and a tool for testing the display of a site in a browser. It allows you to test responsive design, as well as conduct cross-browser testing of the site.
')
Galen Framework (or simply Galen) uses Selenium to open a page in the desired browser, resize the browser window and get information about the elements on the web page (absolute position and size).
As another idea, Galen proposes to develop a frontend with a TDD (Test Driven Development) approach. It's simple: first write the test using the Galen syntax, then implement the frontend, run the tests and refactor. You don’t have to follow this methodology as long as it’s just a concept. Unfortunately, the Galen Framework has not yet been thoroughly tested in combat, and now only begin to attempt to implement it in our company. However, I think it will be interesting to take a look at the very idea of ​​developing a frontend with a TDD approach using Galen tests. So let's get started. Imagine that we are thinking about how we can optimize the site for different devices.
Step 1. We make the site in the usual notebook
Let's try to make a simple sketch for three different devices:
As can be seen from the figure, we identified 3 types of devices: mobile, tablet, desktop. And, judging by our outline, some elements of the site will be displayed differently on different devices.
Step 2. Writing Galen Specs
For simplicity, we will do everything in one file: homepage.spec
We define the names of all elements on the page with which we will work. I will explain the Galen syntax later, and now I will try to use the most simplified test constructions. In the following posts, if users find this tool interesting, I will try to present a more sophisticated approach to testing. You can also see the entire syntax and examples on the official website galenframework.com
Since I am about to imagine in advance how I will build the markup in html, I can specify the "locators" for the test items. In general, in Galen you can use three types of locators: id, css, xpath. I advise you to avoid xpath, use it only in rare cases when id and css do not allow you to select the desired item from the page.
=========================================== header id header header-logo css #header .logo header-caption css #header h1 menu id menu menu-item-* css #menu li a content id content side-panel id side-panel side-panel-caption css #side-panel h2 side-panel-links-* css #side-panel li a article id article article-caption css #article h1 article-text id article-text comments id comments footer id footer ========================================== As you can see, some objects have * in their name. This is necessary when there are many similar elements on the page. Galen in this case will first find all the elements on the given locator, and then create objects with the specified name, only instead of * it will substitute a sequence number starting from 1. Take for example menu-item- *. If our site has 4 menu items, then Galen will create objects accordingly: menu-item-1, menu-item-2, menu-item-3, menu-item-4.
I advise you not to write tests for menu items. It is better to start with a simple one: testing the framework of our site. Let's try to explain to Galen how he should test our objects, depending on the size of the browser. We will use four tags: mobile, tablet, desktop, all. We will introduce the latter for our own convenience so that we don’t have to list all the devices for routine tests.
We will add the next piece of code to our homepage.spec file after the objects are declared, although, in fact, it does not matter where we write the checks ourselves.
@ all ------------------------------------ header width: 100% of screen/width height: 100px above: menu 0px menu width: 100% of screen/width below: header 0px above: content 0px footer width: 100% of screen/width height: > 100px content inside: screen 0px left below: menu 0px @ desktop, tablet ----------------------------------- side-panel width: 300px below: menu 0px inside: screen 0px right near: content 0px right menu height: 50px footer below: content 0px @ mobile ----------------------------------- side-panel width: 100% of screen/width below: content 5px content width: 100% of screen/width above: side-panel 5px footer below: side-panel 0px As you can see, for desktop and tablet tags, the content object is the only one that has no width defined. This is because in Galen it is not yet possible to use arithmetic operations (for example, 100% of screen / width - 300px). But, on the other hand, it would be superfluous. In our test, there is a side-panel check that it is located next to the content object on the left side. Also indicated that the content is attached to the left side of the screen, and the side-panel - to the right. Thus, if the content width is too small or too large, it will be displayed in violation when checking “near: content 0px right”, and we will not miss this defect in any case.
The code above can be optimized, but I do not do this for greater clarity. While this is enough to proceed with the layout of our frame. Here's how to optimize it:
Show optimized code
We can remove unnecessary checks above and below. If we check that the menu is located above main at a distance of 10 pixels, then it makes no sense to check that main is located under the menu at the same distance. We can also combine all the objects that should be stretched across the width of the screen into a single check.
@ all ------------------------------------ header, menu, footer width: 100% of screen/width header height: 100px above: menu 0px menu height: 50px above: content 0px footer height: > 100px content inside: screen 0px left @ desktop, tablet ----------------------------------- side-panel width: 300px below: menu 0px inside: screen 0px right near: content 10px right @ mobile ----------------------------------- side-panel, content width: 100% of screen/width side-panel below: content 5px Step 3. We write Html / CSS
Let's now write our main page. Then I sketched a simple page where you can test our tests samples.galenframework.com/tutorial1/tutorial1.html
Step 4. Run the tests in Galen
Our main page is ready. We can run tests in galen for different devices. By default, the Firefox browser will be used.
Attention! Since Galen uses Selenium, it is highly likely that the latest Firefox browser will not be supported. This is one of the main headaches in the world of Selenium, and I have not yet figured out how to solve it for Galen. All I can do is update the version of Selenium in Galen to the last one and hope that with the next Firefox release, the Mozilla guys will stop making major changes to the API and break the dependency of the selenium. Alternatively, it would be possible to develop your browser based on WebKit, especially for Galen, but it seems to me that this would be a waste of time.
Run three commands from the command line for a separate check in the desktop, tablet and mobile version.
galen check homepage.spec --url "http://samples.galenframework.com/tutorial1/tutorial1.html" --size 1024x768 --include "all,desktop" --htmlreport desktop-reports galen check homepage.spec --url "http://samples.galenframework.com/tutorial1/tutorial1.html" --size 600x800 --include "all,tablet" --htmlreport tablet-reports galen check homepage.spec --url "http://samples.galenframework.com/tutorial1/tutorial1.html" --size 400x600 --include "all,mobile" --htmlreport mobile-reports As a result, Galen will create an html report in which you can find all the perfect checks, as well as see error messages.
If you click on the check highlighted in red, you will get a screenshot with highlighted blocks that violated this very check.
As you can see, the message says “side-panel” is 0px below “content” instead of 5px. Those. in our version of the site there is no specified indent of 5 pixels between the “content” and “side-panel” objects for the mobile version. We need to do something about it. If this is a bug, then we will fix our layout and add this indent. If we do not care how big is the indent, then we can indicate that it should be within reason in this way: below: content 0 to 10 px. Those. the indent can be any in the range from 0 to 10.
After we correct our layout or fix the checks, we can run the tests again. Only a way to start some not very convenient. You have to run different commands separately and read reports in different folders. Let's use another feature of Galen - the so-called Galen Test Suites. Create a file galen-tutorial.test, write the following in it:
@@ Parameterized | device | size | tags | | desktop | 1024x768 | all,desktop | | tablet | 600x800 | all,tablet | | mobile | 400x600 | all,mobile | Tutorial 1 Home page in ${device} device http://samples.galenframework.com/tutorial1/tutorial1.html ${size} check homepage.spec --include "${tags}" Is done. Now we can run our tests and merge all the reports into one.
galen test galen-tutorial.test --htmlreport reports The resulting report can be viewed at this link: samples.galenframework.com/tutorial1/reports/report.html
More complex and detailed checks
Our framework is ready and tested, we can proceed to more complex checks. I will try to use more features of the Galen language for demonstration. Imagine that we have added a site logo and text in the title
@ all ----------------------------------- header-logo inside: header 10px left centered vertically inside: header height: 60px header-caption inside: header 10 to 20px top left near: header-logo ~ 10 px text is: My Awesome Website! As you can see, new checks have appeared: “centered vertically inside” and “text is”. The first one checks that our logo is vertically in the middle and inside the header object, and the second one just checks the text.
You also probably noticed the new "~" symbol, which means that we want to make an approximate comparison within 2 pixels of error (in Galen, you can adjust the error for approximate comparisons).
Now even more difficult check. Let's verify that on the desktop and tablet the menu is laid out horizontally, and on the mobile version by a table in two columns, as we, in fact, conceived in our sketch at the beginning of the article. I will try to explain the syntax of each check in the code comments. If you remember, then at the top we defined menu objects as follows: “menu-item- * css #menu li a”. Those. Galen will find all the menu items; we have only 4 of them, we will end up with menu-item-1, menu-item-2, menu-item-3, menu-item-4 objects.
@ all -------------------------------------- # , menu contains: menu-item-* # 50 menu-item-* height: 50px @ desktop, tablet -------------------------------------- # , (0px) [ 1 - 3 ] menu-item-@ near: menu-item-@{+1} 0px left aligned horizontally: menu-item-@{+1} @ mobile -------------------------------------- # menu-item-* width: 45 to 50 % of menu/width # , ( ) # - # , [ 1, 3 ] menu-item-@ near: menu-item-@{+1} 0px left aligned horizontally: menu-item-@{+1} # , ( ) # - [1, 2] menu-item-@ above: menu-item-@{+2} 0px aligned vertically: menu-item-@{+2} The construction [1 - 9] instructs Galen to parameterize the next object with all its checks and, where the “@” symbol is encountered, to substitute the current value of the parameter. You can specify a range, or you can specify specific values ​​separated by commas [1, 3, 6, 10]. Also, as seen in the upper example, we used the @ {+ 1} construct. These are the simplest arithmetic operations that allow us to point to the next object in the list.
And so you can continue indefinitely. As you can see, you can check almost any arrangement of elements on the page. The main thing is just to choose the method of checks correctly, so that later you do not create problems for yourself with test support.
If you are interested in finding out what other possibilities there are in the Galen Framework, you can visit the official website galenframework.com . Later I will try to post a few more articles (for example, how to automate testing in different browsers using Selenium Grid or how to use component testing correctly).
Briefly list the advantages and disadvantages (shortcomings) of Galen.
What can galen:
- Check the location of the blocks relative to each other
- Check visible text
- Work with Selenium Grid
- Run custom javascript on the test page (if, for example, you need to open the drop-down menu and check its elements)
- Interact with the browser through Selenium in order to get to the desired place on the site, if you can not call it through a direct link
- Component testing. Galen can run separate check files for specified objects. Thus, you can implement checks for duplicate complex elements on the page (for example, search results, comments, etc.)
- Make adjustments to the size and position of objects. This can be useful in two cases: either Selenium gives incorrect information about the location of the object, or we need the so-called “guides” (or virtual binding objects), to which we will refer other objects on the page
- Condition blocks They can be useful when it is impossible to know exactly which element will be displayed on the page (eg banners)
- Test parameterization in test suites. By the way, you can go further and use parametrization of parameterization (a rare case, but you can, for example, run the same test for different sizes and even for different browsers)
What still can not Galen:
- Check the location of the visible text, not the entire block (for example, h1 tags). Plans to implement the definition of the text in the screenshot, but this will only work in Firefox (Selenium has problems with screenshots in Chrome and IE)
- Run in real mobile browsers. This is the problem of Selenium, because it does not allow to obtain information about the location of elements on the page in the Android browser. The solution could be developing your own javascript-based tool (for example, as BusterJS or Selenium 1.0). In this case, you can work with any browser at all. Or you can write your own browser for iOS and Android, which would support communication with Galen. But it will be too difficult for one person, so I’ll put off this idea for now.
- Check the color and compare parts of the image on the screenshots. There are plans to develop a simple comparison of color gamut according to histograms (for example,
color scheme is: 80 to 85 % black, ~10% whiteorcolor scheme like: image-samples/menu.png). Galen will take a screenshot, select the area of ​​the specified element and check its color range. You can also add the simplest comparison of images, however, it seems to me redundant. - Perform complex arithmetic operations in parameterized checks. For now, you can only do this: @ {+ 1}, @ {* 2}, @ {- 1}, @ {/ 10}. But plans to make full arithmetic expressions using brackets
- Perform arithmetic operations in regular checks. For example, it would be nice to do something like
width: 100% of screen/width - 10px - 100% of comments/width + 10% of somethingelse/width - No sane installer. There should be Java 1.6+ on the machine, unfortunately I have no experience in creating installers, but I hope to come up with something soon.
The project is posted on GitHub github.com/ishubin/galen and is distributed under the Apache License, Version 2.0.
Galen is supported by only one person and is not common anywhere. We are trying to implement it in our company, but so far it’s too early to talk about any successes. If you are interested in the project, I will be glad to any feedback, especially constructive criticism, because The project is in beta and needs testing. You can submit defects here at github.com/ishubin/galen/issues . You can download the project at this link galenframework.com/download
Source: https://habr.com/ru/post/203506/
All Articles