Chaos and gremlins: testing web interfaces
There is an unwritten rule in the web development world: it’s almost impossible to predict user behavior. Anyone who has watched someone start working with a new website or web application can confirm this. For example, I already do not remember how many times I have seen this. For example, sometimes it seems that people simply forget how to use the browser on a mobile phone. Sometimes their actions are so illogical that it seems as if you are not a real person working with a real website, but a play in the genre of the theater of the absurd: unusual, sometimes more than attractive, there is something to think about, but ... we are not in the theater .
The point here is that the developer is not able to accurately predict the behavior of users at one time or another working with the program. Let's say a person can be very excited about something, and as a result, she will try to do something too quickly. The way he enters data into the fields and clicks on the buttons will be quite unlike quiet and focused behavior. Unfortunately, when designing and developing software products, too often they are guided precisely by this “ideal version”, and they do not think about what may happen in other cases far from the “ideal”.
Developers tend to create software systems, hoping that the user will understand their internal logic. They see the user as rational, knowing that, let's say, indiscriminate touches of everything on the phone screen can lead to something extremely unpredictable. By the way, oddities are possible even when a completely ordinary person accidentally, perhaps without even looking at the screen of a computer or a phone, clicks something or touches something. Can you remember how many times you did this yourself, say, talking on the phone on the go and trying to answer a tweet or an email in parallel?
')
Tools that allow you to investigate the reaction of computer systems to unpredictable events did not appear yesterday, but we will not delve too much into history. Consider an example close to web developers, when in 2012 Netflix opened the code for its internal service, Chaos Monkey . This system "turns off virtual machines and containers that run inside the production environment." Speaking in ordinary language, Chaos Monkey randomly chops off the server so that you can find out how the system behaves in such a case, to make sure that the entire service does not collapse as a result of the failure of several machines.
The fact that the design of applications should not be limited to their preparation for the "ideal options", everyone knows. But, if we translate this problem into the development plane of web interfaces, there are quite reasonable questions: “How to find unpredictable bad places and destructive for the program sequence of actions of hypothetical users? If something like this is done for servers, is it possible to do the same with interfaces? ”
Before answering this question, let us recall the theorem on infinite monkeys : "Absolutely randomly striking the keys of a typewriter, a hypothetical monkey will sooner or later type one of Shakespeare's plays." If so, then only one monkey, whatever it may be, is fully capable of finding bugs and problems in the web interface.
“Monkey testing” (monkey testing), or, as it is also called, “chaotic testing”, is a method of testing software products, based on simulating random user actions that are not related to the actual use cases of the systems.
If we talk about testing mobile versions of websites, it can be random touch of buttons, different gestures on the touch screen, and the input would be included in the fields. The purpose of such testing is to find the problems of the application or completely disrupt its performance. This is not at all like the usual unit or acceptance testing when writing test scripts that contain sets of actions that occur in a certain sequence and under certain conditions. As a result, such tests, for example, in the field of interfaces, strongly depend on how the developer sees their proper operation.
Programmers have less control over how monkey tests are performed, and since each time they run, in general, exposes the application to a new random test set, testing is not limited to one scenario, it is rather an endless set of unpredictable user-program interaction scenarios.
Although this type of testing is available on most platforms that run a wide variety of applications, it cannot be said that monkey tests are well-received in the web environment. For example, the Android SDK has a built-in UI Exerciser Monkey tool, with which you can test an application using most interface and system events. Looking at Android, it would be quite logical to want to find similar standard functionality, for example, in the sections of the “developer tools” of popular browsers, but there are no “monkeys” yet. The situation was corrected by the Marmelab team, creating a JavaScript library of chaotic testing of the web interfaces Gremlins.js.
Everything points to the fact that, during the development of Gremlins.js, the creators of the library were inspired by a couple of comedy horror films by Joe Dante. “Gremlins” in the library are functions that can make hellish disorder on a poorly designed page. Library “mogwai” are peaceful entities whose main task is to observe what is happening, to report what they see, to support the testing process. There is also a Gizmo. In the library, it plays the role of a mechanism that stops the testing session.
In order to use the library by running a standard random testing procedure, just a couple of lines of code or a few mouse clicks are enough. At the same time, if necessary, the tests can be very finely tuned. In order to get started, you can choose one of three ways: connect Gremlins.js to the page as a standalone library, connect as a module Require.js, run from the bookmarklet. Consider these ways.
The easiest way to create a test environment is to include the library directly on the webpage you want to experience. Thanks to this approach,
If your project uses Require.js, Gremlins.js can be imported in the right place without affecting the global namespace.
If you do not want to integrate the library into your project, choose the path of episodic testing, you can use the booklet, which allows you to experience any page that is open in the browser. Here is a link from the official documentation , from where you can add a bookmark to the bookmarks bar.
If you choose the library use option, which includes its inclusion in the pages or import using Require.js, you can start experimenting with gremlins in the code. Our example shows a call to gremlins.createHorde (). Unleash (). Let's deal with what happens here.
Gremlins.js provides five “breeds” (species in library terminology) “gremlins” - functions that implement various options for user interaction with the page. The standard “pack” (horde), with default settings, includes all five breeds. Namely, these are:
Caused gremlins, performing various actions with the page, leave visible traces on the screen. In addition, their actions, along with additional data associated with one or another kind of gremlins, are logged in the JavaScript console of the browser. These entries look like this:
By default, gremlins are called at random with an interval of 10 milliseconds 1000 times.
As already mentioned, there are not only gremlins in Gremlins.js, which are quite capable of tearing a failed page apart. There are also peaceful mogwy:
For example, a monitor that monitors the frame rate may report an error if the FPS drops below 10. It looks like this:
By logging what is happening and visualizing the actions of the gremlins, the test results can be analyzed and, if necessary, corrective measures can be taken.
In fact, even without any settings, Gremlins.js implements a very decent set of tests.
If, after using tests in a standard configuration, you did not achieve what you wanted, there are enough ways to set up tests. For example, perhaps in each test session, you want to concentrate only on a particular component of the page, rather than constantly testing it all.
Although you cannot customize all types of gremlins, you can limit the
If the standard capacity of the library is not enough for you, in particular, you are not satisfied with the existing breeds of gremlins, this is easy to fix. You can derive your own breed of gremlins, teach them to do all that you expect from the user of your project.
For example, you need to check what happens if someone tries to send a form randomly at any time working with its elements. You can hope that the
To derive new breeds of gremlins, you need to understand a little about how to create and configure DOM events in JavaScript. Therefore, let's consider the process of creating a gremlin sending a form.
If you want to know more about creating custom events, take a look at the Mozilla documentation . In addition, it is worth looking at the source code of the library, to understand how events are created in it. For example, you can start with clickerGremlin .
In fact, the library supports much more ways to tweak and expand, rather than creating new types of gremlins. Here, for example, the addition of new breeds of mogvaev, and the choice of test scenarios, and the execution of the code before and after the tests. In the end, if you want more, you can create your own project based on Gremlins.js, which will be exactly the way you see the JS interface testing library.
Randomly selected test scenarios, a new pack of gremlins appearing after each test run, and a randomly testing page are a great way to find bugs. However, if the error is found, and you have worked on correcting it, how to make sure that it is really fixed? In such a situation, it would be good to have a way to repeat the test that caused it, to find out the new reaction of the system to it. Just for such cases when it is necessary to repeat the same test scenario several times, there is a variant of initialization of the random number generator, on the basis of which the behavior of the pack is simulated. This is done using the
Here I would like to note that the main value of monkey testing implemented with the help of Gremlins.js is its unpredictability. Therefore, although repeating the same scenarios is useful, using only such tests calls into question the very meaning of chaotic testing.
To make assumptions about users, to base on them the design and development of web projects, means to refuse in advance to take into account an incredible number of special cases. Users may have slow communication channels, not the newest devices, and how can a particular person have anything? It is impossible to guarantee that a curious child of about five years old will not reach the parent phone and will not start poking at everything on the touchscreen display with enthusiasm. What this will lead to is a big question. It is impossible to know in advance that someone from the marketing department, having seen a new site for the first time, will not be delighted that it will be like a madman to click everything. Neither order, nor speed, nor repeatability of actions can be predicted. As developers, we are obliged to do for our users so that our services will not collapse for no reason at all simply because we expect people to have a strictly defined model of correct behavior.
The choice of means of monkey testing on the client side is small, but the developers of Gremlins.js managed to grasp the essence. And each of us can contribute to the improvement of this library. The team of creators of Gremlins.js will be glad to any help. Let them know if you have any thoughts on what else gremlins and mogwai can do useful.
The point here is that the developer is not able to accurately predict the behavior of users at one time or another working with the program. Let's say a person can be very excited about something, and as a result, she will try to do something too quickly. The way he enters data into the fields and clicks on the buttons will be quite unlike quiet and focused behavior. Unfortunately, when designing and developing software products, too often they are guided precisely by this “ideal version”, and they do not think about what may happen in other cases far from the “ideal”.
Developers tend to create software systems, hoping that the user will understand their internal logic. They see the user as rational, knowing that, let's say, indiscriminate touches of everything on the phone screen can lead to something extremely unpredictable. By the way, oddities are possible even when a completely ordinary person accidentally, perhaps without even looking at the screen of a computer or a phone, clicks something or touches something. Can you remember how many times you did this yourself, say, talking on the phone on the go and trying to answer a tweet or an email in parallel?
')
Tools that allow you to investigate the reaction of computer systems to unpredictable events did not appear yesterday, but we will not delve too much into history. Consider an example close to web developers, when in 2012 Netflix opened the code for its internal service, Chaos Monkey . This system "turns off virtual machines and containers that run inside the production environment." Speaking in ordinary language, Chaos Monkey randomly chops off the server so that you can find out how the system behaves in such a case, to make sure that the entire service does not collapse as a result of the failure of several machines.
The fact that the design of applications should not be limited to their preparation for the "ideal options", everyone knows. But, if we translate this problem into the development plane of web interfaces, there are quite reasonable questions: “How to find unpredictable bad places and destructive for the program sequence of actions of hypothetical users? If something like this is done for servers, is it possible to do the same with interfaces? ”
Before answering this question, let us recall the theorem on infinite monkeys : "Absolutely randomly striking the keys of a typewriter, a hypothetical monkey will sooner or later type one of Shakespeare's plays." If so, then only one monkey, whatever it may be, is fully capable of finding bugs and problems in the web interface.
Monkeys, in the studio!
“Monkey testing” (monkey testing), or, as it is also called, “chaotic testing”, is a method of testing software products, based on simulating random user actions that are not related to the actual use cases of the systems.
If we talk about testing mobile versions of websites, it can be random touch of buttons, different gestures on the touch screen, and the input would be included in the fields. The purpose of such testing is to find the problems of the application or completely disrupt its performance. This is not at all like the usual unit or acceptance testing when writing test scripts that contain sets of actions that occur in a certain sequence and under certain conditions. As a result, such tests, for example, in the field of interfaces, strongly depend on how the developer sees their proper operation.
Programmers have less control over how monkey tests are performed, and since each time they run, in general, exposes the application to a new random test set, testing is not limited to one scenario, it is rather an endless set of unpredictable user-program interaction scenarios.
Although this type of testing is available on most platforms that run a wide variety of applications, it cannot be said that monkey tests are well-received in the web environment. For example, the Android SDK has a built-in UI Exerciser Monkey tool, with which you can test an application using most interface and system events. Looking at Android, it would be quite logical to want to find similar standard functionality, for example, in the sections of the “developer tools” of popular browsers, but there are no “monkeys” yet. The situation was corrected by the Marmelab team, creating a JavaScript library of chaotic testing of the web interfaces Gremlins.js.
Gremlins are coming!
Everything points to the fact that, during the development of Gremlins.js, the creators of the library were inspired by a couple of comedy horror films by Joe Dante. “Gremlins” in the library are functions that can make hellish disorder on a poorly designed page. Library “mogwai” are peaceful entities whose main task is to observe what is happening, to report what they see, to support the testing process. There is also a Gizmo. In the library, it plays the role of a mechanism that stops the testing session.
In order to use the library by running a standard random testing procedure, just a couple of lines of code or a few mouse clicks are enough. At the same time, if necessary, the tests can be very finely tuned. In order to get started, you can choose one of three ways: connect Gremlins.js to the page as a standalone library, connect as a module Require.js, run from the bookmarklet. Consider these ways.
▍ Autonomous library
The easiest way to create a test environment is to include the library directly on the webpage you want to experience. Thanks to this approach,
gremlins will be in the global namespace, the library can be called from anywhere. It looks like this:
<script src="path/to/gremlins.min.js"></script> <script type="javascript"> // ! gremlins.createHorde().unleash(); </script>
Requ Module Require.js
If your project uses Require.js, Gremlins.js can be imported in the right place without affecting the global namespace.
require.config({ paths: { gremlins: 'scripts/libraries/gremlins.min' } }); require(['gremlins'], function(gremlins) { gremlins.createHorde().unleash(); });
▍Using Bookmarklet
If you do not want to integrate the library into your project, choose the path of episodic testing, you can use the booklet, which allows you to experience any page that is open in the browser. Here is a link from the official documentation , from where you can add a bookmark to the bookmarks bar.
Experiments
If you choose the library use option, which includes its inclusion in the pages or import using Require.js, you can start experimenting with gremlins in the code. Our example shows a call to gremlins.createHorde (). Unleash (). Let's deal with what happens here.
gremlins // , ! .createHorde() // .unleash(); // .
Gremlins.js provides five “breeds” (species in library terminology) “gremlins” - functions that implement various options for user interaction with the page. The standard “pack” (horde), with default settings, includes all five breeds. Namely, these are:
formFillerGremlin
- fills in data entry fields, clicks on checkboxes and radio buttons, interacts with other standard form elements.
clickerGremlin
- simulates mouse clicks everywhere in the visible area of the document.
toucherGremlin
- simulates the touch of the touch screen.
scrollerGremlin
- scrolls the document.
typerGremlin
- randomly enters data from the virtual keyboard.
Caused gremlins, performing various actions with the page, leave visible traces on the screen. In addition, their actions, along with additional data associated with one or another kind of gremlins, are logged in the JavaScript console of the browser. These entries look like this:
gremlin formFiller input 5 in <input type="number" name="age"> gremlin formFiller input pzdoyzshh0k9@o8cpskdb73nmi.r7r in <input type="email" name="email"> gremlin clicker click at 1219 301 gremlin scroller scroll to 100 25
By default, gremlins are called at random with an interval of 10 milliseconds 1000 times.
As already mentioned, there are not only gremlins in Gremlins.js, which are quite capable of tearing a failed page apart. There are also peaceful mogwy:
alertMogwai
- prevents test blocking due to alert () call.
fpsMogwai
- Reports the number of frames per second, which gives the browser.
gizmoMogwai
- stops testing.
For example, a monitor that monitors the frame rate may report an error if the FPS drops below 10. It looks like this:
mogwai fps 12.67 mogwai fps 23.56 err > mogwai fps 7.54 < err mogwai fps 15.76
By logging what is happening and visualizing the actions of the gremlins, the test results can be analyzed and, if necessary, corrective measures can be taken.
In fact, even without any settings, Gremlins.js implements a very decent set of tests.
Advanced Experiments
If, after using tests in a standard configuration, you did not achieve what you wanted, there are enough ways to set up tests. For example, perhaps in each test session, you want to concentrate only on a particular component of the page, rather than constantly testing it all.
Although you cannot customize all types of gremlins, you can limit the
toucherGremlin , clickerGremlin and formFillerGremlin only certain areas of the page. In particular, when setting up, we offer a gremlin to look at the parent of the element that we want to aim at. If this element is included in the list of objects of interest to us, Gremlin will start his work. Otherwise, it will try to find an element with which to interact. We can also set the maximum number of attempts to find a suitable element by setting the maxXbTries parameter.
gremlins.species.clicker().canClick(function(element) { return $(element).parents('.my-component').length; /** , , ? – true . – , false. **/ }).maxNbTries(5); // 5 // toucherGremlin formFillerGremlin gremlins.species.formFiller.canFillElement(/** **/); gremlins.species.toucher.canTouch(/** **/);
Gremlins Breeding
If the standard capacity of the library is not enough for you, in particular, you are not satisfied with the existing breeds of gremlins, this is easy to fix. You can derive your own breed of gremlins, teach them to do all that you expect from the user of your project.
For example, you need to check what happens if someone tries to send a form randomly at any time working with its elements. You can hope that the
clickerGremlin ever click on the submit button, but the best thing is to create your own gremlin, which deals exclusively with sending the form. This will increase the chances of this operation and give us control over how it is performed.
To derive new breeds of gremlins, you need to understand a little about how to create and configure DOM events in JavaScript. Therefore, let's consider the process of creating a gremlin sending a form.
gremlins.createHorde() // .allGremlins() // .gremlin(function() { // , var targetElement, availableForms; availableForms = document.querySelectorAll('form'); // targetElement = availableForms[Math.floor(Math.random()*availableForms.length)]; // // var evt = document.createEvent('HTMLEvents'); // evt.initEvent('submit'); // targetElement.dispatchEvent(evt); // // , , console.log('gremlin submit ', targetElement); }) .unleash();
If you want to know more about creating custom events, take a look at the Mozilla documentation . In addition, it is worth looking at the source code of the library, to understand how events are created in it. For example, you can start with clickerGremlin .
In fact, the library supports much more ways to tweak and expand, rather than creating new types of gremlins. Here, for example, the addition of new breeds of mogvaev, and the choice of test scenarios, and the execution of the code before and after the tests. In the end, if you want more, you can create your own project based on Gremlins.js, which will be exactly the way you see the JS interface testing library.
Non-random test case selection
Randomly selected test scenarios, a new pack of gremlins appearing after each test run, and a randomly testing page are a great way to find bugs. However, if the error is found, and you have worked on correcting it, how to make sure that it is really fixed? In such a situation, it would be good to have a way to repeat the test that caused it, to find out the new reaction of the system to it. Just for such cases when it is necessary to repeat the same test scenario several times, there is a variant of initialization of the random number generator, on the basis of which the behavior of the pack is simulated. This is done using the
seed() function:
var horde = gremlins.createHorde(); horde.seed(1234); horde.unleash();
Here I would like to note that the main value of monkey testing implemented with the help of Gremlins.js is its unpredictability. Therefore, although repeating the same scenarios is useful, using only such tests calls into question the very meaning of chaotic testing.
findings
To make assumptions about users, to base on them the design and development of web projects, means to refuse in advance to take into account an incredible number of special cases. Users may have slow communication channels, not the newest devices, and how can a particular person have anything? It is impossible to guarantee that a curious child of about five years old will not reach the parent phone and will not start poking at everything on the touchscreen display with enthusiasm. What this will lead to is a big question. It is impossible to know in advance that someone from the marketing department, having seen a new site for the first time, will not be delighted that it will be like a madman to click everything. Neither order, nor speed, nor repeatability of actions can be predicted. As developers, we are obliged to do for our users so that our services will not collapse for no reason at all simply because we expect people to have a strictly defined model of correct behavior.
The choice of means of monkey testing on the client side is small, but the developers of Gremlins.js managed to grasp the essence. And each of us can contribute to the improvement of this library. The team of creators of Gremlins.js will be glad to any help. Let them know if you have any thoughts on what else gremlins and mogwai can do useful.
Source: https://habr.com/ru/post/316816/
All Articles