Case Crocs: sales optimization of one hundred million. Part 2
In the last post, we started a story about our optimization of the checkout process for Crocs and got acquainted with the most widely used variations of online testing resources: A / B and multivariate testing. It should be noted that in the case of e-commerce the basis for success in this part is the ability to effectively test the entire site and its individual elements. In turn, the ability to test and improve test results directly depends on the technical capabilities of an individual web resource. In projects with a large turnover testing functionality becomes extremely important. Today we will talk about the organization of the framework for automated regression testing, which was implemented for Crocs, the key technologies used in this process, and the implementation of the code itself.
Organization of a framework for automated regression testing:
To implement the project, the Thucydides + JUnit framework was used. It was this solution that allowed us to provide maximum customization flexibility, implement many auxiliary methods for working with web elements, provide for logging and handling of exceptional situations, and, of course, creating beautiful reports.
')
At this stage, we were able to cover the following localization of Crocs sites: USA, Canada, Europe, Australia, Taiwan, United Kingdom, Poland, Germany, Korea, Japan, Singapore, Hong Kong, Finland, France and Holland.
Since the functionality of individual site localizations is different, the number of tests for sites also varies. In this case, the minimum number was 70, the maximum - 101 test.
Continuous Integration Environment: Jenkins .
Key technologies used in the framework:
1. As a rule, end-to-end automated testing is written separately for one specific site. Multiple differences in the design and logic of the site require a large number of forks in the code, which leads to its poor readability. Therefore, to implement the idea of “one test should be performed on different locales and versions of the site” we decided to use reflection (the mechanism for examining program data during its execution is performed using the Java Reflection API , which consists of the java.lang and java.lang package classes .reflect ). Here's how to do it:
Suppose there is some kind of test. At a high level, a test is a sequence of steps that must be performed regardless of the site being tested. The site will determine the features of the implementation of each step.
The site on which it will run depends on the value of the system property site. This property in Java is passed as a command line parameter. To access it, use the line of code System.getProperty (“site”); Thus, it is necessary to implement the switching steps goHome, goToSite, login and complexStep depending on the site. Let all steps be methods of the AllSteps class.
This class is the Thucydides step library. First of all, it contains links to specific implementations of the steps. Suppose that there are three specific sites: Korea, Taiwan, and the United States. Then
public class AllSteps extends ScenarioSteps {
}
The heart of the AllSteps class is a method called reflector . Its task is to redirect the execution of a step from a common library of steps to one of three specific ones. In the usual situation, the method is called via the object at the point. However, another method is implemented in Java to do the same, that is, to call an object through a point on a method. This action is implemented in java.lang.reflect.Method . The name of the method being invoked is converted to a string, which allows for various manipulations with this name. Suppose that a method that implements a step for a certain site is called the same as the step itself, but with the addition of the site name to the end of the method name. That is, login will turn into loginUS , goHome into goHomeUS, and so on.
For the case of three sites, the reflector can be implemented as follows:
The main feature (and inconvenience) is that for such an "inverted" method call, you need to create a special data structure that will match the signature of the method. In the example above, this is the outwardly passed variable types , which is formed by a wrapper method called from a test located at the topmost level.
The types parameter will always be even if the signature of the method being called is empty (in this case, it must be assigned a null or new Class [0] ). The values of the parameters of the step are transmitted inside the reflector unchanged. Since there can be a variable number of parameters, the reflector is also declared as a method that accepts a variable number of Object parameters.
All wrapper steps that implement the reference to the reflector should be marked with the
To output to the error log of the reflector, use the error method, which is also a Thucydides step. It does nothing, but its parameters (error message) will be included in the report.
The reflector forwards the call to the library of steps of the corresponding site. The typical step is to redirect the call to a PageObject that will work with the page through the web driver. PageObject libraries for the three sites tested in the sample will also be separate. It can be said that the Korea steps library is needed only for these steps to be instantiated by PageObjects from the Korean PageObjects library.
And although the example step is very simple, you can see that it implies the possibility of creating complex steps with a lot of actions (if the programmer considers it necessary to use them).
What can I get from the Thucydides multisite tester?
2. An important moment is data storage. Given the number of sites and the potential for increasing this number, the process of filling XML files will be very time consuming and, even worse, data duplication will reach 60% for some sites.
As a result, nothing else but how to organize all the data that is stored in one file with a hierarchical structure from Apache is hard to come up with. The advantage is that with this type of structure we avoid unnecessary duplication of products, credit cards and similar data for sites where they match. By using the principles of inheritance, it is possible to bring all the common data up, leaving only unique parameters for each site.
3. The Open Commerce API (OCAPI) is a REST API for sites on the Demandware platform, which allows customers, partners and developers to get product information for easy integration. Shop APIs provide easy integration with store data for use by other systems, applications, and so on, with the ultimate goal of making a transaction.
OCAPI allows you to work with the following resources:
Before using OCAPI, you must configure it in Demandware Business Manager. Settings are saved in JSON-format and are unique for each site. Especially for the purposes of test development, the Java OCAPI library was written, which automatically injects ocapi.js into the client page in the browser and handles all of its calls. All communication between the OCAPI java library and ocapi.js passes through the WebDriver JavascriptExecutor interface. This means that before you make a request to ocapi, you need to open the store page. The integration of the OCAPI module into Thucydides significantly speeds up the execution of tests and, thus, the results after the build.
4. Flexible configuration of jobs on Jenkins and elevation of the Selenium Grid server allows you to run tests in multi-threaded mode, speeding up the regression testing procedure.
Results:
After implementing the above described functionality, we were able to ensure timely detection and elimination of problems, which in turn allowed us to implement the process of optimizing and testing the sales funnel, on the basis of which Crocs achieved significant financial success. As a result, after doing the necessary testing and optimization, in 2012 alone, the Crocs marketing team achieved $ 100 million in sales in its online stores, followed by an annual 19 percent increase. Speaking about this, we, of course, remember that the success of an e-commerce site lies not only in qualitative testing - naturally, this is a delicate combination of high-quality design solutions, good marketing, technical know-how and, most importantly, in-demand products, without which such high sales would be difficult to achieve.
The authors:
Konstantin Tishchenko
Vitaly Taradayko
Alexander Mamalyga
Organization of a framework for automated regression testing:
To implement the project, the Thucydides + JUnit framework was used. It was this solution that allowed us to provide maximum customization flexibility, implement many auxiliary methods for working with web elements, provide for logging and handling of exceptional situations, and, of course, creating beautiful reports.
')
At this stage, we were able to cover the following localization of Crocs sites: USA, Canada, Europe, Australia, Taiwan, United Kingdom, Poland, Germany, Korea, Japan, Singapore, Hong Kong, Finland, France and Holland.
Since the functionality of individual site localizations is different, the number of tests for sites also varies. In this case, the minimum number was 70, the maximum - 101 test.
Continuous Integration Environment: Jenkins .
Key technologies used in the framework:
1. As a rule, end-to-end automated testing is written separately for one specific site. Multiple differences in the design and logic of the site require a large number of forks in the code, which leads to its poor readability. Therefore, to implement the idea of “one test should be performed on different locales and versions of the site” we decided to use reflection (the mechanism for examining program data during its execution is performed using the Java Reflection API , which consists of the java.lang and java.lang package classes .reflect ). Here's how to do it:
Suppose there is some kind of test. At a high level, a test is a sequence of steps that must be performed regardless of the site being tested. The site will determine the features of the implementation of each step.
@Test
public void testSomething () {
// options
HashMap <String, String> x = new HashMap <String, String> ();
HashMap <String, String> y = new HashMap <String, String> ();
x.put ("x1", "12");
x.put ("x2", "13");
y.put ("y1", "22");
y.put ("y2", "23");
// test steps
all.goHome ();
all.goToSite ();
all.login ("j.smith", "123456q");
all.complexStep (x, 123, "aaa", "bbb", y);
}
The site on which it will run depends on the value of the system property site. This property in Java is passed as a command line parameter. To access it, use the line of code System.getProperty (“site”); Thus, it is necessary to implement the switching steps goHome, goToSite, login and complexStep depending on the site. Let all steps be methods of the AllSteps class.
@Steps public AllSteps all;
This class is the Thucydides step library. First of all, it contains links to specific implementations of the steps. Suppose that there are three specific sites: Korea, Taiwan, and the United States. Then
public class AllSteps extends ScenarioSteps {
@Steps public KoreaSteps korea;
@Steps public TaiwanSteps taiwan;
@Steps public USSteps us;
}
The heart of the AllSteps class is a method called reflector . Its task is to redirect the execution of a step from a common library of steps to one of three specific ones. In the usual situation, the method is called via the object at the point. However, another method is implemented in Java to do the same, that is, to call an object through a point on a method. This action is implemented in java.lang.reflect.Method . The name of the method being invoked is converted to a string, which allows for various manipulations with this name. Suppose that a method that implements a step for a certain site is called the same as the step itself, but with the addition of the site name to the end of the method name. That is, login will turn into loginUS , goHome into goHomeUS, and so on.
For the case of three sites, the reflector can be implemented as follows:
@Step
public void reflector (String methodName, Class [] types, Object ... args) {
if (System.getProperty ("site"). equals ("US")) {
try {
Method m = us.getClass (). GetMethod (
methodName + "US", types
);
m.invoke (us, args);
}
catch (Exception e) {
error ("US reflection error" + e.getMessage ());
}
}
if (System.getProperty ("site"). equals ("Taiwan")) {
try {
Method m = taiwan.getClass (). GetMethod (
methodName + "Taiwan", types
);
m.invoke (taiwan, args);
}
catch (Exception e) {
error ("Taiwan reflection error" + e.getMessage ());
}
}
}
The main feature (and inconvenience) is that for such an "inverted" method call, you need to create a special data structure that will match the signature of the method. In the example above, this is the outwardly passed variable types , which is formed by a wrapper method called from a test located at the topmost level.
@Step
public void login (String login, String password) {
// method signature
Class [] types = new Class [2];
types [0] = String.class;
types [1] = String.class;
reflector ("login", types, login, password);
}
The types parameter will always be even if the signature of the method being called is empty (in this case, it must be assigned a null or new Class [0] ). The values of the parameters of the step are transmitted inside the reflector unchanged. Since there can be a variable number of parameters, the reflector is also declared as a method that accepts a variable number of Object parameters.
All wrapper steps that implement the reference to the reflector should be marked with the
@Step annotation, which will allow them to appear in the Thucydides report. In the example above, the reflector is also marked with the @Step annotation. This will lead to the fact that the report will also display a reflection as a test step, and the steps themselves will become nested. To prevent this from happening, the @Step annotation can be removed.
To output to the error log of the reflector, use the error method, which is also a Thucydides step. It does nothing, but its parameters (error message) will be included in the report.
@Step
public void error (String message) {}
The reflector forwards the call to the library of steps of the corresponding site. The typical step is to redirect the call to a PageObject that will work with the page through the web driver. PageObject libraries for the three sites tested in the sample will also be separate. It can be said that the Korea steps library is needed only for these steps to be instantiated by PageObjects from the Korean PageObjects library.
@Step
public void loginKorea (String login, String password) {
KoreaStorefrontPage sp = getPages (). Get (KoreaStorefrontPage.class); sp.clickOnLoginLink ();
}
And although the example step is very simple, you can see that it implies the possibility of creating complex steps with a lot of actions (if the programmer considers it necessary to use them).
What can I get from the Thucydides multisite tester?
- Multisite tester will allow you to have independent step libraries for each of the tested sites. Each of these libraries can be developed by a separate programmer, minimally intersecting (and conflicting) with the author of the neighboring library for another site.
- A multisite tester will allow you to have independent PageObject libraries for each of the tested sites. Each of these libraries can be developed by a separate programmer, minimally intersecting (and conflicting) with the author of the neighboring library for another site.
- In addition to the site-specific code, the tester will also have code that applies to all tested sites. Since several testers are physically located within the same program, the question of sharing common libraries and developments is removed.
- In addition to technical benefits, there are also psychological ones: sometimes the customer wants several sites to be tested, and the program to be one, as in our case.
2. An important moment is data storage. Given the number of sites and the potential for increasing this number, the process of filling XML files will be very time consuming and, even worse, data duplication will reach 60% for some sites.
As a result, nothing else but how to organize all the data that is stored in one file with a hierarchical structure from Apache is hard to come up with. The advantage is that with this type of structure we avoid unnecessary duplication of products, credit cards and similar data for sites where they match. By using the principles of inheritance, it is possible to bring all the common data up, leaving only unique parameters for each site.
- Sections - the file is divided into sections, each section starts with a specific declaration. The syntax of the declaration begins with a “[” and ends, respectively, “]”. This section gets the site name.
- Parameters - have a typical format: key = value
- Comments - two options: lines that begin with “#” or “;”
3. The Open Commerce API (OCAPI) is a REST API for sites on the Demandware platform, which allows customers, partners and developers to get product information for easy integration. Shop APIs provide easy integration with store data for use by other systems, applications, and so on, with the ultimate goal of making a transaction.
OCAPI allows you to work with the following resources:
- Account
- Basket
- Category
- Content
- Contentsearch
- Folder
- Product
- ProductSearch
- Promotion
- Site
- Store (13.4+)
- Customer (Data API, 14.2+)
- CustomerSearch (Data API, 14.2+)
Before using OCAPI, you must configure it in Demandware Business Manager. Settings are saved in JSON-format and are unique for each site. Especially for the purposes of test development, the Java OCAPI library was written, which automatically injects ocapi.js into the client page in the browser and handles all of its calls. All communication between the OCAPI java library and ocapi.js passes through the WebDriver JavascriptExecutor interface. This means that before you make a request to ocapi, you need to open the store page. The integration of the OCAPI module into Thucydides significantly speeds up the execution of tests and, thus, the results after the build.
4. Flexible configuration of jobs on Jenkins and elevation of the Selenium Grid server allows you to run tests in multi-threaded mode, speeding up the regression testing procedure.
Results:
After implementing the above described functionality, we were able to ensure timely detection and elimination of problems, which in turn allowed us to implement the process of optimizing and testing the sales funnel, on the basis of which Crocs achieved significant financial success. As a result, after doing the necessary testing and optimization, in 2012 alone, the Crocs marketing team achieved $ 100 million in sales in its online stores, followed by an annual 19 percent increase. Speaking about this, we, of course, remember that the success of an e-commerce site lies not only in qualitative testing - naturally, this is a delicate combination of high-quality design solutions, good marketing, technical know-how and, most importantly, in-demand products, without which such high sales would be difficult to achieve.
The authors:
Konstantin Tishchenko
Vitaly Taradayko
Alexander Mamalyga
Source: https://habr.com/ru/post/235623/
All Articles