Testing with the help of aitraker to perform tasks on the website alget.ru

An example of a report on testing the performance of tasks on the site, performed by I-Traker . The objective of this test was to identify the main inconveniences that target visitors to the site face - people who came to advertise the online repair cost calculator.

This report is published with the permission of the customer.

Since the goal was to conduct a qualitative, rather than a quantitative study, and it was necessary to identify the most significant features and disadvantages, after discussion with the customer, it was decided to limit the minimum group of five respondents. A separate wish was to include one interior designer in the group.

Test conditions
Testing of the site alget.ru was carried out with the use of the ITracking technology, in which the computer tracks the direction of the subject's gaze and registers on which areas of the screen people are looking at at any given time. This allows you to objectively analyze when and what exactly focuses the attention of visitors to the site.
')
Testing was conducted July 15-16, 2010 on the site alget.ru
The respondents were men over 30 with an income above the average. Testing was conducted on the territory of the respondents. 5 people took part in the testing.

As tasks for testing, respondents were offered 4 tasks:

1. The perception of the main page: the respondents had to look at the main page of the site and say what this site is about.
2. Search for contact information: respondents were asked to find the address, telephone number and location map for the real estate repair office on the site.
3. Calculation of the cost of repair: respondents were asked to set the parameters of the room similar to the provided images and calculate the cost of repair using the calculator available on the website.
4. Comments: respondents were asked to leave their comments about the site.

Additional materials

This report includes video recordings of test sessions and a map of attention in the original resolution.

Summary of Results and Recommendations

1. The overall design of the site performs well the functional load - the information blocks are allocated quite well and are noticed by the respondents.
2. The image of the house on the main page is perceived by the respondents as an indication that the company is engaged in countryside real estate. It is recommended to change this image, for example, to a collage of a country house, apartment and office.
3. The combination of different elements makes the respondents feel that the company is engaged in construction. Recommended:
   
   • Change news to less official and more related to repairs
   • Change the format of the dies "Building Materials" - large write "Online Store" and under it "building materials".
   • Perhaps change the animation. Existing animation is more associated with construction than repair.
4. The lower part of the main page is incorrectly displayed in Internet Explorer (the weight of the text is highlighted by the bolt). For visitors who look there, it can cause negative emotions.
5. On the contact page, it is recommended to add signatures to the displayed objects on the map. For example, "Store of building materials", "Office of the company", etc.
6. In the repair calculator, helper prompts pop out of the browser window (in Internet Explorer)
7. It is necessary to alter the interface for choosing the type and color of the material on the first page of the calculator!
8. The button “Calculate cost” covers drop-down lists (in IE, Chrome, FF)
9. It makes sense to automatically determine the incorrect entry of the dimensions of the room and show a tool tip.
10. Many of the terms used are incomprehensible to non-specialists. It makes sense to make a standard hint with explanations.
11. It makes sense to increase the distance between the two columns ("cost of work" and "cost of materials") on the page with the estimate of repair.

Task 1: Perception of the Main Page

When ordering testing, there was a high level of refusals (leaving the page) of visitors who visited the target advertisement “calculating the cost of repairs”. During testing, the following probable reasons for this behavior were identified:

1. All respondents (except for the professional interior designer) called the “construction and renovation of suburban real estate” (with different options - “rural buildings”, etc.) as the purpose of the site. This perception is promoted by a large image of a country house. We would recommend to change this picture to a collage of images of a country house, office, apartment, etc. Also in the picture it makes sense to focus attention on the repair, for example, add animation.
2. The majority of respondents marked the construction site as a destination. Similar to the perception is largely contributed to the news on the page dedicated to the "official" events related to construction. Since these news become visible even in the process of drawing an animated picture of a house under construction and dies with the main sections of the site, they quickly attract the user's attention and its perception of the site is largely based on the content of the news block. We would recommend choosing news that is more related to repairs than to construction. You can also recommend using company news instead of "official" news.
3. The die of the main section “Building materials” also attracts noticeable attention of users and causes associations with construction. We would recommend swapping the title and subtitle of this dice: write “Online Store” large and “Building materials” under it. Perhaps, instead of “Building materials” it is worth writing “Goods for repair”.
4. It should be noted that the text at the bottom of the page is incorrectly displayed in Internet Explorer.

In general, the design of the main page is well done: visitors quickly notice the main sections of the site, the logo, phone, etc. To a certain disadvantage can be attributed excessive attention, which attracts the news section, especially in the second design.

Design 1
Misty mask for the first 10 seconds of viewing by the number of views	Fog mask for the first 10 seconds of viewing by the percentage of time
The heat map for all time viewing by the number of views	Heat map for all viewing time by percentage of time
Design 2
Misty mask for the first 10 seconds of viewing by the number of views	Fog mask for the first 10 seconds of viewing by the percentage of time
The heat map for all time viewing by the number of views	Heat map for all viewing time by percentage of time

Task 2: Search for contact information

Search links on the main page

The search for the “Contacts” link on the main page of the site did not cause difficulties for any of the respondents. Two found the desired link in less than 5 seconds, two more took from 10 to 15 seconds, and two more spent from 30 to 35 seconds. The time scatter is not due to the difficulty of finding the reference, but to the fact that the respondents were distracted by looking at the image of the house and reading the phone numbers.
It should be noted that none of the respondents paid attention to the upper navigation block with icons.
The attention cards of the main page when performing the second task are presented below:

Thermal map by the number of views for all time viewing	Heat map for the percentage of elapsed time for all the time viewing
Map of views for the first 5 seconds of viewing	Fog mask on the percentage of elapsed time for all the time viewing

Search for contact information on the Contacts page

All respondents were able to find the right address, although some confused the addresses of stores and the repair office. The location map caused a lot of difficulties - aerial photograph maps shown on the page are not very informative - they lack signatures about what is shown there. Three go-to respondents opened the google maps map to view the location of the repair office.
This problem is not critical, since visitors interested in the company's services will still call up or be charged off by email anyway.
The attention cards of the Contacts page are listed below:

Heat map for the number of views in the first 5 seconds	Heat map for the number of views in the first 10 seconds
Heat map for the number of views in the first 15 seconds	Thermal map by the number of views for all time

Task 3: calculating the cost of repair

Home page

Thermal map by the number of views for all time viewing

Heat map for the percentage of elapsed time for all the time viewing

The search for a link to the repair calculator on the main page did not cause any difficulties for the respondents, despite the fact that the word “calculator” was missing in the task formulation. The average link search time was about 15 seconds. The long search time for the link is explained by the fact that the respondents simultaneously reviewed the materials for the assignment.

Calculator first page

Heat map for the number of views in the first 10 seconds

Thermal map by the number of views for all time viewing

Filling this page took from respondents from one and a half to 6 minutes. Key notes on this page:

1. The choice of the form of the room is quickly noticed by all respondents, it does not cause problems.
2. The ability to rotate a room was noticed only by one respondent, by and large, by chance.
3. All respondents clicked on the headings of the fields in the material selection table, suggesting that something will appear there. You can hang on these areas some additional functionality (for example, tooltips) without the danger of overloading the interface.
4. Only one respondent drew attention to the “assistant”.
5. In Internet Explorer, the assistant's tooltip crawls out of the browser.
6. A lot of complaints caused drop-down menus with a choice of type and color of materials. This interface element must be redone! Respondents can not understand the logic of the work, do not guess what type of material will be selected, do not notice the inscription about the selected type of material, etc.
7. The button "Calculate the cost" is located on top of the drop-down lists with the choice of material and color, making it difficult to choose.
8. The name of the button “Calculate the cost” does not correspond to the functionality, so it leads to the next page with data entry. It is better to sign on the button the name of the next step, for example, “Go to the indication of dimensions”.

Calculator second page

Heat map for the number of views in the first 10 seconds

Thermal map by the number of views for all time viewing

Respondents successfully coped with the task. Some first indicated dimensions in meters or centimeters, but noticed an error before entering. It may make sense to provide for automatic determination of dimensions or tooltips during the input process (if the focus is lost by the corresponding field).
Some difficulty was caused by the indication of the window size: some respondents after entering the sizes clicked on the plus and then tried to remove the appeared field to enter the second window.

The third page of the calculator

Heat map for the number of views in the first 10 seconds

Thermal map by the number of views for all time viewing

All respondents successfully coped with the task. Three of them forgot to indicate the method of preparing the surface, but after the appearance of the reminder, it was successfully indicated.
When fulfilling the operator's request to add “plywood on the screed”, some respondents chose “plywood on the crate” instead, but quickly corrected their mistake.
No one found information about what a “plywood on the screed” was, although almost everyone found information on what plywood is. Since the site is designed for use by non-professionals, it would make sense to add hints about all the terms and materials used. Instead of (or in addition to) the “helper” used on the site, we would recommend adding standard icons with a question mark and a pop-up window with brief information about the term. In these tips, you can add links to any additional articles or sections of the FAQ.

The fourth page of the calculator

Heat map for the number of views in the first 10 seconds

Thermal map by the number of views for all time viewing

Filling this page did not cause problems for any respondent. The only question was from the interface designer - whether the number of pieces or the number of types of decorative elements is meant.

The fifth page of the calculator (estimate)

Heat map for the number of views in the first 10 seconds	Heat map for the percentage of elapsed time in the first 10 seconds
Thermal map by the number of views for all time viewing	Heat map for the percentage of elapsed time for all the time viewing

When viewing the estimate, only one note was made on the design: the columns “cost of work” and “cost of materials” are poorly separated. When first viewed, the estimate is not read by columns, but by rows.

Task 4: Respondent Comments

Pros:
1. Good color range.
2. Availability of calculator
3. Easily located contact information
Minuses:
1. The lack of prompts "on the fly" when entering incorrect sizes
2. Inconvenient choice of material finishes and colors
3. News is too "fussy", distracting.
Additional comments can be heard when viewing session recordings.


Applications

Dynamic fog mask of the main page during the initial viewing:


Record a session of the respondent when performing the first task:


Record a session of the respondent when performing the second task:


Recording a respondent session in the third assignment (Part 1):


Recording the respondent's session when performing the third task (part 2):


Record the respondent's session when performing the third task (part 3):


Recording a respondent session in the third assignment (Part 4):


Recording the respondent’s session during the third assignment (Part 5):


Recording the respondent’s session during the third assignment (Part 6):


Download report

This report can be downloaded in PDF format.