Communication with the next respondent. Frame from k / f Matrix (The Matrix, 1999)



As you know from our past posts , the Mail.Ru Group has its own UX-lab, which is more than four years old. Now most major IT companies either already have a UX research line or are planning to launch it. Our laboratory works only with Mail.Ru Group products, but we decided to make an exception and opened its doors to external companies and projects that are aware of the values ​​of UX research, but have not yet decided to allocate a budget for their implementation.



To select an external “customer”, we distributed information about our competition project in social networks and arranged a free usability testing draw. The winner was the project team service-centers.ru - this is a site for searching and selecting service centers throughout Russia.

')

In this post, we want to tell those who so far have encountered little UX tests, how they should pass, how to prepare for them, and what benefits they can bring. So, perhaps, the idea of ​​“looking through the eyes of the user” on the product will help someone from the readers not to step on the same rake.

1. Stages of the project

1.1. Training

1.1.1. The first meeting with the customer

Any project begins with a meeting with the customer. It is important for us to find out the customer’s expectations of the project, to disassemble the product together, discuss its audience, tell how the project will go, what stages the customer will need to actively engage in, and agree on further interaction.

1.1.2. Meeting results

About the product



Service-centers.ru allows owners / representatives of service centers ( hereinafter abbreviated as SO ) to place information about the SOs and promote them / promote them, and consumers of the service can find an SO corresponding to their request.



Product audience



The audience of the product is divided into two groups:

• The owners / representatives of the SC are the owners of the SC or responsible for promoting the SC and placing information about them.
• Users of household appliances and electronics - any person who is faced with a breakdown of household appliances ( hereinafter abbreviated as BT ) and / or electronics.

According to the customer, among the users of the site 80% of men and 20% of women. The age range is from 28 to 35 years.



Main customer request



The main wishes of customers were recorded and presented clear requirements for the project:

• Get information about how users choose SOs, what parameters they orient when choosing SOs, what they pay attention to.
• Check whether the behavior and selection criteria for SCs are different for residents of Moscow and residents of the regions (for example, Rostov-on-Don and Kazan).
• Understand how convenient the site has become after redesign for users of BT and electronics.

Product Changes



At the stage of familiarization with the product, we gave the customer several expert recommendations on how to refine the site in order to eliminate obvious problems (for example, the lack of filters).

1.1.3. Project planning

Offer to the customer



Since the customer wanted not only to identify the product's usability problems, but also to learn about how representatives of Central Asia choose SOs and what they pay attention to, an interview block was added to testing, which allowed them to collect the experience of respondents, obtained earlier when confronted with such a problem as breakdown of household appliances or electronics.



Following the meeting, we made a proposal for the project:



Method



Usability testing with interview elements (introductory interview, testing tasks that simulate several reference situations in the SO).



Respondents



The customer was focused on users of BT and electronics, so we began to work with this part of site users.



Respondents who differed by knowledge in the subject area and live in different regions were invited to participate in the study to see if these differences would affect the behavior of respondents when selecting SOs.



Recruitment criteria



Eight respondents representing two groups of the audience - versed in the repair of equipment and not versed.



"Experienced users" - at the household level understand the repair of BT and electronics (they can assemble a computer from the components; having disassembled the electronics, it can be understood which of the parts has failed); independently choose BT and electronics; prefer to independently understand the cause of failure.



"Inexperienced users" - do not understand the repair of BT and electronics; can not choose the BT and electronics independently and seek help; Do not try to determine the cause of the breakdown yourself.



In each group, half of the respondents were residents of Moscow and half were residents of the regions (Kazan and Rostov-on-Don).

	Experienced	Inexperienced
Moscow	2 rep.	2 rep.
Kazan	1 rep.	1 rep.
Rostov-on-Don	1 rep.	1 rep.

Testing tasks



The objectives of testing were typical situations of contacting the SO and checking the main assumptions of the customer about how users choose the SO.



Tasks offered to respondents:

• Free search for SC - respondents were asked to find a SC for repairing the thing that they had broken last.
• Search for a large (networked) SC - respondents were asked to imagine that their friend had told how, after having handed over the phone for repair, could not return it. The respondent was asked to choose the SC, which, in their opinion, can be trusted. We wanted to check which of the parameters would be associated with the reliability of the SC.
• Search for authorized SC - respondents were asked to find a SC approved by the manufacturer of the broken thing. The task was proposed if the respondent had not previously searched for SC for this parameter.
• SC search for urgent repairs - the respondent was immersed in a situation requiring urgent repairs.

1.2. What did we get before going to testing?

Respondents



Recruitment criteria - clearly defined, agreed with the customer and sent to a recruiting agency.



Test schedule



The schedule was calculated at the rate of four people per day, one session lasting an hour, an hour break between sessions. One day was pledged in case of failure of testing sessions.



Note: a one-hour break helps attenuate respondents' non-punctuality and delayed sessions.



Scenario



Scenario consisting of an interview block and testing tasks, agreed with the customer.



Inviting the customer to view testing sessions



The customer was invited to view the test. Representatives of the customer could see the work of the respondents live and record the difficulties encountered.



According to the results of the test day, the test results were discussed with the customer.



Note: it is important to discuss the identified problems with the customer, as the customer may interpret what he saw differently. But this is not our case.

1.3. Testing

Testing organization



Part of the testing took place in our laboratory using eye-tracking.



The respondents from the regions were tested remotely. For these cases, we prefer to use GoToMeeting.



The course of testing, or what problems are



A small list of problems that can greatly complicate testing, if not to insure in advance.



Respondents sometimes:

• do not come - saves a spare day of testing;
• badly late - it saves a big break between sessions;
• they do not understand which software will be tested remotely - it happens that the respondent doesn’t have a computer at hand before testing, it is important to agree very clearly;
• cannot configure the GoToMeeting program — we send the installation instructions and schedule a pre-call day to make sure everything is ready for testing.

Processing results



When processing the data we looked at:

• Is there any parameter that significantly affects the behavior of users when choosing a SO, and / or the parameters to which they are oriented?
• What are the steps in the process of selecting the SC?
• What are the parameters of the SC users consider the main?

The report also presented the identified usability-problems of service-centers.ru.



We talked about the preparation of the project, then we will present some of the results and interesting usability problems.

2. Results

2.1. Interview results

Lecture hall



According to the results of the interview, it can be said that the differences in experience / inexperience and region of residence do not significantly affect the actions of the owners of the equipment when selecting the SOs.



Activities are always divided into three steps:

• search for information on the Internet;
• call / consultation in SC;
• Making a decision on applying to the SC.

Despite the fact that it is impossible to single out groups of the audience that fundamentally differ in the behavior or parameters to which they are oriented, we managed to fix a number of interesting features of behavior.

Respondents belonging to the group "Experienced"	Respondents belonging to the group "Inexperienced"
Rely on consultations in the SC. Carefully and carefully ask about the possible causes of failure. Consultation by phone is more a check of SC and masters. Trust in SC less. Ring up more options.	They are more focused on external signs (photos, site appearance, courtesy of acceptance masters during a telephone conversation) and convenience of location. They call the SC to ask if they will take the repair equipment when they return, they do not test the masters.

The features of SC selection between the residents of the regions and Moscow turned out to be so curious that we decided not to publish them and leave them as the competitive advantage of our customer. :)



SC selection parameters



When telling respondents about the experience of selecting SCs on the Internet, we managed to fix the parameters that users are guided by.



The main parameters of the selection of SC on sites:

• type of technology - 5 of 8;
• opening hours - 4 of 8;
• cost of work - 4 of 8
• location - 4 of 8
  
  (preferred SC near home or work) ;
• manufacturer - 3 of 8.

Additional options:

• nice site - 2 of 8;
• SC specialization - 1 of 8.

In addition to the basic selection parameters, respondents consider:

• repair time - 7 out of 8 (more often, in order to understand when they get the thing back; if the thing is needed urgently, they clarify whether there is an option for urgent repair in the SC).

2.2. Test results

According to the results of testing, the main blocks of service-centers.ru problems were identified:

• navigation;
• filters and sorting;
• view list of SC;
• presentation of information about SC;
• problems card SC.

Consider the problem from the “Navigation” block that “meets” users.

There is no single and universal way to start searching for a service center.

Visitors are invited to use the search lines and sections "Most often in Moscow repair" and "The most popular service centers in Moscow."







Why is this uncomfortable? These navigation tools are good for specific tasks and can be used as additional.



For example, if a broken product belongs to frequently repaired or you need a SC approved by a popular manufacturer (yes, the second block refers to authorized SC, and not the most popular by rating or frequency of hits). The search strings worked well, but only for users who guessed their working principle. But the site does not have a basic, universal navigation tool.



What did the respondents do? The respondents took a long time to look through the main page before going to the SC search. In case of breakage of things that do not fall into the category of often repaired, users experienced difficulties.



Our recommendations. Present the main navigation on the site in the form of a product catalog with the usual categorization (electronics, computers and tablets, household appliances, cars). Since this tool is used in most online stores of household appliances and electronics, it will be more familiar and more convenient for users to choose the category of goods being repaired.



Of course, this is not all the difficulties that respondents encountered on the main page, but in this review we can cover a small number of problems, and it is better to make out the shortcomings of different blocks.



Let us analyze the problem from the "Filters and sorting" block.

Unnoticeable opportunity to view the SC on the map

View of the option "View on the map" before making changes:







Why is this uncomfortable? This visualization of the option made it “invisible”, it was located above the main filtering parameters (manufacturer, product name), almost merged with the background of the site and was invisible even to those respondents who found the filter unit. At the end of the first test day, the customer changed the visualization and location of the option.







Partially the problem was solved, but due to the unsuccessful design of the filter block * two respondents did not notice the possibility to sort the list of SOs by location.



* The filter block is not visually highlighted and merges with the background of the site. As a result, the filter block does not look like a tool belonging to the SC list, is perceived as “noise” and is ignored by users.



Our recommendations. View on the map, in fact, is not a filtering parameter, but a way to view the list of SCs. We recommended to place this functionality at the top of the list. For example, in the form of a switch "View on the map" and "View list".



Consider another problem, now from the block "Presentation of information about SC".

Unnoticeable to see the time of SC

The work schedule of the SC appears as a tooltip when you hover the cursor on the status of the SC ("Now open / close" and the time of closing / opening).







Users were inconvenient to navigate the status of the SC. To choose a time to visit the SC, it is important to know when the SC opens and closes. Most respondents did not notice the opportunity to see the work schedule by hovering over the status of the SC.



The instructions for the task did not indicate when the respondent would contact the SC. Respondents were confronted with the fact of a breakdown and asked to pick up a SC. Respondents independently focused on "tomorrow" or the coming days. When planning the visit time, it was important for the respondents first of all to see the working hours of the SC, in order to understand whether they would be able to call in before or after work.



Our recommendations. Present the work schedule of the SC in the plate and the SC card so that it is visible by default and does not require additional actions to display it. For example, just write a work time.

3. Results

The result of our collaboration was the improved website service-centers.ru. The project helped the customer team to understand its audience, their behavior when searching for SC and the selection priority parameters.



The test classics — the customer understood the language in which his audience spoke, and was able to adapt the product to meet the objectives of Central Asia.



It is important not to forget that the test does not guarantee 100% successful solutions, you need to check the implemented changes.



A good option for finding effective solutions is iterative testing.