How we tested interface usability
UX interface research is a popular trend of the past few years. In our company for this purpose we have created a usability laboratory. In it, we explore the usability of not only the products created in the company, but also conduct an audit of external projects at the request of customers.
In this post we want to talk about how we conduct an audit and usability testing on the example of a CRM-system research for the sale of special equipment. How did you prepare, what work did you do and how much did you get.
At the beginning of the work on the UX-study, we discussed with the customer his audience, found out the expectations of the project and together understood the product, identifying the main sections of the system that should be emphasized during the work.
And here are the results we got.
CRM-system is designed for companies selling and repairing special equipment and spare parts for agriculture.
')
The goal of the system is to provide users with tools that will automate their daily work processes. One of the main requirements for the system - ease of use. Intuitive interface, hints for the user, quick learning how to use the program, familiar web interface.
The audience of the system is very extensive, from the sales manager to the owner of the company and from the product specialist to the HR manager and logistician.
When analyzing the audience, we focused on two main user groups that require special attention:
Having discussed the main wishes of the customer, the following project requirements were formulated:
In order to complete the task in a short time, we, together with the client, have compiled a list of sections that require our special attention.
The audit was carried out by the method of expert evaluation. Based on the list of sections and audience of the application, we have compiled the main usage scenarios and analyzed the system.
The result of this stage was a report on the UX-audit consisting of two blocks:
In the course of the audit, several hypotheses of user behavior were put forward, on the basis of which test scripts were compiled.
Test scenarios have become typical tasks of the manager and the head of the department, which they face every day and spend most of their working time.
To test our recommendations, we developed an interactive prototype with which respondents could perform the same test scenarios.
Each respondent performed tasks on only one test bench: either in the prototype or in the system so that the learning experience does not affect the result.
The method of confirming the hypotheses was selected usability testing with elements of the interview (introductory briefing, filling in the questionnaire, testing, exit interview).
Twenty respondents were selected to participate in the study. They represented two groups of audiences - with experience in CRM-systems and without experience with similar systems.
In the group of “experienced users” we invited managers and managers from special equipment sales companies.
Before the start of testing for the target audience, we conducted a trial test on the company's employees. It was important for us to check whether factors affecting the quality of testing are present:
According to the results of trial testing, we finalized the prototype and changed the order of testing tasks.
Testing takes place in our laboratory, the process of researching audio and video is recorded. Respondents came to our office during office hours and some, of course, were late. Saved a reserve in time between sessions.
During the testing, the moderator remained in the laboratory: he read the tasks from the sheet so that there was no discrepancy in the execution of the task, watched them being executed.
Users could comment on their actions if they wish. If the moderator had questions about the behavior of the user, he asked them after the end of a specific task. The most difficult thing for a moderator is not to prompt users when performing tasks, not to answer their questions. All of this can affect metrics performance.
The following metrics were used to process the results:
Let's take a closer look at one scenario and the resulting results.
We tested a hypothesis about whether a user understands how to add a company address.
The results of testing the initial version showed that 73% of users experienced difficulties in performing this task.
Common user errors in the system
1) Users do not notice the “Find” button on the map. They are looking for the “Add” button or the address entry field.
Our prototype solution
The field for entering the address is placed above the card.
2) After entering the address, users see a mark on the map, but do not understand what to click on it in order to call the add address window.
Our solution for the prototype system
After adding the address and displaying it on the map, a window pops up with the addition of the address.
results
Test results showed that in the original version of the CRM-system, only 17% of users completed this task without errors. And in the prototype, 83% of the respondents successfully completed the task.
The task execution time in the prototype was reduced from 70 to 26 seconds. This clearly demonstrates the existing problem in the system and its effective solution.
The test results showed that the experience of using CRM in different ways affects the performance of tasks. Experienced users have an established understanding of the principles of work in similar systems, which affects the results of the execution of tasks. Tasks where the interface was radically different from their counterparts, inexperienced users performed faster, since this interface is intuitive for them. And experienced users were looking for solutions that were familiar to them, but after sorting out they performed the task and gave a good assessment of this implementation. And when performing tasks with a familiar interface, experienced users showed high results.
When comparing the results of work in the system and the prototype, we received confirmations of our hypotheses and revealed several more problems associated with experienced users and patterns familiar to them.
At the end of the stage a detailed report on the usability testing was compiled. The newly identified problems were analyzed, and the corresponding changes were made in the report on the UX-audit.
The result of our work was an improved CRM-system, which will soon be presented at a specialized exhibition. Usability testing allowed us to evaluate the effectiveness and efficiency of the existing system, identify the difficulties faced by users and offer an effective solution. We were able to verify the solutions we proposed before integrating them into the project, which allowed the client to reduce development costs. The earlier the problem is found in the interface, the lower the cost of fixing it.
In this post we want to talk about how we conduct an audit and usability testing on the example of a CRM-system research for the sale of special equipment. How did you prepare, what work did you do and how much did you get.
Preparatory stage - communication with the customer
At the beginning of the work on the UX-study, we discussed with the customer his audience, found out the expectations of the project and together understood the product, identifying the main sections of the system that should be emphasized during the work.
And here are the results we got.
Find out the customer's vision of your product.
CRM-system is designed for companies selling and repairing special equipment and spare parts for agriculture.
')
The goal of the system is to provide users with tools that will automate their daily work processes. One of the main requirements for the system - ease of use. Intuitive interface, hints for the user, quick learning how to use the program, familiar web interface.
Learned about the product audience
The audience of the system is very extensive, from the sales manager to the owner of the company and from the product specialist to the HR manager and logistician.
When analyzing the audience, we focused on two main user groups that require special attention:
- Sales Manager. This is a man aged 20-30 years. Mostly from the countryside, and there they found him. Simple, usually without universities. The level in the profession is often not high: does not know the product, poorly owns sales technologies. Considers working with CRM as an additional, unnecessary and burdensome burden.
- Sales Manager. A man aged 25–40 years. Out of sellers. It often happens that the branch manager bears the function of the head of the sales department. As a rule, the successful seller himself. Keeps on the territory of the most interesting customers, leaving ordinary sellers to pick up the rest.
Decided on the purpose of the UX-research
Having discussed the main wishes of the customer, the following project requirements were formulated:
- Creation of recommendations on the application of interface elements in standard system pages: a list with a filter panel, a card, a dashboard, a calendar view.
- Search and analysis of places that can produce human errors.
- Acceleration of user interaction with the interface.
- Improving user training when working with the system.
In order to complete the task in a short time, we, together with the client, have compiled a list of sections that require our special attention.
Audit
The audit was carried out by the method of expert evaluation. Based on the list of sections and audience of the application, we have compiled the main usage scenarios and analyzed the system.
The result of this stage was a report on the UX-audit consisting of two blocks:
- Expert advice to help refine the system to fix problems and improve the interface.
- A developer’s guide to help maintain a single solution for generic project pages.
Usability testing
Training
In the course of the audit, several hypotheses of user behavior were put forward, on the basis of which test scripts were compiled.
Test scenarios have become typical tasks of the manager and the head of the department, which they face every day and spend most of their working time.
- Add a new meeting
- Add address to meeting
- Add, edit, delete contact in the company
- Add new address
- Find a deal
- Perform the first stage of the transaction
- Add item to transaction
- Play a deal
To test our recommendations, we developed an interactive prototype with which respondents could perform the same test scenarios.
Each respondent performed tasks on only one test bench: either in the prototype or in the system so that the learning experience does not affect the result.
The method of confirming the hypotheses was selected usability testing with elements of the interview (introductory briefing, filling in the questionnaire, testing, exit interview).
Twenty respondents were selected to participate in the study. They represented two groups of audiences - with experience in CRM-systems and without experience with similar systems.
In the group of “experienced users” we invited managers and managers from special equipment sales companies.
Before the start of testing for the target audience, we conducted a trial test on the company's employees. It was important for us to check whether factors affecting the quality of testing are present:
- All scripts are run in prototype and system. Here we paid special attention to the prototype, since its functionality is limited and any erroneous action could lead to incorrect performance of the task and affect the final result.
- Are there any dependencies in the tasks? Whether the learning ability of previous tasks affects the performance of the current one.
According to the results of trial testing, we finalized the prototype and changed the order of testing tasks.
Testing
Testing takes place in our laboratory, the process of researching audio and video is recorded. Respondents came to our office during office hours and some, of course, were late. Saved a reserve in time between sessions.
During the testing, the moderator remained in the laboratory: he read the tasks from the sheet so that there was no discrepancy in the execution of the task, watched them being executed.
Users could comment on their actions if they wish. If the moderator had questions about the behavior of the user, he asked them after the end of a specific task. The most difficult thing for a moderator is not to prompt users when performing tasks, not to answer their questions. All of this can affect metrics performance.
The following metrics were used to process the results:
- The success of assignments.
- Time to complete assignments.
- Frequency problems.
- Subjective satisfaction.
Let's take a closer look at one scenario and the resulting results.
Scenario “Adding Company Address”
We tested a hypothesis about whether a user understands how to add a company address.
The results of testing the initial version showed that 73% of users experienced difficulties in performing this task.
Common user errors in the system
1) Users do not notice the “Find” button on the map. They are looking for the “Add” button or the address entry field.
Our prototype solution
The field for entering the address is placed above the card.
2) After entering the address, users see a mark on the map, but do not understand what to click on it in order to call the add address window.
Our solution for the prototype system
After adding the address and displaying it on the map, a window pops up with the addition of the address.
results
Test results showed that in the original version of the CRM-system, only 17% of users completed this task without errors. And in the prototype, 83% of the respondents successfully completed the task.
The task execution time in the prototype was reduced from 70 to 26 seconds. This clearly demonstrates the existing problem in the system and its effective solution.
Interface Test Results
The test results showed that the experience of using CRM in different ways affects the performance of tasks. Experienced users have an established understanding of the principles of work in similar systems, which affects the results of the execution of tasks. Tasks where the interface was radically different from their counterparts, inexperienced users performed faster, since this interface is intuitive for them. And experienced users were looking for solutions that were familiar to them, but after sorting out they performed the task and gave a good assessment of this implementation. And when performing tasks with a familiar interface, experienced users showed high results.
When comparing the results of work in the system and the prototype, we received confirmations of our hypotheses and revealed several more problems associated with experienced users and patterns familiar to them.
At the end of the stage a detailed report on the usability testing was compiled. The newly identified problems were analyzed, and the corresponding changes were made in the report on the UX-audit.
Results
The result of our work was an improved CRM-system, which will soon be presented at a specialized exhibition. Usability testing allowed us to evaluate the effectiveness and efficiency of the existing system, identify the difficulties faced by users and offer an effective solution. We were able to verify the solutions we proposed before integrating them into the project, which allowed the client to reduce development costs. The earlier the problem is found in the interface, the lower the cost of fixing it.
Source: https://habr.com/ru/post/315342/
All Articles