Dmitry Satin. Modern technology usability testing
Usability, as an activity aimed at improving the user qualities of the product, has a huge impact on the success of the project. Each dollar spent on usability returns from 10 to 100 dollars of profit or savings. One of the important conditions for this effectiveness is regular usability testing of a product involving users at all phases of a project. Modern technologies not only simplify the procedure for usability testing, but also introduce new features, for example, monitoring the physiological indicators of the respondent or testing mobile devices. Depending on the product and the criticality of its user qualities, usability testing may involve additional technical tools, while remaining unchanged in its structure: the user performs tasks under the supervision of the moderator conducting the testing. (PC World # 05/2007) 
What is usability? What does it affect?
Usability is the degree of efficiency, productivity and satisfaction with which a product can be used by certain users to achieve certain goals in a specific context (ISO 9241-11). Achieving a high level of usability is possible only by applying a user-oriented approach to design and development.
A user-centered design (User-centered design, UCD) is characterized by the following features (ISO 13407):
The use of the UCD approach to achieve high usability results in reducing development costs and increasing product efficiency both in terms of business (additional profit) and user satisfaction (increasing loyalty to the product and the developer).
Analysis of return on investment (ROI) in usability shows that each dollar invested returns from $ 10 to $ 100 in the form of profits and savings associated with cost reduction [1].
The positive contribution of usability can be divided into four main groups of factors:
What is usability testing?
Usability testing is an experimental method built on interviewing users according to a predetermined scenario and aimed at finding out how they use the product.
Test participants (respondents) are selected and hired on the basis of a specialized questionnaire designed to weed out those people who are not included in the target audience of the product.
During each interview, the moderator (facilitator) asks the respondents to carry out tasks solved with the help of the device under study.
The meaning of usability testing is to identify the difficulties faced by users while working with the product. Taking into account the results of testing, it is possible to improve the interfaces so that the target audience of the product works most fruitfully with its final version.
Seeing how people interact with the product allows you to find ways to improve the design. But looking at it impassively is quite difficult. It is inherent to a person to help those who need help, and not to look at their failures. An effective moderator must keep the respondent’s focus on tasks, without helping him to solve these tasks.
Therefore, the moderator and the respondent should be in different rooms, communicating with each other through various communication devices. These rooms are usually separated by a one-way mirror, which allows the moderator and other observers to see how the respondent works.
')
Why do we need specialized usability testing tools?
Conducting usability testing does not seem very difficult, especially since only 5–10 respondents are sufficient for a full-fledged study [5, 6]. The main technical difficulty of this procedure is the large amount of data that needs to be recorded in the testing process for the purposes of subsequent analysis. This is the speech of the moderator and the respondent; facial expression of the respondent (videotaped); image of the computer screen with which the respondent works; various events occurring on the computer associated with the actions of the user: moving the mouse and pressing its keys, using the keyboard, navigating between screens (browser or other program). All these data streams should be synchronized so that when analyzing they can be correlated to each other.
Observers who participate in testing, along with the moderator, take notes as problems are discovered. This information also needs to be synchronized with other data so that later you can easily find a fragment of the record commented in the observer’s note.
The lack of special tools makes the subsequent analysis so time consuming that the time required for it exceeds two thirds of the total time spent on testing, from planning it to the report.
Morae 1.3
TechSmith supplies an affordable and yet highly efficient solution - Morae. This software complex takes 15 minutes to be deployed on the computers of the respondent, the moderator, and the observers, which is especially convenient if the testing takes place outside the company, for example, at the customer’s office.
Morae Recorder is installed on the respondent’s PC — a module that records the respondent’s voice, images of his face (for example, from a webcam), a computer screen, various events on a computer, etc.
The moderator and observers use the Morae Remote Viewer, a module that allows you not only to see what is happening on the respondent’s PC, but also to set markers — records with comments on the problems that arise, which are very helpful in the analysis when the bright impressions of the testing start to fade. .
At the end of the testing session, the recorded data is imported into the Morae Manager. Here they are analyzed and a visual video report is prepared for each of the detected problems or for each subject.
It should be mentioned that TechSmith also offers a remote usability testing tool - UserVue, which is indispensable in cases where it is necessary to provide a wide geographic coverage of respondents. This is especially true when testing websites.
UserVue is a lightweight version of Morae Recorder and is installed on the respondent’s computer with two or three mouse clicks.
At the end of the test, the received data can be imported into Morae Manager, where it will be processed and compiled into a video report.
For now, UserVue is only available in the USA and Canada, but it is obvious that with sufficient demand for this tool in other countries, its distribution area may expand to include Russia.
Convenience Morae is most clearly manifested in its efficiency. If the usability testing session is held in the morning, then by the evening a report on it can be received. Under these conditions, the most laborious is the development of recommendations for overcoming the identified problems, but this process cannot be automated, since it requires a creative approach, which so far is accessible only to humans.
Observer xt
However, this is not the only possible problem. Respondents, despite the promise received from them to be honest and frank, are often silent about the difficulties encountered in the testing process, not wanting to upset the moderator or trying to look more successful.
Typically, such distortions are quickly detected and do not affect the quality of the results obtained. But in cases where the cost of testing errors is too high, it is better to use more complex tools, the purpose of which is not so much in testing as in researching how the user interacts with the product. In this case, the Observer XT solution offered by Noldus and its associated equipment may be optimal.
In addition to the data recorded by Morae, Observer XT, by integrating with special equipment, allows recording physiological parameters: heart rate, pupil size, and when connecting infrared cameras supplied by the Swedish company Tobii, even the eye movement of the respondent. This allows not only to objectify additionally the received data (a change in the heart rhythm can speak about the emotions experienced by the respondent, even if he tries to hide it), but also to establish the relationship between the user’s work and the functional states he experiences. For example, it becomes possible to assess the degree of fatigue when working with a product.
This is not the only advantage of the system. The developers of the Noldus company offered a coding tool for the collected information, for which the unit of analysis is not individual operations, but complex user actions directly related to his work. To do this, it is necessary to collect various events into the structures that the system registers, thus teaching it to recognize complex user actions.
This allows you to analyze not only the interfaces with which it interacts, but also the activity in which these interfaces are involved.
The equipment produced by Noldus can be installed on stationary computers as well as used in the form of mobile usability laboratories that fit in a small suitcase.
Mobile Testing Technologies
User interfaces of mobile devices - phones, communicators and handheld computers - are also available to usability testing.
Due to the technical limitations of the products themselves, it is often impossible to capture an image from the screen in a way that can be done on a desktop or laptop computer.
Therefore, when testing mobile devices, special cameras are used. A simple version of such a camera can be assembled by yourself, using portable devices designed for covert shooting and widely represented on the market today. But such a decision will be associated with the restriction that the test subject will have to work at the table and will be restricted in movement, since the device will have to be fixed on a tripod.
To bring the testing conditions closer to real ones, it is better to use wireless portable cameras with which the user will feel freer - to a certain degree of course.
Nevertheless, despite the limitations that are currently available, usability testing is fully applicable not only to desktops and laptops, but also to mobile devices. In the West, it has been practiced for several years, and more recently, Russian operators have begun to pay a lot of attention to the issues of usability of the products they are developing, which will certainly give impetus to the spread of this methodology in our country.
Conclusion
With a great deal of confidence, we can expect that in the near future we will witness a rapid surge of interest in usability as an activity capable of several times increasing the success of the project and the product being developed.
The experience gained so far allows us to conduct high-quality testing of user characteristics of products in various conditions.
Since 1997, Russian companies began to appear departments whose activities were aimed at improving the user qualities of the developed products. As a natural continuation of this process, usability laboratories equipped with modern equipment and special software are already encountered in Russia today.
All this suggests that the present century will be the century of usability. t
Literature
1. Marcus A. Return on Investment for Usable Design // User Experience Magazine: Winter 2002, from http://upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html .
2. Human Factors International (2001). Some client experiences. Retrieved October 10, 2001, from http://www.humanfactors.com/about/finance.asp .
3. Bias RG & Mayhew DJ (Eds.) (1994). Cost-Justifying usability. San Francisco: Morgan Kaufmann Publishers.
4. Rhodes John S. (2000). Usability can save your company. Retrieved on October 10, 2001, from http://www.webword.com/moving/savecompany.html .
5. Jakob Nielsen's Alertbox, March 19, 2000: http://www.useit.com/alertbox0000319.html .
6. Scott Weiss. Handheld usability. John Wiley & Sons Ltd, 2002, p. 180.
What is usability? What does it affect?
Usability is the degree of efficiency, productivity and satisfaction with which a product can be used by certain users to achieve certain goals in a specific context (ISO 9241-11). Achieving a high level of usability is possible only by applying a user-oriented approach to design and development.
A user-centered design (User-centered design, UCD) is characterized by the following features (ISO 13407):
- active involvement of users in the process of product design and testing;
- clear understanding of user requirements and tasks;
- optimal distribution of functions between users and technologies;
- iterative and multidisciplinary approach.
The use of the UCD approach to achieve high usability results in reducing development costs and increasing product efficiency both in terms of business (additional profit) and user satisfaction (increasing loyalty to the product and the developer).
Analysis of return on investment (ROI) in usability shows that each dollar invested returns from $ 10 to $ 100 in the form of profits and savings associated with cost reduction [1].
The positive contribution of usability can be divided into four main groups of factors:
- development - reducing costs and reducing time;
- sales - increased revenue;
- users - increase efficiency, productivity and satisfaction;
- reducing the cost of implementation (by reducing training) and user support.
What is usability testing?
Usability testing is an experimental method built on interviewing users according to a predetermined scenario and aimed at finding out how they use the product.
Test participants (respondents) are selected and hired on the basis of a specialized questionnaire designed to weed out those people who are not included in the target audience of the product.
During each interview, the moderator (facilitator) asks the respondents to carry out tasks solved with the help of the device under study.
The meaning of usability testing is to identify the difficulties faced by users while working with the product. Taking into account the results of testing, it is possible to improve the interfaces so that the target audience of the product works most fruitfully with its final version.
Seeing how people interact with the product allows you to find ways to improve the design. But looking at it impassively is quite difficult. It is inherent to a person to help those who need help, and not to look at their failures. An effective moderator must keep the respondent’s focus on tasks, without helping him to solve these tasks.
Therefore, the moderator and the respondent should be in different rooms, communicating with each other through various communication devices. These rooms are usually separated by a one-way mirror, which allows the moderator and other observers to see how the respondent works.
')
Why do we need specialized usability testing tools?
Conducting usability testing does not seem very difficult, especially since only 5–10 respondents are sufficient for a full-fledged study [5, 6]. The main technical difficulty of this procedure is the large amount of data that needs to be recorded in the testing process for the purposes of subsequent analysis. This is the speech of the moderator and the respondent; facial expression of the respondent (videotaped); image of the computer screen with which the respondent works; various events occurring on the computer associated with the actions of the user: moving the mouse and pressing its keys, using the keyboard, navigating between screens (browser or other program). All these data streams should be synchronized so that when analyzing they can be correlated to each other.
Observers who participate in testing, along with the moderator, take notes as problems are discovered. This information also needs to be synchronized with other data so that later you can easily find a fragment of the record commented in the observer’s note.
The lack of special tools makes the subsequent analysis so time consuming that the time required for it exceeds two thirds of the total time spent on testing, from planning it to the report.
Morae 1.3
TechSmith supplies an affordable and yet highly efficient solution - Morae. This software complex takes 15 minutes to be deployed on the computers of the respondent, the moderator, and the observers, which is especially convenient if the testing takes place outside the company, for example, at the customer’s office.
Morae Recorder is installed on the respondent’s PC — a module that records the respondent’s voice, images of his face (for example, from a webcam), a computer screen, various events on a computer, etc.
The moderator and observers use the Morae Remote Viewer, a module that allows you not only to see what is happening on the respondent’s PC, but also to set markers — records with comments on the problems that arise, which are very helpful in the analysis when the bright impressions of the testing start to fade. .
At the end of the testing session, the recorded data is imported into the Morae Manager. Here they are analyzed and a visual video report is prepared for each of the detected problems or for each subject.
It should be mentioned that TechSmith also offers a remote usability testing tool - UserVue, which is indispensable in cases where it is necessary to provide a wide geographic coverage of respondents. This is especially true when testing websites.
UserVue is a lightweight version of Morae Recorder and is installed on the respondent’s computer with two or three mouse clicks.
At the end of the test, the received data can be imported into Morae Manager, where it will be processed and compiled into a video report.
For now, UserVue is only available in the USA and Canada, but it is obvious that with sufficient demand for this tool in other countries, its distribution area may expand to include Russia.
Convenience Morae is most clearly manifested in its efficiency. If the usability testing session is held in the morning, then by the evening a report on it can be received. Under these conditions, the most laborious is the development of recommendations for overcoming the identified problems, but this process cannot be automated, since it requires a creative approach, which so far is accessible only to humans.
Observer xt
However, this is not the only possible problem. Respondents, despite the promise received from them to be honest and frank, are often silent about the difficulties encountered in the testing process, not wanting to upset the moderator or trying to look more successful.
Typically, such distortions are quickly detected and do not affect the quality of the results obtained. But in cases where the cost of testing errors is too high, it is better to use more complex tools, the purpose of which is not so much in testing as in researching how the user interacts with the product. In this case, the Observer XT solution offered by Noldus and its associated equipment may be optimal.
In addition to the data recorded by Morae, Observer XT, by integrating with special equipment, allows recording physiological parameters: heart rate, pupil size, and when connecting infrared cameras supplied by the Swedish company Tobii, even the eye movement of the respondent. This allows not only to objectify additionally the received data (a change in the heart rhythm can speak about the emotions experienced by the respondent, even if he tries to hide it), but also to establish the relationship between the user’s work and the functional states he experiences. For example, it becomes possible to assess the degree of fatigue when working with a product.
This is not the only advantage of the system. The developers of the Noldus company offered a coding tool for the collected information, for which the unit of analysis is not individual operations, but complex user actions directly related to his work. To do this, it is necessary to collect various events into the structures that the system registers, thus teaching it to recognize complex user actions.
This allows you to analyze not only the interfaces with which it interacts, but also the activity in which these interfaces are involved.
The equipment produced by Noldus can be installed on stationary computers as well as used in the form of mobile usability laboratories that fit in a small suitcase.
Mobile Testing Technologies
User interfaces of mobile devices - phones, communicators and handheld computers - are also available to usability testing.
Due to the technical limitations of the products themselves, it is often impossible to capture an image from the screen in a way that can be done on a desktop or laptop computer.
Therefore, when testing mobile devices, special cameras are used. A simple version of such a camera can be assembled by yourself, using portable devices designed for covert shooting and widely represented on the market today. But such a decision will be associated with the restriction that the test subject will have to work at the table and will be restricted in movement, since the device will have to be fixed on a tripod.
To bring the testing conditions closer to real ones, it is better to use wireless portable cameras with which the user will feel freer - to a certain degree of course.
Nevertheless, despite the limitations that are currently available, usability testing is fully applicable not only to desktops and laptops, but also to mobile devices. In the West, it has been practiced for several years, and more recently, Russian operators have begun to pay a lot of attention to the issues of usability of the products they are developing, which will certainly give impetus to the spread of this methodology in our country.
Conclusion
With a great deal of confidence, we can expect that in the near future we will witness a rapid surge of interest in usability as an activity capable of several times increasing the success of the project and the product being developed.
The experience gained so far allows us to conduct high-quality testing of user characteristics of products in various conditions.
Since 1997, Russian companies began to appear departments whose activities were aimed at improving the user qualities of the developed products. As a natural continuation of this process, usability laboratories equipped with modern equipment and special software are already encountered in Russia today.
All this suggests that the present century will be the century of usability. t
Literature
1. Marcus A. Return on Investment for Usable Design // User Experience Magazine: Winter 2002, from http://upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html .
2. Human Factors International (2001). Some client experiences. Retrieved October 10, 2001, from http://www.humanfactors.com/about/finance.asp .
3. Bias RG & Mayhew DJ (Eds.) (1994). Cost-Justifying usability. San Francisco: Morgan Kaufmann Publishers.
4. Rhodes John S. (2000). Usability can save your company. Retrieved on October 10, 2001, from http://www.webword.com/moving/savecompany.html .
5. Jakob Nielsen's Alertbox, March 19, 2000: http://www.useit.com/alertbox0000319.html .
6. Scott Weiss. Handheld usability. John Wiley & Sons Ltd, 2002, p. 180.
Source: https://habr.com/ru/post/20066/
All Articles