Through thorns to the stars: why you need to seek access to users, and how to do it
We have already written a lot about the role of qualitative methodology in user research (for example, http://bit.ly/25hnAMQ and http://bit.ly/2jmfkcM ). In one of the articles ( http://bit.ly/2ay3Nzg ) we talked about how qualitative methods are used in UX research to develop IT products in foreign companies. What is happening in Russia?
In this article we will talk about our experience in applying qualitative methodology for the development of IT products in the Russian market. We will talk not only about the specifics of the work, but also about how to proceed with it. How to explain to the customer the value of a qualitative methodology? What to tell him to give the go-ahead to communicate with users? And why does it even become necessary to explain?
In the article we will discuss three key points.
Let's start with the problems of understanding qualitative methodology. In fact, it will be a question of what objections to the conduct of qualitative research the customer puts forward. Organizing research is a work task. The main thing is to start it. And for this you need to get to the customer. What problems can there be?
')
It is still a popular question from Russian customers of various sizes. The situation is changing, but not as fast as we would like. Let's figure out what's the matter.
The customer fears that time and money will be wasted.
Fears are generally understandable: for the customer, qualitative methods and user analytics remain a black box. Recall that in Western countries, the situation is different. And we have a black box, where they ask to put money and time of the project. Do not forget about it when talking with the customer. The situation is complicated by the fact that the output result is unknown.
The customer does not understand what he will receive as a result of the research and how he will work with it.
This is a serious problem related to the presentation of the results of qualitative research. For example, numbers in quantitative studies are understandable. Web analytics is understandable. Job copywriter and designer visible to the naked eye. The results of the qualitative research packaged in the formats of user portraits, a description of the environment of use, life situations and scenarios are not understandable to the customer without a detailed explanation. What do these results give? How does the team work with them? What benefits do they bring? All these customer questions need to be able to answer.
There is another situation where the customer roughly represents ( or thinks that he represents ) what a user analyst is. Probably, I heard about qualitative methods. Why do you need to conduct research, if I can tell everything myself?
The customer believes in his expert knowledge of users.
What remains to us? It is necessary to understand the origins of this knowledge and its depth. For example, it is many years of experience in the industry. It's good. To analyze this experience, we conduct interviews with the customer about users. Who are they? What groups can be identified? What problems do they face? What are their questions? What do they live in anyway? And there are many more questions that qualitative research covers. We do not deny the knowledge of the customer, but consider it as a hypothesis - we check and supplement it with the help of research.
Another possible source of expertise is your own research. This is very good, but you need to understand how the study was conducted? What can we learn about users, their life situations and behavior? Not all studies are equally useful for UX tasks.
There is another facet of belief in their expert knowledge - disbelief in user knowledge.
The customer doubts the user competence and knowledge.
What can they tell you? Will offer the car "features", and they do not need them. The problem in this case is the incorrect understanding of the essence of the qualitative research of users. The researcher studies the user's daily life, his problems, the environment of use, and does not ask about ready-made solutions. Development of the solution is our task. Users do not need competencies in the development of IT products.
And now let's talk about the concerns of the customer on the organization of the study.
The customer is convinced that we cannot organize the study.
The organization of the study may seem to be a difficult process: you need to find users, agree with the company's employees, process the collected data, etc. In practice, the complexity of the organization of the study depends on many factors. Often the active participation of the customer greatly simplifies the process.
The customer is afraid that users will refuse to participate in the study.
Various ways of motivation help to attract users to participate in the study - from delayed gain, personal interest in the research topic to stimulating with money. Sometimes such problems do not arise at all: people are ready to help, if it is clear to explain why we turn to them.
Before analyzing the cases, we will once again list the problems and concerns that the customer has regarding the user research using qualitative methodology:
We now turn to the cases. The main task of this part is:
We specifically selected cases with different project objectives: content preparation, auditing and prototyping a professional interface, researching self-service kiosks. Customers are also different - small business (Set.Design), medium business (Solar Staff) and large business (Sportmaster).
Cases Set.Design and Solar Staff can be viewed on our website. Case Sportmaster, unfortunately, is not available - strict conditions NDA. The main emphasis in the analysis of cases is made on:
Company Set. Design manufactures and sells wooden eco-mosaic, decorative 3D panels, coffered ceilings and other decorative items for interior design. The company has its own designers and its carpentry shop. They make products according to the catalog, adapt someone else's solution or create an original product according to a sketch.
The company came for the texts on the new site. In the course of the discussion, they found out that the company has questions on positioning - whether it is a manufacturer of design materials, or a supplier of solutions based on these materials, or even individual production. Not really the task of the texts, but without this the texts do not go.
There were tight financial constraints on the project, so the best method was to use a semi-structured interview. The allotted time was enough for only three interviews (taking into account the preparation of the text). But even three interviews are better than none.
To determine the target audience, we talked with the customer and found out: we are interested in interior designers. Based on the results of this communication, we compiled a guide of the interview and discussed it with the customer. The customer may offer additional questions or topics - if there is an opportunity, we add them.
One respondent was suggested by the customer - the buyer of his shop. The other is the first designer: "a friend who does everything differently." The snowball method in action. The third respondent was found independently. Interviews were conducted in person and via Skype.
We conducted three interviews - it could have been more, but there was no time left in the project. For each interview, a summary was made, which the customer could read.
Based on the interview, portraits of users were created - two designers and one “independent customer”. A portrait of an independent buyer was compiled according to information shared by the designers and the customer. It would have been nice to have a talk with such a buyer, but the limitations of the project did not allow.
Interview texts helped to formulate the designer’s questions about the company, its work, materials, etc. These questions formed the basis of the content plan and object-information model - a structured list of questions on which the customer was interviewed. After the documents are taken to work, there is no doubt - now they will not forget about the users. The text will answer their questions. Questions of real people.
The interview results helped with positioning. It turned out that own production is a huge plus in the eyes of designers, and this must be borne in mind when shaping the image. Until now, the client was afraid that his company would look like an ordinary carpentry workshop, and was silent about the shop on the old site. The results of qualitative research often go beyond UX-tasks.
Solar Staff is a service for automating workflow and payments to distributed teams and freelancers. The employer registers with the system and invites freelancers involved in the project. An employer sets tasks, negotiates a price, accepts or rejects results, pays for work. And all the bureaucracy takes the service.
Conduct user research and interface auditing to identify key issues for freelancers and executives.
Before the audit, it was decided to conduct a study of freelancers and managers who had experience with the service, using a semi-structured interview.
To develop the guide, we conducted blitz-interviews within the company - we were interested in those who worked as a freelancer or had experience in hiring them. Created guide approved by the customer, and we began to study.
The customer was ready to help with the search for respondents - given the specifics of the project, help was very necessary. Before the start of the survey, customers were afraid that people would refuse to talk about hiring freelancers, because from the point of view of labor law, this is a rather sensitive issue. Of particular concern was the topic of payment for the services of freelancers.
Problems with the selection of respondents arose, but they managed to be solved - the customer approached the search for the right people seriously. Despite the fears, people agreed to discuss the payment of services of freelancers, when they explained the purpose of the study. We were interested in the payment process and the problems that arise, and not the specific amounts and addressees of transfers.
As a result, we managed to conduct ten interviews - five each with freelancers and managers.
We discussed with the respondents not only work with the service, but also experience before the service. How were the freelancer and manager relationships organized before? These are valuable insights that may be reflected in the service. In qualitative research, we always go beyond the boundaries of product use.
The interviews revealed problems in the work with the service of freelancers and managers. The customer guessed about many problems of the service, but he wanted to test his hypotheses. He did not abandon the study: as a result, the list of problems and difficulties was expanded.
The results of the study were decorated in portraits of users. This helped to understand how the service users live and what problems they face. The obtained information can be used for further development of the service.
The interview allowed us to develop realistic scenarios for using the service. These scenarios were combined into two tasks for an audit: the history of a freelancer and the history of a manager. According to these tasks, the interface designer conducted an audit of the service. At the exit, we received a list of user problems and expert assessment of the service interface for real-world scenarios.
The customer received a list of problems and recommendations: some decided to implement independently (the main page), and left us the redesign of the service. The results of the work are available on the site.
Sportmaster does not need long performances - this large retailer specializing in sporting goods has stores in Russia, Belarus, Ukraine, Kazakhstan and even in China.
We had to learn how shoppers and store employees work with self-service kiosks. We had to identify the problems faced by users of kiosks. And not only problems with the interface.
After an interview with customers, we made a list of hypotheses that needed to be tested during the study.
Since the task was difficult, and the questions concerned not only the interfaces, we decided to use several methods:
The study turned out to be large-scale, but the tasks were not easy. Problems with the organization of the customer did not bother. Field work was carried out in two stages: first communicated with customers, and then - with employees. The study involved four stores in different parts of the city - the researcher visited two stores a day. Employees were interviewed directly at the workplace, and they talked with customers while they made purchases.
Buyers were motivated to participate in usability testing and interviews with loyalty card bonuses. There were almost no failures.
Despite the large amount of research procedures, there were no problems with the organization. Field work took four days.
As a result, we conducted and recorded 20 sessions of moderated usability testing on video. In addition, about 40 interviews of various lengths were conducted with customers and store employees.
The results of the observation were recorded in the diary and with the help of photo / video.
The customer received a 50-page report that included a list of confirmed, partially confirmed, and refuted hypotheses. Each conclusion was supported by a reference to the study: photographs, videotaped sessions.
An expert assessment of the interfaces and recommendations on how to improve these interfaces are given. On the basis of analytics and brainstorming, a number of proposals were put forward on how to draw the attention of visitors to the terminals. The report includes seven ideas.
Following the observations and interviews, we collected data on how self-service kiosks fit into the daily routine of employees and customers. This information is especially important for the development of self-service systems.
Let us return to the objections and doubts of the customer and try to answer them using case materials.
Unfortunately, qualitative research is still exotic in the Russian market for the development of IT products. Be prepared for the fact that the customer does not understand the essence of the methodology. And you need to be able to persuade him.
In this article we will talk about our experience in applying qualitative methodology for the development of IT products in the Russian market. We will talk not only about the specifics of the work, but also about how to proceed with it. How to explain to the customer the value of a qualitative methodology? What to tell him to give the go-ahead to communicate with users? And why does it even become necessary to explain?
In the article we will discuss three key points.
- What does the customer think about quality methods in user analytics? Consider the main problems in the understanding of qualitative methodology and the reasons for their appearance.
- How is a qualitative study? Let us show, by the example of our case studies, the organization of research, problems and their solutions, communication with the customer during work.
- What we get as a result of the study? How and who works with these results? Using case studies, we will look at the team’s work with the research results, as well as how the customer can use these results outside of UX tasks.
Let's start with the problems of understanding qualitative methodology. In fact, it will be a question of what objections to the conduct of qualitative research the customer puts forward. Organizing research is a work task. The main thing is to start it. And for this you need to get to the customer. What problems can there be?
')
Is it possible without research and user analytics?
It is still a popular question from Russian customers of various sizes. The situation is changing, but not as fast as we would like. Let's figure out what's the matter.
The customer fears that time and money will be wasted.
Fears are generally understandable: for the customer, qualitative methods and user analytics remain a black box. Recall that in Western countries, the situation is different. And we have a black box, where they ask to put money and time of the project. Do not forget about it when talking with the customer. The situation is complicated by the fact that the output result is unknown.
The customer does not understand what he will receive as a result of the research and how he will work with it.
This is a serious problem related to the presentation of the results of qualitative research. For example, numbers in quantitative studies are understandable. Web analytics is understandable. Job copywriter and designer visible to the naked eye. The results of the qualitative research packaged in the formats of user portraits, a description of the environment of use, life situations and scenarios are not understandable to the customer without a detailed explanation. What do these results give? How does the team work with them? What benefits do they bring? All these customer questions need to be able to answer.
Belief in your own expertise
There is another situation where the customer roughly represents ( or thinks that he represents ) what a user analyst is. Probably, I heard about qualitative methods. Why do you need to conduct research, if I can tell everything myself?
The customer believes in his expert knowledge of users.
What remains to us? It is necessary to understand the origins of this knowledge and its depth. For example, it is many years of experience in the industry. It's good. To analyze this experience, we conduct interviews with the customer about users. Who are they? What groups can be identified? What problems do they face? What are their questions? What do they live in anyway? And there are many more questions that qualitative research covers. We do not deny the knowledge of the customer, but consider it as a hypothesis - we check and supplement it with the help of research.
Another possible source of expertise is your own research. This is very good, but you need to understand how the study was conducted? What can we learn about users, their life situations and behavior? Not all studies are equally useful for UX tasks.
There is another facet of belief in their expert knowledge - disbelief in user knowledge.
The customer doubts the user competence and knowledge.
What can they tell you? Will offer the car "features", and they do not need them. The problem in this case is the incorrect understanding of the essence of the qualitative research of users. The researcher studies the user's daily life, his problems, the environment of use, and does not ask about ready-made solutions. Development of the solution is our task. Users do not need competencies in the development of IT products.
We will not be able to organize a study
And now let's talk about the concerns of the customer on the organization of the study.
The customer is convinced that we cannot organize the study.
The organization of the study may seem to be a difficult process: you need to find users, agree with the company's employees, process the collected data, etc. In practice, the complexity of the organization of the study depends on many factors. Often the active participation of the customer greatly simplifies the process.
The customer is afraid that users will refuse to participate in the study.
Various ways of motivation help to attract users to participate in the study - from delayed gain, personal interest in the research topic to stimulating with money. Sometimes such problems do not arise at all: people are ready to help, if it is clear to explain why we turn to them.
Before going to the cases
Before analyzing the cases, we will once again list the problems and concerns that the customer has regarding the user research using qualitative methodology:
- the customer fears that time and money will be wasted;
- the customer does not understand what he will receive as a result of the research and how he will work with it;
- the customer believes in his expert knowledge of users;
- customer doubts user competences and knowledge;
- the customer is convinced that the study cannot be organized;
- the customer is afraid that users will not participate in the study.
We now turn to the cases. The main task of this part is:
- show how qualitative methodology is used to research users in specific projects;
- show what arguments in favor of qualitative research can be made to the customer.
We specifically selected cases with different project objectives: content preparation, auditing and prototyping a professional interface, researching self-service kiosks. Customers are also different - small business (Set.Design), medium business (Solar Staff) and large business (Sportmaster).
Cases Set.Design and Solar Staff can be viewed on our website. Case Sportmaster, unfortunately, is not available - strict conditions NDA. The main emphasis in the analysis of cases is made on:
- project task;
- organization of research and customer participation in the work;
- collected materials;
- research results and their use.
Set. Design: texts on a site and positioning for small business
Company Set. Design manufactures and sells wooden eco-mosaic, decorative 3D panels, coffered ceilings and other decorative items for interior design. The company has its own designers and its carpentry shop. They make products according to the catalog, adapt someone else's solution or create an original product according to a sketch.
Project task
The company came for the texts on the new site. In the course of the discussion, they found out that the company has questions on positioning - whether it is a manufacturer of design materials, or a supplier of solutions based on these materials, or even individual production. Not really the task of the texts, but without this the texts do not go.
Research organization
There were tight financial constraints on the project, so the best method was to use a semi-structured interview. The allotted time was enough for only three interviews (taking into account the preparation of the text). But even three interviews are better than none.
To determine the target audience, we talked with the customer and found out: we are interested in interior designers. Based on the results of this communication, we compiled a guide of the interview and discussed it with the customer. The customer may offer additional questions or topics - if there is an opportunity, we add them.
One respondent was suggested by the customer - the buyer of his shop. The other is the first designer: "a friend who does everything differently." The snowball method in action. The third respondent was found independently. Interviews were conducted in person and via Skype.
Collected materials
We conducted three interviews - it could have been more, but there was no time left in the project. For each interview, a summary was made, which the customer could read.
The results of the study and their use
Based on the interview, portraits of users were created - two designers and one “independent customer”. A portrait of an independent buyer was compiled according to information shared by the designers and the customer. It would have been nice to have a talk with such a buyer, but the limitations of the project did not allow.
Interview texts helped to formulate the designer’s questions about the company, its work, materials, etc. These questions formed the basis of the content plan and object-information model - a structured list of questions on which the customer was interviewed. After the documents are taken to work, there is no doubt - now they will not forget about the users. The text will answer their questions. Questions of real people.
The interview results helped with positioning. It turned out that own production is a huge plus in the eyes of designers, and this must be borne in mind when shaping the image. Until now, the client was afraid that his company would look like an ordinary carpentry workshop, and was silent about the shop on the old site. The results of qualitative research often go beyond UX-tasks.
Solar Staff: a professional interface for freelancers and executives
Solar Staff is a service for automating workflow and payments to distributed teams and freelancers. The employer registers with the system and invites freelancers involved in the project. An employer sets tasks, negotiates a price, accepts or rejects results, pays for work. And all the bureaucracy takes the service.
Project task
Conduct user research and interface auditing to identify key issues for freelancers and executives.
Research organization
Before the audit, it was decided to conduct a study of freelancers and managers who had experience with the service, using a semi-structured interview.
To develop the guide, we conducted blitz-interviews within the company - we were interested in those who worked as a freelancer or had experience in hiring them. Created guide approved by the customer, and we began to study.
The customer was ready to help with the search for respondents - given the specifics of the project, help was very necessary. Before the start of the survey, customers were afraid that people would refuse to talk about hiring freelancers, because from the point of view of labor law, this is a rather sensitive issue. Of particular concern was the topic of payment for the services of freelancers.
Problems with the selection of respondents arose, but they managed to be solved - the customer approached the search for the right people seriously. Despite the fears, people agreed to discuss the payment of services of freelancers, when they explained the purpose of the study. We were interested in the payment process and the problems that arise, and not the specific amounts and addressees of transfers.
Collected materials
As a result, we managed to conduct ten interviews - five each with freelancers and managers.
We discussed with the respondents not only work with the service, but also experience before the service. How were the freelancer and manager relationships organized before? These are valuable insights that may be reflected in the service. In qualitative research, we always go beyond the boundaries of product use.
The results of the study and their use
The interviews revealed problems in the work with the service of freelancers and managers. The customer guessed about many problems of the service, but he wanted to test his hypotheses. He did not abandon the study: as a result, the list of problems and difficulties was expanded.
The results of the study were decorated in portraits of users. This helped to understand how the service users live and what problems they face. The obtained information can be used for further development of the service.
The interview allowed us to develop realistic scenarios for using the service. These scenarios were combined into two tasks for an audit: the history of a freelancer and the history of a manager. According to these tasks, the interface designer conducted an audit of the service. At the exit, we received a list of user problems and expert assessment of the service interface for real-world scenarios.
The customer received a list of problems and recommendations: some decided to implement independently (the main page), and left us the redesign of the service. The results of the work are available on the site.
"Sportmaster": a study of self-service kiosks
Sportmaster does not need long performances - this large retailer specializing in sporting goods has stores in Russia, Belarus, Ukraine, Kazakhstan and even in China.
Project task
We had to learn how shoppers and store employees work with self-service kiosks. We had to identify the problems faced by users of kiosks. And not only problems with the interface.
Research organization
After an interview with customers, we made a list of hypotheses that needed to be tested during the study.
Since the task was difficult, and the questions concerned not only the interfaces, we decided to use several methods:
- semi-structured interviews with customers and store employees;
- monitoring customer and employee interaction with self-service kiosks;
- usability testing of self-service kiosks with customers and employees.
The study turned out to be large-scale, but the tasks were not easy. Problems with the organization of the customer did not bother. Field work was carried out in two stages: first communicated with customers, and then - with employees. The study involved four stores in different parts of the city - the researcher visited two stores a day. Employees were interviewed directly at the workplace, and they talked with customers while they made purchases.
Buyers were motivated to participate in usability testing and interviews with loyalty card bonuses. There were almost no failures.
Despite the large amount of research procedures, there were no problems with the organization. Field work took four days.
Collected materials
As a result, we conducted and recorded 20 sessions of moderated usability testing on video. In addition, about 40 interviews of various lengths were conducted with customers and store employees.
The results of the observation were recorded in the diary and with the help of photo / video.
The results of the study and their use
The customer received a 50-page report that included a list of confirmed, partially confirmed, and refuted hypotheses. Each conclusion was supported by a reference to the study: photographs, videotaped sessions.
An expert assessment of the interfaces and recommendations on how to improve these interfaces are given. On the basis of analytics and brainstorming, a number of proposals were put forward on how to draw the attention of visitors to the terminals. The report includes seven ideas.
Following the observations and interviews, we collected data on how self-service kiosks fit into the daily routine of employees and customers. This information is especially important for the development of self-service systems.
Conclusion: how to achieve access to users?
Let us return to the objections and doubts of the customer and try to answer them using case materials.
- The customer fears that time and money will be wasted. Qualitative research allows you to get detailed information about users and their behavior. Even three interviews help you gather valuable data about your users.
- The customer does not understand what he will receive as a result of the research and how he will work with it. The research results are used by the project team throughout the work. User questions help write texts on the site. Life situations and scenarios are needed when auditing the interface. The results of observation will allow you to go beyond the interface and see new problems that hinder effective work with your IT product. It is important that the users will not be forgotten. Sometimes the results of qualitative research go beyond the solution of UX-problems. For example, they help determine the positioning of a company or service. These results can also be used by marketers of the company.
- The customer believes in his expert knowledge of users. Hypotheses that are at the beginning of work at the customer or project team are checked during the research. Some of the hypotheses can be confirmed, some refuted. The project team will be able to work with the confirmed hypotheses in the future. We do not refuse customer knowledge, but we check it and supplement it.
- The customer doubts the user competence and knowledge. In research, we study user experience. We collect this experience to make decisions on a product based on it. In this study, we specifically go beyond the product to get valuable insights from the user's daily life, which may be reflected in the product.
- The customer is convinced that the study will not be able to organize. Any research can be organized. Of course, there are problems, but they can be solved. Customer participation in research helps speed up many processes.
- The customer is afraid that users will refuse to participate in the study. Users are ready to talk. You can motivate them in different ways. Of course, financial incentives simplify the data collection process, but you can do without it.
Unfortunately, qualitative research is still exotic in the Russian market for the development of IT products. Be prepared for the fact that the customer does not understand the essence of the methodology. And you need to be able to persuade him.
Source: https://habr.com/ru/post/325258/
All Articles