8 facts about online surveys that every researcher should remember
Hello! My name is Sveta Polovinkina. I lead the audience research department at the Wargaming development center in Minsk.
Today, analysts have access to a wide range of tools that allow them to study user behavior. However, you always want to know what the user thinks about, what he wants, what attracts him in the product.
And here, perhaps, a survey can help. At first glance, everything is simple: all you need is to create a questionnaire and find users who will answer the questions. It is because of the seeming simplicity that it is so easy to lose your head and make wrong decisions based on incorrect data. In this article, I want to give you 8 facts about online surveys that every researcher who has decided to collect data online should remember.
Fact 1: You are not your user.
No one knows the product better than the developer or seller. We remember all the details, possible scenarios of interaction, advantages and disadvantages. Therefore, user reviews are sometimes surprising. What is the matter? Why is it something they do not understand? It's simple: you are not your user. An outside person knows much less about the product, and he relies only on his own experience and emotions. Everything that seems obvious or convenient to you may be unknown or incomprehensible to the user.
A vivid example of how the new user and the developer interact with the product:
Conduct a preliminary study before making a survey questionnaire. Examine user feedback, discussions on forums or social networks, internal statistics and market information. Chat with community managers, support, sales, etc. What you learn will be the source of hypotheses that will be tested by a survey.
Even after careful study of all available information, you should not be completely sure that everything is taken into account. Therefore, before you run a survey, it is worth testing it on real users. This will allow the respondent to see the reaction to the questions, make sure that he understands everything and add relevant response options.
Fact 2: One study - one goal.
If you do not conduct surveys every day, then you probably want to know as much as possible with the help of a single study. And, most likely, you will get either a very long questionnaire that the user will not fill out, or the questions will be very superficial and will not help to reveal the essence of the problem. In order not to get carried away with the compilation of various questions, you should first determine the goals and objectives of your research and concentrate on them.
')
Fact 3: Opinion and facts are not the same thing.
User responses are their opinion. It is subjective and relevant only at a certain moment. Opinion is influenced by personal experience, mood, recent bright events, as well as the formulation of the question. The user may not know something or forget something. In addition, he will tell you only what he wants to tell. Therefore, remember that the results of surveys cannot be viewed as exact figures - there is always an error in this kind of data.
You can find out from the user his attitude to something, find out the level of awareness of something, assess the perception and satisfaction with the product. What you can find out using internal statistics should be taken there.
If any important for you indicators of user behavior are not recorded in the databases, then, asking about this, you should limit yourself to any particular time from the recent past. For example: “How often have you visited megamozg.ru in the last month?” The more time has passed since the event, the less accurate and honest answer you will get.
Fact 4: the user has personal files.
Your invitation to take part in the survey most often finds the user at a time when he is busy with personal affairs. Therefore, not all will answer. If the user agrees to take part in the survey, he will devote a certain amount of time to this, but no more than he wants. Therefore, your task is to make the questionnaire clear, simple and relevant for the user, so that he does not want to close the questionnaire before reaching the “ready” button.
In an online survey situation, a user cannot clarify with you the meaning of a particular question. The questionnaire must be carefully thought out, contain clear questions and relevant response options. An important role is played by the length of the questionnaire. In our experience, users easily fill out a questionnaire of 20–30 questions, with about 85–90% of respondents reaching the end of the questionnaire.
Fact 5: The way to get the data matters.
Correctly conducted survey allows you to distribute data to the entire target audience and understand the scale of the problem. It is not enough to simply collect a large number of answers. It is important to determine the characteristics of the user whose opinion is required to find out, and to choose the optimal method of research.
For example, if you need to get information about the experience of interacting with a new product, then the survey should be conducted among those who used it. If it is required to study not only the first impression, but also to identify the main drawbacks of the product, then when selecting respondents we add indicators of the duration or intensity of use of the product.
Fact 6: The most interested users respond.
The more and longer the user interacts with the product, the more he is interested in helping you improve it. Involved users are happy to respond to polls, because they want their opinion to be taken into account. However, among your audience there are those who use the product rarely or have become your client only recently. Most often, they are not so interested in expressing themselves, but in most cases they are also worth listening to. In order to learn the opinions of the entire audience, it is important to balance the answers of experienced users and beginners. As a result, the sample should be similar to your entire audience in terms of key parameters.
Fact 7: User opinions may vary.
The attitude of the user to your product may depend not only on his gender, age and education, but on the duration and experience of using the product, as well as on the degree of involvement / loyalty of the user. Analysis of user responses in conjunction with data on its behavior with respect to the product, will give you a more complete understanding of the expectations and needs of various user groups. And this, in turn, will allow to find the most relevant offers for each user and influence the audience more effectively.
For example, experienced users can more negatively assess new versions of a product, since they have to change their usual style of using the product / behavior. While beginners may simply not be aware of the innovation or be neutral to them, since they have little experience in using the product to evaluate it.
To combine survey data and statistics, you should find a way to identify each user among the respondents. Thus, you will be able to not only see the differences between users, but also be able to verify that the “correct” respondents answered you.
Fact 8: Polls do not help make the product perfect.
There is always a temptation to directly ask the user what he wants. At the same time, be sure that you will get an answer to any question. However, this information will not always benefit you.
For example, if we talk about consumers of online games, then their wishes are as follows: all players always want to win, quickly and easily achieve their goals, receive premium content for free, etc. But it’s unlikely that anyone will create such a game: nobody will just play it, because players will quickly get bored.
It is much more important to know the needs of users. The most useful information about what is important for users, what difficulties and inconveniences they encounter in the process of interaction with the product, etc.
For example, if you want to diversify the content, then asking users what they want is ineffective: the user's response does not oblige him to anything, and the implementation may differ from their expectations. In such a situation it is better to study what the users are unhappy with, what difficulties they face.
These fairly well-known facts about conducting online surveys must be kept in mind in order to significantly improve the quality of the data and benefit your business. Study the opinions of users, compare data obtained from different sources, check all the information received.
Let's research!
Today, analysts have access to a wide range of tools that allow them to study user behavior. However, you always want to know what the user thinks about, what he wants, what attracts him in the product.
And here, perhaps, a survey can help. At first glance, everything is simple: all you need is to create a questionnaire and find users who will answer the questions. It is because of the seeming simplicity that it is so easy to lose your head and make wrong decisions based on incorrect data. In this article, I want to give you 8 facts about online surveys that every researcher who has decided to collect data online should remember.
Fact 1: You are not your user.
No one knows the product better than the developer or seller. We remember all the details, possible scenarios of interaction, advantages and disadvantages. Therefore, user reviews are sometimes surprising. What is the matter? Why is it something they do not understand? It's simple: you are not your user. An outside person knows much less about the product, and he relies only on his own experience and emotions. Everything that seems obvious or convenient to you may be unknown or incomprehensible to the user.
A vivid example of how the new user and the developer interact with the product:
Conduct a preliminary study before making a survey questionnaire. Examine user feedback, discussions on forums or social networks, internal statistics and market information. Chat with community managers, support, sales, etc. What you learn will be the source of hypotheses that will be tested by a survey.
Even after careful study of all available information, you should not be completely sure that everything is taken into account. Therefore, before you run a survey, it is worth testing it on real users. This will allow the respondent to see the reaction to the questions, make sure that he understands everything and add relevant response options.
Fact 2: One study - one goal.
If you do not conduct surveys every day, then you probably want to know as much as possible with the help of a single study. And, most likely, you will get either a very long questionnaire that the user will not fill out, or the questions will be very superficial and will not help to reveal the essence of the problem. In order not to get carried away with the compilation of various questions, you should first determine the goals and objectives of your research and concentrate on them.
')
Fact 3: Opinion and facts are not the same thing.
User responses are their opinion. It is subjective and relevant only at a certain moment. Opinion is influenced by personal experience, mood, recent bright events, as well as the formulation of the question. The user may not know something or forget something. In addition, he will tell you only what he wants to tell. Therefore, remember that the results of surveys cannot be viewed as exact figures - there is always an error in this kind of data.
You can find out from the user his attitude to something, find out the level of awareness of something, assess the perception and satisfaction with the product. What you can find out using internal statistics should be taken there.
If any important for you indicators of user behavior are not recorded in the databases, then, asking about this, you should limit yourself to any particular time from the recent past. For example: “How often have you visited megamozg.ru in the last month?” The more time has passed since the event, the less accurate and honest answer you will get.
Fact 4: the user has personal files.
Your invitation to take part in the survey most often finds the user at a time when he is busy with personal affairs. Therefore, not all will answer. If the user agrees to take part in the survey, he will devote a certain amount of time to this, but no more than he wants. Therefore, your task is to make the questionnaire clear, simple and relevant for the user, so that he does not want to close the questionnaire before reaching the “ready” button.
In an online survey situation, a user cannot clarify with you the meaning of a particular question. The questionnaire must be carefully thought out, contain clear questions and relevant response options. An important role is played by the length of the questionnaire. In our experience, users easily fill out a questionnaire of 20–30 questions, with about 85–90% of respondents reaching the end of the questionnaire.
Fact 5: The way to get the data matters.
Correctly conducted survey allows you to distribute data to the entire target audience and understand the scale of the problem. It is not enough to simply collect a large number of answers. It is important to determine the characteristics of the user whose opinion is required to find out, and to choose the optimal method of research.
For example, if you need to get information about the experience of interacting with a new product, then the survey should be conducted among those who used it. If it is required to study not only the first impression, but also to identify the main drawbacks of the product, then when selecting respondents we add indicators of the duration or intensity of use of the product.
Fact 6: The most interested users respond.
The more and longer the user interacts with the product, the more he is interested in helping you improve it. Involved users are happy to respond to polls, because they want their opinion to be taken into account. However, among your audience there are those who use the product rarely or have become your client only recently. Most often, they are not so interested in expressing themselves, but in most cases they are also worth listening to. In order to learn the opinions of the entire audience, it is important to balance the answers of experienced users and beginners. As a result, the sample should be similar to your entire audience in terms of key parameters.
Fact 7: User opinions may vary.
The attitude of the user to your product may depend not only on his gender, age and education, but on the duration and experience of using the product, as well as on the degree of involvement / loyalty of the user. Analysis of user responses in conjunction with data on its behavior with respect to the product, will give you a more complete understanding of the expectations and needs of various user groups. And this, in turn, will allow to find the most relevant offers for each user and influence the audience more effectively.
For example, experienced users can more negatively assess new versions of a product, since they have to change their usual style of using the product / behavior. While beginners may simply not be aware of the innovation or be neutral to them, since they have little experience in using the product to evaluate it.
To combine survey data and statistics, you should find a way to identify each user among the respondents. Thus, you will be able to not only see the differences between users, but also be able to verify that the “correct” respondents answered you.
Fact 8: Polls do not help make the product perfect.
There is always a temptation to directly ask the user what he wants. At the same time, be sure that you will get an answer to any question. However, this information will not always benefit you.
For example, if we talk about consumers of online games, then their wishes are as follows: all players always want to win, quickly and easily achieve their goals, receive premium content for free, etc. But it’s unlikely that anyone will create such a game: nobody will just play it, because players will quickly get bored.
It is much more important to know the needs of users. The most useful information about what is important for users, what difficulties and inconveniences they encounter in the process of interaction with the product, etc.
For example, if you want to diversify the content, then asking users what they want is ineffective: the user's response does not oblige him to anything, and the implementation may differ from their expectations. In such a situation it is better to study what the users are unhappy with, what difficulties they face.
These fairly well-known facts about conducting online surveys must be kept in mind in order to significantly improve the quality of the data and benefit your business. Study the opinions of users, compare data obtained from different sources, check all the information received.
Let's research!
Source: https://habr.com/ru/post/293400/
All Articles