How to become a product manager. Part 4 about Data Science and ASO
In the middle of November, our friends from Sports.ru launched a course for those who want to become a product manager of mobile applications. Among the lecturers are employees of Sports.ru, AppFollow, Aviasales, Uber and other cool guys. A student at kirillkobelev teaches us how the training went. Today there are notes on what aspects of analytics the novice product manager should first pay attention to (based on the lecture by Oleg Novikov from Uber ) and how to promote your application in stores without a budget (according to Sergey Sharov from ASO Desk ).
Earlier in the series:
→ Part 1 - who are product managers and a little about design.
→ Part 2 - about the stages of application development.
→ Part 3 - about monetization and management of the team.
')
If you strive to use analytics tools in your products as efficiently as possible, start with the correct formulation of the problem. It is not enough to formally choose two or three powerful tools - you will have to understand why you are going to measure something at all. It's amazing how few people do it.
So why measure anything at all? Variants of answers:
Obviously, in life, each of us uses his combination from the list above, but if you understand to which category of measurements the metric you need, your life will become much easier.
Suddenly, but the fact is: the farther away from the real money is the metric, the less it should worry you. The first step is to measure directly where and how much you earn, as well as where and how you manage to spend it all. Then look at the metrics that indirectly affect the money. Finally, thirdly ... well, it is worth stopping in time.
The key monetary metric is rightly considered Lifetime value (LTV) , that is, how much money the user will bring for the entire time of interaction with the product. This indicator is considered on a breathtakingly complex scheme: LTV = lifetime * ARPU.
LTV describes how much the user likes to work with your product (and how long they are likely to use it), and most importantly, how much money he is willing to spend on you. This indicator is good because a novice product manager can independently build a minimum strategy without spending money on an MBA: either you focus on increasing the lifespan of the product, or work to increase the income from each user. Ideally, you need to do both directions, but we will not in vain reassure ourselves :)
The most important non-monetary indicator can be considered the Retention rate , that is, the proportion of users returning to you. And remember that attracting new users is always more expensive than retaining those who have already come to you.
In addition to the difficult choice, to go from the first prize to the Maldives or the Seychelles, product managers have to think, for example, about the ratio of profit and investment. LTV allows you to correctly determine the amount of investment in attracting new users. Once you have calculated the costs, start experimenting with different acquisition tools.
Any experiment and any testing must begin with a hypothesis. The hypothesis focuses on one particular parameter or one measurable effect. Another important element of the experiment is sufficiency, that is, what minimal effect we will consider sufficient to continue the work. Do you know what the catch is? The exact answer to this question is only with you and in your specific conditions, there is no universal indicator.
The level of statistical significance indicates how many participants need an experiment to be considered valid. Knowing the sample size and audience of your application, you can calculate the duration of the experiment.
If for the selected parameters you have to turn the test for six months to get the necessary sample, most likely, you go too deep and the time for testing has not yet come.
To conduct effective testing, it is necessary to assign users to the groups, rather than their sessions in the application. Need I say that the distribution should be random? Accurate distribution can be quickly and inexpensively done using Google Analytics or Optimizely .
If you have a user base in your hands and no paid tools, hash the required number of records from the base and select some logical distribution method (for example, all even / odd). And of course, do not forget about the control group in order tosee how insignificant the attempts to optimize something, to watch the testing progress.
There are many pitfalls in testing, but this should not diminish its importance. Just approach the organization very carefully and imagine well what result you expect. Check the quality of the sample and the formulation of the problem to the test using the AA test, when the two groups are shown the same options, and generally their results should not be different;)
If for the period of testing there are important events like elections or “Black Friday”, the results can be greatly distorted.
But let's say everything went well and one of the options is really better. Does this mean that you can stop testing and finally return to the Maldives? I think the answer is obvious. A successful test gives a starting point, allows you to delve into the details and see if everyone has become better. If the overall positive result is achieved due to the fact that someone has become much worse, it may be worth refusing the test results.
I said above that knowing how much money your user brings to you on average, you can determine the level of investment in attracting new people. Alas, the price of a paid user is constantly growing, so it is important to pay attention to shareware tools.
Organic search is almost always the most effective, because it responds directly to a query. Both on the web and mobile sites, search traffic is well converted, and users who come from it remain loyal to your product for a long time. This thesis will be demolished in the next lecture, where it will be a question of fichering, but for now we will remain with ours.
Search engine optimization in app stores solves 3 key business problems:
Since the product manager will never get anything for free, you will have to pay for the shareware seo with sweat and blood.
It is important to remember that search results in the App Store are updated several times a day; Google Play is updated once a day, but more significantly. Entering new data into the account always happens through the new version. Apple indexes it immediately, and Google for several days.
When compiling a semantic core there are three main sources:
Accurate data on the frequency of requests in the search stores is almost there. Accordingly, since the number of terms is limited, it is important to accurately know the traffic for each of them. Life hacking : there are related countries and localizations, and this list can be used to fill less important kernel pieces into countries that are not essential to your localization, but still indexing the product.
Text optimization takes about 70% of the work, so it is important to use the right toolkit: ASODesk offers an automated approach when you fill out the list of competitors with your hands, and then the program goes through the stor and selects keywords from the database. Attention gift: enter PRODUCTDEGREE on the ASODesk website and get a 5% discount.
Formally, schoolchildren, bringing the application to the top, do not break anything. They go through the search term, pick up the desired application and so raise it in the issue. Yet the platforms put a lot of effort to separate the wheat from the chaff and living schoolchildren from the bots.
Interestingly, the Stores choose different tactics of protection: Apple warns first, and Google bans left and right. On the other hand, Google allows you to respond to reviews and encourages for this rating. Karma accumulates through all versions of the application and after some time begins to work on the product.
From the compiled semantic core, you will have to choose such a set of keywords so that they, including the service words, cover the maximum of search queries. Changes need to track about 4 weeks. Then compare the lowest position two weeks before the release and within the next two weeks after the release. Based on this information, you can continue to saw the description and prepare the next release. And next time we will talk about business models for applications and about finding the best experts in the product team or how to live in a world without LinkedIn.
Earlier in the series:
→ Part 1 - who are product managers and a little about design.
→ Part 2 - about the stages of application development.
→ Part 3 - about monetization and management of the team.
')
Lecture Five: Analytics and Data Arrays
If you strive to use analytics tools in your products as efficiently as possible, start with the correct formulation of the problem. It is not enough to formally choose two or three powerful tools - you will have to understand why you are going to measure something at all. It's amazing how few people do it.
So why measure anything at all? Variants of answers:
- Assessment of the current state of key indicators (KPI);
- Business status slice (dashboard);
- Detailed study of a given set of metrics (ad hoc studies);
- Comparison of different versions of the project (experiments and testing).
Obviously, in life, each of us uses his combination from the list above, but if you understand to which category of measurements the metric you need, your life will become much easier.
And again about the money
Suddenly, but the fact is: the farther away from the real money is the metric, the less it should worry you. The first step is to measure directly where and how much you earn, as well as where and how you manage to spend it all. Then look at the metrics that indirectly affect the money. Finally, thirdly ... well, it is worth stopping in time.
The key monetary metric is rightly considered Lifetime value (LTV) , that is, how much money the user will bring for the entire time of interaction with the product. This indicator is considered on a breathtakingly complex scheme: LTV = lifetime * ARPU.
LTV describes how much the user likes to work with your product (and how long they are likely to use it), and most importantly, how much money he is willing to spend on you. This indicator is good because a novice product manager can independently build a minimum strategy without spending money on an MBA: either you focus on increasing the lifespan of the product, or work to increase the income from each user. Ideally, you need to do both directions, but we will not in vain reassure ourselves :)
The most important non-monetary indicator can be considered the Retention rate , that is, the proportion of users returning to you. And remember that attracting new users is always more expensive than retaining those who have already come to you.
Experiments
In addition to the difficult choice, to go from the first prize to the Maldives or the Seychelles, product managers have to think, for example, about the ratio of profit and investment. LTV allows you to correctly determine the amount of investment in attracting new users. Once you have calculated the costs, start experimenting with different acquisition tools.
Any experiment and any testing must begin with a hypothesis. The hypothesis focuses on one particular parameter or one measurable effect. Another important element of the experiment is sufficiency, that is, what minimal effect we will consider sufficient to continue the work. Do you know what the catch is? The exact answer to this question is only with you and in your specific conditions, there is no universal indicator.
Young experimenter's checklist
- The parameter you are going to change;
- Hypothesis (if ..., then ...);
- Minimum sufficient effect;
- The level of statistical significance and the minimum sample size;
- Indicator of the probability that the difference is random.
The level of statistical significance indicates how many participants need an experiment to be considered valid. Knowing the sample size and audience of your application, you can calculate the duration of the experiment.
If for the selected parameters you have to turn the test for six months to get the necessary sample, most likely, you go too deep and the time for testing has not yet come.
To conduct effective testing, it is necessary to assign users to the groups, rather than their sessions in the application. Need I say that the distribution should be random? Accurate distribution can be quickly and inexpensively done using Google Analytics or Optimizely .
If you have a user base in your hands and no paid tools, hash the required number of records from the base and select some logical distribution method (for example, all even / odd). And of course, do not forget about the control group in order to
There are many pitfalls in testing, but this should not diminish its importance. Just approach the organization very carefully and imagine well what result you expect. Check the quality of the sample and the formulation of the problem to the test using the AA test, when the two groups are shown the same options, and generally their results should not be different;)
How much can you put up with it?
- One day to get a first impression;
- A certain length of the experiment in order to obtain statistical significance (this is the most cumbersome synonym for the word “week” that I have seen);
- One business cycle (perhaps all the fun will happen at the weekend);
- And a bit more :).
If for the period of testing there are important events like elections or “Black Friday”, the results can be greatly distorted.
But let's say everything went well and one of the options is really better. Does this mean that you can stop testing and finally return to the Maldives? I think the answer is obvious. A successful test gives a starting point, allows you to delve into the details and see if everyone has become better. If the overall positive result is achieved due to the fact that someone has become much worse, it may be worth refusing the test results.
Lecture Six: Search Engine Optimization
I said above that knowing how much money your user brings to you on average, you can determine the level of investment in attracting new people. Alas, the price of a paid user is constantly growing, so it is important to pay attention to shareware tools.
Organic search is almost always the most effective, because it responds directly to a query. Both on the web and mobile sites, search traffic is well converted, and users who come from it remain loyal to your product for a long time. This thesis will be demolished in the next lecture, where it will be a question of fichering, but for now we will remain with ours.
Do I need it?
Search engine optimization in app stores solves 3 key business problems:
- Increasing the number of installations by attracting better traffic,
- Reducing the cost of attraction,
- Increase Retention rate.
Circles of hell
Since the product manager will never get anything for free, you will have to pay for the shareware seo with sweat and blood.
- 70% of labor costs will be in text optimization, that is, the compilation of the semantic core;
- Visual search engine optimization involves working with screenshots and videos that illustrate your product in the app storefront;
- AB testing in stor allows you to consistently improve selected texts / images / creatives.
It is important to remember that search results in the App Store are updated several times a day; Google Play is updated once a day, but more significantly. Entering new data into the account always happens through the new version. Apple indexes it immediately, and Google for several days.
In the beginning was the word
When compiling a semantic core there are three main sources:
- common sense;
- a search suggestion in the store (each term from the first paragraph will receive several associations);
- competitor analysis.
Accurate data on the frequency of requests in the search stores is almost there. Accordingly, since the number of terms is limited, it is important to accurately know the traffic for each of them. Life hacking : there are related countries and localizations, and this list can be used to fill less important kernel pieces into countries that are not essential to your localization, but still indexing the product.
Text optimization takes about 70% of the work, so it is important to use the right toolkit: ASODesk offers an automated approach when you fill out the list of competitors with your hands, and then the program goes through the stor and selects keywords from the database. Attention gift: enter PRODUCTDEGREE on the ASODesk website and get a 5% discount.
3 relatively interesting facts about ASO
- It is necessary to start with the most relevant low-frequency to increase the sustainable organic base. As the downloads increase, the product will go to the top and be able to actively fight in a more competitive environment.
- Black search gives a good impulse of downloads, and then it is already supported by organics. I did not tell you this, but almost all the search leaders at one time or another used this tool.
- Search trends are a local tool, it is tied to a country and hitting it gives an increase in traffic, but for this you need to be in the list of recommended applications and in the top by category.
Prohibited Techniques
Formally, schoolchildren, bringing the application to the top, do not break anything. They go through the search term, pick up the desired application and so raise it in the issue. Yet the platforms put a lot of effort to separate the wheat from the chaff and living schoolchildren from the bots.
Interestingly, the Stores choose different tactics of protection: Apple warns first, and Google bans left and right. On the other hand, Google allows you to respond to reviews and encourages for this rating. Karma accumulates through all versions of the application and after some time begins to work on the product.
To drive or not to drive?
From the compiled semantic core, you will have to choose such a set of keywords so that they, including the service words, cover the maximum of search queries. Changes need to track about 4 weeks. Then compare the lowest position two weeks before the release and within the next two weeks after the release. Based on this information, you can continue to saw the description and prepare the next release. And next time we will talk about business models for applications and about finding the best experts in the product team or how to live in a world without LinkedIn.
Source: https://habr.com/ru/post/319408/
All Articles