How game statistics differ from these playtests
Now, working in a research company, I sometimes meet developers who confidently declare that they do not need any research from the audience, since they have statistics. They have data from hundreds of thousands or even millions of players behind their backs and this gives them confidence. But is everything so clear? It is quite ambiguous, because the statistics show what the players are doing, but does not answer the “why” questions.
In 2013, on my most ambitious project, World of Battles (MMO RTS), we received statistics on each significant player action (and there were hundreds of thousands of them), application and server behavior. A part of the statistics was available “live”, and a part appeared after the fact the next day in the process of automatic data processing on the server.
The project was large and I, like other responsible employees, collected and analyzed this data. These statistics showed us many of the strengths and weaknesses of the game. In the course of receiving answers based on statistics, we have many new questions. In this article I will write about what I lacked to analyze the behavior of game users while working on World of Battles.
')
Game statistics will appear immediately after you have a working client and, if necessary, a server. At first, these will be just logs for debugging, but later you will need to think through and set up a statistics system that will allow you to keep up with your application. After all, the program itself will not tell you what's wrong with it, nor its users.
This picture shows what is known about the work of the game before it began to study and analyze data from an array of users.
Picture of community activity - generalized data on the use of chat rooms, forums, a slice of posts and activity in social networks, information about broadcasts and resources of fans
User metrics - a strategic view on user activity. These are DAU, MAU, ARP (P) U, CCU, Retention Rate, Average Session time and many other indicators.
What can the processed data do?
This picture shows what you will know about the game after reviewing and analyzing data.
For example, while working on World of Battles, thanks to statistics, we learned that certain things are not bought by players, but we did not know the exact reasons - either this is the price, or this is the strength of things, or their appearance. And without playtests, we would have wondered about the reasons for the low purchase of certain things.
We find representatives of the target audience, and let your game play. The whole process is recorded on video, as well as collecting feedback from players on the go and at the end of testing.
What are the main tools of playtests?
Video Recording - this tool allows you to visually see the player's activity in your game. Video of the first game session is very important, it will show how beginners behave - the majority of your users.
On the World of Battles project, a video game showed us that our first interface could only be understood by developers and hardcore players from the 90s and the fact that some important screens players do not open at all.
Questionnaires and interviews - these tools allow you to collect quantitative results and answers to the most concerns of developers, such as estimates of complexity, graphics, music, balance, etc.
They also give answers to the questions “Why?”. For example, why players do not pay? Why stop playing? Will they advise the game to their friends and what will they say about it?
After interviewing the World of Battles respondents, we learned that the speed of battles in the game is slow, and the speed of movement of units is no good at all. Players had to wait for one unit to kill in battle for another, and especially as they move around the map. We got used to the game, did not notice it at all, and no statistics told us about this problem.
Focus group - this tool allows you to deeply and in detail work out any issues of interest to the developer, ranging from the relationship to existing mechanics and ending with plans for future updates. With good organization of the process from the respondents can even get subconscious information.
During the discussion of the World of Battles bots on the focus group, we learned why the players choose one or another elite squad, what they see as giants, as well as which cards and why they seem more interesting than others.
There is another very important advantage of statistics against statistics, and it lies in the fact that the statistics of the gameplay is available only from its projects, or publisher's projects. A playtest can be carried out on any existing product and as a result one can obviously find out which of the mechanics and elements of a successful game are more popular and which are less, whether the balance is good and so on.
The bottom line is that statistics allow you to see problems, and playtests understand their causes and correct them later. Those. only using a combination of statistics and playtests can we see a complete and clear picture of what is happening.
In 2013, on my most ambitious project, World of Battles (MMO RTS), we received statistics on each significant player action (and there were hundreds of thousands of them), application and server behavior. A part of the statistics was available “live”, and a part appeared after the fact the next day in the process of automatic data processing on the server.
The project was large and I, like other responsible employees, collected and analyzed this data. These statistics showed us many of the strengths and weaknesses of the game. In the course of receiving answers based on statistics, we have many new questions. In this article I will write about what I lacked to analyze the behavior of game users while working on World of Battles.
')
When does statistics appear?
Game statistics will appear immediately after you have a working client and, if necessary, a server. At first, these will be just logs for debugging, but later you will need to think through and set up a statistics system that will allow you to keep up with your application. After all, the program itself will not tell you what's wrong with it, nor its users.
This picture shows what is known about the work of the game before it began to study and analyze data from an array of users.
We start collecting game statistics, what is it like?
Performance statistics
- Server statistics — server activity data, such as system status or system load
- Customer statistics - customer performance data, for example, performance, application crashes
Community statistics
Picture of community activity - generalized data on the use of chat rooms, forums, a slice of posts and activity in social networks, information about broadcasts and resources of fans
User statistics
User metrics - a strategic view on user activity. These are DAU, MAU, ARP (P) U, CCU, Retention Rate, Average Session time and many other indicators.
Gaming process statistics
- Player behavior - data on the number of games played (or levels), visits to various screens, levels reached, and so on.
- Economy - data on the in-game economy, both on the Soft economy and Hard- currency
- Balance - the balance of the game has a lot of facets and statistics can cover only some of them, for example, the balance of success in games, the balance of battle, financial balance, the balance of the speed of progress
What can the processed data do?
- Learn the basic metrics of product activity
- Learn about his weakest points.
- Build optimization hypotheses based on the data
This picture shows what you will know about the game after reviewing and analyzing data.
For example, while working on World of Battles, thanks to statistics, we learned that certain things are not bought by players, but we did not know the exact reasons - either this is the price, or this is the strength of things, or their appearance. And without playtests, we would have wondered about the reasons for the low purchase of certain things.
Now try to look at the product through the eyes of users, how to do it?
We find representatives of the target audience, and let your game play. The whole process is recorded on video, as well as collecting feedback from players on the go and at the end of testing.
What are the main tools of playtests?
Video Recording - this tool allows you to visually see the player's activity in your game. Video of the first game session is very important, it will show how beginners behave - the majority of your users.
On the World of Battles project, a video game showed us that our first interface could only be understood by developers and hardcore players from the 90s and the fact that some important screens players do not open at all.
Questionnaires and interviews - these tools allow you to collect quantitative results and answers to the most concerns of developers, such as estimates of complexity, graphics, music, balance, etc.
They also give answers to the questions “Why?”. For example, why players do not pay? Why stop playing? Will they advise the game to their friends and what will they say about it?
After interviewing the World of Battles respondents, we learned that the speed of battles in the game is slow, and the speed of movement of units is no good at all. Players had to wait for one unit to kill in battle for another, and especially as they move around the map. We got used to the game, did not notice it at all, and no statistics told us about this problem.
Focus group - this tool allows you to deeply and in detail work out any issues of interest to the developer, ranging from the relationship to existing mechanics and ending with plans for future updates. With good organization of the process from the respondents can even get subconscious information.
During the discussion of the World of Battles bots on the focus group, we learned why the players choose one or another elite squad, what they see as giants, as well as which cards and why they seem more interesting than others.
There is another very important advantage of statistics against statistics, and it lies in the fact that the statistics of the gameplay is available only from its projects, or publisher's projects. A playtest can be carried out on any existing product and as a result one can obviously find out which of the mechanics and elements of a successful game are more popular and which are less, whether the balance is good and so on.
Total
The bottom line is that statistics allow you to see problems, and playtests understand their causes and correct them later. Those. only using a combination of statistics and playtests can we see a complete and clear picture of what is happening.
Source: https://habr.com/ru/post/302802/
All Articles