We collect and analyze statistics in mobile applications

In our blog, we have already published a number of articles on statistics in games, in which we talked mainly about key indicators and metrics and what tools are available on the market for analysts.

And in today's article we want to share the experience of building directly technical tools for collecting and processing statistical information for mobile games.



There are ready-made tools on the market, such as, for example, Flurry or Kontagent . For them, there are both ready-made SDKs for the main platforms, as well as some tools for working with data in the form of reports / graphs / funnels, etc. However, these tools also have a number of drawbacks.



For example, Flurry has significant delays in information processing and, worse, it has been experimentally established that there are serious data losses (according to our estimates, Flurry “loses” from 30 to 60% of the data). Well, a serious disadvantage of Kontagent is the cost of a license to use it, since it may not suit everyone. Also, almost all ready-made solutions do not provide the necessary flexibility and transparency when working with reports.

Flurry + Splunk

After a series of experiments, trials, errors, we decided to build our own statistical processing system based on Splunk , a fairly powerful tool for processing various kinds of information. By itself, Splunk provides great opportunities to build all sorts of graphs, reports, dashboards, etc. etc. that we are more than satisfied.

')

After choosing a tool for processing statistics, the logical way was the question of how, in fact, we will receive data from the game. The game, which we were interested in, was already published and the Flurry module was already integrated in it (and at that time we didn’t know about Flurry data loss), so we decided to download data from Flurry using their Raw Data API .



This is the Raw Data API, which turned out to be a very ambiguous tool worthy of a separate post. But “regardless of the pain”, we nevertheless set up getting statistics from Flurry and uploading it to Splunk. After that, the system was put into operation: the data is indexed, analysts build reports, everyone is happy, the hall applauds, flowers, music, champagne.



Happiness did not last long. Literally up to several reconciliations of the numbers, which we could verify at least with something else, obviously reliable, in addition to the available statistics - data related to the in-game payments that we had in Splunk and which the mobile phone shows us. Nobody doubted that the data would not be the same (there are always factors of delay in data receipt, possible errors in the generated reports, potential “background” in statistics from cheaters, etc.), but no one expected the difference to be just like this: the data in the statistics was displayed almost two times less than in the store (both in the amount and in the number of payments). Detailed analysis showed that there are no errors in the reports and samples, the data in Splunk are interpreted correctly, the game application also sends statistics to the Flurry SDK regularly and where it is needed, but the raw data from Flurry is “somewhat” less than it actually is. As mentioned above, our estimate of data loss is from 30 to 60 percent.

Kontagent

After some time, we also had the opportunity to compare the live work of our Flurry + Splunk bundle with the Kontagent system. Kontagent provides a much wider range of options, it is stated that it works in almost real-time mode, has integration with a bunch of marketing tools and, in general, is all that kind of thing.

In fact, everything is approximately as it is, however, I repeat, the cost of a license for mobile platforms is unlikely to please small teams: the cost directly depends on the size of your audience and the conversation begins with tens of thousands of dollars a year. A comparative analysis confirmed our hypothesis of data loss on the Flurry side. Apparently, including the completeness of the data Kontagent and asks for its price tag.



So, we are faced with the next dilemma - move to Kontagent or stay with Flurry? A full transition to Kontagent was complicated by a number of points, primarily related to the need for rework in the game, which we really wanted to avoid. At the same time, to remain with Flurry meant to get a deliberately incomplete picture, on the basis of which important decisions are made.

Final decision

As a result, “Solomon's decision” was made - to replace Flurry with our own data transport from the game in Splunk with maximum preservation of the data format and minimal changes in the game code.

From the system required:

• maximum reliability
• maximum light weight / high load resistance
• the minimum delay between receiving data from the client and hitting them in Splunk indexes

To meet these requirements, the following system operation was proposed:

• transfer all parameters to the server in the GET request line
• In this version, the access.log of the web server becomes the easiest, most reliable and cheap from the point of view of server load; the data storage becomes the web server; we use nginx
• To ensure minimal security, the md5 signature of the request is used, which is implemented using the HTTPSecureLink module for nginx
• to ensure scalability, a balancer is placed on the frontend, which can scatter requests across a cluster of logger nodes with a “revolver”
• to minimize delays, access.log is rotated once an hour and the freshest rotated log is sent to Splunk, ensuring hourly data lags in the indexes
• Splunk Universal Forwarder is installed on each Splunk head node for transmitting data to the Splunk instance, which is a native Splunk transport that only transmits data; gives a minimum additional load on the node
• to ensure the consistency of data from one user, distributed across a cluster of node loggers, each request contains a unique user ID and timestamp taken over the time of the user device

As a result, we got a stable, lightweight, scalable system for collecting and transporting Splunk statistical data. In parallel, client SDKs for iOS and Android were implemented, which, for ease of integration, duplicated the API provided by the Flurry SDK. Those. transition to the new SDK took half an hour of programmer time, including tests.



A comparative analysis showed that the amount of data obtained in this way is identical to what Kontagent collects, and, if we are talking about in-game payments separately, the information in mobile sites is also relevant. The cost of developing, supporting, and migrating from Flurry was several times, if not much less, compared to switching to Kontagent.