Simpson's Paradox in Mobile Analytics

Analytics in mobile marketing is not as simple as it may seem at first glance. Some metrics do not always reflect the real state of things, so they should be treated very carefully. Today we will talk about Simpson's paradox or union paradox. This statistical phenomenon describes the conditions when there may be a general trend in several groups of data, but when considering them individually, the situation changes - the trends become opposite.

Consider this on a hypothetical example close to mobile marketing. Suppose there is a group of users, of which 5,000 are iOS users, and 10,000 are Android. The average conversion is 5%: 4% for iOS and 5.5% for Android. Agree that a product manager can make very specific decisions based on this data ... and make a mistake. Consider the data in more detail - broken down by device. Of the five thousand Apple users, 1,500 iPads (6.67% conversion) and 3,500 iPhones (2.86%) are in the group, and 8,000 tablets (6.25%) and 2,000 smartphones (2.5%) are in the Google platform group. That is, it turns out that the conversion in both groups of Apple devices is higher if we consider them separately.


')
Mixing heterogeneous groups of data into a single array (in this example, tablets and smartphones) is a gross mistake. Such oversights are often found in the analysis of freemium applications. For example, when they are trying to derive a conversion rate for all regions.

To avoid such mistakes, do not combine disparate groups. Below is a list of criteria that should be used in mobile marketing to divide data into groups:
- regions or countries;
- types and device platforms;
- sources of information;
- behavioral signals;
- installation date (in case of studying the seasonality of demand).



Source: mobiledevmemo.com

mobiledevmemo.com/avoiding-simpsons-paradox-data-analysis