Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles

Please download to get full document.

View again

of 10
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Internet & Technology

Published:

Views: 0 | Pages: 10

Extension: PDF | Download: 0

Share
Description
2016 IEEE International Conference on Data Science and Advanced Analytics Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles África Periáñez, Alain Saas, Anna
Transcript
2016 IEEE International Conference on Data Science and Advanced Analytics Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles África Periáñez, Alain Saas, Anna Guitart and Colin Magne Game Data Science Department Silicon Studio Ebisu Shibuya-ku, Tokyo, Japan {africa.perianez, alain.saas, anna.guitart, Abstract Reducing user attrition, i.e. churn, is a broad challenge faced by several industries. In mobile social games, decreasing churn is decisive to increase player retention and rise revenues. Churn prediction models allow to understand player loyalty and to anticipate when they will stop playing a game. Thanks to these predictions, several initiatives can be taken to retain those players who are more likely to churn. Survival analysis focuses on predicting the time of occurrence of a certain event, churn in our case. Classical methods, like regressions, could be applied only when all players have left the game. The challenge arises for datasets with incomplete churning information for all players, as most of them still connect to the game. This is called a censored data problem and is in the nature of churn. Censoring is commonly dealt with survival analysis techniques, but due to the inflexibility of the survival statistical algorithms, the accuracy achieved is often poor. In contrast, novel ensemble learning techniques, increasingly popular in a variety of scientific fields, provide high-class prediction results. In this work, we develop, for the first time in the social games domain, a survival ensemble model which provides a comprehensive analysis together with an accurate prediction of churn. For each player, we predict the probability of churning as function of time, which permits to distinguish various levels of loyalty profiles. Additionally, we assess the risk factors that explain the predicted player survival times. Our results show that churn prediction by survival ensembles significantly improves the accuracy and robustness of traditional analyses, like Cox regression. Index Terms social games; churn prediction; ensemble methods; survival analysis; online games; user behavior I. INTRODUCTION The economics of gaming has changed in the recent years with the widespread adoption of social networks and smartphones, leading to a new type of video games: social games. Social games target a new audience of players: casual gamers, with a new monetization model: free-to-play (F2P or freemium), which now largely dominates all the mobile platforms [2], [14]. The freemium model consists in offering a game for free, and monetizing it by charging for in-game content through in-app purchases. For social games, player retention is key for a successful monetization, and to increase the social interactions that in turn help to drive the adoption of the game and retain players. In addition, the cost of acquiring new players is ever increasing Fig. 1. Screenshots of the game chosen to evaluate the churn models, Age of Ishtaria. This game is representative of the successful F2P mobile social role-playing games in Japan. The left panel exhibits characteristic activity in role-playing games, the right panel shows the usual F2P in-app purchases for in-game content. [14] and can significantly exceed the cost of retaining existing ones. This study is motivated by the idea that the ability to predict when a player will leave a game allows to take incentive actions to re-engage her and prevent churn, or move her to another game of the company. Churn prediction has been widely researched in the fields of telecom, finance, retail, pay TV and banking, as shown by the extensive literature review given by [55], [58]. It has also been studied in e-commerce [60], [61] and even in terms of employee retention [51]. In the field of video games, pioneering studies were introduced in [29], [31]. However, they focus on MMORPG (Massively Multiplayer Online Role-Playing Games). MMORPG have been the first successful type of online social games, however they targeted a narrower audience and they are mainly using a subscription-based monetization /16 $ IEEE DOI /DSAA model. This implies the possibility to measure churn as a formal termination of contract, similarly to the sectors mentioned above, at the exception of e-commerce. Free-To-Play (F2P) monetization, which is the main model used by mobile social games, involves a non-contractual relationship. In this context, churn is not clearly determined by an explicit statement ending a contract. For the most active players, we can define churn as a prolonged period of inactivity. However, the problem slightly differs from the churn in e-commerce. It is indeed always possible for inactive users to come back to an e-commerce website, while inactive mobile players can uninstall a game, which would correspond to a well defined and definitive state of churn. However, this information is normally not available. The definition of churn in non-contractual settings has been discussed in [10]. A comprehensive discussion on the definition of churn for F2P applications is beyond the scope of this paper, and is the subject of dedicated studies [10]. The work presented in [19] is the first study investigating churn prediction in F2P games. [19] introduces a general definition of the problem, a selection of game content independent features and a comparison of classifiers. A second study shown in [48] focuses on the churn prediction of high value players in F2P games. [48] investigates in detail the problem definition and classifier evaluation, though it approaches the problem only from a binary classification point of view. It uses an algorithm that assumes a distribution of data that normally does not fit with the common shape of the churn data. Going further, [49] and [47] try to address the temporality of the data for churn prediction in mobile games. The work presented in this paper focuses on predicting churn for high value players who are commonly called whales in the video game industry. A motivation for this focus is that whales behave differently than average players, including in terms of survival curve as we can see in Fig. 2. Since they are often the most active players, i.e. they play nearly every day, we can easily define their churn as a prolonged period of inactivity. Their high level of engagement also allows to collect more data about their activity and makes them more likely to answer positively to actions taken in order to prevent their churn. Finally, from a business perspective, whales, who represent about 0.15% of the players, or 10% of the paying users [28], are particularly important since they are the top spenders who account for 50% of the in-app purchases revenues. The game chosen for this study, Age of Ishtaria developed by Silicon Studio, is representative of the successful mobile social games and has several million players worldwide. A. Our contribution Classical approaches to churn study the problem as a binary classification: whether or not the player connects again to the game (e.g. [3]). Although the binary models are very intuitive, they are not able to predict when the player will stop playing and, moreover, the features are limited to provide static (nontemporal) information. In order to model the time until churn, traditional methods like regressions would be appropriate only when all players have stopped playing the game. The challenge arises for data which contains incomplete information about every users, as some of them still play the game. The present work improves previous studies [19], [48] using an adequate technique that assimilates censored data (observations with incomplete information about churn time) [34] and that captures the temporal dimension of the churn prediction challenge. Our model based on survival ensembles outputs accurate predictions of when players churn, and provides information about the risk factors that affect the exit of players as well. Additionally, the approach suggested in this paper not only gives us a list of possible churners, but also produces, for every player, a survival probability function that will let us know how the probability of churning is varying as a function of time. This feature lets us distinguish various levels of loyalty profiles, upcoming, near-future and far-future churners, and the variables that influence this survival behavior (considering that a player is alive as long as she connects to the social game). From this survival function, the median survival time is extracted and used as a life expectancy threshold. This feature lets us label players as being at risk of churning, take action beforehand to retain valuable players, and ultimately improve game development to enhance player satisfaction. To the best of our knowledge, we are the first to thoroughly model the prediction of churn by using a survival ensemble approach in the social games sector. Our model improves the accuracy, robustness and flexibility of traditional survival methods, like Cox regression, and has been developed with the goal of being usable in an operational business environment. II. SURVIVAL ENSEMBLE MODELS A. Survival Analysis Survival analysis consists of a set of statistical techniques traditionally used to predict lifetime expectancy of individuals in medical and biological research [15], [26], [35]. This group of methods have also been applied in several industries to predict customer attrition, mainly in telecommunication [36], banking [54] and insurance [16]. Survival analysis focuses on studying the time until an event of interest happens and its relationship with different factors. Originally in medical research, an event is the failure or death of an individual, however in our case it is the moment when a player leaves the game. The time-to-event outcome is also known as survival time. A fundamental characteristic of survival analysis is that data are censored. Censoring indicates that observations do not include complete information about the occurrence of the event of interest. It means that for a certain number of players, we do not know the time of event experience (because they did not experience it yet), i.e. measurements only contain information if the event occurs or not before a given time t. 565 The survival function S(t), which is simply the likelihood that a player will survive at a certain time t, can be estimated through the non-parametric Kaplan-Meier estimator [30], where the churn probability can be computed directly from recorded censored survival times. If k players churn during the period of time T of study at different instants t 1 t 2 t 3... t k and, as churn occurrences are supposed to be independent of each other [9], the probabilities of surviving in the game from one time to the next can be multiplied to obtain the cumulative survival probability: ( S(t j )=S(t j 1 ) 1 d ) j (1) n j where S(0) = 1, with n j being the number of players alive before t j, and d j being the number of events at t j. We will get as a result a step function that changes its value at the time of each churn. Further analysis on this topic includes the presence of competing risks [43]. They belong to a special class of timeto-event models where there is more than one possible failure event. These alternative events can prevent the observation of the main event of interest. In this study, we focus on the loss of interest in a game, which is the main cause of churn. However, it can happen that a player stops playing the game because she loses her phone, or dies, which are considered as competing risks events. Additional semi-parametric survival techniques, like the renowned regression method for censored observations, the Cox proportional-hazards model [11] [13], or parametric methods (e.g. accelerated failure time models [38]), are valuable tools to investigate the impact of multiple covariates. The covariates or predictors are expected to be correlated with the player s reason for quitting the game. Following Cox proportional-hazards model, the estimated hazard for k individual players and p covariate vectors x k takes the form h k (t) =h 0 (t)exp ( β 1 x k, β p x k,p ), (2) where the hazard function h k (t) is dependent on the baseline hazard h 0 (t) and the features β p x k,p. The Cox regression is not assumed to follow a particular statistical distribution. It is fitted based on the data and it solves the censoring problem by maximizing the partial likelihood. The Cox model and its extensions [56] allow regressions to work with censored data, and they permit an intuitive interpretation of the impact of the features. However, these techniques assume a fixed link between the output and the variables (assuming them additive and constant over time). This requires an explicit specification of the relationship by the researcher, and involves important efforts in terms of model selection and evaluation. In spite of their semi-parametric nature, these models present difficulties to scale with big data problems, and alternative regularized versions of Cox regression [39] have been proposed to amend this. Nevertheless, they are still based on restrictive assumptions that are not easy to fulfill. In the parametric approaches, like the accelerated failure time models [38], the type of the distribution is previously determined (e.g. Weibull, lognormal, exponential). Though, these methods are suboptimal because it is uncommon that the data follow these specific distribution shapes. In the present paper, we address the drawbacks mentioned above by applying machine learning algorithms to censored data problems. B. Survival Trees and Ensembles 1) Decision Trees: Originally presented in [41], decision trees became popular in the 1980s, when the most relevant algorithms for Classification and Regression Trees (CART) were introduced by [7], [44], [50]. Classification and regression trees are non-parametric techniques where the basic idea is to split the feature space recursively, to group subjects with homogeneous characteristics and to separate those with bigger differences based on the outcome of concern. In order to perform the nodes classification and maximize homogeneity within the nodes, a measure called impurity must be minimized. Common examples of impurity measure are cross-entropy or sum of squared errors. For example, considering a binary split and given a continuous variable X, the split can be performed if X d is fulfilled, with d being a constant. 2) Survival trees: Survival trees are constructed as a set of binary trees that grow by recursive partitioning of the sample space χ, where the q i tree nodes are subspaces of χ. The tree splitting starts in the root node, which concentrates all the data. Based on a survival statistical criterion, such as the cumulative hazard function or Kaplan-Meier estimates, the root node is then divided into two daughter nodes. The principle for partitioning these two branches is to maximize the survival difference between two groups of individuals, which are compressed in the two daughter nodes, maximizing the homogeneity among nodes, based on survival experience. The first idea of using tree-based methods for censored data was initially introduced in [8] and [37]. The first survival tree as we know was presented in [17], where a Kaplan- Meier estimator survival function was computed at every node as a discrepancy measure using Wasserstein metrics. For a comprehensive review about different types of survival trees, check [4]. The best split is achieved by exploring all combinations, considering all the x i predictor variables and all the possible splits, in order to maximize the survival difference. This way, subjects with similar survival characteristics are grouped together. As long as the tree grows, the difference between branches increases, and individuals are gathered in nodes with more homogeneous groups in terms of survival behavior. Despite being a powerful classification tool which is able to model censored data, employing a single tree can produce instability in its predictions. It means that if small changes in data arise, the prediction can differ among computations (the divergences are mainly related with the prediction of risk 566 factors) [33]. This drawback will be fixed if we execute an ensemble of them, instead of using one single tree. 3) Survival ensembles: Using an ensemble of models, instead of a single one, is an accurate prediction tool firstly suggested by [5], [6] with the well-known random forest. Ensembles of tree-based models achieve outstanding predictions in real-world applications [62]. Survival forests are ensemble-based learning methods where the underlying algorithm is a kind of survival tree. A survival ensemble lies in growing a set of survival trees, instead of a single one. The two main survival ensemble techniques are random survival forest, presented in [27], and conditional inference survival ensembles, developed by [24], based on their previous work introduced in [21], [25]. The conditional inference survival ensembles is the method chosen for the predictions shown in Section III. The conditional inference survival ensemble technique uses a weighted Kaplan-Meier function based on the measurements used for the training. The ensemble survival function [40] can be summarized by S conditional (t x i )= ( N n=1 1 T ) n(dt, x i ) N n=1 Q (3) n(t, x i ) where n indicates the number of trees within the ensembles, with n =1,,N, and x i being the covariates. Therefore, in the node where x i is located, T n accounts for the uncensored events until time t, and Q n counts the number of individuals at risk at time t. Moreover, conditional inference survival ensembles introduces additional weight to the nodes where there are more subjects at risk. It uses linear rank statistics as splitting criterion to grow the trees. In contrast, random survival forests [27] are based on Nelson-Aalen estimates (instead of using Kaplan-Meier estimates). The maximum of the log-rank statistical test is used in every node as split criterion, which leads to biased results in favor of covariates with many splits. Conditional inference survival ensembles is a promising approach to deal with the censoring nature of churn prediction. It is a flexible method compared to the traditional statistical Cox regression model and it solves the instability that is present in survival trees. In the selected method for the churn study, overfit is not present in its estimates and provides robust information about the variable importance. This fixes the random survival forest problem [59] of being biased towards predictors with many splits or missing data. III. DATASET We collected data from a major mobile social game between October 2014 and February Several churn predictors or risk factors were investigated. We investigated mainly game-independent features, i.e. features that are not related to the game mechanics and can be measured in any game. This allows us to build a gameindependent churn prediction model that can be applied to other games. Additionally, we want to implement our model in a data science product running in an operational business environment. Thus, the feature selection takes into account limitations in terms of memory and processing capabilities that might not be considered in a pure research environment. Player attention: the time component of the player accessing the game. Time spent per day in the game, including averages over the first weeks and moving average over the last weeks. Lifetime: number of days since registration until churn, in case the player churns. Player loyalty: the frequency of the player access to the game. Number of days with at least one playing session. Loyalty index: ratio of number of days played, divided by lifetime. Days from registration to first purchase. Days since last purchase. Playing intensity: the q
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x