Introduction

Modern Telco companies gather lots of data on their customers. Varying from their gender to their monthly payments to the type of contract they have. One aspect that is especially important to keep track of is the group of customers that ends or will end their contract soon. Cybiant can help Telco companies to obtain valuable insights on how to identify customers that are likely to end their contract and to find which factors are indicating such behaviour.  There are two ways this dataset can be used to study contract ends. The first is determining which factors are typical of customers who end their contract, the second is identifying which customers are likely to end their contract. The focus of this article is on the former.  Therefore we built a showcase model with the powerful combination of Power BI and R.

We use a sample dataset of the customer-base of a telco company for our analysis. The variable of interest is ‘churn’, which has two values: ”yes” if a customer ended their contract in the last month, and “no” otherwise. There are various different characteristics of customers in the dataset which can be divided in three main categories: services that each customer signed up for, customer account information and demographic information about each customer.

Figure 1: Summarizing Customer Data in Power BI

After loading and cleaning the data in Power BI. We add a measure which is called the ‘churn-rate’, which is the percentage of the customers that ended their contract in the last month. The data can be visualized in various ways for different purposes. Some example of insightful visualizations are shown in figure 1.  We can observe the churn-rate of all customers in the top left. At the bottom is shown that the churn rate Fiber optic users is a lot higher that of DSL users or users which have no internet connection at all. This last group could have a lower churn rate, since it is more difficult to end you contract without an internet connection. The chart to the right demonstrates that the customers using Fiber-optic account for the largest part of total monthly income for the Telco company. This means that the largest group in income is also the group which has the highest churn rate. This seems to be a problem the company cannot neglect. At the pie chart becomes clear that the Fiber Optic customers are only 44% of the total number of customers, but they account 63% of monthly charges. It must be that they pay more monthly charges than an average customer.

Attribute Selection: which variables indicate customer churn?

The next step is to determine which attributes are the most important in predicting which customers are going to end their contract. Nowadays businesses have access to such large amounts of data and so many different attributes of information, that it is difficult to determine where to start. Figure 2 shows the importance of each attribute for predicting whether a customer will churn or not. The type of contract is the most important attribute as shown in the figure above. This figure provides a direction to start looking for more information. It makes clear that the contract attribute has the highest predictive value for churn.  Not too much time should be spent on analysing the differences between genders since that attribute has no significant predictive value for churn. Figure 2 is just an example and can be easily extended to datasets with many more attributes.

The used measure is called information gain(IG). We can split the data in different sets according to the categories of an attribute. The whole dataset has a certain purity. If all cases in the set have the same value for churn, the set is perfectly pure. In real-life big data perfect purity is rarely possible. , therefore we use a measure for the gain in purity when segmenting, which is IG. The set of customers can be divided into subsets according to the different values of an attribute. IG measures how much in purity is gained by segmenting on that particular attribute. A more in-depth explanation and the mathematical formulas can be found in the full article.

Back to the Telco data. Figure 2 shows that the type of contract is the most important attribute. The bar chart in figure 1 shows us that people who have a more flexible month-to-month contract, are more likely to churn. The line in the bar chart informs us that the group of people with a flexible contract is responsible for the largest part of income.  The Telco company could use these insights in various ways. For instance try to move people from month-to-month to yearly contracts or to keep more in touch with clients in that group, to make sure that they will be less likely to churn.

Figure 2 – Attribute Selection for Predicting Customer Churn

Correlation of predictive variables

Thanks to the information gain values we know more about the coherence of the attributes with the churn variable. To see how the attributes relate to each other, we have created a plot of “correlation” between the attributes, which is shown in figure 3. The plot below shows how much the attributes are related to each other. The values differ from 1(perfect correlation) to 0(no correlation at all). It is not a correlation as one would normally see with numerical variables, but a different measure for categorical variables, which is called Cramer’s V. In the attribute importance chart can be observed that OnlineSecurity and TechSupport are both important attributes. In the chart above we can see that those are highly correlated. Therefore, it is plausible that one causes another or the two go almost always together. Then they can be viewed as a one combined phenomenon. Besides the value for predicting churn, this chart can be used to develop a better understanding of the behaviour of customers. It can be easily observed which attributes are interrelated.

We used the powerful combination of Power-BI to make business-oriented insightful visualizations on the data. Power BI is used to gather everything in one place, and make visual reports. R provides us with the flexibility to perform more custom-made operations on the data, for the needs of the specific company. The next step is to identify customers who are likely to churn beforehand. In our view both uses of the data described above are essential. And the predictive part would not be complete without understanding the data.

Figure 3 – Correlation between attributes

Read more or download the white paper

Want to learn more about how to predict customer churn, or want to dive into the formulas and underlying theoretical model? Our Predicting Customer Churn Case Study provides an even more in-depth look at at the steps that were taken to generate these results.