AI News, Modeling: Teaching a Machine Learning Algorithm to Deliver Business Value
Modeling: Teaching a Machine Learning Algorithm to Deliver Business Value
Although machine learning algorithms may sound technically complex, implementing them in Python is simple thanks to standard machine learning libraries like Scikit-Learn.
To get the data ready for machine learning, we have to take some basic steps: missing value imputation, encoding of categorical variables, and optionally feature selection if the input dimension is too large (see notebook for full details).
Then, we can create a model with standard modeling syntax: Before applying machine learning, it’s best to establish a naive baseline to determine if machine learning is actually helping.
The metrics for no machine learning, logistic regression, and the random forest with default hyperparameters are shown below: Each model was evaluated using about 30% of the data for holdout testing based on a time-series split.
How We Built Our Machine Learning Model for Churn Prediction
(This content originally appeared in Inside Big Data, and is reprinted here with permission.) With the cost of acquiring new app installs skyrocketing, keeping users engaged who have already installed is critical for maximizing acquisition spend and customer lifetime value.
Here, I provide insight into the process of building a scalable predictive machine learning model over billions of events and address how these predictive capabilities lead to new insights into user behavior, fuel new engagement strategies and impact user retention.
Churn prediction is a straightforward classification problem:go back in time, look at user activity, check to see who remains active after some time point, then come up with a model that separates users who remain active from those who do not.
Looking into the model and which features had the biggest impact, we found some interesting patterns: Now that we created a working model, the next step was to test its ability to scale to thousands of apps and billions of users.
Adding more apps quickly exposed a weak spot: the re-processing of data from a csv (the output from a MapReduce job to create our feature data) to a sparse matrix (format required by the boosted trees model).
For example, if you’re attempting to re-engage at-risk users you can scale up the reward offered if you include only the high-risk audience or scale it down if you include both the high-risk and medium-risk audience.
Once a user has been categorized as high, medium or low risk-to-churn, the data is immediately available through our real-time mobile data stream for analysis or action in other systems, dashboards to view five-week performance, and visualizations to show how effective your efforts are in moving users from high-risk to lower risk states.
By comparing churn prediction and app engagement strategy our customers can identify areas for improvement, effect those changes and compare how churn scores change week over week.
For App B, we would recommend targeting more of their audience, messaging more frequently and expanding their use of messaging strategy to alternative communication such as in-app messaging and message center as well as improve targeting via tags and named user.
How to Apply Machine Learning to Business Problems
It’s easy to see the massive rise in popularity for venture investment, conferences, and business-related queries for “machine learning”
In this article, we’ll break down categories of business problems that are commonly handled by ML, and we’ll also provide actionable advice to begin a ML initiative with the right approach and perspective (even it’s the first such project you’ve undertaken at your company).
For illustrative purposes, it will be helpful to list a number of well-established business use-cases for machine learning so that you (the reader) can churn up your own application ideas:
If you have reams of business data from years ago, it may have no relevance today, particularly in fields where the basic business processes change drastically year-over-year, such as mobile eCommerce).
For example, if you run a door delivery service for pet supplies, and your app, prices, product offerings, and service areas have changed significantly over the last six months, you will need much more recent data to learn from than, say, a company selling homeowners’
While unsupervised learning (see glossary below) allows for a wide degree of applications in making sense of data without labels, it’s usually not advised for companies to “jump into”
One letter difference or one number difference could mean overpaying your bill by 10x the original amount (if the decimal was interpreted to be in the wrong place), or sending money to the wrong company (if an invoicing company name isn’t registered exactly).
As an interesting caveat, there is a San Francisco-based startup called Roger.ai which is aiming to use natural language processing and machine vision to real and pay bills, albeit it pulls humans into the loop before sending funds.
In order to gain additional perspective on the issue of “picking a business problem for machine learning”, we decided to reach out to our network of previous AI podcast interview guests for additional guidance for our business readers: Dr. Ben Waber —
CEO, Calculation Consulting: “The best problems are those in which there is a very large, historical data set that includes both rich features and some kind of direct feedback that can be used to build and algorithm that can be implemented and tested easily and will either decrease operational costs and /or increase revenue immediately.“
CEO, AGI Innovations Inc: (To begin, Peter quotes Dr. Robin Hanson, Professor at George Mason University: “Good CS expert says: Most firms that think they want advanced AI/ML really just need linear regression on cleaned-up data.”) “I think that most businesses don’t justify the investment in ML/DL (of course, ML means many things). Cutting edge stuff that everyone is talking about requires a lot of data and expertise, and is static – i.e.
This leads us into the second major section of this guide: In an off-mic conversation with Dr. Charles Martin (AI consultant in the Bay Area), he mentioned that many companies read about ML with enthusiasm and decide to “find some way to use it.”
Pick a business problem that matters immensely, and seems to have a high likelihood of being solved UBER’s Danny Lange has mentioned from stage that there is one thought process that’s highly likely to yield fruitful machine learning use case ideas: “If we only knew ____.”
could get in the way of developing an effective ML solution: Building a ML solution requires careful thinking and testing in selecting algorithms, selecting data, cleaning data, and testing in a live environment.
Data Science and BD&A, Computer Sciences Corporation: “The most common mistake that businesses make when using ML is that they think that an ML solution is a one-shot process: They send data to data scientists, and data scientists send back THE model.
Log everything, build storage and processing systems, ensure they are accessible, conduct deep analysis and as many experiments as you can on your product, build in intelligence into as much as your product as possible.
The consensus (in the limited number of quotes above, and from dozens of other conversations with business-minded data scientists) is that machine learning is not as much of a mere “tool”
Interested readers might benefit from our recent consensus of 26 machine learning / AI researchers where we asked: “Where should machine learning be applied first in business?” The infographic featured drives home many of the same points highlighted in this article.
The ultimate question for executives remains: When can we have (a) the resources required to invest in machine learning seriously, and (b) a legitimate use case that started from trying to find real business value, not from “trying to find a way to kinda use machine learning.”
- On Wednesday, February 19, 2020
2. Case Study: Churn Prediction
We have an interactive discussion on how to formulate a realistic, but subtly complicated, business problem as a formal machine learning problem. Access the ...
AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)
In this session, we take a specific business problem—predicting Telco customer churn—and explore the practical aspects of building and evaluating an Amazon ...
AWS Summit Tel Aviv 2016: Amazon Machine Learning Case Study: Predicting Customer Churn
Learn More: - - - - - - - - - - - - - - - - - Session Language: English We do a deeper dive into Amazon Machine Learning, using a specific ..
Logistic Regression in R | Machine Learning Algorithms | Data Science Training | Edureka
Data Science Training - ) This Logistic Regression Tutorial shall give you a clear understanding as to how a Logistic ..
Implementing and Training Predictive Customer Lifetime Value Models in Python
Implementing and Training Predictive Customer Lifetime Value Models in Python by Jean-Rene Gauthier, Ben Van Dyke Customer lifetime value models (CLVs) ...
Machine Learning Models in Production
Data Scientists and Machine Learning practitioners, nowadays, seem to be churning out models by the dozen and they continuously experiment to find ways to ...
Predicting Customer Churn Decision Tree Model
In this video you will learn the how to build a Decision Tree to understand data that is driving customer churn using RapidMiner. Learn more at ...
[Webinar] Predict Your App Uninstalls and Prevent your Churning Users using Machine Learning
Predict who is going to Uninstall your app within the next 7 days and prevent them from Churning Learn how to approach this uninstall problem using Machine ...
Kaggle Competition No.1 Solution: Predicting Consumer Credit Default
Contributed by Bernard Ong, Jielei Emma Zhu, Miaozhi Trinity Yu, Nanda Trichy Rajarathinam. They enrolled in the NYC Data Science Academy 12-Week Full ...
Machine Learning Project-Churn Model using logistic regression
In this machine learning project, you will predict the customer churn of telecom sector and find out the key drivers that lead to churn. Learn how the logistic ...