AI News, Explaining behavior of Machine Learning models with eli5 library

Explaining behavior of Machine Learning models with eli5 library

Target audience is ML practitioners who want to 1) get a better quality from their ML pipelines - understanding of why a wrong decision happens is often a first step to improve the quality of an ML solution; 2)

humans trust such models more because they can check if an explanation is consistent with their domain knowledge or gut feeling, understand better shortcomings of the solution and make a more informed decision as a result.

Target audience is ML practitioners who want to 1) get a better quality from their ML pipelines - understanding of why a wrong decision happens is often a first step to improve the quality of an ML solution;

humans trust such models more because they can check if an explanation is consistent with their domain knowledge or gut feeling, understand better shortcomings of the solution and make a more informed decision as a result.

In my last two posts, I wrote about model interpretability, with the goal of trying to understanding what it means and how to measure it.

In this post, I want to discuss a number of desirable properties that have been suggested for model interpretations, and that might be used to judge whether and how much a model or explanation is interpretable.

As I noted previously, while there are many papers in the research literature that describe interpretable models, or ways of adding a layer of interpretability on top of existing models, most of these papers are not explicit about what they mean by interpretability.

For example, if a prediction can be explained by indicating 100 or 1000 factors that contributed to the value, it would be more concise to have the explanation algorithm pick out the top 5 according to some notion of importance, and only present those. Of course, if you make an explanation too concise, it may cease to be useful.

So you’ll always want to balance making an explanation concise against the the other desirable properties, particularly: Faithful. The explanation should accurately describe the way the model made the prediction.

For example, you might try explaining the predictions from a neural network by separately training a linear model, then presenting the features that contributed to the linear model’s prediction (which are relatively easy to determine) as the features that probably had the most impact on the neural net’s prediction.

This is a major theme in the PED paper, since it focuses on debugging models through repeated explanations. An explanation algorithm is consistent if each successive explanation helps the user to better understand later predictions.

For example, if a feature increases a risk prediction for one data point but decreases it for another data point, the explanations should include enough information for the user to understand why.

Engaging. The PED paper notes that an explanation should encourage a user to pay attention to the important details, and they point to research showing (not surprisingly) that users who pay more attention to explanations become familiar with the model faster and ultimately make better decisions.

Interpretable ML Symposium

Complex machine learning models, such as deep neural networks, have recently achieved outstanding predictive performance in a wide range of applications, including visual object recognition, speech perception, language modeling, and information retrieval.

There has since been an explosion of interest in interpreting the representations learned and decisions made by these models, with profound implications for research into explainable ML, causality, safe AI, social science, automatic scientific discovery, human computer interaction (HCI), crowdsourcing, machine teaching, and AI ethics.

Predictions on their own produce insights, but by interpreting the learned structure of the model, we can gain more important new insights into the processes driving crime, enabling us to develop more effective public policy.

One of the panels will have a moderated debate format where arguments are presented on each side of key topics chosen prior to the symposium, with the opportunity to follow-up each argument with questions.

We invite researchers to submit their recent work on interpretable machine learning from a wide range of approaches, including (1) methods that are designed to be more interpretable from the start, such as rule-based methods, (2) methods that produce insight into existing ML models, and (3) perspectives either for or against interpretability in general.

Looking inside machine learning black boxes

For a long time (the whole time I’ve had this job, 2 years), I’ve struggled with a bunch of questions about complicated machine learning models.

I’m not going to explain in depth what a random forest is here (it’s basically a collection of decision trees which you let vote on your outcome), but they’re more complicated.

5:20pm AND departing airport != LAX AND [ten more things] But what the random forest chose to assign a given probability of lateness to my flight is actually totally explainable by I

Then one week, over a couple of days, my awesome product manager Isaac implemented (in javascript!!) a tool to explain to you why a random forest model made a given choice.

So it turns out that if you just do the simplest possible thing (get the random forest to report exactly why it made the choice that it did), it’s actually surprisingly helpful in helping debug!

talked to someone at a conference a while ago who worked on automated trading systems, and we were talking about how machine learning approaches can be really scary because you fundamentally don’t know whether the ML is doing a thing because it’s smart and correct and better than you, or because there’s a bug in the data.

Airbnb has one called Unboxing the random forest classifier, Sift Science has Large Scale Decision Forests: Lessons Learned, and this short paper called A Model Explanation System discusses a general system for explaining black box models (I’m actually, unusually, very excited about that paper) The more I learn about machine learning, the more I think that debugging tools &

a clear understanding of how the algorithms you’re using work are totally essential for making your models better (actually, I don’t understand how they wouldn’t be –

it makes no sense to me.) I imagine the people who build amazing neural nets and things like AlphaGo have an extremely strong understanding of the foundations of how neural networks work, and some sense for how the algorithms are translating the data into choices.

For an example of what I mean by model debugging tools, check out this toy notebook where I train an overfit model, investigate a specific instance of something it predicted poorly, and find out why.

I’d love to hear about what work you’re doing in explaining / debugging / untangling complex machine learning models, and especially if you’ve written anything about it.

Mikhail Korobov - Explaining behavior of Machine Learning models with eli5 library

Target audience is ML practitioners who want to 1) get a better quality from their ML pipelines - understanding of why a wrong decision happens is often a first step to improve the quality of an ML solution;2) explain ML model behavior to clients or stakeholders - inspectable ML pipelines are easier to “sell” to a client;

humans trust such models more because they can check if an explanation is consistent with their domain knowledge or gut feeling, understand better shortcomings of the solution and make a more informed decision as a result.License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/...Please see our speaker release agreement for details: https://ep2017.europython.eu/en/speak...

Mikhail Korobov - Explaining behavior of Machine Learning models with eli5 library

"Explaining behavior of Machine Learning models with eli5 library [EuroPython 2017 - Talk - 2017-07-13 - Anfiteatro 2] [Rimini, Italy] ML estimators don't have to ...

Visualizing a Decision Tree - Machine Learning Recipes #2

Last episode, we treated our Decision Tree as a blackbox. In this episode, we'll build one on a real dataset, add code to visualize it, and practice reading it - so ...

Finale Doshi-Velez: "A Roadmap for the Rigorous Science of Interpretability" | Talks at Google

With a growing interest in interpretability, there is an increasing need to characterize what exactly we mean by it and how to sensibly compare the interpretability ...

Deep Attention Mechanism for Multimodal Intelligence: Perception, Reasoning, & Expression

We have long envisioned that machines one day can perform human-like perception, reasoning, and expression across multiple modalities including vision and ...