AI News, Microsoft Research Podcast

Microsoft Research Podcast

You may have heard the phrase, necessity is the mother of invention, but for Dr. Nicolo Fusi, a researcher at the Microsoft Research lab in Cambridge, Massachusetts, the mother of his invention wasn’t so much necessity as it was boredom: the special machine learning boredom of manually fine-tuning models and hyper-parameters that can eat up tons of human and computational resources, but bring no guarantee of a good result.

Automate machine learning with a meta-model that figures out what other models are doing, and then predicts how they’ll work on a given dataset.

On today’s podcast, Dr. Fusi gives us an inside look at Automated Machine Learning – Microsoft’s version of the industry’s AutoML technology – and shares the story of how an idea he had while working on a gene editing problem with CRISPR/Cas9 turned into a bit of a machine learning side quest and, ultimately, a surprisingly useful instantiation of Automated Machine Learning –

(music plays) Host: You’re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting-edge of technology research and the scientists behind it.

Host: You may have heard the phrase, necessity is the mother of invention, but for Dr. Nicolo Fusi, a researcher at the Microsoft Research lab in Cambridge, Massachusetts, the mother of his invention wasn’t so much necessity as it was boredom: the special machine learning boredom of manually fine-tuning models and hyper-parameters that can eat up tons of human and computational resources, but bring no guarantee of a good result.

Automate machine learning with a meta-model that figures out what other models are doing, and then predicts how they’ll work on a given dataset.

On today’s podcast, Dr. Fusi gives us an inside look at Automated Machine Learning – Microsoft’s version of the industry’s AutoML technology – and shares the story of how an idea he had while working on a gene editing problem with CRISPR-Cas9, turned into a bit of a machine learning side quest, and, ultimately, a surprisingly useful instantiation of Automated Machine Learning – now a feature of Azure Machine Learning – that reduces dependence on intuition and takes some of the tedium out of data science at the same time.

And so I did a lot of work in computational biology and then during that work, I kind of figured, oh, there are so many machine learning problems that you can solve that are interesting and apply to a wide range of things.

So, most recently, as you said, I’m working on automated machine learning which is a field where the goal is to kind of automate as much of the machine learning process as possible.

So those are three sort of divergent paths – well there’s some overlaps on Venn diagram – but how did those all come together for you?

Nicolo Fusi: I started in machine learning, and you know machine learning you can do either applied work – uh, you pick a problem, you apply machine learning to it, it always comes with its own set of challenges – or you can pick something more theoretical and maybe advance the way people do modeling or infer the parameters of a model, for instance.

And when it came to starting my PhD, I had the choice of problems from both fields and, due to personal circumstances, I felt like I needed to do something that had an effect on human health.

So, I started working on solving questions in computational biology using machine learning with the goal of later, kind of going from computational biology to medicine, again using machine learning.

Before we launch into your specific work in the field, let’s talk a little more generally about automated machine learning.

And now we are starting to move up the hierarchy and kind of combine classes and families of models into meta-models, and that kind of incorporates all that is going on underneath them.

Nicolo Fusi: In my mind, the traditional machine learning workflow, which is also the data science workflow – people use different names – you start with some question and you define what kind of data do I need to answer that question, what kind of metrics measure my success and what’s the closest numerically computable metric that I can pair and I can measure to see whether my model is doing well?

And every time you go down one path, you pick a way to transform your features, you pick one model, you pick a set of parameters and then you test it and then you go back.

You keep doing this loop, over and over again, and then, at the end, you basically produce one model that maybe you deploy, maybe you inspect to see whether the predictions are correct or fair or stuff like that.

I don’t think we’ll ever be able to automate the “phrasing the question” or “deciding the metric,” because that’s what the human should be doing, really.

So, depending on which value you set one hyper-parameter to, all the values for another hyper-parameter completely change meaning or change the scale at which they are relevant.

If you do kind of parameter sweeps, which is a lot of what the industry is doing, you kind of waste a ton of computational resources, and you have no guarantee of finding anything good.

(music plays) Host: According to legend – I use that term loosely – you were using machine learning and getting mired in the drudgework of data science and, basically, thinking there’s got to be an app for this, or something.

We did all the deciding which metric we want to optimize, and then we spent six months, maybe, of our own time, almost full-time, trying different ways to slice and dice the model space, the parameter space.

In previous work, we basically figured out, again, a machine-learning model to predict, given the many ways you can edit the gene, because you can do it in different ways, what’s the most successful edit?

And in this follow-up work, we were investigating the issue of, given that I want to perform this edit, what’s the likelihood that I mess up something else in the genome that I didn’t want to touch?

You can imagine if I want to remove a gene, or silence a gene that was causing a disease, I don’t want to suddenly give you a different disease because an unintended edit happened somewhere else.

Nicolo Fusi: So, the idea was ultimately deciding which algorithm, which set of hyper-parameters to use for a given problem, you’re kind of trying to recommend a series of things and then I evaluate them and I tell you how well they work.

So, implementing this… Nicolo Fusi: Yeah… So because I was the first user, I didn’t want just something that you could you know academically show, “Oh, you know, we beat random,” because random is a strong baseline for AutoML, surprisingly.

Is there anybody who knows machine learning who can help me out with this data analysis question?” And so, we thought, ok, so if nobody’s responding to that email, maybe we should just blast out an email to the entire mailing list saying, “We have fifty spots.

And if you want to use something that kind of finds a model automatically, you phrase the question and we kind of search your model space for you and give you a pipeline you can use at the end.

And it sounds to me like the scenario you’re painting is something that would help validate your research and actually help the people that are doing projects within the hackathon itself.

So, it’s a joint collaboration at this point between MSR, you know we did the original kind for proof-of-concept, and a huge amount of work went in from people within Azure.

Nicolo Fusi: Uhhh, we struggled a little bit, because in the early validation phase, let’s call it, the research prototype, “let’s give it away” phase, we got a lot of different kind of people approaching us.

And if you don’t know what to do, this kind of solution gives you a good model in a short amount of time, relative to the size of your data so you can just use it.

Host: So, going back to your comment about data scientists being a customer, why wouldn’t a data scientist be a little bit worried that this AutoML might be taking over their job?

Like, if you change, let’s say, the way you featurize your data, you don’t have to start from scratch with the tuning, you just set an AutoML run going and you just move on.

Um… So, we had done work on the on-target problem, which was how do you find the optimal, for some notion of optimality, how do you find the best edit to perform to make sure you disable a gene you want to disable?

And so, we decided, why don’t we just pre-compute everything for the human genome which took an exorbitant amount of CPU time on Azure, but we could just pre-populate a giant database table and then search it almost instantly.

You can put in the gene you want to edit, and you get a “least of possible” guide with a score that tells you how likely the edit is to be successful and how likely each off-target is… Host: To happen?

And I want to make sure that, as we build a capability to generate better and better predictors, we are also thinking of ways to make sure that the predictions are well-explained, that the biases are auditable and visible to the person who’s deploying these systems.

There was a program where the idea, I think, of to represent it correctly, was to kind of combine some mathematically-minded people and some biology/medicine-minded people to see what kind of collaborations arise.

But we were in the address book so sometimes Microsoft sales people would come to our office intending to access the corporate network, but they didn’t understand that we were not on the corporate network even.

Nicolo Fusi: Yes, so… I think after a couple of years in LA, different people moved in different places, and Jen Listgarten came to Boston and you know, I heard this Boston lab is incredible.

And it’s not like predictions of the future, but more just sort of as you look at the landscape, what are the exciting challenges, hard problems, that are still out there for young researchers who might be, yourself, a few years ago, trying to decide where do I want to land?

And, you know, self-driving cars and all these things will need the notion of raising the red flag and saying, “I don’t know what’s going on.

If you’re entering the game now, and you’re trying to just maximize predictive accuracy, where predictive accuracy is basically like a root mean square.

But I think, thinking more end-to-end, “what is the end goal of this machine learning system” is going to be a much more interesting area in the future.

Nicolo Fusi

I lead the automated machine learning research team at Microsoft Research in Cambridge, MA.

Broadly speaking, we work on scalable probabilistic models to search through complex optimization spaces.

My own research interests include Gaussian processes, Bayesian nonparametrics and scalable inference methods.

In computational biology, I have worked on statistical methods to perform genome-wide association studies and predictive models for CRISPR/Cas9 gene editing.

New CRISPR off-target and end-to-end guide design paper : The paper is now out in Nature Biomedical Engineering.

This work complements our earlier on-target work published in Nature Biotechnology.

New arXiv preprint : Probabilistic matrix factorization for automated machine learning

From time to time we host computational biology talks at MSR New England.

To subscribe to the talk announcement list, click here

Fusi, Rishit Sheth, Huseyn Melih Elibol to appear in NIPS, 2018 arXiv preprint Prediction of off-target activities for end-to-end CRISPR guide design J

Listgarten, M Weinstein, B Kleinstiver, AA Sousa, JK Joung, J Crawford, K Gao, M Elibol, L Hoang, J Doench, N Fusi (equal contributions and co-corresponding) Nature Biomedical Engineering, in press (2018) Orthologous CRISPR–Cas9 enzymes for combinatorial genetic screens F.

Optimized sgRNA design to maximize activity and minimize off-target effects for genetic screens with CRISPR-Cas9 J.

This pre-print has been largely (though not entirely) absorbed into the Nature Biotechnology paper above.

genome-to-genome analysis of associations between human genetic variation, HIV-1 sequence diversity, and viral control. István

Whole genome transcriptome analysis identifies indices of fast and slow disease progression in two mouse models of amyotrophic lateral sclerosis. G.

Unravelling the enigma of selective vulnerability in neurodegeneration: motor neurons resistant to degeneration in ALS show distinct gene expression characteristics and decreased susceptibility to excitotoxicity. A.

Microsoft brings new brains to Azure AI at Ignite conference

Microsoft researcher Nicolo Fusi shows data from the company's automated machine learning technology.

Artificial intelligence, which these days typically refers to technology called neural networks or machine learning modeled loosely on human brains, is moving from the exotic to the mainstream in the computing world.

Automated machine learning is designed to zero in on the best way to construct an AI model, whether that's for identifying what's in a video, analyzing medical scans, looking for manufacturing glitches or flagging fraudulent credit card transactions.

AI jobs are different, so Microsoft's new service is designed to try out lots of possible approaches, tweak the 'hyperparameterrs' that govern how they run, and rapidly winnow out the dud approaches.

And it's expanded the number of AI services that'll run in accelerated form on  programmable chips, called FPGAs, that are at the heart of the Microsoft's Brainwave AI work.  Taking It to Extremes: Mix insane situations -- erupting volcanoes, nuclear meltdowns, 30-foot waves -- with everyday tech.

Facebook Machine Learning Announcement: New AI Lab Launch

I was lucky enough to be invited to a major announcement by Facebook about their new new research lab dedicated to AI through Machine Learning. They've ...

GATK4: Live Launch Event

On January 9th, 2018, the Broad Institute Data Sciences Platform unleashed the latest version of GATK. GATK4 is completely open source. This recording of the ...

Introduction to Astronomy: Crash Course Astronomy #1

Welcome to the first episode of Crash Course Astronomy. Your host for this intergalactic adventure is the Bad Astronomer himself, Phil Plait. We begin with ...

Yelawolf - Till It’s Gone

iTunes: Sign up for updates: Music video by Yelawolf performing Till It's Gone. (C) 2014 Interscope .

Креатин на сушке. Можно ли принимать креатин моногидрат во время (в период) сушки?

reassured screamed liter favoring traction wondered reconsider realizing plow nap brain's ebb manifests CVD HDL ..

Набор мышечной массы. Набор массы. Как набрать вес?

reassured screamed liter favoring traction wondered reconsider realizing plow nap brain's ebb manifests CVD HDL ..

Сушка. Сушка тела для девушек. Бешеная сушка

reassured screamed liter favoring traction wondered reconsider realizing plow nap brain's ebb manifests CVD HDL ..

Google I/O 2014 - Maps for good: Saving trees and saving lives with petapixel-scale computing

Speaker(s): Rebecca Moore Description: Satellites have been systematically collecting imagery of our changing planet for more than 40 years, yet until recently ...

Yelawolf - Daddy's Lambo

Sign up for updates: Music video by Yelawolf performing Daddy's Lambo. (C) 2011 DGC Records Best of Yelawolf: ..