AI News, MachineLearning

MachineLearning

Hyper-parameter optimization has already been found to be a useful way to (partially) automate the search for good configurations in deep learning.

One of the main early contributors to this line of work (before it was applied to machine learning hyper-parameter optimization) is Frank Hutter (along with collaborators), who devoted his PhD thesis (2009) to algorithms for optimizing knobs that are typically set by hand in general in software systems.

http://jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf We then proposed using for deep learning the kinds of algorithms Hutter had developed for other contexts, called sequential optimization and this was published at NIPS'2011, in collaboration with another PhD student who devoted his thesis to this work, Remi Bardenet, and his supervisor Balazs Kegl (previously a prof in my lab, now in France).

http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips.pdf Snoek et al put out a software that has since been used by many researchers, called 'spearmint', and I found out recently that Netflix has been using it in their new work aiming to take advantage of deep learning for movie recommendations: http://techblog.netflix.com/2014/02/distributed-neural-networks-with-gpus.html

The Future of Deep Learning Research

Back-propagation is fundamental to deep learning. Hinton (the inventor) recently said we should "throw it all away and start over". What should we do?

TensorFlow, Deep Learning, and Modern Convolutional Neural Nets, Without a PhD (Cloud Next '18)

The hottest topics in computer science today are machine learning and deep neural networks. Many problems deemed ""impossible"" only five years ago have ...

Lecture 5: Backpropagation and Project Advice

Lecture 5 discusses how neural networks can be trained using a distributed gradient descent technique known as back propagation. Key phrases: Neural ...

MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars

This is lecture 1 of course 6.S094: Deep Learning for Self-Driving Cars taught in Winter 2017. Course website: Lecture 1 slides: ..

Scalable Trust-Region Method for Deep Reinforcement Learning Using Kronecker-Factored Approximation

In this work, we propose to apply trust region optimization to deep reinforcement learning using a recently proposed Kronecker-factored approximation to the ...

Training Performance: A user’s guide to converge faster (TensorFlow Dev Summit 2018)

Brennan Saeta walks through how to optimize training speed of your models on modern accelerators (GPUs and TPUs). Learn about how to interpret profiling ...

Lecture 6: Dependency Parsing

Lecture 6 covers dependency parsing which is the task of analyzing the syntactic dependency structure of a given input sentence S. The output of a dependency ...

Lesson 4: Practical Deep Learning for Coders

COLLABORATIVE FILTERING, EMBEDDINGS, AND MORE When we ran this class at the Data Institute, we asked what students were having the most trouble ...

How to Do Mathematics Easily - Intro to Deep Learning #4

Let's learn about some key math concepts behind deep learning shall we? We'll build a 3 layer neural network and dive into some key concepts that makes ...

Effective TensorFlow for Non-Experts (Google I/O '17)

TensorFlow is Google's machine learning framework. In this talk, you will learn how to use TensorFlow effectively. TensorFlow offers high level interfaces like ...