AI News, Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters

Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters

It is hence a good method for meta-optimizing a neural network which is itself an optimisation problem: tuning a neural network uses gradient descent methods, and tuning the hyperparameters needs to be done differently since gradient descent can’t apply.

Therefore, Hyperopt can be useful not only for tuning hyperparameters such as the learning rate, but also to tune more fancy parameters in a flexible way, such as changing the number of layers of certain types, or the number of neurons in a layer, or even the type of layer to use at a certain place in the network given an array of choices, each with nested tunable hyperparameters.

Hyperparameter Optimization - The Math of Intelligence #7

Hyperparameters are the magic numbers of machine learning. We're going to learn how to find them in a more intelligent way than just trial-and-error. We'll go ...

3. Bayesian Optimization of Hyper Parameters

Video from Coursera - University of Toronto - Course: Neural Networks for Machine Learning:

Hyperopt: A Python library for optimizing machine learning algorithms; SciPy 2013

Hyperopt: A Python library for optimizing the hyperparameters of machine learning algorithms Authors: Bergstra, James, University of Waterloo; Yamins, Dan, ...