AI News, Baidu Tech Blog

Baidu Tech Blog

Today Baidu’s Silicon Valley AI Lab (SVAIL) released Warp-CTC, open source software for the machine learning community. Warp-CTC can be used to solve supervised problems that map an input sequence to an output sequence, such as speech recognition.  Get Warp-CTC and read more here  Warp-CTC QA Q.

A. CTC is an objective function that can be used while doing supervised training for sequence prediction, without knowing an alignment between the input and output.The CTC algorithm was developed at IDSIA by Alex Graves, Santiago Fernandez, Faustino Gomez and Jürgen Schmidhuber.  Q.

We found that currently available implementations of CTC generally required significantly more memory and/or were tens to hundreds of times slower.

A lot of open source software for deep learning exists, but previous code for training end-to-end networks for sequences (like our Deep Speech engine) has been too slow.

SVAIL Tech Notes: Accelerating Recurrent Neural Networks by Stashing Weights On-Chip

Learn how to make RNNs 30 times faster at small mini-batch sizes - allowing data parallel scaling to 16 times more GPUs - and enabling training of 11 times deeper networks.