AI News, Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter
- On Saturday, February 17, 2018
- By Read More
Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter
At the end of 2013, when Mark Zuckerberg decided to create Facebook AI Research, the organization I direct, Facebook was about to turn 10 years old.
The company thought about what “connecting people” would entail 10 years in the future, and realized that AI would play a key role.
Facebook can potentially show each person on Facebook about 2,000 items per day: posts, pictures, videos, etc.
Doing a good job at this requires understanding people, their tastes, interests, relationships, aspirations and even goals in life.
It also requires understanding content: understanding what a post or a comment talks about, what an image or a video contains, etc.
In a way, doing a perfect job at this is an “AI-complete” problem: it requires understanding people, emotions, culture, art.
Much of our work at Facebook AI focuses on devising new theories, principles, methods, and systems to make machines understand images, video, speech, and language—and then to reason about them.
Facebook recently announced a face-verification algorithm called “DeepFace,” with results that were widely reported to involve near-human accuracy in facial recognition.
Whereas lay people might think the computers are looking at the same sorts of random pictures that people look at every day.
Spectrum: How well will Deep Learning do in areas beyond image recognition, especially with issues associated with generalized intelligence, like natural language?
How do we combine the advantages of Deep Learning, with its ability to represent the world through learning, with things like accumulating knowledge from a temporal signal, which happens with language, with being able to do reasoning, with being able to store knowledge in a different way than current Deep Learning systems store it.
There’s a lot of work at Facebook, at Google, and at various other places where we’re trying to have a neural net on one side, and then a separate module on the other side that is used as a memory.
These systems are based on the idea of representing words and sentences with continuous vectors, transforming these vectors through layers of a deep architecture, and storing them in a kind of associative memory.
A lot of people are working on what’s called “recurrent neural nets.” These are networks where the output is fed back to the input, so you can have a chain of reasoning.
Consequently, with few exceptions, we mostly hire research scientists with PhDs and a couple years of post-doctoral research experience.
The type of scientists we recruit are not primary recruiting targets for startups in search of engineers.
After graduation, many of these fresh PhDs will strengthen the local startup ecosystem, having been exposed to the latest techniques in an industrial context.
Fourth, the prospect of getting a research position at FAIR will motivate talented students to go into research and study AI by doing a research-oriented Master and a PhD.
So the net effect of the creation of FAIR-Paris will undoubtedly be positive in terms of the availability of talents for the local tech ecosystem.
Why Deep Learning Is Suddenly Changing Your Life
Over the past four years, readers have doubtlessly noticed quantum leaps in the quality of a wide range of everyday technologies.
To gather up dog pictures, the app must identify anything from a Chihuahua to a German shepherd and not be tripped up if the pup is upside down or partially obscured, at the right of the frame or the left, in fog or snow, sun or shade.
Medical startups claim they’ll soon be able to use computers to read X-rays, MRIs, and CT scans more rapidly and accurately than radiologists, to diagnose cancer earlier and less invasively, and to accelerate the search for life-saving pharmaceuticals.
They’ve all been made possible by a family of artificial intelligence (AI) techniques popularly known as deep learning, though most scientists still prefer to call them by their original academic designation: deep neural networks.
Programmers have, rather, fed the computer a learning algorithm, exposed it to terabytes of data—hundreds of thousands of images or years’ worth of speech samples—to train it, and have then allowed the computer to figure out for itself how to recognize the desired objects, words, or sentences.
“You essentially have software writing software,” says Jen-Hsun Huang, CEO of graphics processing leader Nvidia nvda , which began placing a massive bet on deep learning about five years ago.
What’s changed is that today computer scientists have finally harnessed both the vast computational power and the enormous storehouses of data—images, video, audio, and text files strewn across the Internet—that, it turns out, are essential to making neural nets work well.
“We’re now living in an age,” Chen observes, “where it’s going to be mandatory for people building sophisticated software applications.” People will soon demand, he says, “ ‘Where’s your natural-language processing version?’ ‘How do I talk to your app?
The increased computational power that is making all this possible derives not only from Moore’s law but also from the realization in the late 2000s that graphics processing units (GPUs) made by Nvidia—the powerful chips that were first designed to give gamers rich, 3D visual experiences—were 20 to 50 times more efficient than traditional central processing units (CPUs) for deep-learning computations.
Its chief financial officer told investors that “the vast majority of the growth comes from deep learning by far.” The term “deep learning” came up 81 times during the 83-minute earnings call.
I think five years from now there will be a number of SP 500 CEOs that will wish they’d started thinking earlier about their AI strategy.” Even the Internet metaphor doesn’t do justice to what AI with deep learning will mean, in Ng’s view.
Yann LeCun (/ləˈkʌn/; born 1960) is a computer scientist with contributions in machine learning, computer vision, mobile robotics and computational neuroscience.
He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets. He is also one of the main creators of the DjVu image compression technology (together with Léon Bottou and Patrick Haffner).
Jackel, where he developed a number of new machine learning methods, such as a biologically inspired model of image recognition called Convolutional Neural Networks, the 'Optimal Brain Damage' regularization methods, and the Graph Transformer Networks method (similar to conditional random field), which he applied to handwriting recognition and OCR. The bank check recognition system that he helped develop was widely deployed by NCR and other companies, reading over 10% of all the checks in the US in the late 1990s and early 2000s.
He is also a professor at the Tandon School of Engineering. At NYU, he has worked primarily on Energy-Based Models for supervised and unsupervised learning, feature learning for object recognition in Computer Vision, and mobile robotics. In 2012, he became the founding director of the NYU Center for Data Science. On December 9, 2013, LeCun became the first director of Facebook AI Research in New York City, and stepped down from the NYU-CDS directorship in early 2014.
- On Monday, March 25, 2019
Signature - A Change is Gonna Come [Sam Cooke cover] (2017 Midwinter)
Signature Performed live at the Lila Cockrell Theater San Antonio, TX 2017 Barbershop Harmony Society Midwinter Convention.
Invertible Conditional GANs for Image Editing, NIPS 2016 | Guim Perarnau, Computer Vision Centre UAB
Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, Jose M. Álvarez NIPS 2016 Workshop on Adversarial Training Spotlight Generative Adversarial Networks..
Counterfactual Multi-Agent Policy Gradients
Many real-world problems, such as network packet routing and the coordination of autonomous vehicles, are naturally modelled as cooperative multi-agent systems. In this talk, I overview some...
Fairness in Machine Learning
Machine learning is increasingly being adopted by various domains: governments, credit, recruiting, advertising, and many others. Fairness and equality are critical aspects, especially in light...
Marcus Weldson, Dr. Renee Armour, J. Randy MacDonald
Marcus Weldon, President of Bell Labs, explains how people can run their lives more effectively using technology to perform mundane tasks. Steve Adubato talks with Dr. Renee Armour, Breast...