AI News, How AI detectives are cracking open the black box of deep learning

How AI detectives are cracking open the black box of deep learning

Jason Yosinski sits in a small glass box at Uber’s San Francisco, California, headquarters, pondering the mind of an artificial intelligence.

Like many of the AIs that will soon be powering so much of modern life, including self-driving Uber cars, Yosinski’s program is a deep neural network, with an architecture loosely inspired by the brain.

This particular AI has been trained, using a vast sum of labeled images, to recognize objects as random as zebras, fire trucks, and seat belts.

And every year, this gap is going to get a bit larger.” Each month, it seems, deep neural networks, or deep learning, as the field is also called, spread to another scientific discipline.

Marco Ribeiro, a graduate student at the University of Washington in Seattle, strives to understand the black box by using a class of AI neuroscience tools called counter-factual probes.

On the basis of thousands of tests, LIME can identify the words—or parts of an image or molecular structure, or any other kind of data—most important in the AI’s original judgment.

But Mukund Sundararajan, another computer scientist at Google, devised a probe that doesn’t require testing the network a thousand times over: a boon if you’re trying to understand many decisions, not just a few.

Instead of varying the input randomly, Sundararajan and his team introduce a blank reference—a black image or a zeroed-out array in place of text—and transition it step-by-step toward the example being tested.

Sundararajan compares the process to picking out the key features that identify the glass-walled space he is sitting in—outfitted with the standard medley of mugs, tables, chairs, and computers—as a Google conference room.

“When the lights become very dim, only the biggest reasons stand out.” Those transitions from a blank reference allow Sundararajan to capture more of the network’s decisions than Ribeiro’s variations do.

According to a directive from the European Union, companies deploying algorithms that substantially influence the public must by next year create “explanations” for their models’ internal logic.

The Defense Advanced Research Projects Agency, the U.S. military’s blue-sky research arm, is pouring $70 million into a new program, called Explainable AI, for interpreting the deep learning that powers drones and intelligence-mining operations.

As a graduate student in the 1990s at Carnegie Mellon University in Pittsburgh, Pennsylvania, he joined a team trying to see whether machine learning could guide the treatment of pneumonia patients.

But disturbingly, he saw that a simpler, transparent model trained on the same records suggested sending asthmatic patients home, indicating some flaw in the data.

“What really terrifies me is what else did the neural net learn that’s equally wrong?” Today’s neural nets are far more powerful than those Caruana used as a graduate student, but their essence is the same.

Those data are sucked into a network with a dozen or more computational layers, in which neuron-like connections “fire” in response to features of the input data.

Her guiding principle is monotonicity—a relationship between variables in which, all else being equal, increasing one variable directly increases another, as with the square footage of a house and its price.

She wires those tables into neural networks, effectively adding an extra, predictable layer of computation—baked-in knowledge that she says will ultimately make the network more controllable.

To develop a model that would match deep learning in accuracy but avoid its opacity, he turned to a community that hasn’t always gotten along with machine learning and its loosey-goosey ways: statisticians.

But GAMs can also handle trickier relationships by finding multiple operations that together can massage data to fit on a regression line: squaring a set of numbers while taking the logarithm for another group of variables, for example.

(It would have made the same optimistic error for pneumonia patients who also had chest pain and heart disease.) Caruana has started touting the GAM approach to California hospitals, including Children’s Hospital Los Angeles, where about a dozen doctors reviewed his model’s results.

He then wired that translation network into his original game-playing network, producing an overall AI that would say, as it waited in a lane, “I’m waiting for a hole to open up before I move.” The AI could even sound frustrated when pinned on the side of the screen, cursing and complaining, “Jeez, this is hard.” Riedl calls his approach “rationalization,” which he designed to help everyday users understand the robots that will soon be helping around the house and driving our cars.

Then, he and his colleagues fed it colored static and sent a signal back through it to request, for example, “more volcano.” Eventually, they assumed, the network would shape that noise into its idea of a volcano.

This time, when told to create “more volcano,” the GAN took the gray mush that the classifier learned and, with its own knowledge of picture structure, decoded it into a vast array of synthetic, realistic-looking volcanoes.

After feeding altered images into the original caption model, he realized that the caption writers who trained it never described trees and a branch without involving a bird.

ENHANCE! Upscaling Images with Neural Networks

When characters on a TV show “enhance!” a blurry image, you probably laugh and tell your friends that it's impossible to do that in real life. But over the past year ...

A friendly introduction to Convolutional Neural Networks and Image Recognition

A friendly explanation of how computer recognize images, based on Convolutional Neural Networks. All the math required is knowing how to add and subtract ...

Research at NVIDIA: AI Reconstructs Photos with Realistic Results

Researchers from NVIDIA, led by Guilin Liu, introduced a state-of-the-art deep learning method that can edit images or reconstruct a corrupted image, one that ...

Full Resolution Image Compression With Recurrent Neural Networks

George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, Michele Covell This paper presents a set of full-resolution lossy ...

How to force unsupervised neural networks to discover the right representation of images

One appealing way to design an object recognition system is to define objects recursively in terms of their parts and the required spatial relationships between ...

Deep Learning Lecture 10: Convolutional Neural Networks

Slides available at: Course taught in 2015 at the University of Oxford by Nando de Freitas with ..

AI evolves to compose 3 hours of jazz!

3 hours of Computery's music: The training data was all composed by Doug McKenzie at ..

Deep Neural Networks are Easily Fooled

A video summary of the paper: Nguyen A, Yosinski J, Clune J. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable ...

Neural Network that Changes Everything - Computerphile

Years of work down the drain, the convolutional neural network is a step change in image classification accuracy. Image Analyst Dr Mike Pound explains what it ...

Convolutional Neural Networks And Unconventional Data - Predicting The Stock Market Using Images

This video is about taking numerical data, transforming it into images and modeling it with convolutional neural networks. In other words, we are creating ...