AI News, 'Minimalist machine learning' algorithms analyze images from very little data

'Minimalist machine learning' algorithms analyze images from very little data

Daniël Pelt and James Sethian of Berkeley Lab's Center for Advanced Mathematics for Energy Research Applications (CAMERA) turned the usual machine learning perspective on its head by developing what they call a 'Mixed-Scale Dense Convolution Neural Network (MS-D)' that requires far fewer parameters than traditional methods, converges quickly, and has the ability to 'learn' from a remarkably small training set.

Their approach is already being used to extract biological structure from cell images, and is poised to provide a major new computational tool to analyze data across a wide range of research areas.

'The breakthrough resulted from realizing that the usual downscaling and upscaling that capture features at various image scales could be replaced by mathematical convolutions handling multiple scales within a single layer,' said Pelt, who is also a member of the Computational Imaging Group at the Centrum Wiskunde &

'This new approach has the potential to radically transform our ability to understand disease, and is a key tool in our new Chan-Zuckerberg-sponsored project to establish a Human Cell Atlas, a global collaboration to map and characterize all cells in a healthy human body.'

Using this vast database of cross-referenced images, convolutional neural networks and other machine learning methods have revolutionized our ability to quickly identify natural images that look like ones previously seen and catalogued.

Mixed-Scale Dense Convolution Neural Networks Many applications of machine learning to imaging problems use deep convolutional neural networks (DCNNs), in which the input image and intermediate images are convolved in a large number of successive layers, allowing the network to learn highly nonlinear features.

To achieve accurate results for difficult image processing problems, DCNNs typically rely on combinations of additional operations and connections including, for example, downscaling and upscaling operations to capture features at various image scales.

Instead, the new 'Mixed-Scale Dense' network architecture avoids many of these complications and calculates dilated convolutions as a substitute to scaling operations to capture features at various spatial ranges, employing multiple scales within a single layer, and densely connecting all intermediate images.

New Applications Pelt and Sethian are taking their approach to a host of new areas, such as fast real-time analysis of images coming out of synchrotron light sources and reconstruction problems in biological reconstruction such as for cells and brain mapping.

Berkeley Lab ‘Minimalist Machine Learning’ Algorithms Analyze Images From Very Little Data

Daniël Pelt and James Sethian of Berkeley Lab’s Center for Advanced Mathematics for Energy Research Applications (CAMERA) turned the usual machine learning perspective on its head by developing what they call a “Mixed-Scale Dense Convolution Neural Network (MS-D)” that requires far fewer parameters than traditional methods, converges quickly, and has the ability to “learn” from a remarkably small training set.

Their approach is already being used to extract biological structure from cell images, and is poised to provide a major new computational tool to analyze data across a wide range of research areas.

“The breakthrough resulted from realizing that the usual downscaling and upscaling that capture features at various image scales could be replaced by mathematical convolutions handling multiple scales within a single layer,” said Pelt, who is also a member of the Computational Imaging Group at the Centrum Wiskunde &

“This new approach has the potential to radically transform our ability to understand disease, and is a key tool in our new Chan-Zuckerberg-sponsored project to establish a Human Cell Atlas, a global collaboration to map and characterize all cells in a healthy human body.” The National Center for X-ray Tomography is located at the Advanced Light Source, a DOE Office of Science national user facility at Berkeley Lab.

Using this vast database of cross-referenced images, convolutional neural networks and other machine learning methods have revolutionized our ability to quickly identify natural images that look like ones previously seen and catalogued.

applications of machine learning to imaging problems use deep convolutional neural networks (DCNNs), in which the input image and intermediate images are convolved in a large number of successive layers, allowing the network to learn highly nonlinear features.

To achieve accurate results for difficult image processing problems, DCNNs typically rely on combinations of additional operations and connections including, for example, downscaling and upscaling operations to capture features at various image scales.

Instead, the new “Mixed-Scale Dense” network architecture avoids many of these complications and calculates dilated convolutions as a substitute to scaling operations to capture features at various spatial ranges, employing multiple scales within a single layer, and densely connecting all intermediate images.

and Sethian are taking their approach to a host of new areas, such as fast real-time analysis of images coming out of synchrotron light sources and reconstruction problems in biological reconstruction such as for cells and brain mapping.

“By reducing the amount of required training images and increasing the size of images that can be processed, the new architecture can be used to answer important questions in many research fields.” CAMERA is supported by the offices of Advanced Scientific Computing Research and Basic Energy Sciences in the Department of Energy’s Office of Science.

Automated Training of Deep Convolutional Neural Networks for Cell Segmentation

Human lung carcinoma cells (H299) were seeded to low density on a glass bottom 96 well-plate (MoBiTec #5241-20) and cultured for 6 h in RPMI medium (RPMI 1640, Gibco #42401) supplemented with 10% Fetal Bovine Serum (Invitrogen, #10270-106).

Images were acquired using the NIS software and a 10x/0.45 air objective on an inverted Nikon Ti microscope, motorized in xy and z (stage piezo, Mad City Labs) and equipped with a full size incubator (Live Imaging Service), a Halogen lamp as light source for bright-field light imaging, LEDs (SpectraX, Lumencor) as light source for fluorescence imaging (Hoechst: ex 395/25;

To help the repositioning of the plate after labeling the cells at the end of the time-lapse acquisition, the plate was pushed to the top right corner of the stage insert and the lid was then placed on top without moving the plate.

Bright-field images were recorded every 10 min overnight (total imaging time 17 h) until the recording was stopped in the morning immediately before the final staining and imaging steps.

The focus was maintained using the Nikon Perfect Focus System set to focusing where the filopodia at the edges of the cells were best contrasted and a z-stack of 2 additional z-planes upwards (z-step 4 μm) was acquired for each image.

In the morning, the acquisition was stopped, the lid of the plate removed and 150 μl of medium from each well were vortexed with 1 μl of blue nuclear dye and 1 μl of green cytoplasmic dye (Hoechst 33342, Invitrogen, CellTracker CMFDA, #C7025, reconstituted according to the manufacturer’s instructions) and added back to each well.

The plate was repositioned, the lid placed on top and after a total of 10 min after the end of the time-lapse, images in bright-field as well as blue and green fluorescence were acquired at the same positions and with the same z-stack to serve as input for creation of ground truth.

1 and the end of the sequence is shown as t

N

.

N+1 shows that the images were acquired after staining.

Each site in this dataset was imaged at five different wavelengths distinguishing different intracellular structures such as nucleus, endoplasmic reticulum, nucleoli, golgi apparatus plasma membrane, and mitochondria (Supplementary Fig.5).

After the dilation, we used the nuclear regions as seeds and applied a seeded watershed segmentation on the dilated cytoplasms image to get the final segmentation mask of the cells (Fig.1b) We repeated the above process on all the three datasets using the same CellProfiler pipeline provided as Supplementary Software.

In addition to the automatic ground truth we also created manual annotations to evaluate the performance of our network trained on automatically generated ground truth for the time-lapse dataset.

The long skip connections from the downward path were element-wise summed and batch normalized at the receiving end of the upward path as shown in Supplementary Fig.2a as right pointed arrows.

The data preprocessing step involved processing both the input bright-field images and the ground truth labels created automatically by our image processing pipeline in CellProfiler (Supplementary Software).

The data augmentation step consisted of random flipping in left-right and up-down directions followed by the creation of an image that was nine times the input image by extending the image on all sides by mirroring.

The choice of this parameter depended on the dataset, based on the density of cells in the image, and could be automatically determined based on the proportion of cells covering the training image.

We set the initial DCNN learning rate to 0.001, trained the network for 60000 iterations and reduced the learning rate to 1/10 of the current value after every 5000 iterations.

To solve this, we performed a second pass through the image after shifting the tile positions to half the tile size, i.e., 120 pixels to the left and up compared to the initial pass.

iii.

Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks

First, the good news is that our “8” recognizer really does work well on simple images where the letter is right in the middle of the image: But now the really bad news: Our “8” recognizer totally fails to work when the letter isn’t perfectly centered in the image.

We can just write a script to generate new images with the “8”s in all kinds of different positions in the image: Using this technique, we can easily create an endless supply of training data.

But once we figured out how to use 3d graphics cards (which were designed to do matrix multiplication really fast) instead of normal computer processors, working with large neural networks suddenly became practical.

It doesn’t make sense to train a network to recognize an “8” at the top of a picture separately from training it to recognize an “8” at the bottom of a picture as if those were two totally different objects.

Instead of feeding entire images into our neural network as one grid of numbers, we’re going to do something a lot smarter that takes advantage of the idea that an object is the same no matter where it appears in a picture.

Here’s how it’s going to work, step by step — Similar to our sliding window search above, let’s pass a sliding window over the entire original image and save each result as a separate, tiny picture tile: By doing this, we turned our original image into 77 equally-sized tiny image tiles.

We’ll do the exact same thing here, but we’ll do it for each individual image tile: However, there’s one big twist: We’ll keep the same neural network weights for every single tile in the same original image.

It looks like this: In other words, we’ve started with a large image and we ended with a slightly smaller array that records which sections of our original image were the most interesting.

We’ll just look at each 2x2 square of the array and keep the biggest number: The idea here is that if we found something interesting in any of the four input tiles that makes up each 2x2 grid square, we’ll just keep the most interesting bit.

So from start to finish, our whole five-step pipeline looks like this: Our image processing pipeline is a series of steps: convolution, max-pooling, and finally a fully-connected network.

For example, the first convolution step might learn to recognize sharp edges, the second convolution step might recognize beaks using it’s knowledge of sharp edges, the third step might recognize entire birds using it’s knowledge of beaks, etc.

Here’s what a more realistic deep convolutional network (like you would find in a research paper) looks like: In this case, they start a 224 x 224 pixel image, apply convolution and max pooling twice, apply convolution 3 more times, apply max pooling and then have two fully-connected layers.

Phase recovery and holographic image reconstruction using deep learning in neural networks

Our deep neural network approach for phase retrieval and holographic image reconstruction is schematically described in Figure 1 (see also Supplementary Figs.

However, due to this relatively short sample-to-sensor distance, the twin-image artifact of the in-line holography, which is a result of the lost phase information, is strong and severely obstructs the spatial features of the sample in both the amplitude and phase channels, as illustrated in Figures 1 and 2.

This training involves learning the statistical transformation between a complex-valued image that results from the back-propagation of a single intensity-only hologram of the object and the same object’s image that is reconstructed using a multi-height phase retrieval algorithm (treated as the gold standard for the training phase).

This training/learning process (which is performed only once) results in a fixed deep neural network that is used to blindly reconstruct the phase and amplitude images of any object, free from twin-image and other undesired interference-related artifacts, using a single hologram intensity.

In our holographic imaging experiments, we used three different types of samples: blood smears, Pap smears and breast tissue sections, and separately trained three convolutional neural networks for each sample type, although the network architecture was identical in each case, as shown in Figure 1.

These reconstructed phase and amplitude images clearly demonstrate the success of our deep neural network-based holographic image reconstruction approach to blindly infer artifact-free phase and amplitude images of the objects, matching the performance of the multi-height phase recovery.

Table 1 further compares the structural similarity54 (SSIM) of our neural network output images (using a single input hologram, that is, Nholo=1) against the results obtained with a traditional multi-height phase retrieval algorithm using multiple holograms (that is, Nholo=2, 3,…,8) acquired at different sample-to-sensor distances.

A comparison of the SSIM index values reported in Table 1 suggests that the imaging performance of the deep neural network using a single hologram is comparable to that of multi-height phase retrieval, closely matching the SSIM performance of Nholo=2 for both Pap smear and breast tissue samples and the SSIM performance of Nholo=3 for blood smear samples.

Next, to evaluate the tolerance of the deep neural network and its holographic reconstruction framework to axial defocusing, we digitally back-propagated the hologram intensity of a breast tissue section to different depths, that is, defocusing distances within a range of z=[−20 μm, +20 μm] with Δz=1 μm increments.

Although the deep neural network was trained with in-focus images, Figure 4 demonstrates the ability of the network to blindly reconstruct defocused holographic images with a negligible drop in image quality across the imaging system’s depth of field, which is ~4 μm.

This is why, for in-line holographic imaging of such strongly scattering and structurally dense samples, self-interference-related terms, in addition to twin-image terms, form strong image artifacts in both the phase and amplitude channels of the sample, making it difficult to apply object support-based constraints for phase retrieval.

Another important property of this deep neural network-based holographic reconstruction framework is that it significantly suppresses out-of-focus interference artifacts, which frequently appear in holographic images due to dust particles or other imperfections in various surfaces or optical components of the imaging setup.

From the perspective of our trained neural network, this property to suppress out-of-focus interference artifacts stems from the fact that these holographic artifacts fall into the same category as twin-image artifacts due to the spatial defocusing operation, helping the trained network reject such artifacts in the reconstruction process.

<?xml version="1.0" encoding="UTF-8"?>Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments

Live-cell imaging, in which living cells are imaged over a period of time using phase contrast and/or fluorescence microscopy, is a powerful method for interrogating living systems.

This class of experiments has shed light on numerous biological problems, which include transcriptional regulation in both bacteria and eukaryotes as well as information transmission in mammalian signaling networks [1–8].

One common insight these single-cell measurements have been able to provide is the role cellular heterogeneity plays in these systems, as they have the ability to capture the differences between cells and observe how these differences evolve in time.

Two notable examples are Ilastik and Microscopy Image Browser, two freely available programs that use supervised machine learning (edge and texture filters with random forest classification) to perform segmentation [12, 17–19].

The specific time cost is rarely, if ever, reported, but work by our lab on NF-κ B signaling in mammalian cells, which required segmenting images of fluorescent nuclei, required on the order of 100+ hours of manual curation per manuscript [4, 5].

Segmentation methods exist for the mammalian cytoplasm, but they typically require either imaging a cytoplasmic fluorescent protein (which removes a fluorescence channel) or imaging multiple focal planes (which increases acquisition time) [21–26].

however, a robust method to identify the cytoplasm of mammalian cells or bacterial micro-colonies with single-cell resolution directly from phase microscopy images has remained elusive [17, 26, 30, 31].

However, the overall lack of sharable segmentation solutions means the cost of entering this field requires a significant—and often unanticipated—computational investment, beyond the obvious costs associated with the microscopy itself.

Recent advances in supervised machine learning, namely deep convolutional neural networks (referred to here as conv-nets) have shown remarkable performance for the task of image classification—that is, assigning descriptive labels to images [32, 33].

Prior work has shown that in addition to functioning as image classifiers, conv-nets can also perform semantic segmentation—the assignment of class labels to each individual pixel of an image rather than to the whole image itself—in a computationally efficient manner [34–36].

While these recent developments have the potential to be revolutionary, our experience both as experimentalists and as users of deep learning demonstrated that there was still work to be done before conv-nets can supplant prior image segmentation methods in this space.

We highlight particular features of our work—image normalization, segmentation refinement with active contours, and receptive field size—which were critical for conv-nets to perform robustly on live-cell imaging data using relatively small training data sets.

Lecture 11 | Detection and Segmentation

In Lecture 11 we move beyond image classification, and show how convolutional networks can be applied to other core computer vision tasks. We show how ...

Build a TensorFlow Image Classifier in 5 Min

In this episode we're going to train our own image classifier to detect Darth Vader images. The code for this repository is here: ...

How to Predict Stock Prices Easily - Intro to Deep Learning #7

Only a few days left to signup for my Decentralized Applications course! We're going to predict the closing price of the S&P 500 using a ..

Deep Neural Network Learns Van Gogh's Art | Two Minute Papers #6

Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc).

What is 1G, 2G, 3G, 4G, 5G of Cellular Mobile Communications - Wireless Telecommunications

FREE TRIAL Get all courses in Prime Membership Telecom (5G,4G,3G,2G) Free for 30 Days!

The Best Way to Visualize a Dataset Easily

Only a few days left to signup for my Decentralized Applications course! In this video, we'll visualize a dataset of body metrics collected ..

V1: Fundamentals of Telecom 1 - Introduction and Preview

Teracom DVD Video Course V1 Fundamentals of Telecom 1 The PSTN • Telephony • The Telecom Industry • Telecom Equipment Length 142 minutes. 55-page ...

Flow Cytometry Tutorials: Doublet Discrimination

Learn to exclude doublets from your Flow Cytometry data analysis. By the end of this tutorial, you should be able to understand: What doublets and clumps are?

But what *is* a Neural Network? | Chapter 1, deep learning

Subscribe to stay notified about new videos: Support more videos like this on Patreon: Special .

How computers are learning to be creative | Blaise Agüera y Arcas

We're on the edge of a new frontier in art and creativity — and it's not human. Blaise Agüera y Arcas, principal scientist at Google, works with deep neural ...