AI News, Retraining a state-of-the-art image classification Neural Network to classify your own images in TensorFlow

Retraining a state-of-the-art image classification Neural Network to classify your own images in TensorFlow

tl;dr I contributed code to the Google TensorFlow project on GitHub that adds TensorBoard visualizations to the existing TensorFlow “How to Retrain Inception’s Final Layer for New Categories” tutorial.

The Google TensorFlow project has a great tutorial which shows you how to quickly get started retraining the Inception v3 model to classify images of flowers and then repurpose the code for your own image classification needs.

Before I added TensorBoard summaries to the TensorFlow image classification tutorial, it was not possible to visualize the model architecture or compare model training performance over many training steps.

To take advantage of the TensorBoard visualization capabilities, I added code to the retraining script that allows you to visualize the model training statistics and overall model architecture.

Once you execute the retraining according to the tutorial, visualizing the retrain process and model architecture is as simple as: Once TensorBoard is running, selecting the EVENTS tab allows you to visualize the change in model statistics such as accuracy and cross entropy.

NOTE: The following examples are run from the tensorflow/examples/image_retraining directory of the TensorFlow GitHub project Example run 1 In this training run, let’s set the learning_rate to 0.01.

How to Retrain an Image Classifier for New Categories

Modern image recognition models have millions of parameters.

requires a lot of labeled training data and a lot of computing power (hundreds

of this by taking a piece of a model that has already been trained on a related

extraction capabilities from powerful image classifiers trained on ImageNet

for many applications, works with moderate amounts of training data (thousands,

not millions of labeled images), and can be run in as little as thirty

example script on your own images, and will explain some of the options you have

Image by Kelly Sikkema Before you start any training, you'll need a set of images to teach the network about

how to prepare your own images, but to make it easy we've created an archive

of flower photos, run these commands: Once you have the images, you can download the example code from GitHub (it

is not part of the library installation): In the simplest cases the retrainer can then be run like this (takes

You can get a full listing with: This script loads the pre-trained module and trains a new classifier on top for

magic of transfer learning is that lower layers that have been trained to distinguish

The script can take thirty minutes or more to complete, depending on the speed of

This penultimate layer has been trained to output a set of values

our final layer retraining can work on new classes is that it turns out the

Because every image is reused multiple times during training and calculating each

bottleneck takes a significant amount of time, it speeds things up to cache

By default they're stored in the /tmp/bottleneck directory, and if

you rerun the script they'll be reused so you don't have to wait for this part

Once the bottlenecks are complete, the actual training of the top layer of the network

what percent of the images used in the current training batch were labeled

the training accuracy is based on images that the network has been able to

learn from so the network can overfit to the noise in the training data.

measure of the performance of the network is to measure its performance on a

data set not contained in the training data -- this is measured by the validation

low, that means the network is overfitting and memorizing particular features

so you can tell if the learning is working by keeping an eye on whether

the loss keeps trending downwards, ignoring the short-term noise.

By default this script will run 4,000 training steps.

at random from the training set, finds their bottlenecks from the cache, and

compared against the actual labels to update the final layer's weights through

reported accuracy improve, and after all the steps are done, a final test accuracy

evaluation is run on a set of images kept separate from the training and

value of between 90% and 95%, though the exact value will vary from run to

percent of the images in the test set that are given the correct label after

The script includes TensorBoard summaries that make it easier to understand, debug, and optimize the retraining.

For example, you can visualize the graph and statistics, such as how the weights or accuracy varied during training.

To launch TensorBoard, run this command during or after retraining: Once TensorBoard is running, navigate your web browser to localhost:6006

The script will write out the new model trained on your categories to /tmp/output_graph.pb,

read in, so you can start using your new model immediately.

the top layer, you will need to specify the new name in the script, for example

color values in the fixed range [0,1], so you do not need to set the --input_mean

You should see a list of flower labels, in most cases with daisy on top (though

If you've managed to get the script working on the flower example images, you can

Here's what the folder structure of the flowers archive looks like, to give you and

The first place to start is by looking at the images you've gathered, since the most

common issues we see with training come from the data that's being fed in.

For example, if you take all your photos indoors against a blank wall and

your users are trying to recognize objects outdoors, you probably won't see good

end up basing its prediction on the background color, not the features of the

only things you'll ever be asked to categorize are the classes of object you know

example: pictures tagged #daisy might also include people and characters named

If you're happy with your images, you can take a look at improving your results by

common way of improving the results of image training is by deforming, cropping,

of expanding the effective size of the training data thanks to all the possible

variations of the same images, and tends to help the network learn to cope

that the bottleneck caching is no longer useful, since input images are never reused

it's recommended you try this as a way of polishing your model only after you

mirror half of the images horizontally, which makes sense as long as those

images as a check to make sure that overfitting isn't occurring, since if we

split is to put 80% of the images into the main training set, keep 10% aside

test set, since they are likely to merely reflect more general problems in the

By default the script uses an image feature extraction module with a pretrained instance

it provides high accuracy results with moderate running time for the retraining

On the other hand, if you intend to deploy your model on mobile devices or other resource-constrained

with the module URL, for example: This will create a 9 MB model file in /tmp/output_graph.pb with a model that uses

number of weights (and hence the file size and speed) shrinks with the square

will need to specify the image size that your model expects, for example: For more information on deploying the retrained model to a mobile device, see the

TensorBoard has bad graphs after deleting the model directory and retraining #652

Looking into this more, I think it's best resolved by making event purging for out-of-order steps work again, which was reported separately in #803 and which I have a PR out to implement.

The graphs look weird because when the model script re-evaluates, the new data gets picked up as a continuation of the old data, even though they're using the same step sequence, resulting in the data series 'zig-zagging' back to 0 on the horizontal axis and then appearing to overlay over the old data.

Note that strictly speaking this is a different condition because we're actually redoing the whole dataset from scratch, so it would be more correct to dump the entire set of data TensorBoard has loaded, not just data at a higher step count (since in theory, the new run might not start at step 1).

TensorFlow: saving/restoring and mixing multiple models

Before going any further, make sure you read the very small primer I made on TF here Why start with those informations?

Because, it is of tremendous importance to understand what can be saved at the different level of your code to avoid messing around cluelessly… Any interaction with your filesystem to save persistent data in TF needs a Saver object and a Session object.

The Saver constructor allows you to control many things among which 1 is important: The Session constructor allows you to control 3 things: The Saver, can handle the saving and loading (called restoring) of your Graph metadata and your Variables data.

Here is an example: If you look at your folder, it actually creates 3 files per save call and a checkpoint file, i’ll go into more details about this in the annexe.

It means that loading a .meta checkpoint will also restore all empty variables, operations and collections associated with your Graph (for example, it will restore the optimiser and its learning rate).

Yes, but there is a drawback to this: I don’t know yet a way to make the gradient flow easily between graphs, as you will have to evaluate the first graph, get the results and feed it to the next graph.

Many more things can be done with Protobufs, if you are interested check the tutorials here Getting back to TF, when you save your data the usual way, you end up with 5 different type of files: Let’s take a break here.

Train an Image Classifier with TensorFlow for Poets - Machine Learning Recipes #6

Monet or Picasso? In this episode, we'll train our own image classifier, using TensorFlow for Poets. Along the way, I'll introduce Deep Learning, and add context ...

How To Train an Object Detection Classifier Using TensorFlow 1.5 (GPU) on Windows 10

These instructions work for newer versions of TensorFlow too! This tutorial shows you how to train your own object detector for multiple objects using Google's ...

TensorFlow How to Retrain Inception's Final Layer for New Categories Tutorial

This tutorial demonstrates how to setup and use the "How to Retrain Inception's Final Layer for New Categories" example on the TensorFlow website. The code ...

Training/Testing on our Data - Deep Learning with Neural Networks and TensorFlow part 7

Welcome to part seven of the Deep Learning with Neural Networks and TensorFlow tutorials. We've been working on attempting to apply our recently-learned ...

Build a TensorFlow Image Classifier in 5 Min

In this episode we're going to train our own image classifier to detect Darth Vader images. The code for this repository is here: ...

Training model in Python and Loading into TensorFlow.js - TensorFlow.js p.4

Welcome to part 4 of the TensorFlow.js series, where we're going to be working on the challenge of training a model in Python, and then loading that trained ...

How to Deploy a Tensorflow Model to Production

Once we've trained a model, we need a way of deploying it to a server so we can use it as a web or mobile app! We're going to use the Tensorflow Serving ...

Adapting to video feed - TensorFlow Object Detection API Tutorial p.2

Welcome to part 2 of the TensorFlow Object Detection API tutorial. In this tutorial, we're going to cover how to adapt the sample code from the API's github repo to ...

TensorFlow Tutorial #08 Transfer Learning

How to use the pre-trained Inception model on the CIFAR-10 data-set using Transfer Learning. NOTE: This video is still useful but you may want to use Keras for ...

TensorFlow Tutorial #04 Save & Restore

How to use save and restore a Neural Network in TensorFlow. Also shows how to do Early Stopping using the validation set. NOTE: This is much easier using ...