AI News, Retraining a state-of-the-art image classification Neural Network to classify your own images in TensorFlow
Retraining a state-of-the-art image classification Neural Network to classify your own images in TensorFlow
tl;dr I contributed code to the Google TensorFlow project on GitHub that adds TensorBoard visualizations to the existing TensorFlow “How to Retrain Inception’s Final Layer for New Categories” tutorial.
The Google TensorFlow project has a great tutorial which shows you how to quickly get started retraining the Inception v3 model to classify images of flowers and then repurpose the code for your own image classification needs.
Before I added TensorBoard summaries to the TensorFlow image classification tutorial, it was not possible to visualize the model architecture or compare model training performance over many training steps.
To take advantage of the TensorBoard visualization capabilities, I added code to the retraining script that allows you to visualize the model training statistics and overall model architecture.
Once you execute the retraining according to the tutorial, visualizing the retrain process and model architecture is as simple as: Once TensorBoard is running, selecting the EVENTS tab allows you to visualize the change in model statistics such as accuracy and cross entropy.
NOTE: The following examples are run from the tensorflow/examples/image_retraining directory of the TensorFlow GitHub project Example run 1 In this training run, let’s set the learning_rate to 0.01.
How to Retrain an Image Classifier for New Categories
Modern image recognition models have millions of parameters.
requires a lot of labeled training data and a lot of computing power (hundreds
of this by taking a piece of a model that has already been trained on a related
extraction capabilities from powerful image classifiers trained on ImageNet
for many applications, works with moderate amounts of training data (thousands,
not millions of labeled images), and can be run in as little as thirty
example script on your own images, and will explain some of the options you have
Image by Kelly Sikkema Before you start any training, you'll need a set of images to teach the network about
how to prepare your own images, but to make it easy we've created an archive
of flower photos, run these commands: Once you have the images, you can download the example code from GitHub (it
is not part of the library installation): In the simplest cases the retrainer can then be run like this (takes
You can get a full listing with: This script loads the pre-trained module and trains a new classifier on top for
magic of transfer learning is that lower layers that have been trained to distinguish
The script can take thirty minutes or more to complete, depending on the speed of
This penultimate layer has been trained to output a set of values
our final layer retraining can work on new classes is that it turns out the
Because every image is reused multiple times during training and calculating each
bottleneck takes a significant amount of time, it speeds things up to cache
By default they're stored in the /tmp/bottleneck directory, and if
you rerun the script they'll be reused so you don't have to wait for this part
Once the bottlenecks are complete, the actual training of the top layer of the network
what percent of the images used in the current training batch were labeled
the training accuracy is based on images that the network has been able to
learn from so the network can overfit to the noise in the training data.
measure of the performance of the network is to measure its performance on a
data set not contained in the training data -- this is measured by the validation
low, that means the network is overfitting and memorizing particular features
so you can tell if the learning is working by keeping an eye on whether
the loss keeps trending downwards, ignoring the short-term noise.
By default this script will run 4,000 training steps.
at random from the training set, finds their bottlenecks from the cache, and
compared against the actual labels to update the final layer's weights through
reported accuracy improve, and after all the steps are done, a final test accuracy
evaluation is run on a set of images kept separate from the training and
value of between 90% and 95%, though the exact value will vary from run to
percent of the images in the test set that are given the correct label after
The script includes TensorBoard summaries that make it easier to understand, debug, and optimize the retraining.
For example, you can visualize the graph and statistics, such as how the weights or accuracy varied during training.
To launch TensorBoard, run this command during or after retraining: Once TensorBoard is running, navigate your web browser to localhost:6006
The script will write out the new model trained on your categories to /tmp/output_graph.pb,
read in, so you can start using your new model immediately.
the top layer, you will need to specify the new name in the script, for example
color values in the fixed range [0,1], so you do not need to set the --input_mean
You should see a list of flower labels, in most cases with daisy on top (though
If you've managed to get the script working on the flower example images, you can
Here's what the folder structure of the flowers archive looks like, to give you and
The first place to start is by looking at the images you've gathered, since the most
common issues we see with training come from the data that's being fed in.
For example, if you take all your photos indoors against a blank wall and
your users are trying to recognize objects outdoors, you probably won't see good
end up basing its prediction on the background color, not the features of the
only things you'll ever be asked to categorize are the classes of object you know
example: pictures tagged #daisy might also include people and characters named
If you're happy with your images, you can take a look at improving your results by
common way of improving the results of image training is by deforming, cropping,
of expanding the effective size of the training data thanks to all the possible
variations of the same images, and tends to help the network learn to cope
that the bottleneck caching is no longer useful, since input images are never reused
it's recommended you try this as a way of polishing your model only after you
mirror half of the images horizontally, which makes sense as long as those
images as a check to make sure that overfitting isn't occurring, since if we
split is to put 80% of the images into the main training set, keep 10% aside
test set, since they are likely to merely reflect more general problems in the
By default the script uses an image feature extraction module with a pretrained instance
it provides high accuracy results with moderate running time for the retraining
On the other hand, if you intend to deploy your model on mobile devices or other resource-constrained
with the module URL, for example: This will create a 9 MB model file in /tmp/output_graph.pb with a model that uses
number of weights (and hence the file size and speed) shrinks with the square
will need to specify the image size that your model expects, for example: For more information on deploying the retrained model to a mobile device, see the
TensorBoard has bad graphs after deleting the model directory and retraining #652
Looking into this more, I think it's best resolved by making event purging for out-of-order steps work again, which was reported separately in #803 and which I have a PR out to implement.
The graphs look weird because when the model script re-evaluates, the new data gets picked up as a continuation of the old data, even though they're using the same step sequence, resulting in the data series 'zig-zagging' back to 0 on the horizontal axis and then appearing to overlay over the old data.
Note that strictly speaking this is a different condition because we're actually redoing the whole dataset from scratch, so it would be more correct to dump the entire set of data TensorBoard has loaded, not just data at a higher step count (since in theory, the new run might not start at step 1).
TensorFlow: saving/restoring and mixing multiple models
Before going any further, make sure you read the very small primer I made on TF here Why start with those informations?
Because, it is of tremendous importance to understand what can be saved at the different level of your code to avoid messing around cluelessly… Any interaction with your filesystem to save persistent data in TF needs a Saver object and a Session object.
The Saver constructor allows you to control many things among which 1 is important: The Session constructor allows you to control 3 things: The Saver, can handle the saving and loading (called restoring) of your Graph metadata and your Variables data.
Here is an example: If you look at your folder, it actually creates 3 files per save call and a checkpoint file, i’ll go into more details about this in the annexe.
It means that loading a .meta checkpoint will also restore all empty variables, operations and collections associated with your Graph (for example, it will restore the optimiser and its learning rate).
Yes, but there is a drawback to this: I don’t know yet a way to make the gradient flow easily between graphs, as you will have to evaluate the first graph, get the results and feed it to the next graph.
Many more things can be done with Protobufs, if you are interested check the tutorials here Getting back to TF, when you save your data the usual way, you end up with 5 different type of files: Let’s take a break here.
- On Monday, June 17, 2019
Train an Image Classifier with TensorFlow for Poets - Machine Learning Recipes #6
Monet or Picasso? In this episode, we'll train our own image classifier, using TensorFlow for Poets. Along the way, I'll introduce Deep Learning, and add context ...
How To Train an Object Detection Classifier Using TensorFlow 1.5 (GPU) on Windows 10
These instructions work for newer versions of TensorFlow too! This tutorial shows you how to train your own object detector for multiple objects using Google's ...
TensorFlow How to Retrain Inception's Final Layer for New Categories Tutorial
This tutorial demonstrates how to setup and use the "How to Retrain Inception's Final Layer for New Categories" example on the TensorFlow website. The code ...
Training/Testing on our Data - Deep Learning with Neural Networks and TensorFlow part 7
Welcome to part seven of the Deep Learning with Neural Networks and TensorFlow tutorials. We've been working on attempting to apply our recently-learned ...
Build a TensorFlow Image Classifier in 5 Min
In this episode we're going to train our own image classifier to detect Darth Vader images. The code for this repository is here: ...
Training model in Python and Loading into TensorFlow.js - TensorFlow.js p.4
Welcome to part 4 of the TensorFlow.js series, where we're going to be working on the challenge of training a model in Python, and then loading that trained ...
How to Deploy a Tensorflow Model to Production
Once we've trained a model, we need a way of deploying it to a server so we can use it as a web or mobile app! We're going to use the Tensorflow Serving ...
Adapting to video feed - TensorFlow Object Detection API Tutorial p.2
Welcome to part 2 of the TensorFlow Object Detection API tutorial. In this tutorial, we're going to cover how to adapt the sample code from the API's github repo to ...
TensorFlow Tutorial #08 Transfer Learning
How to use the pre-trained Inception model on the CIFAR-10 data-set using Transfer Learning. NOTE: This video is still useful but you may want to use Keras for ...
TensorFlow Tutorial #04 Save & Restore
How to use save and restore a Neural Network in TensorFlow. Also shows how to do Early Stopping using the validation set. NOTE: This is much easier using ...