Deep Learning Blog | Bitfusion

Training a Bird Classifier with Tensorflow and TFLearn

Written by sono | Aug 31, 2016 5:50:25 PM

If you are new to our AMIs, head over to our Tensorflow README on how to get started, or check out our previous blog entry on getting started with TensorFlow

Intro

This entry is a walkthrough using the our latest Tenorflow AMI to train a model based on the example in Adam Geighty’s Medium article on Machine Learning . I am specifically using a g2.2xlarge EC2 instance to train the model to show the training benefits of using GPU instance over using a CPU instance.

Adam Geighty’s article articulated a number of things really well - his code example split out the different steps needed to train the model and the steps matched with sections of the article itself, allowing you to get a good understanding of what he was explaining. The example he used is based on the Cifar-10 example code and uses a combination of datasets to train a bird classifier. You can read more about Cifar datasets here and the referenced TFLearn code example here.

 

Creating the Classifier

Before we start, create a directory named bird_classifier in the ubuntu users home directory. We will carry out all operations in this directory as the ubuntu user.

mkdir ~/bird_classifier
cd ~/bird_classifier

Next we need our dataset to work with. You can download the dataset referenced in the article from S3. It’s a combination of the the Cifar 10 dataset and Caltech-UCSD Birds-200–2011 data set. In total there are ~74K images.

wget https://s3-us-west-2.amazonaws.com/ml-is-fun/data.zip
unzip data.zip

From this you will get the dataset: full_dataset.pkl

Download the Training Code

Next we need to get the code used in the article. I have provided a couple options to obtain it below:

Option 1 - use wget: The code below will pull from a gist and save it as bird_classifier.py.

wget -O bird_classifier.py https://gist.githubusercontent.com/sono-bfio/89a91da65a12175fb1169240cde3a87b/raw/b859d1673e0a81ebd42d7799d7c1df71517c175b/bird_classifier.py

Option 2 - copy the code below to a file that is in the same directory as full_dataset.pkl. In my case, I copied it to a file called bird_classifier.py.

from __future__ import division, print_function, absolute_import

# Import tflearn and some helpers
import tflearn
from tflearn.data_utils import shuffle
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.estimator import regression
from tflearn.data_preprocessing import ImagePreprocessing
from tflearn.data_augmentation import ImageAugmentation
import pickle

# Load the data set
X, Y, X_test, Y_test = pickle.load(open("full_dataset.pkl", "rb"))

# Shuffle the data
X, Y = shuffle(X, Y)

# Make sure the data is normalized
img_prep = ImagePreprocessing()
img_prep.add_featurewise_zero_center()
img_prep.add_featurewise_stdnorm()

# Create extra synthetic training data by flipping, rotating and blurring the
# images on our data set.
img_aug = ImageAugmentation()
img_aug.add_random_flip_leftright()
img_aug.add_random_rotation(max_angle=25.)
img_aug.add_random_blur(sigma_max=3.)

# Define our network architecture:

# Input is a 32x32 image with 3 color channels (red, green and blue)
network = input_data(shape=[None, 32, 32, 3],
                     data_preprocessing=img_prep,
                     data_augmentation=img_aug)

# Step 1: Convolution
network = conv_2d(network, 32, 3, activation='relu')

# Step 2: Max pooling
network = max_pool_2d(network, 2)

# Step 3: Convolution again
network = conv_2d(network, 64, 3, activation='relu')

# Step 4: Convolution yet again
network = conv_2d(network, 64, 3, activation='relu')

# Step 5: Max pooling again
network = max_pool_2d(network, 2)

# Step 6: Fully-connected 512 node neural network
network = fully_connected(network, 512, activation='relu')

# Step 7: Dropout - throw away some data randomly during training to prevent over-fitting
network = dropout(network, 0.5)

# Step 8: Fully-connected neural network with two outputs (0=isn't a bird, 1=is a bird) to make the final prediction
network = fully_connected(network, 2, activation='softmax')

# Tell tflearn how we want to train the network
network = regression(network, optimizer='adam',
                     loss='categorical_crossentropy',
                     learning_rate=0.001)

# Wrap the network in a model object
model = tflearn.DNN(network, tensorboard_verbose=0, checkpoint_path='bird-classifier.tfl.ckpt')

# Train it! We'll do 100 training passes and monitor it as it goes.
model.fit(X, Y, n_epoch=100, shuffle=True, validation_set=(X_test, Y_test),
          show_metric=True, batch_size=96,
          snapshot_epoch=True,
          run_id='bird-classifier')

# Save model when training is complete to a file
model.save("bird-classifier.tfl")
print("Network trained and saved as bird-classifier.tfl!")
 

Train it!

At this point, all we need to do is run our python script. The script carries out the following functions:

  • Will run through our dataset 100 times (epoch=100)
  • Takes roughly ~60 minutes (This is on a g2.2xlarge – EC2 Instance with a single GPU)
  • Produce our model file: bird-classifier.tfl.
$ python2 bird_classifier.py

# OUTPUT BELOW

I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so locally
..
.....
........ Concatenated output ...........
.....
..

I tensorflow/core/common_runtime/gpu/gpu_device.cc:806] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GRID K520, pci bus id: 0000:00:03.0)
---------------------------------
Run id: bird-classifier
Log directory: /tmp/tflearn_logs/
---------------------------------
Preprocessing... Calculating mean over all dataset (this may take long)...

..
.....
........ Concatenated output ...........
.....
..

--
Training Step: 59200  | total loss: 0.16163
| Adam | epoch: 100 | loss: 0.16163 - acc: 0.9332 | val_loss: 0.24135 - val_acc: 0.9387 -- iter: 56780/56780
--
Network trained and saved as bird-classifier.tfl!

Inference (Let’s test some images)

The script above created out trained model bird-classifier.tfl. Next, we will download the inference script provided in the article and some images from the internet and test it.

The code below will save the inference script as infer.py

 

wget -O infer.py https://gist.githubusercontent.com/ageitgey/a40dded08e82e59724c70da23786bbf0/raw/7c78536295f1ab8cce62d5c63ed57212cafd8950/r_u_a_bird.py

Next, we will create a directory to store our test images and download some creative commons images from the net. The test set has a total of 6 images – three that are birds and three that are not.

 

mkdir -p test_images

cd test_images

wget -O bird_bullocks_oriole.jpg http://www.5ensesmag.com/wp-content/uploads/2013/03/800px-Bullocks_Oriole.jpg

wget -O bird_mount_bluebird.jpg  http://climate.audubon.org/sites/default/files/bird_images/Mountain_Bluebird_FlickrCC_1.jpg

wget -O bird_african_fish_eagle.jpg http://www.nature.org/cs/groups/webcontent/@web/@africa/documents/media/african-fish-eagle-720x400.jpg

wget -O not_a_bird_stop_sign.jpg https://upload.wikimedia.org/wikipedia/commons/f/fd/Free_creative_commons_Rural_Stop_Landscape,_Antelope_Island,_Utah_\(4594258122\).jpg

wget -O not_a_bird_airplane.jpg http://blogs.voanews.com/student-union/files/2012/01/airplane-flickr-shyb.jpg

wget -O not_a_bird_creativecommons_logo.jpg https://www.raspberrypi.org/wp-content/uploads/2014/03/creative_commons.j

Let’s run our inference script against the images. I created a simple loop to go through and test each image below:

# Make sure you are in the directory where you downloaded infer.py to

for f in test_images/*.jpg; do echo "File: ${f}"; python2 infer.py ${f} 2>/dev/null; echo  ""; done

Here’s our output:

File: test_images/bird_african_fish_eagle.jpg
That's a bird!

File: test_images/bird_bullocks_oriole.jpg
That's a bird!

File: test_images/bird_mount_bluebird.jpg
That's a bird!

File: test_images/not_a_bird_airplane.jpg
That's not a bird!

File: test_images/not_a_bird_creativecommons_logo.jpg
That's not a bird!

File: test_images/not_a_bird_stop_sign.jpg
That's a bird!

Pretty good, we got one false positive in the bunch (“not_a_bird_stop_sign.jpg”). I left this in here as it is reveals an interesting anomoly, Adam’s article has a section that speaks to this: “How accurate is 95% accurate?” Lastly, if you would like run the inference script with a single image rather than using the bash forloop I have above, run the following:

python2 infer.py test_images/bird_african_fish_eagle.jpg

GPU vs. CPU Performance

To see the value of a GPU, I ran the training overnight with a CPU instance (C4.4xl). The results are below:

c4.4xl - $0.838 per Hour

  • 16 cores (hyper threaded) maxed out
  • ~ 123 Minutes

g2.2xl - $0.65 per Hour

  • single GPU
  • ~ 65 Minutes
 

Conclusion

There you have it - a trained bird classifier based on the Medium Article using Bitfusion’s Tensorflow AMI. If you are interested in scientific computing or deep learning, I encourage you to take a look our AMI offerings. They are sure to speed up your development, prototyping and GPU cluster creation. Additionally, if you have trained models and are looking for solid infrastructure to serve them, contact us here.

Questions or comments? Please post them in the comment section below or join our community Bitfusion-AWS Slack Channel.

Get Started!

 

Are you currently developing AI applications, but spending too much time wrangling machines and setting up your infrastructure? We are currently offering a Free 30-Day Trial of Bitfusion Flex!