Transfer Learning Approach | Pre-trained Models | Classifying ImageNet classes with ResNet50 in Python

Hello Everyone, In this post, we will learn about Transfer Learning and the pre-trained models in Keras and try to predict classes using the ImageNet dataset. You must be now wondering what are these “PRE-TRAINED” Models? Let me tell you. With the need to access a hundred GB VRAM on GPUs in order to run a super complex supervised machine learning problem that would be much costly. We have discovered a concept known as “Transfer Learning”.

Transfer Learning enables us to use the pre-trained models from other people by making small relevant changes. Basically, Transfer Learning (TL)  is a Machine Learning technique that trains a new model for a particular problem based on the knowledge gained by solving some other problem.  For example, the knowledge gained while learning to recognize trucks could be applied to recognize cars.

Analogy Between Neural Network and Transfer Learning

We can also compare TL with the relationship of a teacher and a student. The teacher has experience in a particular domain, say Mathematics. Now, the knowledge the teacher has is delivered to students as a part of his/her teaching and so the student doesn’t need to learn everything from scratch. So here we see a transfer of knowledge from teacher to the student which is the main idea behind Transfer Learning.

Now, applying this analogy and analyzing the concept of Neural Networks, we find that we train a neural network model on some data. The knowledge gained is stored in the form of “weights” of the network. Next, we can just extract the already trained weights and transfer to any other neural network. Thus, instead of training the other neural network, we transfer the already learnt features and this simplifies our work as well. Also, TL is an optimization technique, as it improves performance when modeling the second task.

There are 2 common approaches for using TL –

1. Develop Model Approach 
2. Pre-trained Model Approach

In this post, we will understand the approach using pre-trained models. Normally, we perform TL with predictive modeling problems using image dataset. To model a large and challenging image classification task, such as the ImageNet – classifying 1000’s of classes we use pre-trained models. Let me guide you on using these models.

What is a pre-trained model?

So, a pre-trained model is not something very difficult to understand. It’s simply a model that was initially created by someone else to solve a similar kind of problem, and we use that already trained model on our problem as a starting point. Just like the packages and libraries already created for us, import them and we are good to go. These models are useful for prediction, feature extraction, and fine-tuning.

Why we should use pre-trained models?

Consider a scenario that you are trying to build a self-learning car model. Now, if you think of building the model from scratch you will apparently require a huge set of images for image recognition and it would take years to train a model on such large datasets. Whereas if you take the help of an existing pre-trained model, your half work will be done in no time. As the weights are already optimized, it saves your time by converging the model fast.

How to use a pre-trained model in Keras?

It is very easy to use pre-trained models. We can find all the pre-trained models in the application module of Keras. First, we import the pre-trained model. Then to add the pre-trained model we have 2 ways – Sequential way or functional API.

Keras has various pre-trained models. The most used ones are:
1. VGG-16
2. Inception
3. ResNet50
4. EfficientNet

We have a family of sub-models of these models as well. For knowing about the different models click here.

As we are aware now of various models, lets try to import one of these models and try to classify images. I will use ResNet50 for classifying ImageNet classes.

1: Import the necessary packages and ResNet50 model.

# import the ResNet50
from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy np

2: Build the model on ImageNet data.

model = ResNet50(weights='imagenet')

3: Assign the image path to access the image.

I am using 3 different object images for classification model. Let’s assign their paths for easy access.

img_path1 = 'tiger.jpg'
img_path2 = 'car.jpg'
img_path3 = 'sail boat.jpg'

With the below code we can display the images in our ipython notebook for our reference.

from pylab import imread,subplot,imshow,show
import matplotlib.pyplot as plt

image1 = imread(img_path1)  # choose image location
plt.imshow(image1)  # display the image

 

image2 = imread(img_path2)  # choose image location
plt.imshow(image2)  # display the image

 

image3 = imread(img_path3)  # choose image location
plt.imshow(image3)  # display the image

4: Convert image to array.

In this step, we convert a new image that we have stored in our system to use for classification, into numpy array.

img = image.load_img(img_path1, target_size=(224, 224))
x = img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

img = image.load_img(img_path2, target_size=(224, 224))
y = img_to_array(img)
y = np.expand_dims(y, axis=0)
y = preprocess_input(y)

img = image.load_img(img_path3, target_size=(224, 224))
z = img_to_array(img)
z = np.expand_dims(z, axis=0)
z = preprocess_input(z)

5: Predict the class.

preds = model.predict(x)

# decode the results into a list of tuples (class, description, probability)
print('Predicted:', decode_predictions(preds, top=3)[0])

Output:

Predicted: [('n02129604', 'tiger', 0.8297287), ('n02123159', 'tiger_cat', 0.1646096), ('n02127052', 'lynx', 0.002560344)]

Tiger image predicted with 82% accuracy.

preds = model.predict(y)

# decode the results into a list of tuples (class, description, probability)
print('Predicted:', decode_predictions(preds, top=3)[0])

Output:

Predicted: [('n04285008', 'sports_car', 0.9737299), ('n04037443', 'racer', 0.013672035), ('n03100240', 'convertible', 0.00819998)]

Sports car also recognized with an accuracy of 97%

preds = model.predict(z)

# decode the results into a list of tuples (class, description, probability)
print('Predicted:', decode_predictions(preds, top=3)[0])

Output:

Predicted: [('n04612504', 'yawl', 0.8056376), ('n02981792', 'catamaran', 0.07685501), ('n04147183', 'schooner', 0.06392856)]

So, now we see that all the images were classifies accurately using the already trained ResNet50 model of Keras.

Here, we come to the end of our today’s tutorial. We got to learn about a classification model which performed well on the test images we provided.

The images I used can be found here.

Conclusion

I hope you learnt something new and valuable today. Try to implement the Transfer Learning approach in your project. Also, don’t forget to be extra careful while choosing a pre-trained model for your problem statement. If the problem statement is based on object detection and you choose a pre-trained model which was previously trained for speech recognition problem, would prove to be a disaster for your project.