Rock Paper Scissor prediction using Keras backend

rows and columns images in the dataset printed

In this tutorial, I have trained a simple CNN for predicting between Rock Paper and Scissor from hand images. I have made this tutorial as an intro to CNN in Keras/TensorFlow.

I have previously made another tutorial on how to classify if an x-ray of chest contains signs of covid-19 or not, to check that tutorial please follow the following link:⇒ here

Before you start following along, please make sure you have compatible keras>2.0 and tensorflow>2.0 installed and working in your system.

Now to start with the coding tutorial:

Import libraries:

import os
import PIL
from PIL import Image
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as npimg
import random
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input

We will first import all the required libraries for the specific tutorial. Most crucial being PIL for using an image in python. Keras for deep-learning framework application for TensorFlow backend. Matplotlib for observing the statistics of our model.

Getting the Data:

For this project, I have utilized the Kaggle rock paper scissor dataset. The dataset can be found here.:⇒ https://www.kaggle.com/sanikamal/rock-paper-scissors-dataset.

Declaring the paths:

Now we will use the paths of the images as a variable to make our job easy.

rockDir_train = os.path.join('dataset/Rock-Paper-Scissors/train/rock')
paperDir_train = os.path.join('dataset/Rock-Paper-Scissors/train/paper')
scissorDir_train = os.path.join('dataset/Rock-Paper-Scissors/train/scissors')
print('Total training rock images: ', len(os.listdir(rockDir_train)))
print('Total training paper images: ', len(os.listdir(paperDir_train)))
print('Total training scissor images: ', len(os.listdir(scissorDir_train)))

 

The number of images in training set

Here we have printed the number of images that consists of the training folder for each of the classes.

 

rockDir_test = os.path.join('dataset/Rock-Paper-Scissors/test/rock')
paperDir_test = os.path.join('dataset/Rock-Paper-Scissors/test/paper')
scissorDir_test = os.path.join('dataset/Rock-Paper-Scissors/test/scissors')
print('Total validation rock images: ', len(os.listdir(rockDir_test)))
print('Total validation paper images: ', len(os.listdir(paperDir_test)))
print('Total validation scissor images: ', len(os.listdir(scissorDir_test)))
number of images in test directory

The number of images that are inside the test folder prints out.

As you can see there are about 800+ images in the training dataset and around 124+ for the testing dataset(For each of the conditions)

We can say that this is a nice amount of data for a decent performance model. Now we will look at some of the images from the dataset.

I have written the below code to make it simpler to print random images from the dataset for observations.

 

rockTestName=os.listdir(rockDir_test)
paperTestName=os.listdir(paperDir_test)
scissorDirName=os.listdir(scissorDir_test)

#Declaring parameters for rows and columns
n_rows=3
n_columns=4

#Picking random no. of images for total 
rock_pic_index =  random.randrange(1,836)
paper_pic_index = random.randrange(1,836)
scissors_pic_index = random.randrange(1,836)

# Set up a matploit fig and size it to fit 4*3
fig = plt.gcf()
fig.set_size_inches(16,16)

next_rock = [os.path.join(rockDir_train, fname)
             for fname in rockTrainName[rock_pic_index:rock_pic_index+4]]

next_paper = [os.path.join(paperDir_train, fname)
             for fname in paperTrainName[paper_pic_index:paper_pic_index+4]]

next_scissors = [os.path.join(scissorDir_test, fname)
             for fname in scissorsTrainName[scissors_pic_index:scissors_pic_index+4]]


for i, img_path in enumerate(next_rock+ next_paper+ next_scissors):
    sp=plt.subplot(n_rows,n_columns,i+1)
    sp.axis('off')# Grid type not shown
    img = npimg.imread(img_path)
    plt.imshow(img)
    
plt.show()

Running the above code will return you with a set of 12 images from the dataset with 4 images for each case. The below image here depicts how the dataset should look like.

rows and columns images in the dataset printed

The sample of images present in the dataset

Defining the model:

I have explained this in much more depth in my previous tutorial, but we will use Conv2D and MaxPooling2D layer to train our images, we will use flatten to convert the output into 1d array and dropout to reduce overfitting.

model = tf.keras.models.Sequential([tf.keras.layers.Conv2D(64,(3,3),activation = 'relu', input_shape=(150,150,3)),
                                   tf.keras.layers.MaxPooling2D(2,2),
                                   tf.keras.layers.Conv2D(128,(3,3),activation = 'relu'),
                                   tf.keras.layers.MaxPooling2D(2,2),
                                   tf.keras.layers.Conv2D(128,(3,3),activation = 'relu'),
                                   tf.keras.layers.MaxPooling2D(2,2),
                                   tf.keras.layers.Flatten(),
                                   tf.keras.layers.Dropout(0.5),
                                   tf.keras.layers.Dense(512, activation='relu'),
                                   tf.keras.layers.Dense(3, activation='softmax')])

model.summary()

The model should look like this:

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 148, 148, 64)      1792      
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 74, 74, 64)        0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 72, 72, 128)       73856     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 36, 36, 128)       0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 34, 34, 128)       147584    
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 17, 17, 128)       0         
_________________________________________________________________
flatten (Flatten)            (None, 36992)             0         
_________________________________________________________________
dropout (Dropout)            (None, 36992)             0         
_________________________________________________________________
dense (Dense)                (None, 512)               18940416  
_________________________________________________________________
dense_1 (Dense)              (None, 3)                 1539      
=================================================================
Total params: 19,165,187
Trainable params: 19,165,187
Non-trainable params: 0

In the next step, we will now prepare for the compilation of our project. We will use the rmsprop instead of adam to observe the error in terms of a beginner, please be sure to check all the other optimizers out there. We will measure loss with categorical cross-entropy as the number of classes in our use case is more than 2. Lastly, we wanna get hold of only the accuracy.

from tensorflow.keras.optimizers import RMSprop
model.compile(optimizer= 'rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])

Data Augmentation:

As I had said earlier, we would require augmentation to be done in the dataset so that we will not run into overfitting, we will use the ImageDataGenerator function from Keras preprocessing library with the following settings.

from tensorflow.keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale=1./225,
                                   zoom_range=0.2,
                                   width_shift_range=0.2,
                                   height_shift_range=0.2,
                                   rotation_range=40,
                                   shear_range=0.2,
                                   horizontal_flip=True,
                                   fill_mode='nearest')
validation_datagen = ImageDataGenerator(rescale=1./225)

And now will pass our training and validation data set into the system so we can get augmented results.

training_set = train_datagen.flow_from_directory('dataset/Rock-Paper-Scissors/train/',target_size=(150,150),class_mode='categorical')

validation_set = train_datagen.flow_from_directory('dataset/Rock-Paper-Scissors/test/',target_size=(150,150),class_mode='categorical')

Start Training:

We will use the .fit_generator command to start the training process of the model.

We will run the training for 20 epochs as that should be enough to attain a nice amount of accuracy.

history=model.fit_generator(training_set,epochs=20,
                                 validation_data = validation_set,verbose=1)

Observing the accuracy:

Now that training has been finished, I am using the matplotlib library to plot the graph for my accuracy with each epoch. Run the following block of code to do the same.

#Varation in this graph is due to overfitting to reduce overfitting use droupout
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))
plt.plot(epochs, acc, 'r',label = 'Training Accuracy')
plt.plot(epochs, val_acc, 'b', label = 'Validation Accuracy')
plt.title('Training and Validation Accuracy')
plt.legend(loc=0)
plt.figure()
plt.show()
The accuracy Graph

This is the matplotlib graph

 

As you can see the accuracy started dipping near the very end, we can retrain the model with the newly acquired information but for now, we will let it slide.

 

Testing the Model:

Now that you have completed training the model, we will now use PIL image class to it with unseen images.

test_image = image.load_img('dataset/Rock-Paper-Scissors/validation/rock5.png', target_size=(150,150))
test_image

The following code will return a rock image

Rock image test

We need to first change the dimensions to the required inputs for the model so we can run inference. I have done that below:

x=image.img_to_array(test_image)
x=np.expand_dims(x,axis=0)



#x=preprocess_input(x)   # preprocessing with vgg16
images = np.vstack([x])
classes = model.predict(images,batch_size=10)

classAll=['paper','rock','scissors']
def tell(classes):
    print("It is a "+classAll[np.argmax(classes)])

tell(classes)

If all works properly, we will have the prediction printed like this!

It is a rock

 

Leave a Reply

Your email address will not be published. Required fields are marked *