AlexNet implementation in TensorFlow using Python

In this tutorial, I will teach you about the implementation of AlexNet, in TensorFlow using Python. AlexNet is first used in a public scenario and it showed how deep neural networks can also be used for image classification tasks. Click here for an in-depth understanding of AlexNet. Click here if you want to check the CIFAR10 dataset in detail.

 

I will provide the implementation of the tutorial in the snippets below.

1. Installing Dependencies

import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow import keras
import os
import time

The above snippet consists of the dependencies that you have to load into your system. Coming to the dataset, I have decided to use the CIFAR10 dataset. The reason I chose this dataset was that it has a lot of images to train on and a good number of classes. I chose CIFAR10 dataset as you can load it directly using Keras.

Look at the Python code below:

(traindata, trainoutput), (testdata, testoutput) = keras.datasets.cifar10.load_data()
CLASS_NAMES= ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog','horse', 'ship', 'truck']
validationdata, validationoutput = traindata[:5000], trainoutput[:5000]
traindata, trainoutput= traindata[5000:], trainoutput[5000:]
train_ds = tf.data.Dataset.from_tensor_slices((traindata, trainoutput))
test_ds = tf.data.Dataset.from_tensor_slices((testdata, testoutput))
validation_ds = tf.data.Dataset.from_tensor_slices((validationdata,validationoutput))

#Preprocessing

def process_images(image, label):
    image = tf.image.per_image_standardization(image)
    image = tf.image.resize(image, (227,227))
    return image, label

2. Loading Data

The above snippet gives you details about loading the dataset and preprocessing the data. In the next snippet, I will teach you about the pipeline that I created for the implementation.

train_data_size = tf.data.experimental.cardinality(train_data).numpy()
test_data_size = tf.data.experimental.cardinality(test_data).numpy()
validation_data_size = tf.data.experimental.cardinality(validation_data).numpy()

train_data = (train_data
                  .map(process_images)
                  .batch(batch_size=64, drop_remainder=True)
                  .shuffle(buffer_size=train_data_size))
test_data = (test_data
                  .map(process_images)
                  .batch(batch_size=64, drop_remainder=True)
                  .shuffle(buffer_size=train_data_size))
validation_data = (validation_data
                  .map(process_image)
                  .batch(batch_size=64, drop_remainder=True)
                  .shuffle(buffer_size=train_data_size))

I have provided the component of pipelines in the above snippet. I have not only preprocessed the images but also divided them into batches and shuffled them so that I can get beter accuracy from the model. In the next snippet, I coded the architectural design of the AlexNet formed using TensorFlow and Keras. As this is an already created network, this architecture might be seen in other sources as well.

3. Creating the Architecture

model = keras.models.Sequential([
    keras.layers.Conv2D(filters=96, kernel_size=(11,11), strides=(4,4), activation='relu', input_shape=(227,227,3)),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),
    keras.layers.Conv2D(filters=256, kernel_size=(5,5), strides=(1,1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),
    keras.layers.Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.Conv2D(filters=384, kernel_size=(1,1), strides=(1,1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.Conv2D(filters=256, kernel_size=(1,1), strides=(1,1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3,3), strides=(2,2)),
    keras.layers.Flatten(),
    keras.layers.Dense(4096, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(4096, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(10, activation='softmax')
])

model.compile(loss='sparse_categorical_crossentropy', optimizer=tf.optimizers.SGD(lr=0.001), metrics=['accuracy'])
model.summary()

The above snippet explains to you about the AlexNet in a more in-depth manner. I have created the AlexNet architecture using the neural networks that are present with TensorFlow and Keras.

As there were more than 2 output labels, I have used softmax as the activation layer for the last network layer. After doing trial and error, I have concluded that SGD was a better optimizer for this problem and sparse categorical cross entropy a better loss function. After running the model, I achieved around 80%, which is exceptional when taking deep learning and huge data information.