How to get the output of each layer in Keras

In the time of implementing a Deep Learning Model we build a Neural Network which has a variety of layers within it. These are a complex network of perceptrons that performs more reliable calculations as compared to a Machine Learning Model.

Ever wondered how the output on each layer in the Neural Network might look like? I hope you would like to dig into this.
In this particular example, we will be building a Neural Network Model for the very popular dataset which is also known as Hello World of the NLP (Natural Language Processing) with the Keras Python library. Keras is the deep learning API built on top of TensorFlow.

We will be looking at multiple Handwritten numbers from 0 to 9 and predicting the number.
After that, visualize what the Output looks like at the intermediate layer, look at its Weight, count params, and look at the layer summary.
We will actually be visualizing the result after each Activation Layer.

Python Modules required are for this tutorial:

• NumPy
• Tensorflow 2.0.0
• Keras
• MatplotLib

In case if you don’t have the following modules, we need to install the following Libraries by typing in the Python virtual environment:

pip install numpy

pip install tensorflow=2.0.0

pip install keras

pip install matplotlib

Let’s Dig into the Code:

We will be Importing all the Required Libraries.

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

print("All the Required Modules have been Imported")
print("The Tensorflow Version we are using is: "+tf.__version__)

Output:

All the Required Modules have been Imported
The Tensorflow Version we are using is: 2.0.0

Note: The Tensorflow Version we are using is 2.0.0. This might not work in a different version of TensorFlow.

Let us Load the MNIST Dataset.

(x_train,y_train),(x_test,y_test)=tf.keras.datasets.mnist.load_data()
print("MNIST Dataset has been Loaded")

Output:

MNIST Dataset has been Loaded

Now we have split the MNIST dataset into Training and Testing Data.
To have a Look at the image of training data.

plt.imshow(x_train[4])

Output:

Let us look at what the Binary Image looks like.

plt.imshow(x_train[4], cmap=plt.cm.binary)

Output:

Let us Explore the shape and size of the “x_train”

print("Shape of the x_train "+str(x_train.shape)+" Type is: "+str(type(x_train[2])))

Output:

Shape of the x_train (60000, 28, 28) Type is: <class 'numpy.ndarray'>

Next, lets reshape the train and test data and convert the data type into float.

x_train=x_train.reshape(x_train.shape[0],28,28,1)
x_test=x_test.reshape(x_test.shape[0],28,28,1)

x_train=x_train.astype('float32')
x_test=x_test.astype('float32')

x_train=x_train/255
x_test=x_test/255

After that, let us now build a model for this dataset and look at its summary.

input_shape=(28,28,1)


model=tf.keras.models.Sequential()

model.add(tf.keras.layers.Conv2D(32,(3,3),activation='relu',
                              input_shape=input_shape,name='input_layer'))
model.add(tf.keras.layers.Conv2D(64,(3,3),activation='relu',name='conv_1'))

model.add(tf.keras.layers.MaxPooling2D(pool_size=(2,2),name='pool_1'))
model.add(tf.keras.layers.Dropout(0.25,name='dropout_1')) 

model.add(tf.keras.layers.Flatten(name='flate_1'))

model.add(tf.keras.layers.Dense(128,activation='relu',name='dense_1'))
model.add(tf.keras.layers.Dropout(0.5,name='dropout_2')) 

model.add(tf.keras.layers.Dense(10,activation='softmax',name='output_layer'))
print("Model is now Ready to Use")
print("----------------------------------------------------------------------------------------------------------------------")

model.summary()

After running the above program, below is the output response:

We will now compile and fit the model

model.compile(loss=tf.keras.losses.sparse_categorical_crossentropy,optimizer='adam',metrics=['accuracy'])
#Fit the Model
model.fit(x_train,y_train,batch_size=128,epochs=5,validation_data=(x_test,y_test))

Output:

Let us Evaluate the Model:

score=model.evaluate(x_test,y_test)
model.layers[0]._name='conv_0'  # we have changed the name of the layer from input_layer to conv_0
print('Test loss',score[0])
print('Test accuracy',score[1])

Output:

Test loss 0.026050921789544372
Test accuracy 0.9917

Now, let us create a function that will enable us to visualize the resulting output of each layer. The function looks like this.

def visualize_conv_layer(layer_name):
  
  layer_output=model.get_layer(layer_name).output  #get the Output of the Layer

  intermediate_model=tf.keras.models.Model(inputs=model.input,outputs=layer_output) #Intermediate model between Input Layer and Output Layer which we are concerned about

  intermediate_prediction=intermediate_model.predict(x_train[4].reshape(1,28,28,1)) #predicting in the Intermediate Node
  
  row_size=4
  col_size=8
  
  img_index=0

  print(np.shape(intermediate_prediction))
    #---------------We will subplot the Output of the layer which will be the layer_name----------------------------------#
  
  fig,ax=plt.subplots(row_size,col_size,figsize=(10,8)) 

  for row in range(0,row_size):
    for col in range(0,col_size):
      ax[row][col].imshow(intermediate_prediction[0, :, :, img_index], cmap='gray')

      img_index=img_index+1 #Increment the Index number of img_index variable
        
print("Function to Visualize the Output has been Created")

Output:

Function to Visualize the Output has been Created

Now we have trained the model and seen the test Accuracy which is 0.9917 which is quite Impressive.
In case you wish to know the weight of perceptrons in the Layer we can type the following Python program as given below:

print("These are the weights of a Layer")
print("----------------------------------------------------------------------")
model.layers[0].get_weights()

I would highly recommend my readers of this article to run this part of code on your own and examine the Output of this program.

Now it’s time to look at the Output of the Intermediate First activation layer. Below is how we can perform it:

visualize_conv_layer('conv_0')

Output:

 

Let’s visualize the output of the Second Activation Layer, we will be replacing the conv_0 with the desired layer name mentioned in the model.summary().
If you are wondering where did the conv_0 came from, refer the evaluate section above.
Replace conv_0 with conv_1 to visualize the output of the Second Activation layer.

visualize_conv_layer('conv_1')

Output:

 

I hope you had a great learning journey.
I highly recommend my readers to play with the program by replacing the name of the layer in the “visualize_conv_layer” function and visualizing the Intermediate layer output. It will improve your skill and help you to become more fluent in the machine learning field.

So we have learned how to get the output of each layer in the Keras Python library. I hope, it will be helpful to you.
Happy Coding  ????????