How to get the output of each layer in Keras

Hey Everyone,
While implementing a Deep Learning Model we build a Neural Network which has a variety of layers within. These are a complex network of perceptrons that performs more reliable calculations as compared to a Machine Learning Model.
Ever wondered how the output on each layer in the Neural Network might look like?.
In this particular example, We will be building a Neural Network Model for the very popular dataset which also known as Hello World of the NLP (Natural Language Processing) with the Keras Python library.

We will be looking at multiple Handwritten numbers from 0 to 9 and predicting the number.
Visualize what the Output looks like at the intermediate layer, Look at its Weight, Count Params, and Look at the layer Summary.
We will actually be visualizing the result after each Activation Layer.

Modules required are:

  • NumPy
  • Tensorflow 2.0.0
  • Keras
  • MatplotLib

In case if you don’t have the following modules, we need to install the following Libraries by typing in the Python virtual environment:

pip install numpy
pip install tensorflow=2.0.0
pip install keras
pip install matplotlib

Let’s Hop into the Code:

We will be Importing all the Required Libraries.

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

print("All the Required Modules have been Imported")
print("The Tensorflow Version we are using is: "+tf.__version__)

Output:

All the Required Modules have been Imported
The Tensorflow Version we are using is: 2.0.0

Note: The Tensorflow Version we are using is 2.0.0. This might not work in a different version of TensorFlow.

Let us Load the MNIST Dataset.

(x_train,y_train),(x_test,y_test)=tf.keras.datasets.mnist.load_data()
print("MNIST Dataset has been Loaded")

Output:

MNIST Dataset has been Loaded

Now we have split the MNIST dataset into Training and Testing Data.
To have a Look at the image of training data.

plt.imshow(x_train[4])

Output:

x_train(color)

Let us look at how the Binary Image looks like.

plt.imshow(x_train[4], cmap=plt.cm.binary)

Output:

x_train(bw)

Let us Explore the shape and size of the “x_train”

print("Shape of the x_train "+str(x_train.shape)+" Type is: "+str(type(x_train[2])))

Output:

Shape of the x_train (60000, 28, 28) Type is: <class 'numpy.ndarray'>

Let us reshape the train and test data and convert the data type into float.

x_train=x_train.reshape(x_train.shape[0],28,28,1)
x_test=x_test.reshape(x_test.shape[0],28,28,1)

x_train=x_train.astype('float32')
x_test=x_test.astype('float32')

x_train=x_train/255
x_test=x_test/255

Let us now build a model for this dataset and look at its summary.

input_shape=(28,28,1)


model=tf.keras.models.Sequential()

model.add(tf.keras.layers.Conv2D(32,(3,3),activation='relu',
                              input_shape=input_shape,name='input_layer'))
model.add(tf.keras.layers.Conv2D(64,(3,3),activation='relu',name='conv_1'))

model.add(tf.keras.layers.MaxPooling2D(pool_size=(2,2),name='pool_1'))
model.add(tf.keras.layers.Dropout(0.25,name='dropout_1')) 

model.add(tf.keras.layers.Flatten(name='flate_1'))

model.add(tf.keras.layers.Dense(128,activation='relu',name='dense_1'))
model.add(tf.keras.layers.Dropout(0.5,name='dropout_2')) 

model.add(tf.keras.layers.Dense(10,activation='softmax',name='output_layer'))
print("Model is now Ready to Use")
print("----------------------------------------------------------------------------------------------------------------------")

model.summary()

Output:

summary

We will now compile and fit the model

model.compile(loss=tf.keras.losses.sparse_categorical_crossentropy,optimizer='adam',metrics=['accuracy'])
#Fit the Model
model.fit(x_train,y_train,batch_size=128,epochs=5,validation_data=(x_test,y_test))

Output:

epochs

Let us Evaluate the Model:

score=model.evaluate(x_test,y_test)
model.layers[0]._name='conv_0'  # we have changed the name of the layer from input_layer to conv_0
print('Test loss',score[0])
print('Test accuracy',score[1])

Output:

Test loss 0.026050921789544372
Test accuracy 0.9917

Let us create a function that will enable us to visualize the resulting output of each layer. The function looks like this.

def visualize_conv_layer(layer_name):
  
  layer_output=model.get_layer(layer_name).output  #get the Output of the Layer

  intermediate_model=tf.keras.models.Model(inputs=model.input,outputs=layer_output) #Intermediate model between Input Layer and Output Layer which we are concerned about

  intermediate_prediction=intermediate_model.predict(x_train[4].reshape(1,28,28,1)) #predicting in the Intermediate Node
  
  row_size=4
  col_size=8
  
  img_index=0

  print(np.shape(intermediate_prediction))
    #---------------We will subplot the Output of the layer which will be the layer_name----------------------------------#
  
  fig,ax=plt.subplots(row_size,col_size,figsize=(10,8)) 

  for row in range(0,row_size):
    for col in range(0,col_size):
      ax[row][col].imshow(intermediate_prediction[0, :, :, img_index], cmap='gray')

      img_index=img_index+1 #Increment the Index number of img_index variable
        
print("Function to Visualize the Output has been Created")

Output:

Function to Visualize the Output has been Created

Now we have trained the model and seen the test Accuracy which is 0.9917 which is quite Impressive.
In case if you wish to know the weight of perceptrons in the Layer we can type the following code.

print("These are the weights of a Layer")
print("----------------------------------------------------------------------")
model.layers[0].get_weights()

I would highly recommend my readers of this article to run this part of code on your own and examine its Output.

Let us look at the Output of the Intermediate First activation layer

visualize_conv_layer('conv_0')

Output:

 

Let’s visualize the output of the Second Activation Layer, we will be replacing the conv_0 with the desired layer name mentioned in the model.summary().
If you are wondering where did the conv_0 came from, refer the evaluate section above.
Replace conv_0 with conv_1 to visualize the output of the Second Activation layer.

visualize_conv_layer('conv_1')

Output:

 

I hope you had a great learning journey.
I highly recommend my readers to play with the program by replacing the name of the layer in the “visualize_conv_layer” function and the visualizing the Intermediate layer output.

So we have learned how to get the output of each layer in the Keras Python library. I hope, it will be helpful to you.
Happy Coding  💻📈

Leave a Reply

Your email address will not be published. Required fields are marked *