Create a simple Recurrent Neural Network using Keras

Hello learners! Welcome to the tutorial on creation of a simple RNN (Recurrent Neural Network) using Keras TensorFlow API with the help of Python programming. We will first begin by understanding RNN.

 

What is RNN?

Previously in the feedforward neural network, there was only a single output, and it did not have ‘feedback’ loops. Due to which it faces a lot of problem in those inputs which are sequential.

 

The best example is that the feed-forward neural network usage used to recognize any animal which is at rest. However, when it starts to move then it becomes difficult for that neural network to determine whether it is moving or not. Hence to solve that, we will need ‘feedback’ of previous output. When the feedback is compared with the current input, then the model can decide whether the animal is moving or not. Hence, this is where the Recurrent Neural Network is used. RNN works with sequential data and has a sequential memory.

 

Types of RNNs

1. One to Many: Used for Image recognition
2. Many to one: Used for sentiment analysis (which means from a given text it analyses the mood of the person).
3. Many to many: It is widely used in language translation

 

This is a single RNN representation. Here the input along with its weight is given and we get the output along with its weight.

 

Here we will use gradient descent and also back-propagation method through time.

 

Hands-on RNN creation

In this hands-on tutorial, you are required to open you Google Colab notebook where you will be available with the Tensorflow and Keras libraries. In this tutorial, we will find the clear sine function out of a noisy sine function used RNN.

 

Steps involved:

Generating the data set –> Prepare the dataset for training –> Train the model –> Plot the model.

Python Code

Here we will begin writing the codes for our RNN.

We will first begin with installing the Python libraries:

import numpy as np
import matplotlib.pyplot as plt

Then in the next step, we will generate the dataset. For generating the dataset, we will begin with the creation of the sine wave function. Below is given the Python program for this task:

np.random.seed(0)
#We generate the seed so that the random value is reproducable
t = np.arange(0,1500)
x = np.sin(0.02*t) + np.random.uniform(low = -1, high = 1, size = (1500,))
#x represents the noisy sine function

plt.plot(x) #plots the noisy sine wave

#we will also print the clear clear sine wave
x_not_noisy = np.sine(0,1500)
plt.plot(x_not_noisy)

Here is the noisy waveform with the yellow wave representing the clear sine wave function.

 

Now we will try to gather the data from the noisy sine wave function.

#We will begin by importing the MaxMinScaler
from sklearn.preprocessing import MinMaxScaler

#then we will gather the data
normalizer = MinMaxScaler(feature_range=(0, 1))
x = (np.reshape(x, (-1, 1)))
x = normalizer.fit_transform(x)

#putting condition for the train and test set

train = x[0:1000]
test = x[1000:]

We will then create the dataset

def createDataset(data, step):
    X, Y =[], []
    for i in range(len(data)-step): 
        X.append(data[i:i+step])
        Y.append(data[i+step])
    return np.array(X), np.array(Y)
 
step = 10
trainX,trainY = createDataset(train,step)
testX,testY = createDataset(test,step)

step = 10
trainX,trainY = createDataset(train,step)
testX,testY = createDataset(test,step)

We will then import the libraries for Keras.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, SimpleRNN

Now it is time to train the data set:

model = Sequential()
model.add(SimpleRNN(units=1, activation="tanh"))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
history = model.fit(trainX,trainY, epochs=500, batch_size=16, verbose=2)

Now we have to wait till the gradient descent happens:

 

After this, we will find out the loss curve which occurred during gradient descent.

import matplotlib.pyplot as plt
 
loss = history.history['loss']
 
plt.plot(loss)

Here is the gradient descent curve using matplotlib.

We will transform and concatenate the train and test predictions into a single variable predict:

trainPredict = normalizer.inverse_transform(model.predict(trainX))
testPredict= normalizer.inverse_transform(model.predict(testX))
predicted= np.concatenate((trainPredict,testPredict))
x = normalizer.inverse_transform(x)

And here we come to our final step of plotting the desired output:

plt.plot(x)
plt.plot(predicted)

Output:

 

Hence, we reach to the end of our tutorial. Thank you for reading. Happy Learning!!!