Fake News Classifier using LSTM In Python
By Prachi ShrivastavaIn this article, We are going to discuss building a fake news classifier. For this task, we will use LSTM(Long Short- Term Memory). We will use LSTM because these networks are great in dealing with long term dependencies. The classifier will give an output 0(Fake News),1(Real News).In a world full of information where some information can be quite misleading, it’s essential to know the authenticity. So let’s get straight to it.
You can get the dataset from https://www.kaggle.com/clmentbisaillon/fake-and-real-news-dataset
ABOUT LSTM
LSTM(Long Short-Term Memory) is mainly used when we need to deal with sequential data. A simple vanilla neural network does not have a memory state. But when we are dealing with sequences it becomes important to incorporate data from the previous timestamp as well. LSTM has a memory state and is useful for dealing with long sequences of information. Therefore it is very useful.
DATA PREPARATION
IMPORTING LIBRARIES
!pip install plotly
!pip install --upgrade nbformat
!pip install nltk
!pip install spacy # spaCy is an open-source software library for advanced natural language processing
!pip install WordCloud
!pip install gensim # Gensim is an open-source library for unsupervised topic modeling and natural language processing
import nltk
nltk.download('punkt')
import tensorflow as tf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from wordcloud import WordCloud, STOPWORDS
import nltk
import re
from nltk.stem import PorterStemmer, WordNetLemmatizer
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize, sent_tokenize
import gensim
from gensim.utils import simple_preprocess
from gensim.parsing.preprocessing import STOPWORDS
# import keras
from tensorflow.keras.preprocessing.text import one_hot, Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Embedding, Input, LSTM, Conv1D, MaxPool1D, Bidirectional
from tensorflow.keras.models import Model
That is a whole lot of libraries but we will implement each and learn their use. Libraries like spacy,gensim,nltk are very important in NLP(Natural Language Processing).
DATA ANALYSIS
Let’s start by reading the data from the files.
df_true = pd.read_csv("True.csv")
df_fake = pd.read_csv("Fake.csv")
Adding a target column to classify fake news and real news distinctly.
df_true['isfake'] = 1 df_true.head()
OUTPUT
df_fake['isfake'] = 0 df_fake.head()
OUTPUT
To make our task easier we merge both the data frames containing fake and real news.
df = pd.concat([df_true, df_fake]).reset_index(drop = True) df
OUTPUT
We will make some changes to the data by dropping the date column and combining the title and text columns.
df.drop(columns = ['date'], inplace = True) df['original'] = df['title'] + ' ' + df['text']
df['original'][0]
OUTPUT
'As U.S. budget fight looms, Republicans flip their fiscal script WASHINGTON (Reuters) - The head of a conservative Republican faction in the U.S. Congress, who voted this month for a huge expansion of the national debt to pay for tax cuts, called himself a “fiscal conservative” on Sunday and urged budget restraint in 2018. In keeping with a sharp pivot under way among Republicans, U.S. Representative Mark Meadows, speaking on CBS’ “Face the Nation,” drew a hard line on federal spending, which lawmakers are bracing to do battle over in January. When they return from the holidays on Wednesday, lawmakers will begin trying to pass a federal budget in a fight likely to be linked to other issues, such as immigration policy, even as the November congressional election campaigns approach in which Republicans will seek to keep control of Congress. President Donald Trump and his Republicans want a big budget increase in military spending, while Democrats also want proportional increases for non-defense “discretionary” spending on programs that support education, scientific research, infrastructure, public health and environmental protection. “The (Trump) administration has already been willing to say: ‘We’re going to increase non-defense discretionary spending ... by about 7 percent,’” Meadows, chairman of the small but influential House Freedom Caucus, said on the program.
DATA CLEANING
Now comes a crucial task of data cleaning where we will make the model suitable for training.
nltk.download("stopwords")
Stop words are usually the words which the model does not consider useful for training. Some examples can be
(a, an, the).
from nltk.corpus import stopwords
stop_words = stopwords.words('english')
stop_words.extend(['from', 'subject', 're', 'edu', 'use'])
We can extend our understanding of stop words by putting additional words in the corpus. After this, we will remove the stop words from our data.
def preprocess(text):
result = []
for token in gensim.utils.simple_preprocess(text):
if token not in gensim.parsing.preprocessing.STOPWORDS and len(token) > 3 and token not in stop_words:
result.append(token)
return result
Now let’s apply the above function to our data.
df['clean'] = df['original'].apply(preprocess)
We will see that the data is cleaned and stop words are removed.
print(df['clean'][0])
OUTPUT
['budget', 'fight', 'looms', 'republicans', 'flip', 'fiscal', 'script', 'washington', 'reuters', 'head', 'conservative', 'republican', 'faction', 'congress', 'voted', 'month', 'huge', 'expansion', 'national', 'debt', 'cuts', 'called', 'fiscal', 'conservative', 'sunday', 'urged', 'budget', 'restraint', 'keeping', 'sharp', 'pivot', 'republicans', 'representative', 'mark', 'meadows', 'speaking', 'face', 'nation', 'drew', 'hard', 'line', 'federal', 'spending', 'lawmakers', 'bracing', 'battle', 'january', 'return', 'holidays', 'wednesday', 'lawmakers', 'begin', 'trying', 'pass', 'federal', 'budget', 'fight', 'likely', 'linked', 'issues', 'immigration', 'policy', 'november', 'congressional', 'election', 'campaigns', 'approach', 'republicans', 'seek', 'control', 'congress', 'president', 'donald', 'trump', 'republicans', 'want', 'budget', 'increase', 'military', 'spending', 'democrats', 'want', 'proportional', 'increases', 'defense', 'discretionary', 'spending', 'programs', 'support', 'education', 'scientific', 'research', 'infrastructure', 'public', 'health', 'environmental', 'protection', 'trump', 'administration', 'willing', 'going', 'increase', 'defense', 'discretionary', 'spending', 'percent', 'meadows', 'chairman', 'small', 'influential', 'house', ]
Let’s now see the number of words we are dealing with.
list_of_words = []
for i in df.clean:
for j in i:
list_of_words.append(j)
list_of_words
OUTPUT
['budget', 'fight', 'looms', 'republicans', 'flip', 'fiscal', 'script', 'washington', 'reuters', 'head', 'conservative', 'republican', 'faction', 'congress',....]
Next,we will add the cleaned data to our data frame after the stop words have been removed.
df['clean_joined'] = df['clean'].apply(lambda x: " ".join(x)) df
OUTPUT
VISUALISE THE DATA
We will use a powerful visualisation technique called word cloud which is used in NLP. The basic use of a word cloud is to visualise the frequently used words. The larger the text corresponds to the more number of times it is being used.
plt.figure(figsize = (15,15))
wc = WordCloud(max_words = 1800 , width = 1500 , height = 700 , stopwords = stop_words).generate(" ".join(df[df.isfake == 1].clean_joined))
plt.imshow(wc, interpolation = 'bilinear')
OUTPUT
TOKENIZATION AND PADDING
Now we will convert the sentences into tokens. We will use a tokenizer and these words are going to be converted into integers to train the model. Also, We know that the classifier cannot train on text data therefore tokenizing is important.
Before tokenizing, we are going to split the data into training and test set using scikit-learn.
from sklearn.model_selection import train_test_split x_train, x_test, y_train, y_test = train_test_split(df.clean_joined, df.isfake, test_size = 0.2)
Also, Let’s Tokenise now.
from nltk import word_tokenize tokenizer = Tokenizer(num_words = total_words) tokenizer.fit_on_texts(x_train) train_sequences = tokenizer.texts_to_sequences(x_train) test_sequences = tokenizer.texts_to_sequences(x_test)
Let’s now explore padding. Our training and testing sentences vary in length, therefore, we will use padding to reduce the size of the longer sentences. For this, we are going to specify a maximum length also the shorter sentences will be stretched.
padded_train = pad_sequences(train_sequences,maxlen = 40, padding = 'post', truncating = 'post') padded_test = pad_sequences(test_sequences,maxlen = 40, truncating = 'post')
for i,doc in enumerate(padded_train[:2]):
print("The padded encoding for document",i+1," is : ",doc)
OUTPUT
The padded encoding for document 1 is : [ 2365 558 332 2311 2716 42 972 27 11043 950 513 120
258 57 30 558 332 6402 972 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0]
The padded encoding for document 2 is : [ 49 183 5 3537 231 75 4423 20 877 694 751 4037
16 12 20 278 316 694 751 838 38 204 23342 844
1023 694 49568 9060 4 4423 348 4631 352 98 45 20
521 694 751 355]
As we can see that there are 0’s inserted. This is going to stretch the shorter sentences.
BUILDING AND TRAINING THE MODEL
model = Sequential() model.add(Embedding(total_words, output_dim = 128)) model.add(Bidirectional(LSTM(128))) model.add(Dense(128, activation = 'relu')) model.add(Dense(1,activation= 'sigmoid')) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc']) model.summary()
We are now going to explore the layers:
Embedding Layer: It transforms the words into vectors. It is also useful to find the relationship between similar words.
Bidirectional LSTM: After observing the model’s performance we will find that it is better with bidirectional LSTM.This is because it is going to deal with data from both the previous and future time stamps.
Sigmoid Activation: This is used because our output is a binary classification problem and only one neuron is used as output neuron.
OUTPUT
Model: "sequential_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= embedding_3 (Embedding) (None, None, 128) 13914112 _________________________________________________________________ bidirectional_3 (Bidirection (None, 256) 263168 _________________________________________________________________ dense_6 (Dense) (None, 128) 32896 _________________________________________________________________ dense_7 (Dense) (None, 1) 129 ================================================================= Total params: 14,210,305 Trainable params: 14,210,305 Non-trainable params: 0 _____________________________
y_train = np.asarray(y_train)
In the next step, we are going to train the model.
model.fit(padded_train, y_train, batch_size = 64, validation_split = 0.1, epochs = 2)
Here we are also going to observe that we are using only 2 epochs this is how powerful bidirectional LSTM is.validation_split is used for checking if we are overfitting.
OUTPUT
Train on 32326 samples, validate on 3592 samples Epoch 1/2 32326/32326 [==============================] - 321s 10ms/sample - loss: 0.0421 - acc: 0.9815 - val_loss: 0.0073 - val_acc: 0.9992 Epoch 2/2 32326/32326 [==============================] - 316s 10ms/sample - loss: 0.0016 - acc: 0.9997 - val_loss: 0.0096 - val_acc: 0.9981
We get a validation accuracy of 99.8% which is very good for our model to make predictions.
pred = model.predict(padded_test)
prediction = []
for i in range(len(pred)):
if pred[i].item() > 0.5:
prediction.append(1)
else:
prediction.append(0)
We then create a list. Where if the model predicts a value greater than 0.5 it is real news else it is fake.
Thanks for reading! You can also check out
Building A Movie Review Classifier Using Tensorflow And Keras
Leave a Reply