# Logistic Regression in TensorFlow

Logistic Regression is one of the most used Algorithms in Machine Learning. In this tutorial, Let’s Implement it using TensorFlow with Python programming.

## Introduction

Unlike linear regression, logistic regression is used to predict categories. Clearly, we use the so-called logistic function or sigmoid. This function takes a value between 0 and 1. Indeed, we can define a threshold to predict the output. It helps us in solving supervised learning problems.

import matplotlib.pyplot as plt import numpy as np import math x = np.linspace(-10, 10, 100) z=1/(1+np.exp(-x)) plt.plot(x, z) plt.xlabel("x") plt.ylabel("Sigmoid(X)") plt.show()

Firstly, the weights are initialized with random values at the beginning of the training. Then, we find the error in our model with a cost function. We try to minimize it using gradient descent. Finally, the model is trained multiple times known as “Epochs”.

## TensorFlow Implementation with Python

Now we are going to implement the Logistic Regression in TensorFlow with the Python code. So let’s follow the steps with me…

First of all, import libraries and datasets. Using the MNIST dataset for implementation.

import input_data import tensorflow as tf import matplotlib.pyplot as plt #Import Dataset mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

Next, we define model parameters.

training_epochs=25 learning_rate = 0.01 batch_size = 100 display_step = 1

## The Model

By now, you would be familiar with the MNIST dataset. If not, you can learn about it from:

x = tf.placeholder("float", [None, 784]) y = tf.placeholder("float", [None, 10]) # Create model # Set model weights W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) # Construct model activation = tf.nn.softmax(tf.matmul(x, W) + b)

Our aim is to predict the digit written. Obviously, when programming, we will deal with vectors. To assign probabilities, we have used the softmax function. Firstly, we define a weight tensor of the size of 784×10. We could use the distance function for calculating error. But we should prefer cross-entropy with neural networks.

# Minimize error using cross entropy cross_entropy = y*tf.log(activation) cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(actv), reduction_indices=1)) # OPTIMIZER learning_rate = 0.01 optm = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) #Used Gradient Descent Algorithm

Finally, we will compile the model and plot the output:

with tf.Session() as sess: # Initializing the Variables sess.run(init) cost_history, accuracy_history = [], [] # Iterating through all the epochs for epoch in range(epochs): cost_per_epoch = 0 # Running the Optimizer sess.run(optimizer, feed_dict = {X : x, Y : y}) # Calculating cost on current Epoch c = sess.run(cost, feed_dict = {X : x, Y : y}) # Calculating accuracy on current Epoch correct_prediction = tf.equal(tf.argmax(Y_hat, 1), tf.argmax(Y, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) # Storing Cost and Accuracy to the history cost_history.append(sum(sum(c))) accuracy_history.append(accuracy.eval({X : x, Y : y}) * 100) # Displaying result on current Epoch if epoch % 100 == 0 and epoch != 0: print("Epoch " + str(epoch) + " Cost: " + str(cost_history[-1])) Weight = sess.run(W) # Optimized Weight Bias = sess.run(b) # Optimized Bias # Final Accuracy correct_prediction = tf.equal(tf.argmax(Y_hat, 1), tf.argmax(Y, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) print("\nAccuracy:", accuracy_history[-1], "%")

## Why Logistic Regression?

For a machine learning beginner, understanding Logistic Regression is very important. You might be eventually using other algorithms like Support Vector Machines, KNN, K-Means, Decision Trees, etc. Logistic regression makes it easy to understand the basics of these algorithms. Not only it introduces you to supervised learning, but it also involves a lot of maths used in many machine learning models.

Feel Free to ask your doubts in the comments section.

## Leave a Reply