IoU implementation in Keras

What is IoU?

IoU or Intersection over Union is a metric used to evaluate the accuracy of any trained model for a particular dataset. It is one of the common evaluation metrics used for semantic image segmentation. IoU is typically used for CNN object detectors which are basically algorithms that can have predicted bounding boxes for performance evaluation. IoU as the name suggests as the formula depiction as follows:

IoU = (Intersection ) / (Union ) = ( Area of overlapped region) / ( Area of union region )

Why IoU is used?

The expected output values as one-hot encoded are termed as y_true and the predicted softmax output values are termed as y_pred. 

Intersection = y_true * y_pred

Union = y_true  + y_pred – Intersection

IoU is an excellent metric because, in most of the cases of class imbalance, high segmentation accuracy does not imply superior segmentation ability. There are chances that although the segmentation accuracy lies in the high range (~ 90%) but in reality, it has hardly segmented and detected any object. It accurately shows that those predicted bounding boxes that heavily overlap have higher scores than those bounding boxes with less overlap. 

More about IoU

It lies between 0 and 1 (i.e. 0 and 100 %). IoU ∈ [0,1] If IoU is 0, then it means that there is no overlap area and it’s almost a garbage output. On the other hand, if IoU is 1, then it implies that it has a perfectly segmented output image. Although the exact match of x and y coordinates are difficult, there is a possibility to find a near accurate match. IoUmean is used for binary or multiclass segmentation. The following formula depicts IoUmean where IoUi is metric for each semantic class.

IoUmean = (IoU1 + IoU2 + ………. + IoU)/N

Algorithm for IoU implementation

To implement Intersection over Union as a function in Keras, let’s first go over the following algorithm:

1. Return a single scalar absolute value product of y_true and y_pred as the intersection
2. Return a scalar value of region including y_true and y_pred but not the intersecting area as the union
3. Introduce a smoothening value to prevent division by zero error.
4. Calculate IoU by dividing Intersection and Union

Implementation of IoU in Keras

Following is the implementation of IoU in Keras with Python programming. I have taken reference from here.

IoU is computed with respect to two segmentation masks using the NumPy library.

import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
import tensorflow as tf
from keras import backend as K

def IoU_coeff(y_true, y_pred):
    axes = (1,2) 
    intersection = np.sum(np.abs(y_pred * y_true), axis=axes) 
    mask = np.sum(np.abs(y_true), axis=axes) + np.sum(np.abs(y_pred), axis=axes)
    union = mask - intersection
    smooth = .001
    iou = (intersection + smooth) / (union + smooth)
    return iou

def mean_class(mask, metric):
    if drop_last:
        metric = metric[:,:-1]
        mask = mask[:,:-1]

    if mean_per_class:
        if naive:
            return np.mean(metric, axis=0)
        else:
            return (np.sum(metric * mask, axis=0) + smooth)/(np.sum(mask, axis=0) + .001)
    else:
        if naive:
            return np.mean(metric)
        else:
            class_count = np.sum(mask, axis=0)
            return np.mean(np.sum(metric * mask, axis=0)[class_count!=0]/(class_count[class_count!=0]))
  
def metrics_np(y_true, y_pred, metric_name, metric_type='standard', drop_last = True, mean_per_class=False):
    soft = (metric_type == 'soft') 
    naive = (metric_type == 'naive')
    num_classes = y_pred.shape[-1]
    drop_last = drop_last and num_classes>1
    
    if not soft:
        if num_classes>1:
            y_pred = np.array([ np.argmax(y_pred, axis=-1)==i for i in range(num_classes) ]).transpose(1,2,3,0)
            y_true = np.array([ np.argmax(y_true, axis=-1)==i for i in range(num_classes) ]).transpose(1,2,3,0)
        else:
            y_pred = (y_pred > 0).astype(int)
            y_true = (y_true > 0).astype(int)
    
    iou=IoU_coeff(y_true, y_pred)
    metric = {'iou': iou}[metric_name]
    mask =  np.not_equal(union, 0).astype(int)
    mean_class(mask, metric)
        
def mean_iou(y_true, y_pred):
    return metrics_np(y_true, y_pred, metric_name='iou')

def circle(xy=(0,0), r=4, factor=0.8):
    x0, y0 = xy
    max = factor * r**2
    circle = np.minimum([1], np.maximum([0], r**2 - (x-x0)**2 - (y-y0)**2)/max)    
    return circle

fine_grid = np.meshgrid(np.arange(-7,7.1,0.05), np.arange(-7,7.1,0.05))
x,y = fine_grid
fig, axes = plt.subplots(1,3, figsize = (13,4))
params = [((0,0), 4), ((2,0), 4, ), ((2,0), 2) ]
y_true = circle(factor=0.01)

for i in range(len(axes)):
    axes[i].scatter(0,0, c='g')
    axes[i].add_artist(plt.Circle((0, 0), 4.05, lw=2, edgecolor='b', facecolor=(0,0,0,0.3), zorder=1))
    xy, r = params[i]
    axes[i].scatter(*xy, c='y')
    axes[i].add_artist(plt.Circle(xy, r, lw=2, ls='--', edgecolor='b', facecolor=(1,1,0,0.3), zorder=1))
    
    smooth = 0.001
    y_pred=circle(xy, r, 0.01)
    intersection = np.sum(np.logical_and(y_true, y_pred))
    union = np.sum(np.logical_or(y_pred, y_true))
    iou = np.mean((intersection)/union)
    axes[i].text(0,5, f'IoU={iou:1.2f}', ha='center')
    axes[i].set_axis_off()
    axes[i].set(aspect=1, xlim=(-5,6.1), ylim=(-5,6))

Output:

As we can see from the output that the IoU metric is calculated based on the overlapping areas of the ground truth and predicted truth.