Semantic and Instance Segmentation with PixelLib
In this tutorial, we will learn how to do semantic and instance image segmentation with help of the PixelLib library.
Some computer vision challenges need a more in-depth comprehension of the visual information. These challenges may not be amenable to classification and object identification. Image Segmentation was created in response to a demand for a reliable method for resolving some difficult computer vision challenges.
A collection of pixel values makes up every image. Image segmentation segments the image at a pixel level. By separating a picture into various segments based on the classes given to each of the pixel values included in the image, a machine may more efficiently analyze it.
Image Segmentation is useful for tackling key computer vision issues because of the unique approach it employs. These are issues that need a great deal of research. Because of the unique approach used in Image Segmentation, it may be used to solve key computer vision challenges. These are issues that need precise information on the objects in an image, information that cannot be obtained simply by categorizing the entire image or giving bounding boxes for the items in the image. Image Segmentation has made significant contributions in the following areas:
- For a better grasp of the broad landscape, autonomous automobiles need a good visual system.
- Medical image segmentation is used to segment body sections to perform diagnostic testing.
- Satellite images’ analysis.
There are two major types of Image Segmentation:
- Semantic Segmentation: In semantic segmentation, colormaps are used to segment objects that are categorized with the same pixel values.
- Instance Segmentation: Instance segmentation differs from semantic segmentation in that various color mappings are used to separate distinct instances of the same item.
It is a library designed to make Image Segmentation in real-world issues simple to implement. PixelLib is a versatile library that may be used in software solutions that need Image Segmentation.
Let’s import all the required libraries for image segmentation.
import pixellib from pixellib.semantic import semantic_segmentation import tensorflow as tf
Let’s create an instance of a class for performing semantic segmentation from pixellib and call the function to load the xception model trained on pascal voc.
This xception model is trained on the pascal voc dataset, a dataset with 20 object categories.
Download the Xception model from here.
The line of code that performs image segmentation is done in the pascalvoc’s color format. This function takes in two parameters:
path_to_image: the path to the image to be segemented.
path_to_output_image: the path to save the output image. The image will be saved in your current working directory.
segment=semantic_segmentation() segment.load_pascalvoc_model("deeplabv3_xception_tf_dim_ordering_tf_kernels.h5") segment.segmentAsPascalvoc("test.jpg",output_image_name="output2.jpg")
Your saved image with all the objects present is segmented.
You can obtain an image with segmentation overlay on the objects with a modified code below.
segment_image.segmentAsPascalvoc("test.jpg", output_image_name = "output3.jpg", overlay = True)
The Instance segmentation is based on the Masked R-CNN framework.
The Mask R_CNN model is trained on the Microsoft Coco dataset, a dataset with 80 common object categories.
Let’s perform instance segmentation which we imported earlier.
segment_intimage = instance_segmentation() segment_intimage.load_model("mask_rcnn_coco.h5") segment_intimage.segmentImage("sample2.jpg", output_image_name = "image_new.jpg")
The above code downloads the RCNN model to perform instance segmentation
Download the RCNN model from here.
This code performs instance segmentation on an image takes two parameters:
path_to_image: The path to the image to be predicted by the model.
output_image_name: The path to save the segmentation result. It will be saved in your current working directory.
We can also implement instance segmentation with bounding boxes also. It can be achieved by modifying the code:
segment_intimage.segmentImage("sample2.jpg", output_image_name = "image_new.jpg",show_bboxes = True)
Here we add an extra parameter that is show_bboxes=True , the segmentation masked are produced with bounding boxes.