Driver Drowsiness Detection using OpenCV and Keras in Python

In this tutorial. we will try to make a Driver Drowsiness Detection model with OpenCV and Keras in Python. For this particular model, we will try to detect whether the eyes of the person are open or closed. If the eyes remain closed for a prolonged amount of time we would raise an alarm to alert the driver.

The classification trained on over 1000s of images is available here. You should download this dataset and create a new folder named “models” and paste it into it.
The next step is to download the ‘haar cascade files’. These files help us to detect the face and eyes of the person. These files detect the face, right eye, and left eye of the person and classify these into closed or open.

Following are the Prerequisites for this model:

  1. OpenCV – Face and eye detection.
  2. Keras – To build our classification model.
  3. TensorFlow – Keras uses TensorFlow as a backend.
  4. Pygame – To play an alarm sound.

Driver Drowsiness Detection

We will be using the CNN type for this model as these perform well on the classification of images. Here is a sample:
open (not sleeping)

Firstly do make sure that you have the following python libraries installed on your system.

import cv2
import os
from keras.models import load_model
import numpy as np
from pygame import mixer
import time

Now we need to access the ‘haar cascade files’ through this program.

face = cv2.CascadeClassifier('haar cascade files\haarcascade_frontalface_alt.xml')
leye = cv2.CascadeClassifier('haar cascade files\haarcascade_lefteye_2splits.xml')
reye = cv2.CascadeClassifier('haar cascade files\haarcascade_righteye_2splits.xml')

Next, we access the alarm file needed in this model.

mixer.init()
sound = mixer.Sound('attention.wav')

Now it is time for us to import the classification model that we have downloaded.

lbl=['Close','Open']

model = load_model('models/cnn.h5')
path = os.getcwd()
cap = cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_COMPLEX_SMALL
count=0
score=0
thicc=2
rpred=[99]
lpred=[99]

Capture a pic from the camera:

Our first step is to capture an image from the camera module so we need to access the camera (webcam in this case) to do so.
Firstly we use cv2.VideoCapture(0) function to access the camera module. The object in the frame is then read by the cap.read() function and the image is then stored in a variable.
The next step would be to create a rectangle around the detected facial region.

# Start of while

while(True):
    ret, frame = cap.read()
    height,width = frame.shape[:2] 

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
    faces = face.detectMultiScale(gray,minNeighbors=5,scaleFactor=1.1,minSize=(25,25))
    left_eye = leye.detectMultiScale(gray)
    right_eye =  reye.detectMultiScale(gray)

    cv2.rectangle(frame, (0,height-50) , (200,height) , (0,0,0) , thickness=cv2.FILLED )

    for (x,y,w,h) in faces:
        cv2.rectangle(frame, (x,y) , (x+w,y+h) , (100,100,100) , 1 )

Detecting eyes and classify them as open or closed :

The next step is to detect the eyes in the captured image and hence narrow the concentration region. We need to separately detect both the right eye and the left eye for this.

# while continued
    for (x,y,w,h) in right_eye:
        r_eye=frame[y:y+h,x:x+w]
        count=count+1
        r_eye = cv2.cvtColor(r_eye,cv2.COLOR_BGR2GRAY)
        r_eye = cv2.resize(r_eye,(24,24))
        r_eye= r_eye/255
        r_eye=  r_eye.reshape(24,24,-1)
        r_eye = np.expand_dims(r_eye,axis=0)
        rpred = model.predict_classes(r_eye)
        if(rpred[0]==1):
            lbl='Open' 
        if(rpred[0]==0):
            lbl='Closed'
        break

    for (x,y,w,h) in left_eye:
        l_eye=frame[y:y+h,x:x+w]
        count=count+1
        l_eye = cv2.cvtColor(l_eye,cv2.COLOR_BGR2GRAY)  
        l_eye = cv2.resize(l_eye,(24,24))
        l_eye= l_eye/255
        l_eye=l_eye.reshape(24,24,-1)
        l_eye = np.expand_dims(l_eye,axis=0)
        lpred = model.predict_classes(l_eye)
        if(lpred[0]==1):
            lbl='Open'   
        if(lpred[0]==0):
            lbl='Closed'
        break

Drowsy or not:

The next step is to classify whether the driver is drowsy or not. We might an issue here.
Closed eye need not mean that the driver is drowsy, he/she might just be blinking. So to tackle this we would make a score and increase it every time the eyes are found closed in continuation. And only if then score goes past 15 the alarm needs to alert the driver.

# while continued    
    if(score<0):
        score=0   
    cv2.putText(frame,'Score:'+str(score),(100,height-20), font, 1,(255,255,255),1,cv2.LINE_AA)
    if(score>15):
        
        cv2.imwrite(os.path.join(path,'image.jpg'),frame)
        try:
            sound.play()
            
        except:  
            pass
        if(thicc<16):
            thicc= thicc+2
        else:
            thicc=thicc-2
            if(thicc<2):
                thicc=2
        cv2.rectangle(frame,(0,0),(width,height),(0,0,255),thicc) 
    cv2.imshow('frame',frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# End of while

Now we are almost done. After classifying the driver as drowsy or not, the final step is to close the windows.

cap.release()
cv2.destroyAllWindows()

Here are some samples of the output of the program on my system.

Our model is working perfectly fine.

Congratulations to you, I hope you would go ahead and use this model, and hence save some lives.
Hope you enjoyed this session with me. Have a good day and happy learning.

Leave a Reply

Your email address will not be published.