Real-Time Driver Drowsiness Detection Using GRU with CNN Features

Abstract

There are many visual characteristics linked with drowsiness like eye closure duration, blinking frequency, gaze, pose, and yawning. In this paper, we propose a robust deep learning approach to detect a driver’s drowsiness. We start by extracting the mouth region from incoming frames of the video stream using Dlib’s frontal face detector and a custom dlib landmark detector. Then, a deep convolutional neural network (DCNN) is used to extract deep features.. Lastly, yawning is detected using a yawn detector consisting of 1D-DepthWise Separable-CNN and Gated Recurrent Unit (GRU) which learns the mapping or patterns from temporal information of the sequence of features extracted from frames and predicts yawning/not yawning. Yawn detector was able to reach training and validation accuracy of 99.99% and 99.97% respectively. We were able to achieve an inference speed of ~30FPS on our host machine on live video recording and ~23 FPS on an embedded board for the same. To check the robustness of our model, testing was done on the YawDD test set where we were able to detect yawning successfully. On the other hand, while testing NTHU data many false positive was observed. Thus, our approach can be effectively used for real-time driver drowsiness detection on an embedded platform.