Real-time detection method of driver fatigue state based on deep learning of face video

Abstract

The use of face video information for driver fatigue detection has received extensive attention because of its low cost and non-invasiveness. However, the current vehicle-mounted embedded device has insufficient memory and limited computing power, which cannot complete the real-time detection of driver fatigue based on deep learning. Therefore, this paper designs a lightweight neural network model to solve this problem. The model includes object detection and fatigue detection. First, a lightweight object detection network is designed, which can quickly identify the opening and closing states of the driver’s eyes and mouth in the time series video. Secondly, the EYE-MOUTH (EM) driver fatigue detection model is designed, which encodes the driver’s eye and mouth opening and closing state, and calculates the driver’s PERCLOS (Percentage of Eyelid Closure over the Pupil) and FOM (Frequency of Open Mouth) according to the coding sequence. Finally, the multi-feature fusion judgment algorithm is used to realize the judgment of the driver’s fatigue state. The experimental results show that our method has an accuracy rate of 98.30% for drowsiness and yawning behaviors in a real vehicle environment, and a detection speed of 27FPS, which is better than other advanced methods and meets the requirements of real-time detection.