Driver fatigue detection based on convolutional neural network and face alignment for edge computing device

Abstract

Most current vision-based fatigue detection methods don’t have high-performance and robust face detector. They detect driver fatigue using single detection feature and cannot achieve real-time efficiency on edge computing devices. Aimed at solving these problems, this paper proposes a driver fatigue detection system based on convolutional neural network that can run in real-time on edge computing devices. The system firstly uses the proposed face detection network LittleFace to locate the face and classify the face into two states: small yaw angle state “normal” and large yaw angle state “distract.” Secondly, the speed-optimized SDM algorithm is conducted only in the face region of the “normal” state to deal with the problem that the face alignment accuracy decreases at large angle profile, and the “distract” state is used to detect driver distraction. Finally, feature parameters EAR, MAR and head pitch angle are calculated from the obtained landmarks and used to detect driver fatigue respectively. Comprehensive experiments are conducted to evaluate the proposed system and the results show its practicality and superiority. Our face detection network LittleFace can achieve 88.53% mAP on AFLW test set at 58 FPS on the edge computing device Nvidia Jetson Nano. Evaluation results on YawDD, 300 W, and DriverEyes show the average detection accuracy of the proposed system can reach 89.55%.