Facial feature fusion convolutional neural network for driver fatigue detection

Abstract

Driver fatigue detection is critical for preventing traffic accidents. However, in real driving scenarios, low-quality input situations often occur due to occlusion, large-turn head deformation, and inconsistent eye states, which introduce noise and degrade the performance of single-stream networks. To address this, we propose a multi-stream facial feature fusion convolution neural network (FFF–CNN) that enhances performance on low-quality inputs. The FFF-CNN includes a global face stream and two local eye streams, with feature fusion used for eye state classification. To facilitate feature fusion, we implemented two preprocessing modules, the feature attention module (FAM) and the stream interaction module (SIM). FAM emphasizes important features in the global face stream, while SIM corrects information from the two local eye streams. The proposed FFF-CNN achieved good performance on both our dataset and the public Closed Eyes in the Wild (CEW) dataset. Compared to single-stream networks, the FFF-CNN improved performance by 4.95% and 4.27% on our dataset and the CEW dataset, respectively. Furthermore, it achieved state-of-the-art performance on the CEW dataset with an accuracy of 98.35%. The effectiveness of the proposed FAM and SIM has been verified by experiments. Notably, our experimental results show that the proposed FFF-CNN effectively improved fatigue detection performance on low-quality input, approaching the performance on normal situations.