FERNET: An Integrated Hybrid DCNN Model for Driver Stress Monitoring via Facial Expressions

Abstract

Drivers undergo a lot of stress that might cause distraction and might lead to an unfortunate incident. Emotional recognition via facial expressions is one of the most important field in the human–machine interface. The goal of this paper is to analyze the drivers’ facial expressions in order to monitor their stress levels. In this paper, we propose FERNET — a hybrid deep convolutional neural network model for driver stress recognition through facial emotion recognition. FERNET is an integration of two DCNNs, pre-trained ResNet101V2 CNN and a custom CNN, ConvNet4. The experiments were carried out on the widely used public datasets CK[Formula: see text], FER2013 and AffectNet, achieving the accuracies of 99.70%, 74.86% and 70.46%, respectively, for facial emotion recognition. These results outperform the recent state-of-the-art methods. Furthermore, since a few specific isolated emotions lead to higher stress levels, we analyze the results for stress- and nonstress-related emotions for each individual dataset. FERNET achieves stress prediction accuracies of 98.17%, 90.16% and 84.49% for CK[Formula: see text], FER2013 and AffectNet datasets, respectively.