Stress Detection through Compound Facial Expressions Using Neural Networks

Abstract

Human face is the most dynamic part of the body that conveys information about the instant emotions. Facial expression analysis starts from early 1900s where later on scientists identify the six basic facial expressions as Anger, Disgust, Fear, Happiness, Sadness, Surprise and Neutral with the pioneering studies of psychologists. In the last decades, the acceleration in artificial intelligence and computer vision research makes it possible to automatically detect facial expressions through images. Furthermore, micro expressions, muscle movements and compound facial expressions; that are the combinations of the basic expressions can be also analyzed with computer vision algorithms. The main motivation in automatic facial expression analysis is to support human-computer. Furthermore, facial expression analysis can be a driver for automatic emotion analysis. In this study, we propose a novel method to detect stress indicators on the frontal face images. The detection procedure is based on compound facial expression analysis. 49 couples of 6 basic facial expressions where one is dominating, and the other is the complementary expression are employed. iCV-MEFED facial expression dataset is used in the experiments where video and image samples are provided for every compound facial expression class. The training and testing of compound facial expressions are done using a deep neural network. The robust representations of faces are achieved using a fusion method that combines deep texture features and the action units on the face. Then, through the appropriate grouping of the compound expressions, the system can detect the signs of stress. The proposed approach obtains encouraging results, and it is open to further improvements.