Micro-facial Expressions Research Articles

Relationship between low mood and micro-expression processing: evidence of negative bias in interpreting fleeting facial expressions.

Depression affects the recognition of emotion in facial expressions by reducing the detection accuracy and adding a bias towards negativity. However, no study has examined associations between depression and the recognition of microfacial expressions (fleeting facial cues of emotions in people's faces). Thus, we investigated associations between low mood and micro-expression processing using video stimuli of micro-expressions. We examined whether (i) individuals with low mood had trouble recognizing emotions, (ii) were more likely to perceive happy facial expressions as neutral and neutral facial expressions as sad, and (iii) recognized sad emotional expressions better than control subjects (n = 349). We found that participants with low mood showed poorer performance when judging emotions in faces (p = 0.03). Furthermore, there was a specific deficit among them in recognizing happiness. Lastly, participants with low moods were more likely to perceive neutral faces as sad (p = 0.042). However, no evidence was found that individuals with low moods confused happy faces as neutral or were better than the control group at recognizing sad faces. Our results show that mood affects the perception of emotions in facial expressions, which has the potential to negatively affect interpersonal interactions and ultimately quality of life.

Deep Learning-Based Micro-Expression Recognition Algorithm Research

In order to improve the accuracy and speed of micro-expressions, a modified model based on densenet and eca is proposed. Microfacial expression is a brief, weak facial change, its characteristics are similar, dense, difficult to extract and identify, and the improved model can be adapted to the characteristics and location of the interest. In particular, the eca attention module was added after the densenet model, using the densenet network to extract the rich characteristics of micro-expressions, and the eca attention module to recalibrate the feature channel and focus on the more subtle expression changes. In order to verify the validity of this method, the experiment was conducted in the micro-emotive data set, and compared with the resnet network and the densenet network, the experimental results showed that the improved model significantly improved the performance of micro-expression recognition, and had strong generalized ability and robustness.

SSGNN: A Macro and Microfacial Expression Recognition Graph Neural Network Combining Spatial and Spectral Domain Features

Emotion recognition from macroexpression and microexpression has been widely used in applications such as human–computer interaction, learning status evaluation, and mental disorder diagnosis. However, due to the complexity of human macroexpressions, recognizing macroexpressions with high accuracy is a challenging task. Moreover, the short duration and low movement intensity of microexpressions make its recognition more difficult. For MM-FER (macro and microfacial expression recognition), the key information can be more efficiently expressed by a graph. In this article, a novel framework based on graph neural network named SSGNN (spatial and spectral domain features based on a graph neural network) is designed to extract spatial and spectral domain features from facial images for MM-FER, which can efficiently recognize both macroexpressions and microexpressions under the same model. SSGNN consists of two parts, SPAGNN and SPEGNN, which are used to extract spectral and spatial domain features, respectively. Experiments proved that jointly using the spectral and spatial information extracted by SSGNN can largely improve the performance of MM-FER when the training sample is limited. First, the influences of different neighbors and samples to the model performance was analyzed. Then, the contribution of SPAGNN and SPEGNN were evaluated. It was discovered that fusing the result of SPAGNN and SPEGNN at decision level further improved the performance of MM-FER. Experiment proved that SSGNN can recognize microexpression acquired by various sensors with higher accuracy under different image resolutions and image formats than the compared state-of-the-art methods in most cases. A cross-dataset experiment demonstrated the generalization ability of SSGNN.

Analysis of Micro Facial Expression by Machine and Deep Learning Methods: Haar, CNN, and RNN

Facial expressions are due to the actions of the facial muscles located at different facial regions. These expressions are two types: Macro and Micro expressions. The second one is more important in computer vision. Analysis of micro expressions categorized by disgust, happiness, anger, sadness, surprise, contempt, and fear are challenging because of very fast and subtle facial movements. This article presents one machine learning method: Haar and two deep learning methods: Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) to perform recognition of micro-facial expression analysis. First, Haar Cascade Classifier is used to detect the face as a pre-image-processing step. Secondly, those detected faces are passed through series of Convolutional Neural Network (CNN) layers for the features extraction. Thirdly, the Recurrent Neural Network (RNN) classifies micro facial expressions. Two types of data sets are used for training and testing of the proposed method: Chinese Academy of Sciences Micro-Expression II (CSAME II) and Spontaneous Actions and Micro-Movements (SAMM) database. The test accuracy of SAMM and CASME II are obtained as 84.76%, and 87% respectively. In addition, the distinction between micro facial expressions and non- micro facial expressions are analyzed by the ROC curve.

Mechanisms of Realistic Facial Expressions for Anthropomorphic Social Agents

Three-dimensional facial animation has been extensively studied, but the achievement of realistic, human-like performance has not yet been decided. This article discusses various approaches for generating animated facial expressions controlled by speech. Combining the considered approaches for both facial animation, and the identification of emotions and the creation of micro-facial expressions in one system, we get a solution suitable for tasks such as game video, avatars of virtual reality or any scenario in which a speaker, speech or language is not known in advance.

Wiser Reasoning and Less Disgust Have the Potential to Better Achieve Suicide Prevention.

Background: High school and university teachers need to advise students against attempting suicide, the second leading cause of death among 15-29-year-olds. Aims: To investigate the role of reasoning and emotion in advising against suicide. Method: We conducted a study with 130 students at a university that specializes in teachers' education. Participants sat in front of a camera, videotaping their advising against suicide. Three raters scored their transcribed advice on "wise reasoning" (i.e., expert forms of reasoning: considering a variety of conditions, awareness of the limitation of one's knowledge, taking others' perspectives). Four registered psychologists experienced in suicide prevention techniques rated the transcripts on the potential for suicide prevention. Finally, using the software Facereader 7.1, we analyzed participants' micro-facial expressions during advice-giving. Results: Wiser reasoning and less disgust predicted higher potential for suicide prevention. Moreover, higher potential for suicide prevention was associated with more surprise. Limitations: The actual efficacy of suicide prevention was not assessed. Conclusion: Wise reasoning and counter-stereotypic ideas that trigger surprise probably contribute to the potential for suicide prevention. This advising paradigm may help train teachers in advising students against suicide, measuring wise reasoning, and monitoring a harmful emotional reaction, that is, disgust.

Manifold feature integration for micro-expression recognition

Recognition of micro-expressions depends on the key features provided in the form of the temporal information. It needs considerable effort, however, to manually design useful characteristics. Subtle or micro-facial expressions are much difficult than regular facial expressions rich in emotional expressions in a true environment to be identified. An easy solution is discussed in this paper to recognise facial micro-expressions that utilizes an algorithm mix for facial identification, feature extraction and classification. The technique proposed is a framework which incorporates handcrafted features and deep features. Local Binary Pattern-Three Orthogonal Planes (LBP-TOP) is the handcraft feature which combines spatial and time analysis to encapsulate regional facet movements. The deep feature model is a micro-expression fine-tuned model based on Convolutional Neural Network (CNN). Two classifiers, i.e. SVM and Softmax are trained with combined feature vectors produced by LBP-TOP and CNN functionalities. All seven widely-used micro-expression databases are evaluated in an experiment. Our research can be claimed as the first extensive experimental study on a big amount of the datasets to train and test the suggested model. The findings in the document show that the method proposed, although simple and straightforward, achieves a substantial increase in precision relative to other commonly recognized micro-expression techniques, which are trained and tested with just a few datasets.

Future-Oriented Happiness: Its Nature and Role in Consumer Decision-Making for New Products

Cognitive evaluations only partially explain the consumer purchasing patterns, especially when consumers approach a product for the first time. In such an encounter, consumers anticipate the emotions they might experience as a result of their decision, as they cannot realistically evaluate product performances. The work investigates the nature and the influence of these future-oriented emotions, namely anticipated and anticipatory happiness, in the first encounter with new products. Through a first laboratory study, adopting both physiological (micro-facial expressions analysis) and self-reported measures, we confirm the distinction between anticipated and anticipatory happiness. We further show the differential impact of these two emotional constructs on the consumer decision-making process by grounding on the Theory of Planned Behavior (TPB). Through a second study, based on a questionnaire, we further investigate the role of anticipated happiness within the TPB. We show that anticipated happiness is a pervasive emotional construct that influences all stages of the intention formation process. We discuss how these findings enrich existing knowledge on the interplay between cognitive and affective components of the decision-making process for new products. Moreover, we offer a methodological contribution to the use of physiological methods to assess emotions.

Micro-Facial Expression Recognition in Video Based on Optimal Convolutional Neural Network (MFEOCNN) Algorithm

Facial expression is a standout amongst the most imperative features of human emotion recognition. For demonstrating the emotional states facial expressions are utilized by the people. In any case, recognition of facial expressions has persisted a testing and intriguing issue with regards to PC vision. Recognizing the Micro-Facial expression in video sequence is the main objective of the proposed approach. For efficient recognition, the proposed method utilizes the optimal convolution neural network. Here the proposed method considering the input dataset is the CK+ dataset. At first, by means of Adaptive median filtering preprocessing is performed in the input image. From the preprocessed output, the extracted features are Geometric features, Histogram of Oriented Gradients features and Local binary pattern features. The novelty of the proposed method is, with the help of Modified Lion Optimization (MLO) algorithm, the optimal features are selected from the extracted features. In a shorter computational time it has the benefits of rapidly focalizing and effectively acknowledging with the aim of getting an overall arrangement or idea. Finally the recognition is done by Convolution Neural network (CNN). Then the performance of the proposed MFEOCNN method is analyzed in terms of false measures and recognition accuracy. This kind of emotion recognition is mainly used in medicine, marketing, E-learning, entertainment, law and monitoring. From the simulation, we know that the proposed approach achieves maximum recognition accuracy of 99.2% with minimum Mean Absolute Error (MAE) value. This results are compared with the existing for Micro-Facial Expression Based Deep-Rooted Learning (MFEDRL), Convolutional Neural Network with Lion Optimization (CNN+LO) and Convolutional Neural Network (CNN) without optimization. The simulation of the proposed method is done in the working platform of MATLAB.

Person-independent facial expression recognition method based on improved Wasserstein generative adversarial networks in combination with identity aware

Since the distinction between two expressions is fairly vague, usually a subtle change in one part of the human face is enough to change a facial expression. Most of the existing facial expression recognition algorithms are not robust enough because they rely on general facial features or algorithms without considering differences between facial expression and facial identity. In this paper, we propose a person-independent recognition method based on Wasserstein generative adversarial networks for micro-facial expressions, where a facial expression recognition network and a facial identity recognition network are established to improve the accuracy and robustness of facial expression recognition via inhibition of intra-class variation. Extensive experimental results demonstrate that 90% average recognition accuracy of facial expression has been reached on a mixed dataset composed of CK+, Multi-PIE, and JAFFE. Moreover, our method achieves 96% accuracy of person-independent recognition on CK+. A 4.5% performance gain is achieved with the novel identity-inhibited expression feature. The proposed algorithm in this paper has been successfully applied to Haikang Visual Integrated Management Platform (iVMS-8700). At present, it runs well and can effectively recognize facial expressions.

Micro-Facial Expression Recognition Based on Deep-Rooted Learning Algorithm

Facial expressions are important cues to observe human emotions. Facial expression recognition has attracted many researchers for years, but it is still a challenging topic since expression features vary greatly with the head poses, environments, and variations in the different persons involved. In this work, three major steps are involved to improve the performance of micro-facial expression recognition. First, an Adaptive Homomorphic Filtering is used for face detection and rotation rectification processes. Secondly, Micro-facial features were used to extract the appearance variations of a testing image-spatial analysis. The features of motion information are used for expression recognition in a sequence of facial images. An effective Micro-Facial Expression Based Deep-Rooted Learning (MFEDRL) classifier is proposed in this paper to better recognize spontaneous micro-expressions by learning parameters on the optimal features. This proposed method includes two loss functions such as cross entropy loss function and center loss function. Then the performance of the algorithm will be evaluated using recognition rate and false measures. Simulation results show that the predictive performance of the proposed method outperforms that of the existing classifiers such as Convolutional Neural Network (CNN), Deep Neural Network (DNN), Artificial Neural Network (ANN), Support Vector Machine (SVM), and k-Nearest Neighbors (KNN) in terms of accuracy and Mean Absolute Error (MAE).

SAMM: A Spontaneous Micro-Facial Movement Dataset

Micro-facial expressions are spontaneous, involuntary movements of the face when a person experiences an emotion but attempts to hide their facial expression, most likely in a high-stakes environment. Recently, research in this field has grown in popularity, however publicly available datasets of micro-expressions have limitations due to the difficulty of naturally inducing spontaneous micro-expressions. Other issues include lighting, low resolution and low participant diversity. We present a newly developed spontaneous micro-facial movement dataset with diverse participants and coded using the Facial Action Coding System. The experimental protocol addresses the limitations of previous datasets, including eliciting emotional responses from stimuli tailored to each participant. Dataset evaluation was completed by running preliminary experiments to classify micro-movements from non-movements. Results were obtained using a selection of spatio-temporal descriptors and machine learning. We further evaluate the dataset on emerging methods of feature difference analysis and propose an Adaptive Baseline Threshold that uses individualised neutral expression to improve the performance of micro-movement detection. In contrast to machine learning approaches, we outperform the state of the art with a recall of 0.91. The outcomes show the dataset can become a new standard for micro-movement data, with future work expanding on data representation and analysis.