Facial Emotion Identification Research Articles

Electrocardiogram-based Emotion Recognition using Convolutional Neural Networks

Introduction: Electrocardiogram-based facial emotion identification has been a widely spread field in the last few decades. Due to its non-linearity, non-stationary and noisy properties, it is a very difficult job to create a framework that is capable of recognizing emotions with a high recognition rate. background: Electrocardiogram based facial emotion identification has been widely spread field in last some decades. Due to its non-linearity, non-stationary and noisy properties, it is very difficult job to create a framework that is capable of recognizing emotions with high recognition rate. Method: In this work, we introduce a new framework for facial emotion detection based on feature creation using a topographic representation of ECG signal properties. The feature map is created using deep learning techniques, and further, extricated features are then used for classification techniques to detect facial emotion recognition. Result: The recognition results are achieved on two publicly available facial expression datasets, i.e., Ascertain and Dreamer. We illustrated the usefulness of our framework by comparing results with other existing methods. Conclusion: The recognition results prove that the introduced framework can enhance the identifying rate on various given datasets.

AI-Based Facial Emotion Recognition

Facial expression, age, and gender detection have received a lot of interest in recent years due to their wide range of applications in fields including healthcare, security, marketing, and entertainment. With the proliferation of artificial intelligence (AI) techniques, especially deep learning, facial analysis systems' accuracy and efficiency have improved significantly. This paper provides a comprehensive analysis of the most recent advances in AI-based facial expression, age, and gender identification algorithms. We explore the constraints and limitations of current techniques, such as the necessity for big annotated datasets, biases in training data, and interpretability issues in deep learning models. This study seeks to give academics, practitioners, and policymakers, a thorough grasp of the most recent state-of-the-art methodologies and trends in AI-based facial emotion, age, and gender identification, thereby supporting future advancements and responsible usage of these technologies. Keywords: Facial expression, age, gender, identification, deep learning, artificial intelligence

CNN Based Face Emotion Recognition System for Healthcare Application

INTRODUCTION: Because it has various benefits in areas such psychology, human-computer interaction, and marketing, the recognition of facial expressions has gained a lot of attention lately.
 OBJECTIVES: Convolutional neural networks (CNNs) have shown enormous potential for enhancing the accuracy of facial emotion identification systems. In this study, a CNN-based approach for recognizing facial expressions is provided. METHODS: To boost the model's generalizability, transfer learning and data augmentation procedures are applied. The recommended strategy defeated the existing state- of-the-art models when examined on multiple benchmark datasets, including the FER-2013, CK+, and JAFFE databases. 
 RESULTS: The results suggest that the CNN-based approach is fairly excellent at properly recognizing face emotions and has a lot of potential for usage in detecting facial emotions in practical scenarios.
 CONCLUSION: Several diverse forms of information, including oral, textual, and visual, maybe applied to comprehend emotions. In order to increase prediction accuracy and decrease loss, this research recommended a deep CNN model for emotion prediction from facial expression.

Facial and Body Posture Emotion Identification in Deaf and Hard-of-Hearing Young Adults

Facial and Body Posture Emotion Identification in Deaf and Hard-of-Hearing Young Adults

Negative symptoms and social cognition as mediators of the relationship between neurocognition and functional outcome in schizophrenia.

In this study we assessed the contribution of psychopathology, including the two domains of negative symptoms (motivational deficit and expressive deficit), processing speed as an index of neurocognition, and emotion recognition, as an index of social cognition, to poor functional outcomes in people with schizophrenia. The Positive and Negative Syndrome Scale was used to evaluate positive symptoms and disorganization and the Brief Negative Symptom Scale to assess negative symptoms. The Symbol Coding and the Trail Making Test A and B were used to rate processing speed and the Facial Emotion Identification Test to assess emotion recognition. Functional outcome was assessed with the Personal and Social Performance Scale (PSP). Regression analyses were performed to identify predictors of functional outcome. Mediation analyses was used to investigate whether social cognition and negative symptom domains fully or partially mediated the impact of processing speed on functional outcome. One hundred and fifty subjects from 8 different European centers were recruited. Our data showed that the expressive deficit predicted global functioning and together with motivational deficit fully mediated the effects of neurocognition on it. Motivational deficit was a predictor of personal and social functioning and fully mediated neurocognitive impairment effects on the same outcome. Both motivational deficit and neurocognitive impairment predicted socially useful activities, and the emotion recognition domain of social cognition partially mediated the impact of neurocognitive deficits on this outcome. Our results indicate that pathways to functional outcomes are specific for different domains of real-life functioning and that negative symptoms and social cognition mediate the impact of neurocognitive deficits on different domains of functioning. Our results suggest that both negative symptoms and social cognition should be targeted by psychosocial interventions to enhance the functional impact of neurocognitive remediation.

Is impairment of facial emotion recognition independent of cognitive dysfunction in multiple sclerosis?

BackgroundEmotions expressed on the face play a key role in social cognition and communication by providing inner emotional experiences. This study aimed to evaluate facial emotion identification and discrimination and empathy abilities in patients with MS and whether it is related to cognitive dysfunction.MethodsOne hundred twenty patients with relapsing–remitting MS and age- and sex-matched 120 healthy controls were enrolled in the study. All the subjects were evaluated with the Facial Emotion Identification Test (FEIT), Facial Emotion Discrimination Test (FEIDT), and Empathy Quotient (EQ). We used the Beck Depression Inventory (BDI) for depression and detailed cognitive tests, including the Montreal Cognitive Assessment (MoCA), the Symbol Digit Modalities Test (SDMT), and the Paced Auditory Serial Addition Test (PASAT). The quality of life was assessed with Multiple Sclerosis Quality of Life-54 (MSQL-54).ResultsPatients with MS were 37.6 ± 9.5 years old, had a mean disease duration of 8.8 ± 6.6 (8–28) years, and a mean EDSS score of 1.6 ± 1.3 (0–4.5). We found significant differences in the identification of facial emotions, discrimination of facial emotions, and empathy in MS patients compared to controls (p < 0.05). Especially the recognition of feelings of sadness, fear, and shame was significantly lower in MS patients. The multivariate logistic regression analysis showed low SDMT and FEIDT scores which showed an independent association with MS.ConclusionsOur findings indicate that facial emotion recognition and identification deficits are remarkable among patients with MS and emotion recognition is impaired together with and independently of cognitive dysfunction in MS patients.

Effect of mindfulness-based cognitive therapy vs. psychoeducational intervention on plasma brain-derived neurotrophic factor and cognitive function in bipolar patients: a randomized controlled trial.

Few controlled trials have assessed the benefits of Mindfulness Based Cognitive Therapy (MBCT) on cognitive functions and brain-derived neurotrophic factor (BDNF) in bipolar disorder (BD). This study aims to evaluate the impact of MBCT adjunctive treatment on these variables. Main hypothesis was that MBCT would improve cognitive functioning and BDNF more than Psychoeducation and TAU. Randomized, multicenter, prospective and single-blinded trial. Included BD outpatients randomly assigned to three treatment arms: MBCT plus treatment as usual (TAU), Psychoeducation plus Tau and TAU. Cognitive functions were assessed with Continuous Performance Test-III, Stroop Test, Trail Making Test, Digit Span and Letter-Number Sequencing from Wechsler Adult Intelligence Scale III, Face Emotion Identification Task and Face Emotion Discrimination Task. BDNF serum level was measured with ELISA. Patients were assessed at baseline, 8 weeks and 6 months. Eighty-four patients were recruited (TAU = 10, Psychoeducation = 34, MBCT = 40). No significant differences between treatment groups were found. MBCT does not achieve better results than Psychoeducation or TAU. Being Psychoeducation and TAU efficient interventions, as well as the scarce duration of a more complex intervention, such as MBCT, are suggested as explanatory variables of these results. ClinicalTrials.gov: NCT02133170. Registered 04/30/2014.

Facial emotion recognition is associated with executive functions and depression scores, but not staging of dementia, in mild-to-moderate Alzheimer's disease.

Although deficits in facial emotion recognition (FER) significantly affect interpersonal communication and social functioning, there is no consensus on how FER affects Alzheimer's disease (AD). In this study, we aimed to investigate the clinical and neuropsychological factors affecting the possible deficits in the FER abilities of patients with AD. This cross-sectional study included 37 patients with mild [clinical dementia rating (CDR) scale score=1] or moderate (CDR=2) AD, in whom vascular dementia and depression were excluded, and 24 cognitively normal (CDR=0) subjects. FER ability was determined using the facial emotion identification test (FEIT) and facial emotion discrimination test (FEDT). All participants underwent mini-mental state examination (MMSE), frontal assessment battery (FAB), and geriatric depression scale (GDS). The neuropsychiatric inventory-clinician rating scale (NPI-C), Katz index of independence in activities of daily living, and Lawton instrumental activities of daily living were also administered to patients with AD. The FEIT and FEDT total scores showed that patients with mild and moderate AD had significant FER deficits compared to healthy controls. However, no significant difference was observed between patients with mild and moderate AD in the FEIT and FEDT total scores. FEIT and FEDT scores were not correlated with the MMSE and NPI-C total and subscales scores in patients with AD. Linear regression indicated that FEIT and FEDT total scores were significantly related to age and FAB scores. The GDS score negatively moderated the relationship between FAB and FEDT. This study demonstrated a decreased FER ability in patients with AD. The critical point in FER deficits is the presence of dementia, not the dementia stage, in AD. It has been determined that executive functions and depression (even at a subsyndromal level), which have limited knowledge, are associated with FER abilities.

Real-Time Facial Expression Recognition: Advances, Challenges, and Future Directions

Facial emotion recognition (FER) is the technology or process of identifying and interpreting human emotions based on the analysis of facial expressions. It involves using computer algorithms and machine learning techniques to detect and classify emotional states from images or videos of human faces. Further, FER plays a vital role in recognizing and understanding human emotions to better interpret someone’s feelings, intentions, and attitudes. In the present time, it is widely used in various fields such as healthcare, human–computer interaction, law enforcement, security, and beyond. FER has a wide range of practical applications across various industries including Emotion Monitoring, Adaptive Learning, and Virtual Assistants. This paper presents a comparative analysis of FER algorithms, focusing on deep learning approaches. The performance of different datasets, including FER2013, JAFFE, AffectNet, and Cohn–Kanade, is evaluated using convolutional neural networks (CNNs), deep face, attentional convolutional networks (ACNs), and deep belief networks (DBNs). Among the tested algorithms, DBNs outperformed other algorithms, reaching the highest accuracy of 98.82%. These results emphasize the effectiveness of deep learning techniques, particularly DBNs, in FER. Additionally, outlining the advantages and disadvantages of current research on facial emotion identification might direct future research efforts in the direction of the most profitable directions.

Unleashing the Power of VGG16: Advancements in Facial Emotion Recognization

In facial emotion detection, researchers are actively exploring effective methods to identify and understand facial expressions. This study introduces a novel mechanism for emotion identification using diverse facial photos captured under varying lighting conditions. A meticulously pre-processed dataset ensures data consistency and quality. Leveraging deep learning architectures, the study utilizes feature extraction techniques to capture subtle emotive cues and build an emotion classification model using convolutional neural networks (CNNs). The proposed methodology achieves an impressive 97% accuracy on the validation set, outperforming previous methods in terms of accuracy and robustness. Challenges such as lighting variations, head posture, and occlusions are acknowledged, and multimodal approaches incorporating additional modalities like auditory or physiological data are suggested for further improvement. The outcomes of this research have wide-ranging implications for affective computing, human-computer interaction, and mental health diagnosis, advancing the field of facial emotion identification and paving the way for sophisticated technology capable of understanding and responding to human emotions across diverse domains.

Emotion identification in human faces through ensemble of deep learning models

Investigation of human face images forms an important facet in affective analysis. The work, a DL-based ensemble is proposed for this purpose. Seven pre-trained models namely Facenet, Facenet2018, VGG16, Resnet-50, Senet-50, Arcface and Openface that have been developed for face verification have been exploited and customized for emotion identification. To each of these models, each all over interaction with softmax method to classification groups are augmented and entire network is then trained completely for emotion recognition. After training all the models individually, the probabilities for each of the class by each of the model are summed to derive at the final value. The class that holds the highest of this value is finalized as the predicted emotion. Thus, the proposed methodology involves image collection, image pre-processing comprising of contrast enhancement, face detection and extraction, face alignment, image augmentation facilitating rotation, shifting, flipping and zooming transformations and appropriate resizing and rescaling, feature extraction and classification through ensemble of customized afore-mentioned pre-trained convolutional neural networks, evaluation and evolving of best weights for emotion recognition from face images with enhanced accuracy. The proposed methodology is evaluated on the well-established FER-2013 dataset. The methodology achieves a validation accuracy of 74.67% and test accuracy of 76.23%. Further, similar images of another dataset (Face Expression Recogniton dataset) are included for training the models and the impact of extra training is assessed to see if there is improvement in performance. The experiments reveal marked improvement in face emotion identification performance reaching values of 94.98% for both validation and test set of FER-2013 dataset and 94.99% on validation set of Face Expression Recognition dataset.

Multi-Network Feature Fusion Facial Emotion Recognition using Nonparametric Method with Augmentation

Facial expression emotion identification and prediction is one of the most difficult problems in computer science. Pre-processing and feature extraction are crucial components of the more conventional methods. For the purpose of emotion identification and prediction using 2D facial expressions, this study targets the Face Expression Recognition dataset and shows the real implementation or assessment of learning algorithms such as various CNNs. Due to its vast potential in areas like artificial intelligence, emotion detection from facial expressions has become an essential requirement. Many efforts have been done on the subject since it is both a challenging and fascinating challenge in computer vision. The focus of this study is on using a convolutional neural network supplemented with data to build a facial emotion recognition system. This method may use face images to identify seven fundamental emotions, including anger, contempt, fear, happiness, neutrality, sadness, and surprise. As well as improving upon the validation accuracy of current models, a convolutional neural network that takes use of data augmentation, feature fusion, and the NCA feature selection approach may assist solve some of their drawbacks. Researchers in this area are focused on improving computer predictions by creating methods to read and codify facial expressions. With deep learning's striking success, many architectures within the framework are being used to further the method's efficacy. We highlight the contributions dealt with, the architecture and databases used, and demonstrate the development by contrasting the offered approaches and the outcomes produced. The purpose of this study is to aid and direct future researchers in the subject by reviewing relevant recent studies and offering suggestions on how to further the field. An innovative feature-based transfer learning technique is created using the pre-trained networks MobileNetV2 and DenseNet-201. The suggested system's recognition rate is 75.31%, which is a significant improvement over the results of the prior feature fusion study.

Predicting Autistic Traits Using Eye Movement during Visual Perspective Taking and Facial Emotion Identification.

Autistic traits are broad in severity and difficult to measure quantitatively. Quantitative measurement would be helpful in determining the effectiveness of training and therapy for autistic traits. The development of eye-tracking technology has made it easier to understand autistic traits. Previous works showed that autistic traits can be predicted from eye movements in the facial emotion identification task. It may be possible to measure autistic traits more accurately than the only prediction from facial emotion identification ability. In this study, we used a visual perspective taking task. The results showed that the Social Responsiveness Scale-2 score, which is associated with autistic traits, was predicted at 0.414 in Spearman's correlation coefficient by using eye movements obtained from the two tasks.

FACIAL EXPRESSION RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS [CNN

While humans have traditionally excelled at discerning emotions through facial expressions, achieving the same capability with a computer program has proven challenging. However, recent advancements in computer vision and machine learning have made it feasible to recognize emotions accurately. This paper introduces a novel facial emotion identification technique known as Convolutional Neural Networks for Facial Emotion Recognition (FERC). FERC utilizes a twopart convolutional neural network (CNN) architecture: The paper consists of two main sections. The first section is dedicated to removing the background from the image, while the second section focuses on extracting the facial feature vector. The FERC model utilizes an expressional vector (EV) to identify five different regular facial expressions. With its single-level CNN approach, FERC offers improved accuracy compared to commonly used techniques. Moreover, prior to constructing the EV, a novel background removal strategy is implemented to address various potential challenges, such as distance from the camera. The application of FERC in emotion detection holds promise for a wide range of applications, including student predictive learning and lie detection.

Pembelajaran Mendalam Pengklasifikasi Ekspresi Wajah Manusia dengan Model Arsitektur Xception pada Metode Convolutional Neural Network

Deep learning is a neural network that creates innovations that give computer-implanted problem-solving expertise. One of the principles of computer vision is a detection system with a vision framework that can identify things encountered in the same manner as a human vision system. Using an artificial intelligence-based Convolutional Neural Network (CNN) model with deep learning techniques, we present a face emotion identification system. The categorization of facial expressions will be utilized as the basis for a face recognition system trained using CNN. The applications are intended to use the OpenCV, Keras, and TensorFlow libraries as the backend. We were discussing the study on the best use of xception architectural models in facial expression recognition systems. Based on the results of these tests, the study obtained an increased accuracy value in training and data testing on an xception architecture model trained for facial expressions using the FER-2013 dataset, resulting in an accuracy value of 66% as well as the value of each average for precision (76%), recall (65%), and F1 score (63%).

An automated hyperparameter tuned deep learning model enabled facial emotion recognition for autonomous vehicle drivers

The progress of autonomous driving cars is a difficult movement that causes problems regarding safety, ethics, social acceptance, and cybersecurity. Currently, the automotive industry is utilizing these technologies to assist drivers with advanced driver assistance systems. This system helps different functions to careful driving and predict drivers' ability of stable driving behavior and road safety. A great number of researches have shown that the driver's emotion is the major factor that handles the emotions, resulting in serious vehicle collisions. As a result, continuous monitoring of drivers' behavior could assist to evaluate their behavior to prevent accidents. The study proposes a new Squirrel Search Optimization with Deep Learning Enabled Facial Emotion Recognition (SSO-DLFER) technique for Autonomous Vehicle Drivers. The proposed SSO-DLFER technique focuses mainly on the identification of driver facial emotions in the AVs. The proposed SSO-DLFER technique follows two major processes namely face detection and emotion recognition. The RetinaNet model is employed at the initial phase of the face detection process. For emotion recognition, the SSO-DLFER technique applied the Neural Architectural Search (NASNet) Large feature extractor with a gated recurrent unit (GRU) model as a classifier. For improving the emotion recognition performance, the SSO-based hyperparameter tuning procedure is performed. The simulation analysis of the SSO-DLFER technique is tested under benchmark datasets and the experimental outcome was investigated under various aspects. The comparative analysis reported the enhanced performance of the SSO-DLFER algorithm on recent approaches.

Educational Facial Emotion Recognition in Children With Autism Spectrum Disorder: A Clinical Trial Study

Objectives: The disability to recognize facial emotions is one of the behavioral problems in autistic children. This study was designed to evaluate the effect of education on the promotion of face recognition. Methods: This single-blind clinical trial study was conducted on children with autism. The participants were allocated with random sampling to the two groups. Autistic children in the intervention group were educated in 10 sessions with 40 facial emotions images, each session lasted 20 min. Autistic children in the control group underwent routine teaching. The assistant researcher completed the Benton questionnaire for each child before and after the study. The obtained data were analyzed with SPSS software, version 22. Results: This study showed that education on facial emotions has good effects on the recognition of facial expressions by children with autism. In addition, a significant difference is identified between the two study groups in identifying facial emotions (P&lt;0.001). Discussion: Education to children with autism improved facial emotion identification. Therefore, according to the increase in children with autism all over the world such as in Iran, it seems that caregivers and health managers should review and use our study results for providing a supportive environment to promote recognition emotional states of the face and ultimately to improve the child’s treatment.

Eye-movement analysis on facial expression for identifying children and adults with neurodevelopmental disorders.

Experienced psychiatrists identify people with autism spectrum disorder (ASD) and schizophrenia (Sz) through interviews based on diagnostic criteria, their responses, and various neuropsychological tests. To improve the clinical diagnosis of neurodevelopmental disorders such as ASD and Sz, the discovery of disorder-specific biomarkers and behavioral indicators with sufficient sensitivity is important. In recent years, studies have been conducted using machine learning to make more accurate predictions. Among various indicators, eye movement, which can be easily obtained, has attracted much attention and various studies have been conducted for ASD and Sz. Eye movement specificity during facial expression recognition has been studied extensively in the past, but modeling taking into account differences in specificity among facial expressions has not been conducted. In this paper, we propose a method to detect ASD or Sz from eye movement during the Facial Emotion Identification Test (FEIT) while considering differences in eye movement due to the facial expressions presented. We also confirm that weighting using the differences improves classification accuracy. Our data set sample consisted of 15 adults with ASD and Sz, 16 controls, and 15 children with ASD and 17 controls. Random forest was used to weight each test and classify the participants as control, ASD, or Sz. The most successful approach used heat maps and convolutional neural networks (CNN) for eye retention. This method classified Sz in adults with 64.5% accuracy, ASD in adults with up to 71.0% accuracy, and ASD in children with 66.7% accuracy. Classifying of ASD result was significantly different (p<.05) by the binomial test with chance rate. The results show a 10% and 16.7% improvement in accuracy, respectively, compared to a model that does not take facial expressions into account. In ASD, this indicates that modeling is effective, which weights the output of each image.

Identification of Facial Emotions Using Reinforcement model under Deep Learning

This paper addresses the identification of facial emotions using a reinforcement model under deep learning. Close-to-perception ability presents a more exhaustive recommendation on human-machine interaction (HMI). Because of the Transfer Self-training (TST), and the Representation Reinforcement Network (RRN), this study offers an active FER arrangement. Two modules are considered for depiction support arranging such as Surface Representation Reinforcement (SurRR) and Semantic Representation Reinforcement (SemaRR). SurRR highlights are detracting component communication centers in feature maps and match face attributes in different facets. Worldwide face settings are semantically sent in channel and dimensional facets of a piece. RRN has a limit concerning involved origin when the edges and computational complication are considerably belittled. Our technique was tried on informational indexes from CK+, RaFD, FERPLUS, and RAFDB, and it was viewed as 100 percent, 98.62 percent, 89.64 percent, and 88.72 percent, individually. Also, the early application exploration shows the way that our strategy can be utilized in HMI.

Predictive Effect of Social Cognitive Skills on Social Anxiety

The main purpose of this study is to examine the relationship between social anxiety and social cognition skills (emotion identification/discrimination, theory of mind, attribution bias, social functioning) and to investigate the predictive power of social cognition skills on social anxiety. In the study, it was also investigated whether social anxiety, social cognition skills and depression scores differed according to gender, age, reported diagnosis status and anxiety level. The sample of the study consisted of a total of 385 participants between the ages of 18-60. The research data were collected from the participants online using the Liebowitz Social Anxiety Scale, the Facial Emotion Identification and Facial Emotion Discrimination Tests, Reading the Mind in the Eyes Test, the Internal, Personal and Situational Attributes Scale, the Social Functioning Scale, the Beck Depression Inventory, and the Sociodemographic Information Form. It was determined that there was a negative relationship between social anxiety and emotion recognition, emotion discrimination and mind reading, and a positive relationship with externalization bias and depression. It was found that depression, emotion recognition and discrimination, mind reading, and antecedent social activities scale scores significantly predicted social anxiety. These variables explain 52% of the total variance related to social anxiety. In addition, when the depression effect was controlled, it was seen that social cognition skills explained 32% of the variance related to social anxiety. It was determined that the deterioration in social cognition skills and depression scores were associated with an increase in social anxiety. According to the results obtained from the research, suggestions were made to prevent social anxiety. It is suggested that it would be beneficial to include activities to develop social cognition skills, which are important for individuals' interpersonal communication, within intervention programs.