Identification Of Facial Expressions Research Articles

AI-Based Facial Emotion Recognition

Facial expression, age, and gender detection have received a lot of interest in recent years due to their wide range of applications in fields including healthcare, security, marketing, and entertainment. With the proliferation of artificial intelligence (AI) techniques, especially deep learning, facial analysis systems' accuracy and efficiency have improved significantly. This paper provides a comprehensive analysis of the most recent advances in AI-based facial expression, age, and gender identification algorithms. We explore the constraints and limitations of current techniques, such as the necessity for big annotated datasets, biases in training data, and interpretability issues in deep learning models. This study seeks to give academics, practitioners, and policymakers, a thorough grasp of the most recent state-of-the-art methodologies and trends in AI-based facial emotion, age, and gender identification, thereby supporting future advancements and responsible usage of these technologies. Keywords: Facial expression, age, gender, identification, deep learning, artificial intelligence

Improving recognition accuracy for facial expressions using scattering wavelet

One of the most evident and meaningful feedback about people’s emotions is through facial expressions. Facial expression recognition is helpful in social networks, marketing, and intelligent education systems. The use of Deep Learning based methods in facial expression identification is widespread, but challenges such as computational complexity and low recognition rate plague these methods. Scatter Wavelet is a type of Deep Learning that extracts features from Gabor filters in a structure similar to convolutional neural networks. This paper presents a new facial expression recognition method based on wavelet scattering that identifies six states: anger, disgust, fear, happiness, sadness, and surprise. The proposed method is simulated using the JAFFE and CK+ databases. The recognition rate of the proposed method is 99.7%, which indicates the superiority of the proposed method in recognizing facial expressions.

Facial expressions and identities recognition in Parkinson disease

Parkinson's Disease (PD) is associated with motor and non-motor symptoms. Among the latter are deficits in matching, identification, and recognition of emotional facial expressions. On one hand, this deficit has been attributed to a dysfunction in emotion processing. Another explanation (which does not exclude the former) links this deficit with reduced facial expressiveness in these patients, which prevents them from properly understanding or embodying emotions. To disentangle the specific contribution of emotion comprehension and that of facial expression processing in PD's observed deficit with emotions we performed two experiments on non-emotional facial expressions. In Experiment 1, a group of PD patients and a group of Healthy Controls (HC) underwent a task of non-emotional expression recognition in faces of different identity and a task of identity recognition in faces with different expression. No differences were observed between the two groups in accuracies. In Experiment 2, PD patients and Healthy Controls underwent a task where they had to recognize the identity of faces encoded through a non-emotional facial expression, through a rigid head movement, or as neutral. Again, no group differences were observed. In none of the two experiments hypomimia scores had a specific effect on expression processing. We conclude that in PD patients the observed impairment with emotional expressions is likely due to a specific deficit for emotions to a greater extent than for facial expressivity processing.

Disrupted properties of functional brain networks in major depressive disorder during emotional face recognition: an EEG study via graph theory analysis.

Previous neuroimaging studies have revealed abnormal brain networks in patients with major depressive disorder (MDD) in emotional processing. While any cognitive task consists of a series of stages, little is yet known about the topology of functional brain networks in MDD for these stages during emotional face recognition. To address this problem, electroencephalography (EEG)-based functional brain networks of MDD patients at different stages of facial information processing were investigated in this study. First, EEG signals were collected from 16 patients with MDD and 18 age-, gender-, and education-matched normal subjects when performing an emotional face recognition task. Second, the global field power (GFP) method was employed to divide group-averaged event-related potentials into different stages. Third, using the phase transfer entropy (PTE) approach, the brain networks of MDD patients and normal individuals were constructed for each stage in negative and positive face processing, respectively. Finally, we compared the topological properties of brain networks of each stage between the two groups using graph theory approaches. The results showed that the analyzed three stages of emotional face processing corresponded to specific neurophysiological phases, namely, visual perception, face recognition, and emotional decision-making. It was also demonstrated that depressed patients showed abnormally decreased characteristic path length at the visual perception stage of negative face recognition and normalized characteristic path length in the stage of emotional decision-making during positive face processing compared to healthy subjects. Furthermore, while both the MDD and normal groups' brain networks were found to exhibit small-world network characteristics, the brain network of patients with depression tended to be randomized. Moreover, for patients with MDD, the centro-parietal region may lose its status as a hub in the process of facial expression identification. Together, our findings suggested that altered emotional function in MDD patients might be associated with disruptions in the topological organization of functional brain networks during emotional face recognition, which further deepened our understanding of the emotion processing dysfunction underlying MDD.

Facial expression recognition with computer vision

Facial Expression Recognition (FER) is a specialized field within the domains of computer vision and pattern recognition, which is dedicated to the automated identification and examination of facial expressions. Facial expression recognition (FER) has attracted considerable scholarly interest in recent years owing to its diverse array of applications and its potential ramifications across multiple disciplines, such as psychology, human-computer interaction, marketing, and security systems. The objective of this study is to present a thorough examination of the scholarly progression of FER, elucidating the significant achievements, approaches, and obstacles encountered by researchers in this domain. The study presents a selection of databases that are appropriate for Facial Expression Recognition (FER) and conducts a comparative analysis of these databases. The primary methodologies are examined, and recommendations are provided for each stage. In conclusion, this research presents several suggestions for addressing both obstacles and potential in future research endeavors.

Multi-class Facial Emotion Expression Identification Using DL-Based Feature Extraction with Classification Models

Facial expression detection from images and videos has recently gained attention due to the wide variety of applications it has found in the field of computer vision such as advanced driving assistance systems (ADAS), augmented and virtual reality (AR/VR), video retrieval, and security systems. Facial terms, body language, hand gestures, and eye contact have all been researched as a means of deciphering and understanding human emotions. Automated facial expression recognition (FER) is a significant visual recognition procedure because human emotions are a worldwide signal used in non-verbal communication. The six primary universal manifestations of emotion are characterized as happiness, sadness, anger, contempt, fear, and surprise. While the accuracy of deep learning (DL)-based approaches has improved significantly across many domains, automated FER remains a difficult undertaking, especially when it comes to real-world applications. In this research work, two publicly available datasets such as FER2013 and EMOTIC are considered for validation process. Initially, pre-processing includes histogram equalization, image normalization and face detection using Multi-task Cascaded Convolutional Network (MT-CNN) is used. Then, DL-based EfficinetNetB0 is used to extract the features of pre-processed images for further process. Finally, the Weighted Kernel Extreme Learning Machine (WKELM) is used for classification of emotions, where the kernel parameters are optimized by Red Fox Optimizer (RFO). From the experimental analysis, the proposed model achieved 95.82% of accuracy, 95.81% of F1-score and 95% of recall for the testing data.

Facial emotion recognition in individuals with mild cognitive impairment: An exploratory study.

Understanding facial emotions is fundamental to interact in social environments and modify behavior accordingly. Neurodegenerative processes can progressively transform affective responses and affect social competence. This exploratory study examined the neurocognitive correlates of face recognition, in individuals with two mild cognitive impairment (MCI) etiologies (prodromal to dementia - MCI, or consequent to Parkinson's disease - PD-MCI). Performance on the identification and memorization of neutral and emotional facial expressions was assessed in 31 individuals with MCI, 26 with PD-MCI, and 30 healthy controls (HC). Individuals with MCI exhibited selective impairment in recognizing faces expressing fear, along with difficulties in remembering both neutral and emotional faces. Conversely, individuals with PD-MCI showed no differences compared with the HC in either emotion recognition or memory. In MCI, no significant association emerged between the memory for facial expressions and cognitive difficulties. In PD-MCI, regression analyses showed significant associations with higher-level cognitive functions in the emotional memory task, suggesting the presence of compensatory mechanisms. In a subset of participants, voxel-based morphometry revealed that the performance on emotional tasks correlated with regional changes in gray matter volume. The performance in the matching of negative expressions was predicted by volumetric changes in brain areas engaged in face and emotional processing, in particular increased volume in thalamic nuclei and atrophy in the right parietal cortex. Future studies should leverage on neuroimaging data to determine whether differences in emotional recognition are mediated by pathology-specific atrophic patterns.

Emotion Recognition based on Facial Expression Identification using Deep Learning Algorithm for Automation Music Healing Application

Daily activities, particularly in the workplace or other societal settings, can cause stress and pressure for those who must manage them. This stress can lead to decreased performance and achievement in daily tasks, and in more severe cases, individuals may seek consultation with a psychiatrist to address their stress and pressure. This work presents a system for emotion identification based on face expression recognition system. The system processes the face expression image using Convolution Neural Network (CNN). The face expression image is extracted and modeled based on CNN method. The identification result sent to database which transferred to the Android application played the song based on emotion identification. The system is installed on an Android cell phone, making it flexible and portable. The system has achieved 70% and 80% accuracy in emotion detection during training and testing, respectively.

Neurobiology of emotions in animal relationships: Facial expressions and their biological functions in mammals

This article aims to analyze the biological and communicative function of facial expressions and their relation to emotions in mammals. Facial expressions and their causes constitute an important yet largely unexplored field of scientific research. While the clinical usefulness of these expressions for recognizing pain in many species has been demonstrated, there is evidence that animals can also emit facial movements with other connotations. Reports show that facial expressions have a biological function by predicting and promoting social interaction in species that form complex social groups, but their meaning depends on the social context. For animals, the unconscious identification of facial expressions also implies an emotional value by modifying the compensatory physiological response, as occurs in species that have close relationships with humans. This corpus of evidence suggests that facial expressions have a kind of communicative function and may transmit emotions and, therefore, participate in affiliative or adverse relations as animals develop. Hence, exploring whether this nonverbal behavior may perform such a dual function is necessary.

Предикторы невербального опознания эмоциональных экспрессий лица

<p>The work was carried out in the paradigm of analyzing the structure of erroneous responses when performing the identification of emotional facial expressions. The subjects performed the task of non-verbal recognition of facial expressions at tachistoscopic exposure time (16 ms, 8 ms, 4 ms). The search for predictors that determine the partial proportions of responses for different combinations (response, expression shown) was performed among the subscales of the Boyko empathy questionnaire, the EmIn questionnaire, the analytical-holistic scale, and the Alekseev thinking styles questionnaire. The study involved students from Moscow universities. Sample size: 49 people, 9 men, 40 women, average age m=22.2 years, sd=4.45.The identified predictors are specific (not universal) in nature, different for different combinations of the presented image and the assessment performed. We believe that this result should be interpreted as indicating the presence of competing parallel processes associated with testing for the presence of different emotional expressions. The selected system of predictors is not complete. A promising direction for further research is expanding the scope of the search for predictors through methods that measure individual personality characteristics and the level of anxiety.</p>

Facial emotion recognition based on improved ResNet

A common and difficult task in computer vision is the identification of facial expressions of emotion. Facial emotion recognition has shown great promise in many areas including healthcare, robotic communication and customer service. However, the variability of human appearance and muscle movement leads to difficulties in facial emotion recognition. Therefore, deeper convolutional neural networks are introduced to recognize facial emotions. The residual network (ResNet) can be built in a deep architecture that can solve the degradation problem when the depth of the network increases. In this paper, an improved ResNet50 is proposed to implement facial emotion recognition. Specifically, the proposed model appends two blocks consisting of fully connected layers to the ResnNet50. The layers are stacked with shortcut connections to solve the degradation problem and to make the training process smoother. The accuracy achieved by the improved model on the Facial Emotion Recognition 2013 dataset (FER-2013) is 13.31% higher than that of the ResNet50. Experimental data indicate that the improved model performs efficiently in facial emotion recognition due to shortcut connectivity and the addition of fully connected layers. Meanwhile, the degradation and gradient disappearance problems are improved.

Assessing the influence of emotional expressions on perceptual sensitivity to faces overcoming interocular suppression.

Detecting faces and identifying their emotional expressions are essential for social interaction. The importance of expressions has prompted suggestions that some emotionally relevant facial features may be processed unconsciously, and it has been further suggested that this unconscious processing yields preferential access to awareness. Evidence for such preferential access has predominantly come from reaction times in the breaking continuous flash suppression (bCFS) paradigm, which measures how long it takes different stimuli to overcome interocular suppression. For instance, it has been claimed that fearful expressions break through suppression faster than neutral expressions. However, in the bCFS procedure, observers can decide how much information they receive before committing to a report, so although their responses may reflect differential detection sensitivity, they may also be influenced by differences in decision criteria, stimulus identification, and response production processes. Here, we employ a procedure that directly measures sensitivity for both face detection and identification of facial expressions, using predefined exposure durations. We apply diverse psychophysical approaches-forced-choice localization, presence/absence detection, and staircase-based threshold measurement; across six experiments, we find that emotional expressions do not alter detection sensitivity to faces as they break through CFS. Our findings constrain the possible mechanisms underlying previous findings: faster reporting of emotional expressions' breakthrough into awareness is unlikely to be due to the presence of emotion affecting perceptual sensitivity; the source of such effects is likely to reside in one of the many other processes that influence response times. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Comparison of the Identification of Emotional Facial Expressions from Full Faces with only the Eyes and the Mouth Regions in Young Indian Adults: an Exploratory Study.

Identification of emotional facial expressions (EFEs) is important in interpersonal communication. Six 'universal' EFEs are known, though accuracy of their identification varies. EFEs involve anatomical changes in certain regions of the face, especially eyes and mouth. But whether other areas of the face are just as important in their identification is still debated. This study was conducted to compare the accuracy of identification of universal EFEs under full-face and partial face conditions (only showing the eyes and the mouth regions). An analytical cross-sectional study was conducted among 140 young Indian adults. They were divided into two equal groups and shown the six universal EFEs in two sets, one with full-face images, and the other with images showing just the eyes and the mouth regions on a computer screen. The participants were asked to identify each of the EFE and their responses were analyzed. Mean age was 21.3 1.7 years for full face group, and 21.2 1.6 years for the partial face group. Most were men, from rural areas and from upper socioeconomic status families, and many of them were students. EFE identification was significantly higher for part-face group compared to full-face group (p-value .0007). Participants of both groups identified happiness the best (100%). For other EFEs, part-face images were identified more accurately than full-face images, except for disgust. These differences were statistically significant except for anger and fear. Among young Indian adults, accuracy of identification of universal EFEs was high, which was significantly enhanced for all except disgust, when only combinations of eyes and mouth were shown, suggesting that other facial regions serve as distractors in EFE identification. 1. Identification of universal EFEs was higher from partial faces (combination of eyes and mouth) as compared to full-face EFEs for all emotions except disgust. 2. This suggests that other regions of face serve as potential distractors in the identification of emotions, except for disgust, where these regions provide more information.

Increased N250 elicited by facial familiarity: An ERP study including the face inversion effect and facial emotion processing

The present study aims to explore how familiarity modulates the neural processing of faces under different conditions: upright or inverted, neutral or emotional. To this purpose, 32 participants (25 female; age: M = 27.7 years, SD = 9.3) performed two face/emotion identification tasks during EEG recording. In the first task, to study facial processing, three different categories of facial stimuli were presented during a target detection task: famous familiar faces, faces of loved ones, and unfamiliar faces. To explore the face inversion effect according to each level of familiarity, these facial stimuli were also presented upside down. In the second task, to study emotional face processing, an emotional identification task on personally familiar and unfamiliar faces was conducted. The behavioural results showed an improved performance in the identification of facial expressions of emotion with the increase of facial familiarity, consistent with the previous literature. Regarding electrophysiological results, we found increased amplitudes of the P100, N170, and N250 for inverted compared to upright faces, independently of their degree of familiarity. Moreover, we did not find familiarity effects at the P100 and N170 time-windows, but we found that N250 amplitude was larger for personally familiar compared to unfamiliar faces. This result supports the reasoning that the facial familiarity increases the neural activity during the N250 time-window, which may be explained by the processing of additional information prompted by the viewing of our loved ones faces, in contrast to what happens with unfamiliar individuals.

Emotional Analysis using Deep Learning

Emotions are mental states that accompany physiological changes in the face, resulting in facial expressions. Sympathy, anger, worry, joy, fright, and other significant emotions are a few examples. Facial expressions play a significant role in non-verbal communication because they encapsulate a person's emotions. There has been a great deal of research done on computer modelling of human emotions. Computer modelling of human emotions has been made possible by computer technology. However, it is still in its infancy. The authors attempted to overcome limitations and create new opportunities as well as gain a better understanding and implement this simple form of human interaction in proposed computer-aided world. It has been made possible to evaluate and interpret genuine facial expressions in real time thanks to new techniques for collecting facial expressions and quick, highresolution pictures. The FER (Facial Expression Recognition) method currently relies on motionless frames, which makes it very hard to recognize foreground from background in the absence of motion information. This study describes a real-time facial expression identification system that detects faces using HAAR cascading classification and classifies facial expressions using convolutional neural networks. The system utilizes a webcam to dynamically display emotion text and accurately categorizes seven major emotions, including anger, disgust, fear, happiness, sadness, surprise, and neutrality. Real-time facial expression recognition may be utilised in a number of real-world applications, including as airport security, trade, and medical monitoring.

Using AI to Detect Pain through Facial Expressions: A Review.

Pain assessment is a complex task largely dependent on the patient's self-report. Artificial intelligence (AI) has emerged as a promising tool for automating and objectifying pain assessment through the identification of pain-related facial expressions. However, the capabilities and potential of AI in clinical settings are still largely unknown to many medical professionals. In this literature review, we present a conceptual understanding of the application of AI to detect pain through facial expressions. We provide an overview of the current state of the art as well as the technical foundations of AI/ML techniques used in pain detection. We highlight the ethical challenges and the limitations associated with the use of AI in pain detection, such as the scarcity of databases, confounding factors, and medical conditions that affect the shape and mobility of the face. The review also highlights the potential impact of AI on pain assessment in clinical practice and lays the groundwork for further study in this area.

Enhancing Feature Extraction Technique Through Spatial Deep Learning Model for Facial Emotion Detection

Automatic facial expression analysis is a fascinating and difficult subject that has implications in a wide range of fields, including human–computer interaction and data-driven approaches. Based on face traits, a variety of techniques are employed to identify emotions. This article examines various recent explorations into automatic data-driven approaches and handcrafted approaches for recognising face emotions. These approaches offer computationally complex solutions that provide good accuracy when training and testing are conducted on the same datasets, but they perform less well on the most difficult realistic dataset, FER-2013. The article's goal is to present a robust model with lower computational complexity that can predict emotion classes more accurately than current methods and aid society in finding a realistic, all-encompassing solution for the facial expression system. A crucial step in good facial expression identification is extracting appropriate features from the face images. In this paper, we examine how well-known deep learning techniques perform when it comes to facial expression recognition and propose a convolutional neural network-based enhanced version of a spatial deep learning model for the most relevant feature extraction with less computational complexity. That gives a significant improvement on the most challenging dataset, FER-2013, which has the problems of occlusions, scale, and illumination variations, resulting in the best feature extraction and classification and maximizing the accuracy, i.e., 74.92%. It also maximizes the correct prediction of emotions at 99.47%, and 98.5% for a large number of samples on the CK+ and FERG datasets, respectively. It is capable of focusing on the major features of the face and achieving greater accuracy over previous fashions.

Face-Selective Patches in Marmosets Are Involved in Dynamic and Static Facial Expression Processing.

The correct identification of facial expressions is critical for understanding the intention of others during social communication in the daily life of all primates. Here we used ultra-high-field fMRI at 9.4 T to investigate the neural network activated by facial expressions in awake New World common marmosets from both male and female sex, and to determine the effect of facial motions on this network. We further explored how the face-patch network is involved in the processing of facial expressions. Our results show that dynamic and static facial expressions activate face patches in temporal and frontal areas (O, PV, PD, MD, AD, and PL) as well as in the amygdala, with stronger responses for negative faces, also associated with an increase of the respiration rates of the monkey. Processing of dynamic facial expressions involves an extended network recruiting additional regions not known to be part of the face-processing network, suggesting that face motions may facilitate the recognition of facial expressions. We report for the first time in New World marmosets that the perception and identification of changeable facial expressions, vital for social communication, recruit face-selective brain patches also involved in face detection processing and are associated with an increase of arousal.SIGNIFICANCE STATEMENT Recent research in humans and nonhuman primates has highlighted the importance to correctly recognize and process facial expressions to understand others' emotions in social interactions. The current study focuses on the fMRI responses of emotional facial expressions in the common marmoset (Callithrix jacchus), a New World primate species sharing several similarities of social behavior with humans. Our results reveal that temporal and frontal face patches are involved in both basic face detection and facial expression processing. The specific recruitment of these patches for negative faces associated with an increase of the arousal level show that marmosets process facial expressions of their congener, vital for social communication.

A VGG16 Based Hybrid Deep Convolutional Neural Network Based Real-Time Video Frame Emotion Detection System for Affective Human Computer Interaction

Abstract: Facial emotion recognition is field of computer vision that deals with recognizing the emotions of the user by facial expression analysis. The development of facial emotion recognition technology has the potential to enhance human-computer interaction by making machines more responsive to human emotions. For recognizing facial emotions, a variety of machine learning methods have been used, such as deep neural networks, support vector machines, decision trees, and among others, Convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their combinations have all been suggested as designs, which have proven especially effective in this field. However, despite substantial advancements in the area, there are still difficulties in attaining accurate and consistent facial expression identification in real-time everyday situations. The effectiveness and usability of face expression detection systems for diverse applications must therefore be improved and efficient systems be developed to expand the focus on further applications of emotions in achieving affective Human computer interaction. My main objective in this paper will be to focus on being able to accurately identify user’s emotion in a real world scenario. I will be a designing a hybrid deep learning CNN architectures for the real world implementation of the real-time emotion recognition system for improving Human Computer Interaction

Emotion processing in maltreated boys and girls: Evidence for latent vulnerability

Evidence of alterations in emotion processing in maltreated youth has been hypothesized to reflect latent vulnerability for psychopathology. However, previous studies have not systematically examined the influence of psychopathology on the results. Here, we examined emotion recognition and learning in youth who differed in terms of presence vs. absence of maltreatment and psychopathology and tested for potential sex effects. Maltreatment and psychopathology were assessed in 828 youth (514 females) aged 9–18 years using diagnostic interviews and self- and parent-report questionnaires. Emotion recognition was assessed via identification of morphed facial expressions of six universal emotions. For emotion learning, reward and punishment values were assigned to novel stimuli and participants had to learn to correctly respond/withhold response to stimuli to maximize points. A three-way interaction of maltreatment by psychopathology by emotion indicated that when psychopathology was low, maltreated youth were less accurate than non-maltreated youth for happy, fear and disgust. A three-way interaction of sex, maltreatment and emotion indicated that maltreated girls and boys were impaired for fear, but girls showed an impairment for happy, while boys for disgust. There were no effects of maltreatment, psychopathology, or sex on reward learning. However, a two-way interaction between sex and maltreatment showed that maltreated girls were worse at learning from punishment relative to non-maltreated girls, while maltreated boys were better than non-maltreated boys. The study provides the first clear evidence of latent-vulnerability in emotion recognition in maltreated youth and suggests that girls and boys might be characterized by distinct profiles of emotion recognition and learning following maltreatment.