Expression Recognition Research Articles

Enhancing feature selection for multi-pose facial expression recognition using a hybrid of quantum inspired firefly algorithm and artificial bee colony algorithm

Facial expression recognition (FER) has advanced applications in various disciplines, including computer vision, Internet of Things, and artificial intelligence, supporting diverse domains such as medical escort services, learning analysis, fatigue detection, and human-computer interaction. The accuracy of these systems is of utmost concern and depends on effective feature selection, which directly impacts their ability to accurately detect facial expressions across various poses. This research proposes a new hybrid approach called QIFABC (Hybrid Quantum-Inspired Firefly and Artificial Bee Colony Algorithm), which combines the Quantum-Inspired Firefly Algorithm (QIFA) with the Artificial Bee Colony (ABC) method to enhance feature selection for a multi-pose facial expression recognition system. The proposed algorithm uses the attributes of both the QIFA and ABC algorithms to enhance search space exploration, thereby improving the robustness of features in FER. The firefly agents initially move toward the brightest firefly until identified, then search transition to the ABC algorithm, targeting positions with the highest nectar quality. In order to evaluate the efficacy of the proposed QIFABC algorithm, feature selection is also conducted using QIFA, FA, and ABC algorithms. The evaluated features are utilized for classifying face expressions by utilizing the deep neural network model, ResNet-50. The presented FER system has been tested using multi-pose facial expression benchmark datasets, including RaF (Radboud Faces) and KDEF (Karolinska Directed Emotional Faces). Experimental results show that the proposed QIFABC with ResNet50 method achieves an accuracy of 98.93%, 94.11%, and 91.79% for front, diagonal, and profile poses on the RaF dataset, respectively, and 98.47%, 93.88%, and 91.58% on the KDEF dataset.

Enhancing facial expression recognition in uncontrolled environment: a lightweight CNN approach with pre-processing

Enhancing facial expression recognition in uncontrolled environment: a lightweight CNN approach with pre-processing

Breathable, superhydrophobic and multifunctional Janus nanofibers for dual-mode passive thermal management/facial expression recognition with deep learning

Breathable, superhydrophobic and multifunctional Janus nanofibers for dual-mode passive thermal management/facial expression recognition with deep learning

Developing a Machine Learning-Based Algorithm for Facial Expression Recognition

With the rise of artificial intelligence, machine learning (ML) is increasingly integrated into daily life. Facial expressions, lasting about 1/20th of a second and difficult to conceal, convey nuanced emotions beyond words. They are categorized as macro expressions, displayed under normal circumstances, and micro expressions, fleeting and subconscious. However, recognizing expressions in photos is challenging due to varied backgrounds, appearances, age, and race, impacting accuracy. Addressing this, our research focuses on a machine learning-based facial expression recognition algorithm. Our experimental results, using CASIA and NTERFACE databases, demonstrate that ML outperforms KNN and ANN with recognition rates 41.54% and 26.94% higher, respectively. By adaptively learning global and local features from complete expression images and face parts, our algorithm offers practical and significant advancements in expression recognition.

ADMF-ER: a novel approach for wild expression recognition integrating adaptive dropout and multi-level features

ADMF-ER: a novel approach for wild expression recognition integrating adaptive dropout and multi-level features

Information gap based knowledge distillation for occluded facial expression recognition

Facial Expression Recognition (FER) with occlusion presents a challenging task in computer vision because facial occlusions result in poor visual data features. Recently, the region attention technique has been introduced to address this problem by researchers, which make the model perceive occluded regions of the face and prioritize the most discriminative non-occluded regions. However, in real-world scenarios, facial images are influenced by various factors, including hair, masks and sunglasses, making it difficult to extract high-quality features from these occluded facial images. This inevitably limits the effectiveness of attention mechanisms. In this paper, we observe a correlation in facial emotion features from the same image, both with and without occlusion. This correlation contributes to addressing the issue of facial occlusions. To this end, we propose a Information Gap based Knowledge Distillation (IGKD) to explore the latent relationship. Specifically, our approach involves feeding non-occluded and masked images into separate teacher and student networks. Due to the incomplete emotion information in the masked images, there exists an information gap between the teacher and student networks. During training, we aim to minimize this gap to enable the student network to learn this relationship. To enhance the teacher’s guidance, we introduce a joint learning strategy where the teacher conducts knowledge distillation on the student during the training of the teacher. Additionally, we introduce two novel constraints, called knowledge learn and knowledge feedback loss, to supervise and optimize both the teacher and student networks. The reported experimental results show that IGKD outperforms other algorithms on four benchmark datasets. Specifically, our IGKD achieves 87.57% on Occlusion-RAF-DB, 87.33% on Occlusion-FERPlus, 64.86% on Occlusion-AffectNet, and 73.25% on FED-RO, clearly demonstrating its effectiveness and robustness.

Design and Research of Facial Expression Recognition System Based on Key Point Extraction

Design and Research of Facial Expression Recognition System Based on Key Point Extraction

Sample-Cohesive Pose-Aware Contrastive Facial Representation Learning

Abstract Self-supervised facial representation learning (SFRL) methods, especially contrastive learning (CL) methods, have been increasingly popular due to their ability to perform face understanding without heavily relying on large-scale well-annotated datasets. However, analytically, current CL-based SFRL methods still perform unsatisfactorily in learning facial representations due to their tendency to learn pose-insensitive features, resulting in the loss of some useful pose details. This could be due to the inappropriate positive/negative pair selection within CL. To conquer this challenge, we propose a Pose-disentangled Contrastive Facial Representation Learning (PCFRL) framework to enhance pose awareness for SFRL. We achieve this by explicitly disentangling the pose-aware features from non-pose face-aware features and introducing appropriate sample calibration schemes for better CL with the disentangled features. In PCFRL, we first devise a pose-disentangled decoder with a delicately designed orthogonalizing regulation to perform the disentanglement; therefore, the learning on the pose-aware and non-pose face-aware features would not affect each other. Then, we introduce a false-negative pair calibration module to overcome the issue that the two types of disentangled features may not share the same negative pairs for CL. Our calibration employs a novel neighborhood-cohesive pair alignment method to identify pose and face false-negative pairs, respectively, and further help calibrate them to appropriate positive pairs. Lastly, we devise two calibrated CL losses, namely calibrated pose-aware and face-aware CL losses, for adaptively learning the calibrated pairs more effectively, ultimately enhancing the learning with the disentangled features and providing robust facial representations for various downstream tasks. In the experiments, we perform linear evaluations on four challenging downstream facial tasks with SFRL using our method, including facial expression recognition, face recognition, facial action unit detection, and head pose estimation. Experimental results show that PCFRL outperforms existing state-of-the-art methods by a substantial margin, demonstrating the importance of improving pose awareness for SFRL. Our evaluation code and model will be available at https://github.com/fulaoze/CV/tree/main.

Real-Time Sensor-Based and Self-Reported Emotional Perceptions of Urban Green-Blue Spaces: Exploring Gender Differences with FER and SAM

Urban green-blue spaces (UGBS) are increasingly recognized for their benefits to physical and mental well-being. However, research on real-time gender-specific emotional responses to UGBS remains limited. To address this gap, a dual-method approach combining facial expression recognition (FER) and self-reported measures to investigate gender differences in real-time emotional evaluations of UGBS was developed. Using static images from Google Street View as stimuli, a self-reporting experiment involving 108 participants provided insights into subjective emotional experiences. Subsequently, a FER experiment, utilizing 360-degree video stimuli, captured over two million data points, validating the feasibility and advantages of real-time emotion monitoring. The findings revealed distinct gender-specific emotional patterns: women experienced stronger pleasant emotions and preferred scenes evoking higher arousal, while men demonstrated sharper responses and rated scenes with peak valence emotions more favorably. Grass elicited relaxation and delight in women and arousal in men, whereas blue spaces induced calmness across genders, with men reporting greater relaxation as water content increased. The study underscores the potential of FER technology in assessing real-time emotional responses, providing actionable insights for inclusive urban planning. By integrating advanced tools and participatory design approaches, urban planners can develop strategies that enhance emotional well-being and create livable cities that support diverse user needs.

Can artificial intelligence understand our emotions? Deep learning applications with face recognition

Abstract Objective The aim of this study is to evaluate the ability to detect emotions from human facial expressions via facial recognition technologies and analyze the effectiveness of deep learning models in this process. Method This research was conducted between 01.04 and 01.07.2024. The data of the study were taken from the open access site https://www.kaggle.com/datasets/msambare/fer2013 (Kaggle, 2024). Python 3.8 is used in this study. The FER-2013 (Facial Expression Recognition 2013) dataset is a comprehensive collection of facial images labeled with various emotions. The dataset contains 35,887 grayscale facial images. Each image has a size of 48 × 48 pixels. The dataset consists of images belonging to 7 emotion categories: anger, disgust, fear, happy, sad, confused, and neutral. Results In our experiments on the FER2013 dataset, we evaluated the performance of three different models: MobileNetV3-L, EfficientNetV2-L, and our proposed EfficientMobileNet. The evaluation criteria were based on sensitivity, specificity, accuracy, and F1 scores to assess the effectiveness of each model comprehensively. The EfficientMobileNet model outperformed MobileNetV3-L and EfficientNetV2-L in all measured performance metrics. EfficientMobileNet was the most successful model for predicting emotions, with an accuracy of 77.6%. Conclusion The impressive results obtained by EfficientMobileNet on the Fer2013 dataset show potential for wider application, especially in image classification tasks involving low-quality or small-scale images. This performance supports the idea of the potential for further improvements in neural network architecture and model efficiency and accuracy. Future work should focus on optimizing the model for more challenging datasets, studying the impact of different architectural adjustments, and investigating the scalability of EfficientMobileNet across various domains and applications.

Automatic facial expression localization and recognition across a large range of emotions

Automatic facial expression localization and recognition across a large range of emotions

Recognition and classification of facial expression using artificial intelligence as a key of early detection in neurological disorders.

The recognition and classification of facial expressions using artificial intelligence (AI) presents a promising avenue for early detection and monitoring ofneurodegenerative disorders. This narrative review critically examines the current state of AI-driven facial expression analysis in the context of neurodegenerative diseases, such as Alzheimer's and Parkinson's. We discuss thepotential of AI techniques, including deep learning andcomputer vision, to accurately interpret and categorize subtle changes in facial expressions associated with thesepathological conditions. Furthermore, we explore theroleof facial expression recognition as a noninvasive, cost-effective tool for screening, disease progression tracking, and personalized intervention in neurodegenerative disorders. The review also addresses the challenges, ethical considerations, and future prospects of integrating AI-based facial expression analysis into clinical practice forearly intervention and improved quality of life for individuals at risk ofor affected by neurodegenerative diseases.

Categorization processes in emotion expression recognition: The roles of language and essentialism.

We examined categorical processing biases in the perception and recognition of facial expressions of emotion across two studies. In both studies, participants first learned to discriminate between two ambiguous facial expressions of emotion selected from the middle of a continuous array of blended expressions (i.e., an array created from morphing images of two facial expressions together, with still images selected in equidistant increments). Participants were then asked to recognize the specific expressions they were trained to discriminate. In Study 1, target expressions labeled with emotion words (e.g., more angry face) during discrimination and recognition tasks were misremembered as more perceptually distinct from one another and therefore more perceptually similar to the stereotypical expression for their labeled emotion category than they were in reality. Critically, in Study 2, these recognition biases were significantly reduced or absent in conditions where the target expressions were not labeled with emotion words (e.g., Face A), demonstrating the role of emotion words in promoting categorical processing biases in emotion recognition. Moreover, in the absence of emotion labels, peoples' beliefs about the nature of emotion categories were related to the extent to which they employed categorical processing during emotion perception and recognition. Specifically, people with more essentialist beliefs about emotion categories-believing emotion categories are more innate, biologically-based, and immutable-exhibited more pronounced categorical processing biases during emotion recognition. Findings shed light on the critical role of language and cognition in constructing emotion and add to empirical findings on categorical processing in emotion perception. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Real-Time Facial Expression Recognition Based on Image Processing in Virtual Reality

More virtual reality (VR) scenarios have become more prevalent in recent years. More and more people are getting into VR, meaning that objective physiological measures to assess a user's emotional state automatically are becoming more critical. Individuals’ emotional states impact their behaviour, opinions, emotions, and decisions. They may be used to analyze VR experiences and make systems react to and engage with the user’s emotions. VR environments require users to wear head-mounted displays (HMDs), blocking off their upper faces. That makes traditional Facial Expression Recognition (FER) approaches very limited in their usefulness. Thus, a Deep Learning (DL) solution combined with image processing is utilized to classify universal emotions: sadness, happiness, disgust, anger, fear and surprise. Hence, this paper suggests the Deep Automatic Facial Expression Recognition Model (DAFERM) for interactive virtual reality (VR) applications such as intelligent education, social networks, and virtual training. Two main parts comprise the system: one that uses deep neural networks (DNNs) for facial emotion identification and another that automatically tracks and segments faces. The system begins by following a marker on the front of the head-mounted display (HMD). With the help of the spatial data that has been retrieved, the positions and rotations of the face are estimated to segment the mouth. Finally, the system interacts with DNN using the pixels processed by the lips. It obtains the facial expression results in real time using an adaptive method for histogram-based mouth segmentation.

Facial Expression Recognition of Al-Qur'an Memorization Students Using Convolutional Neural Network

Facial expression recognition technology has advanced significantly and has become an intriguing topic of study. This research focuses on the facial expressions of Al-Qur’an memorization students, which naturally reveal various aspects of their engagement, understanding, and emotional barriers about the verses being memorized. The issue is that facial expression recognition still lacks optimal accuracy, and the need for a better algorithmic model to improve accuracy is evident. Therefore, an intelligent computing system is required to address this problem. This study aims to enhance the accuracy of facial expression recognition in Al-Qur’an memorization students using the Convolutional Neural Network (CNN) method, classifying facial expressions such as happy, neutral, and tired based on collected facial image data, achieving improved accuracy. The first stage involves capturing image data via CCTV, followed by preprocessing, training the CNN model, result analysis, and model evaluation. By using the CNN method to recognize the facial expressions of Al-Qur’an memorization students, a high accuracy of 84% was achieved with a loss value of 14.9.

A facial expression recognition network using hybrid feature extraction.

Facial expression recognition faces great challenges due to factors such as face similarity, image quality, and age variation. Although various existing end-to-end Convolutional Neural Network (CNN) architectures have achieved good classification results in facial expression recognition tasks, these network architectures share a common drawback that the convolutional kernel can only compute the correlation between elements of a localized region when extracting expression features from an image. This leads to difficulties for the network to explore the relationship between all the elements that make up a complete expression. In response to this issue, this article proposes a facial expression recognition network called HFE-Net. In order to capture the subtle changes of expression features and the whole facial expression information at the same time, HFE-Net proposed a Hybrid Feature Extraction Block. Specifically, Hybrid Feature Extraction Block consists of parallel Feature Fusion Device and Multi-head Self-attention. Among them, Feature Fusion Device not only extracts the local information in expression features, but also measures the correlation between distant elements in expression features, which helps the network to focus more on the target region while realizing the information interaction between distant features. And Multi-head Self-attention can calculate the correlation between the overall elements in the feature map, which helps the network to extract the overall information of the expression features. We conducted a lot of experiments on four publicly available facial expression datasets and verified that the Hybrid Feature Extraction Block constructed in this paper can improve the network's recognition ability for facial expressions.

Supplemental Material for Categorization Processes in Emotion Expression Recognition: The Roles of Language and Essentialism

Supplemental Material for Categorization Processes in Emotion Expression Recognition: The Roles of Language and Essentialism

Ultrasensitive iontronic pressure sensor based on microstructure ionogel dielectric layer for wearable electronics

Flexible pressure sensors show great promise for applications in such fields as electronic skin, healthcare, and intelligent robotics. Traditional capacitive pressure sensors, however, face the problem of low sensitivity, which limits their wider application. In this paper, a flexible capacitive pressure sensor with microstructured ionization layer is fabricated by a sandwich-type process, with a low-cost and simple process of inverted molding with sandpapers being used to form a thermoplastic polyurethane elastomer ionic film with double-sided microstructure as the dielectric layer of the sensor, with silver nanowires as electrodes. The operating mechanism of this iontronic pressure sensor is analyzed using a graphical method, and the sensor is tested on a pressure platform. The test results show that the sensor has ultrahigh pressure sensitivities of 3.744 and 1.689 kPa−1 at low (0–20 kPa) and high (20–800 kPa) pressures, respectively, as well as a rapid response time (100 ms), and it exhibits good stability and repeatability. The sensor can be used for sensitive monitoring of activities such as finger bending, and for facial expression (smile, frown) recognition, as well as speech recognition.

Enhancing Facial Expression Recognition with Robust CNN Architectures and Adaptive Preprocessing Techniques

Facial expression recognition (FER) is an essential technology at the intersection of artificial intelligence (AI), computer vision, and psychology. This study proposes a novel framework for FER, aiming to improve system robustness and generalization, especially under variable real-world conditions. Using the FER2013 dataset, this research combines an adaptive preprocessing pipeline with a custom Convolutional Neural Network (CNN) architecture. Key preprocessing steps include normalization, rotation, and flipping to improve data quality and diversity. The CNN architecture combines regularization methods, including dropout and L2 regularization. Dynamic hyperparameter tuning and early stopping optimize performance and prevent overfitting. Normalized confusion matrix indicating strong recognition for well-represented emotions, such as happiness with 86% accuracy, and challenges with underrepresented categories like disgust. This research aims to contribute to the ongoing development of facial expression recognition systems by enhancing their robustness and generalization. While further refinement is needed, this work provides a step toward more accurate and adaptable FER models, with the potential to support advancements in human-computer interaction and various real-world applications.

Skin-Inspired and Self-Powered Piezoionic Sensors for Smart Wearable Applications.

Bio-inspired by tactile function of human skin, piezoionic skin sensors recognize strain and stress through converting mechanical stimulus into electrical signals based on ion transfer. However, ion transfer inside sensors is significantly restricted by the lack of hierarchical structure of electrode materials, and then impedes practical application. Here, a durable nanocomposite electrode is developed based on carbon nanotubes and graphene, and integrated into piezoionic sensors for smart wearable applications, such as facial expression and exercise posture recognitions. The nanocomposite electrode provides abundant channels for ion transfer because of its hierarchically porous structure. Carbon nanotubes not only prevent restacking of graphene nanolayers, but also connect them across out-plane dimension. The piezoionic skin sensors present a high degree of linearity in a wide strain range with high sensitivity, and long cycling life with bending strains beyond 20000 s. Further, a smart bracelet based on flexible sensors is fabricated for accurate posture recognition of badminton exercise, valuable to athlete training.