Field Of Facial Expression Recognition Research Articles

Optimizing Facial Expression Recognition with Image Augmentation Techniques: VGG19 Approach on FERC Dataset

In the field of facial expression recognition (FER), the availability of balanced and representative datasets is key to success in training accurate models. However, Facial Expression Recognition Challenge (FERC) datasets often face the challenge of class imbalance, where some facial expressions have a much smaller number of samples compared to others. This issue can result in biased and unsatisfactory model performance, especially in recognizing less common facial expressions. Data augmentation techniques are becoming an important strategy as they can expand the dataset by creating new variations of existing samples, thus increasing the variety and diversity of the data. Data augmentation can be used to increase the number of samples for less common facial expression classes, thus improving the model's ability to recognize and understand diverse facial expressions. The augmentation results are then combined with balancing techniques such as SMOTE coupled with undersampling to improve model performance. In this study, VGG19 is used to support better model performance. This will provide valuable guidelines for optimizing more advanced CNN models in the future and may encourage further research in creating more innovative augmentation techniques.

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Driven by advancements in deep learning technologies, substantial progress has been achieved in the field of facial expression recognition over the past decade, while challenges remain brought about by occlusions, pose variations and subtle expression differences in unconstrained (wild) scenarios. Therefore, a novel multiscale feature extraction method is proposed in this paper, that leverages convolutional neural networks to simultaneously extract deep semantic features and shallow geometric features. Through the mechanism of channel-wise self-attention, prominent features are further extracted and compressed, preserving advantageous features for distinction and thereby reducing the impact of occlusions and pose variations on expression recognition. Meanwhile, inspired by the large cosine margin concept used in face recognition, a center cosine loss function is proposed to avoid the misclassification caused by the underlying interclass similarity and substantial intra-class feature variations in the task of expression recognition. This function is designed to enhance the classification performance of the network through making the distribution of samples within the same class more compact and that between different classes sparser. The proposed method is benchmarked against several advanced baseline models on three mainstream wild datasets and two datasets that present realistic occlusion and pose variation challenges. Accuracies of 89.63%, 61.82%, and 91.15% are achieved on RAF-DB, AffectNet and FERPlus, respectively, demonstrating the greater robustness and reliability of this method compared to the state-of-the-art alternatives in the real world.

NA

NA

Facial expression recognition based on Feature Pyramid Network

Facial expression recognition with significant implications across fields such as psychology, computer science, and artificial intelligence. This paper proposes a combination of a Feature Pyramid Network (FPN) and a Residual Network (ResNet) to construct a recognition model. The main objective of the proposed model is to refine the multi-level feature representation of facial expressions. This approach aims to provide a more holistic understanding of the diverse and complex nature of facial expressions, recognizing the intricate interplay between macro and micro-expressions. Experimental results underscore the model's considerable superiority over traditional methods, particularly in terms of accuracy and adaptability to objects of varying sizes and complexities. This comprehensive approach to facial expression recognition showcases the potential of integrating different neural network architectures, furthering our understanding of the subtleties of facial expressions. The research, therefore, presents a significant contribution to the field of facial expression recognition, demonstrating the efficacy of integrating multi-scale feature extraction techniques to improve model performance. It sets the stage for future research directions in this domain, paving the way for more sophisticated emotion recognition systems that can be deployed in real-world applications.

Emotional Classification Based on Facial Expression Recognition Using Convolutional Neural Network Method

In recent years, the development of human-computer interaction technology has reached remarkable levels, particularly in the field of facial expression recognition. This technology utilizes human facial images to identify and classify emotional expressions such as happiness, sadness, fear, and more through computer image processing. Active research in facial expression recognition yields substantial benefits for individual and societal advancement, especially in the context of its application within Smart City environments. This study demonstrates that well- configured Convolutional Neural Network (CNN) models empowered by TensorFlow exhibit higher accuracy compared to models utilizing PyTorch. The TensorFlow model achieves the highest accuracy of 93% in recognizing emotional expressions, whereas the PyTorch model achieves 69% accuracy. The TensorFlow model also displays lower accuracy loss and shorter training times compared to the PyTorch model. In the context of calculating happiness indices within Smart City environments, the appropriate choice of technology significantly influences measurement accuracy and efficiency. Therefore, the TensorFlow platform, proven to deliver superior performance in this study, can be a strategic choice for integrating facial expression detection technology into happiness index measurements in such locations

Investigation into facial expression recognition methods: a review

Facial expression recognition (FER) is a rapidly emerging topic in computer vision that has gotten a lot of interest because of its numerous applications in fields including psychology, sociology, human-computer interaction (HCI), and security. FER seeks to recognise and analyse human facial expressions in order to determine emotions and other mental states. Several strategies, including feature-based, kernel-based, and deep learning-based methods, have been developed and implemented in FER in recent years. FER’s major goal is to extract and identify the most discriminating elements that accurately represent the emotions expressed by facial expressions. The literature reviewed in this field shows that deep learning-based methods have outperformed traditional feature-based and kernel-based methods in terms of accuracy and robustness in recognizing facial expressions. However, these deep learning-based methods also pose several challenges, such as the need for large labeled-data-sets, robustness to different facial poses and illumination conditions, and generalization to unseen data. Despite these challenges, the field of FER is expected to continue growing, and future research will likely focus on addressing these challenges and improving the accuracy and robustness of FER systems.

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition

In recent years, facial expression recognition (FER) has garnered significant attention within the realm of computer vision research. This paper presents an innovative network called the Dual-Direction Attention Mixed Feature Network (DDAMFN) specifically designed for FER, boasting both robustness and lightweight characteristics. The network architecture comprises two primary components: the Mixed Feature Network (MFN) serving as the backbone, and the Dual-Direction Attention Network (DDAN) functioning as the head. To enhance the network’s capability in the MFN, resilient features are extracted by utilizing mixed-size kernels. Additionally, a new Dual-Direction Attention (DDA) head that generates attention maps in two orientations is proposed, enabling the model to capture long-range dependencies effectively. To further improve the accuracy, a novel attention loss mechanism for the DDAN is introduced with different heads focusing on distinct areas of the input. Experimental evaluations on several widely used public datasets, including AffectNet, RAF-DB, and FERPlus, demonstrate the superiority of the DDAMFN compared to other existing models, which establishes that the DDAMFN as the state-of-the-art model in the field of FER.

Facial emotion recognition using deep learning detector and classifier

Numerous research works have been put forward over the years to advance the field of facial expression recognition which until today, is still considered a challenging task. The selection of image color space and the use of facial alignment as preprocessing steps may collectively pose a significant impact on the accuracy and computational cost of facial emotion recognition, which is crucial to optimize the speed-accuracy trade-off. This paper proposed a deep learning-based facial emotion recognition pipeline that can be used to predict the emotion of detected face regions in video sequences. Five well-known state-of-the-art convolutional neural network architectures are used for training the emotion classifier to identify the network architecture which gives the best speed-accuracy trade-off. Two distinct facial emotion training datasets are prepared to investigate the effect of image color space and facial alignment on the performance of facial emotion recognition. Experimental results show that training a facial expression recognition model with grayscale-aligned facial images is preferable as it offers better recognition rates with lower detection latency. The lightweight MobileNet_v1 is identified as the best-performing model with WM=0.75 and RM=160 as its hyper-parameters, achieving an overall accuracy of 86.42% on the testing video dataset.

Summary of Research on Facial Expression Recognition

Facial expression recognition allows computers to understand expressions according to human thinking. It is an important component of human-computer interaction. However, with the rapid development of deep learning, the research of deep learning technology in the field of facial expressions has also become a research hotspot, so is deep learning. It is necessary to analyze the application of technology in facial expression recognition and the results obtained. Firstly, several commonly used expression data sets are summarized; then the expression recognition methods based on deep learning are classified from the aspects of feature extraction and feature classification, and several network improvements in expression recognition based on deep learning are analyzed from the aspect of network improvement. Method; Finally, the challenges and future development in the field of facial expression recognition are described.

Reconocimiento de expresiones faciales con vision transformer

The identification of emotions through the reading of non-verbal signals, such as gestures and facial expressions, has generated a new application in the field of Facial Expression Recognition (FER) and human-computer interaction. Through the recognition of facial expressions, it would be possible to improve industrial equipment by making it safer through social intelligence that has excellent applications in the area of industrial security. That is why this research proposes to classify a series of images from the database called FER-2013, which contains data on seven different emotions, which are anger, disgust, fear, joy, sadness, surprise, neutral. For the recognition of expressions, a Vision Transformer architecture was implemented, of which 87% precision was obtained, while the top test accuracy was 99%.

Construction of Facial Expression Recognition Technology of Intelligent Service Robot Based on Nanofiber Flexible Wearable Electronic Skins

Recognizing human features is a highly significant and popular research topic in academia today, especially with regard to human intervention and computer intelligence. If you want your computer to have real intelligence, then the computer must first learn to understand human feelings, and facial expressions often contain rich emotional information. Clearly, recognizing human features is indispensable for the future intelligent development of computers. There are always a few components and many researchers are involved. Compared with the traditional facial expression recognition method, which needs to manually define features and artificially perform appropriate feature extraction, a great advantage of deep learning is to remove the disadvantages that require a lot of human intervention, thereby reducing the uncertainty of expression recognition. In recent years, deep processing has led to outstanding gains in numerous domains, including robotics, computer graphics, and natural language processing. Therefore, this article applies a popular recognition algorithm based on deep learning, combined with a nanofiber flexible wearable electronic skin, and applies it to the field of facial expression recognition. Functional expression identification is an essential goal of the external analysis of human features. Mature facial expression recognition technology has great application values.

Multi-feature fusion network for facial expression recognition in the wild

Facial expression recognition (FER) has been one of the research focuses in recent years due to its significance in human-computer interactions. However, there are still challenges in the field of FER caused by the diversity and variation of facial expressions in real scenes, the singleness of feature type and the lack of enough discriminant features cannot effectively improve the recognition performance. To solve these problems, we propose a Multi-feature Fusion Network (MFNet) with dual-branch based on deep learning. Firstly, the MFNet uses the pyramid parallel multiscale residual network structure with progressive max-pooling of channel attention to extract multi-level facial features and enhance the discrimination of features; In the meantime, a shallow Gabor convolutional network is designed to enhance the adaptation of learned features to the orientation and scale changes and improve the ability to capture local details features; Finally, the maximum expression features obtained by the above two networks are fused to make more effective expression recognition. Experiments on three public large-scale wild FER datasets (RAF-DB, FERPlus, and AffectNet) show that our MFNet has a superior recognition performance than other recognition methods.

Real Time Facial Expression Recognition for Online Lecture

In order to better improve online teaching during the epidemic, teachers can adjust the teaching according to the students’ understanding. During the epidemic period, online teaching has become a basic way for teachers to teach. One big problem with online teaching is that the lecturers find it hard to see the students’ facial expressions. According to the study, the students’ facial expressions are essential for the instructor to understand the students’ understanding of the course material. There are some relevant studies on the recognition of human facial expressions. However, most of them did not consider some expressions such as “Enlightened,” “Confused,” or “Bored.” This study helps teachers to improve the quality of teaching by designing a program to provide them with students’ facial expressions. The study, implemented on students’ computers, can capture their faces from a camera, identify some common and useful facial expressions, and send them to lecturers regularly during online lectures. The main research methods used in this paper are as follows: (1) Use the face recognition algorithm provided by OpenCV to capture faces. (2) Train a convolutional neural network model to recognize the expression “Happy,” “Surprised,” “Neutral,” “Enlightened,” “Confusion,” and “Boredom.” (3) Use the message protocol EMQX to transmit the expression information. This study can successfully capture faces, and about 80% accuracy can identify expressions and successfully transmit expression information. This study contains some expressions rarely studied by others and is innovative in the field of facial expression recognition. Datasets, deep learning models, and results are available for other research teams.

Deep learning for facial emotion recognition using custom CNN architecture

Human facial expressions are an indication of true emotions. To recognize facial expressions accurately is useful in the field of Artificial Intelligence, Computing, Medical, e-Education, and many more. The facial expression recognition (FER) system detects emotion through facial expression. But, it is challenging to detect facial emotions accurately. However, recent advancements in technology, research, and availability of facial expression datasets have led to the development of many FER systems which can accurately detect facial emotions. Past research in the field of FER indicates With Convolutional Neural Networks (CNNs), deep learning techniques are the most advanced presently. Custom CNN Architecture is used to implement basic facial emotion recognition in static images in this paper. A K-fold cross-validation method was used to train them using FER13, CK+, and the JAFFE data set. On the seven classes of fundamental emotions, including anger, disgust, fear, happiness, neutrality, sorrow, and surprise, the FER13, CK+, and JAFFE datasets had an accuracy rate of 91.58 percent. Given the difficulty of developing unique CNN architecture, this study’s accurate findings contrast well with those of previous studies.

Face Recognition Method Based on Residual Convolution Neural Network

With the advancement of information technology and societal growth, social security has become more important than ever. Face recognition, as compared to other traditional recognition methods like fingerprint recognition, palm recognition, etc, has the benefit that it is contact less, and now it is becoming one among the most prominent technologies in development. Although there are numerous recognition systems that use DNNs in the field of facial expression recognition, their accuracy and practicality are still insufficient for real-world applications. A facial recognition approach based on Resnet 152 v2 has been proposed in this work. In this paper, a residual learning approach is presented to make the training of networks that are far deeper than previously employed networks easier. The proposed method, employs the AT&T face dataset, and supposing that normalization and segmentation are complete, we concentrate on the subtask of person verification and recognition, demonstrating performance using a testing database comprising illumination, pose, expression and occlusion variations. SoftMax is the activation function that has been used, which adjusts the output sum up to one allowing it to be understood as probabilities. Then, the model would generate a judgment depending on which option has a strong likelihood. This system employs Adam as an optimizer to control the learning rate through training and categorical cross entropy as its loss function. The proposed approach has a 97 percent face recognition accuracy on AT&T dataset, showing its efficacy after a significant number of analyses and experimental verification.

Estimation of Learning Affects Experienced by Learners: An Approach Using Relational Reasoning and Adaptive Mapping

Various studies have measured and analyzed learners’ emotions in both traditional classroom and e-learning settings. Learners’ emotions can be estimated using their text input, speech, body language, or facial expressions. The presence of certain facial expressions has shown to indicate a learner’s levels of concentration in both traditional and e-learning environments. Many studies have focused on the use of facial expressions in estimating the emotions experienced by learners. However, little research has been conducted on the use of analyzed emotions in estimating the learning affect experienced. Previous studies have shown that online learning can enhance students’ motivation, interest, attention, and performance as well as counteract negative emotions, such as boredom and anxiety, that students may experience. Thus, it is crucial to integrate modules into an existing e-learning platform to effectively estimate learners’ learning affect (LLA), provide appropriate feedback to both learner and lecturers, and potentially change the overall online learning experience. This paper proposes a learning affect estimation framework that employs relational reasoning for facial expression recognition and adaptive mapping between recognized emotions and learning affect. Relational reasoning and deep learning, when used for autoanalysis of facial expressions, have shown promising results. The proposed methodology includes estimating a learner’s facial expressions using relational reasoning; mapping the estimated expressions to the learner’s learning affect using the adaptive LLA transfer model; and analyzing the effectiveness of LLA within an online learning environment. The proposed research thus contributes to the field of facial expression recognition enhancing online learning experience and adaptive learning.

Efficient CNN Approach for Facial Expression Recognition

In the last decade, the Facial Expression Recognition field has been studied widely and become the base for many researchers, and still challenging in computer vision. Machine learning technique used in facial expression recognition facing many problems, since human emotions expressed differently from one to another. Nevertheless, Deep learning that represents a novel area of research within machine learning technology has the ability for classifying people’s faces into different emotion classes by using a Deep Neural Network (DNN). The Convolution Neural Network (CNN) method has been used widely and proved as very efficient in the facial expression recognition field. In this study, a CNN technique for facial expression recognition has been presented. The performance of this study has been evaluated using the fer2013 dataset, the total number of images has been used. The accuracy of each epoch has been tested which is trained on 29068 samples, validate on 3589 samples. The overall accuracy of 69.85% has been obtained for the proposed method.

Design and Implementation of Face Emotion Recognition System Based on CNN Mini_Xception Frameworks

In the new era, the rapid development of information technology makes the face recognition technology in artificial intelligence also develop rapidly, among which, face expression recognition has become a research hotspot. In recent years, due to deep learning, convolutional neural network and multilayer perceptron and other related algorithms have become the research focus of scholars, so their wide application in the field of facial expression recognition is also the direction of face emotion recognition exploration and research. In addition, computer software is widely used in human daily life, so it is also very important to design and implement an intelligent, real-time and universally applicable UI interface for facial emotion recognition system. Therefore, this paper firstly completed the facial expression recognition model based on the Mini_Xception framework of CNN by training the FER2013 expression database. Secondly, the system UI interface is designed and realized through PyQT5, OpenCV, Keras and other libraries. The final results show that the system is based on the algorithm model, and the UI interface designed and realized can not only recognize the saved pictures, but also recognize the real-time face emotions through the camera, and the overall effect of the system is outstanding.

RECOGNIZE HAPPY AND SADNESS EMOTIONS USING MUTUAL INFORMATION AND SUPPORT VECTOR CLASSIFICATION METHODS

In the past decade, the field of facial expression recognition has attracted the attention of scientists who play an important role in enhancing interaction between human and computers. The issue of facial expression recognition is not a simple matter of machine learning, because expression of the individual differs from one person to another based on the various contexts, backgrounds and lighting. The goal of the current system was to achieve the highest rate for two facial expressions ("happy" and "sad") The objective of the current work was to attain the highest rate in classification with computer vision algorithms for two facial expressions ("happy" and "sad"). This was accomplished through several phases started from image pre-processing to the Gabor filter extraction, which was then used for the extraction of important characteristics with mutual information. The expression was finally recognized by a support vector classifier. Cohn-Kanade database and JAFFE data base have been trained and checked. The rates achieved by the qualified data package were 81.09% and 92.85% respectively.

Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition. Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision