Expression Recognition Algorithm Research Articles

Human Emotion Recognition Based on Spatio-Temporal Facial Features Using HOG-HOF and VGG-LSTM

Human emotion recognition is crucial in various technological domains, reflecting our growing reliance on technology. Facial expressions play a vital role in conveying and preserving human emotions. While deep learning has been successful in recognizing emotions in video sequences, it struggles to effectively model spatio-temporal interactions and identify salient features, limiting its accuracy. This research paper proposed an innovative algorithm for facial expression recognition which combined a deep learning algorithm and dynamic texture methods. In the initial phase of this study, facial features were extracted using the Visual-Geometry-Group (VGG19) model and input into Long-Short-Term-Memory (LSTM) cells to capture spatio-temporal information. Additionally, the HOG-HOF descriptor was utilized to extract dynamic features from video sequences, capturing changes in facial appearance over time. Combining these models using the Multimodal-Compact-Bilinear (MCB) model resulted in an effective descriptor vector. This vector was then classified using a Support Vector Machine (SVM) classifier, chosen for its simpler interpretability compared to deep learning models. This choice facilitates better understanding of the decision-making process behind emotion classification. In the experimental phase, the fusion method outperformed existing state-of-the-art methods on the eNTERFACE05 database, with an improvement margin of approximately 1%. In summary, the proposed approach exhibited superior accuracy and robust detection capabilities.

Recognizing facial emotion leveraging convolutional neural network

Facial expression recognition technology is a technology based on the combination of artificial intelligence technology and biology, and it is interdisciplinary research. The appearance of this technology shows the diversity of the development direction and application fields of computer technology, but it also means that the development of facial expression recognition technology needs not only the support of computer technology but also the exploration and progress of biology. At present, to recognize facial expressions a system usually contains three stages, namely, face detection stage, feature extraction stage, and face expression recognition stage. The facial expression recognition algorithm is transformed into a classification problem, and a convolutional neural network (CNN) is adopted, where features of the facial expressions could be automatically extracted. It takes as input a preprocessed image and can directly produce encoded features and predictions, leveraging the end-to-end strategy. It could discard the complicated intermediate modeling process of traditional machine learning. In this work a CNN is implemented for the recognition of facial expression on Fer2013 dataset. Moreover, the effectiveness of different numbers of CNN layers are testified on this problem. It could be concluded that with deeper architectures, the CNN tends to perform better.

A dynamic fusion of features from deep learning and the HOG-TOP algorithm for facial expression recognition

A dynamic fusion of features from deep learning and the HOG-TOP algorithm for facial expression recognition

Analysis and Recognition of Voluntary Facial Expression Mimicry Based on Depressed Patients.

Many clinical studies have shown that facial expression recognition and cognitive function are impaired in depressed patients. Different from spontaneous facial expression mimicry (SFEM), 164 subjects (82 in a case group and 82 in a control group) participated in our voluntary facial expression mimicry (VFEM) experiment using expressions of neutrality, anger, disgust, fear, happiness, sadness and surprise. Our research is as follows. First, we collected a large amount of subject data for VFEM. Second, we extracted the geometric features of subject facial expression images for VFEM and used Spearman correlation analysis, a random forest, and logistic regression-based recursive feature elimination (LR-RFE) to perform feature selection. The features selected revealed the difference between the case group and the control group. Third, we combined geometric features with the original images and improved the advanced deep learning facial expression recognition (FER) algorithms in different systems. We propose the E-ViT and E-ResNet based on VFEM. The accuracies and F1 scores were higher than those of the baseline models, respectively. Our research proved that it is effective to use feature selection to screen geometric features and combine them with a deep learning model for depression facial expression recognition.

A high‐performance and lightweight framework for real‐time facial expression recognition

AbstractFacial expression recognition technology has become a powerful tool for conveying human emotions and intentions and is widely used in areas such as assisted driving and intelligent medical care. Due to the limited computing power of current hardware devices and the real‐time requirements of application scenarios, this paper proposes a high‐performance and lightweight framework for real‐time facial expression recognition framework to solve the problem of real‐time completion of expression recognition tasks under low hardware costs. To address these issues, this paper first designs a RepVGG and mobileNetV2 dual‐channel structure in the feature extraction. It is then input into the MobileViT Block for global feature modelling. Finally, the position vector of the capsule network is used to replace the output of the global pooling, preserving the spatial relationship of the salient features and enhancing the classification effect. Compared with the mainstream facial expression recognition algorithm that cannot get good classification results under low complexity conditions, the model has a significant accuracy improvement while ensuring lightweight. With only 294.60M FLOPS and 0.95M parameters, it achieved an accuracy of 97.53% on the KDEF dataset and 85.56% on the RAF‐DB, demonstrating the advanced nature of the algorithm.

A facial expression recognition algorithm incorporating SVM and explainable residual neural network

A facial expression recognition algorithm incorporating SVM and explainable residual neural network

Deep Learning Approach for Facial Expression Recognition

Abstract: In recent decades, facial expression recognition has emerged as a hot topic with significant implications in the realm of human-computer interaction. The simplest way for humans to express their emotions is through facial expressions. Non-verbal communication relies heavily on facial expression. This study outlines deep learning-based Facial Expression Recognition (FER) algorithm. The performance of the FER approach is compared based on the number of expressions detected and the difficulty of CNN algorithms. The FER2013, CK+ databases were used in testing the design. CNNs (Convolutional Neural Networks) have as of late gained fame in the field of profound learning because of its superb plan and capacity to convey clever outcomes without the requirement for manual feature extraction from raw information. The suggested algorithm achieves a higher rate of recognition on four datasets.

A novel facial expression recognition algorithm using geometry β –skeleton in fusion based on deep CNN

Facial expression recognition (FER) methods based on single-source facial data often suffer from reduced accuracy or unpredictability due to facial occlusion or illumination changes. To address this, a new technique called Fusion-CNN is proposed. It improves accuracy by extracting hybrid features using a β-skeleton undirected graph and an ellipse with parameters trained using a 1D-CNN. In addition, a 2D-CNN is trained on the same image. The outputs from these two subnetworks are fused, and their features are concatenated to create a feature vector for classification in a deep neural network. The proposed method is evaluated on four public face datasets: the extended Cohn-Kanade (CK+) dataset, the Japanese Female Facial Expression (JAFFE) dataset, Karolinska Directed Emotional Faces (KDEF), and Oulu-CASIA. The experimental results show that Fusion-CNN outperforms other algorithms, achieving recognition accuracy of 98.22%, 93.07%, 90.30%, and 90.13% for the CK+, JAFFE, KDEF, and Oulu-CASIA datasets, respectively.

Learning Transferable Sparse Representations for Cross-Corpus Facial Expression Recognition

An assumption widely used in traditional facial expression recognition algorithms is that the training and testing are conducted on the same dataset. However, this assumption does not hold in practice, in which the training data and testing data are often from different datasets. In this scenario, directly deploying these algorithms would lead to severe information loss and performance degradation due to the domain shift. To address this challenging problem, in this paper, we propose a novel transferable sparse subspace representation method (TSSR) for cross-corpus facial expression recognition. Specifically, in order to reduce the cross-corpus mismatch, inspired by sparse subspace clustering, we advocate reconstructing the source and target samples using the source data points based on L1-norm sparse representation. Each data point in source and target corpora can be ideally represented as a combination of a few other source points from its own subspace. Moreover, we take into account the local geometrical information within the cross-corpus data by adopting a graph Laplacian regularizer, which can efficiently preserve the local manifold structure and better transfer knowledge between two corpora. Finally, experimental results on several facial expression datasets demonstrate the superiority of the proposed method over some state-of-the-art methods.

A CNN-Based Facial Expression Recognition System

Facial expression recognition has long attracted researchers from a variety of domains, such as human-computer interaction, emotion analysis, intelligent medical care, and so on. Various human-designed features have been used in FER to extract image appearance features, but human-designed methods are difficult to be used to recognize new face images, which poses challenges for FER in uncontrolled environments. Recently, CNNs and FNNs have been used for facial expression recognition. The new CNN-based recognition approach yields excellent FER results. The numerous convolution and pooling layers that CNN possesses allow it to extract higher and multi-level features of the entire face or local regions. Additionally, CNN has excellent classification performance when it comes to the feature extraction of facial expression pictures. However, the mechanism through which FNN remains largely controversial. In this work, the experiments are carried out on the comparison of three facial expression recognition algorithms: K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), and the experimental results show that, using CNN can get more stable recommendation results of better quality.

Face expression recognition based on NGO-BILSTM model.

Facial expression recognition has always been a hot topic in computer vision and artificial intelligence. In recent years, deep learning models have achieved good results in accurately recognizing facial expressions. BILSTM network is such a model. However, the BILSTM network's performance depends largely on its hyperparameters, which is a challenge for optimization. In this paper, a Northern Goshawk optimization (NGO) algorithm is proposed to optimize the hyperparameters of BILSTM network for facial expression recognition. The proposed methods were evaluated and compared with other methods on the FER2013, FERplus and RAF-DB datasets, taking into account factors such as cultural background, race and gender. The results show that the recognition accuracy of the model on FER2013 and FERPlus data sets is much higher than that of the traditional VGG16 network. The recognition accuracy is 89.72% on the RAF-DB dataset, which is 5.45, 9.63, 7.36, and 3.18% higher than that of the proposed facial expression recognition algorithms DLP-CNN, gACNN, pACNN, and LDL-ALSG in recent 2 years, respectively. In conclusion, NGO algorithm effectively optimized the hyperparameters of BILSTM network, improved the performance of facial expression recognition, and provided a new method for the hyperparameter optimization of BILSTM network for facial expression recognition.

Performance Evaluation of Fusion Based Efficient Algorithm for Facial Expression Recognition

Objectives: To develop face expression recognition system using JAFFE database and to evaluate the performance of the face expression recognition models. Methods: This study used the FER model based on modified-HoG (Histogram of oriented gradient), LBP (Local Binary Patterns) and Fast Key point detector and BRIEF descriptor (FKBD) to extract the significant features of JAFFE dataset. The features extracted using HoG, LBP and FKBD techniques form a feature vector. Then, the fusion of all the features is carried out at the feature level. The multiclass SVM and KNN classifiers are used to recognize the facial expressions, effectively. Findings: In this work, an effort is made to develop a robust FER model using JAFFE database. It is recorded that, based on the experimental results, the proposed model suits better with a performance rate of 98.26% for SVM and 96.51% for KNN, when compared with the different state-of-the-art methods. Novelty: Many FER models have been developed and adopted for enhancing their quality and to extract the facial features using transform and frequency domains. It is observed that, maximum approaches are based on generating the texture features. The fusion at the feature level using modified HoG, LBP and FKBD is performed and the SVM model is more compatible when compared with other classifiers and it supports one-to-one and one-to-many comparisons’ technique. Keywords: Face Expression Recognition; Local Binary Pattern; Emotions; Nearest Neighbor; Histogram of Gradients

Statement of Removal

ABSTRACT Aiming at the problem that uneven illumination easily affects the facial expression recognition rate and reduces the facial expression recognition rate, a facial expression recognition algorithm based on local features and deep belief network (DBN) is proposed. Firstly, the non-uniform illumination invariant feature of LSH in facial expression image is extracted; Secondly, Gauss Laplace operator (log) is used to extract the edge detail features of facial expression; Then, the facial expression features fused by LSH and log are constructed; Finally, the DBN model is used to train the facial expression features fused by LSH and log, and the trained DBN model is used for facial expression recognition. Experiments on Jaffe facial expression database and Uighur facial expression database show that the algorithm has strong robustness and effectiveness.

Energetic optimization of an autonomous mobile socially assistive robot for autism spectrum disorder.

The usage of socially assistive robots for autism therapies has increased in recent years. This novel therapeutic tool allows the specialist to keep track of the improvement in socially assistive tasks for autistic children, who hypothetically prefer object-based over human interactions. These kinds of tools also allow the collection of new information to early diagnose neurodevelopment disabilities. This work presents the integration of an output feedback adaptive controller for trajectory tracking and energetic autonomy of a mobile socially assistive robot for autism spectrum disorder under an event-driven control scheme. The proposed implementation integrates facial expression and emotion recognition algorithms to detect the emotions and identities of users (providing robustness to the algorithm since it automatically generates the missing input parameters, which allows it to complete the recognition) to detonate a set of adequate trajectories. The algorithmic implementation for the proposed socially assistive robot is presented and implemented in the Linux-based Robot Operating System. It is considered that the optimization of energetic consumption of the proposal is the main contribution of this work, as it will allow therapists to extend and adapt sessions with autistic children. The experiment that validates the energetic optimization of the proposed integration of an event-driven control scheme is presented.

Facial expression recognition based on improved residual network

AbstractFacial expressions are an important part of human emotional signals and their recognition has become an important topic of research in the field of pattern recognition. Deep learning based methods have achieved great success in the recognition of facial expressions. However, with the evolution of convolution neural networks and the increased network depth, these methods suffer from problems such as degraded network performance and loss of feature information. To address these problems, a novel facial expression recognition algorithm based on an improved residual neural network is proposed. First, a residual neural network is designed to extract deep features while retaining the shallow ones. This can effectively prevent the degradation of network performance. Moreover, when the gradient of the Rectified Linear Units activation function used in the residual module is 0, it will inactivate the neurons and cause a loss of feature information. To address this, the Mish activation function is used instead. The slight allowance for negative values in Mish improves the gradient flow. Next, an inception module is introduced to obtain richer feature information under the same receptive field. Finally, by conducting verification experiments on the public datasets CK+ and KDEF, the authors manage to solve the problems of degraded network performance and insufficient information from extracted features, achieving recognition accuracy rates of 96.37% and 93.38% on the two datasets, respectively.

Facial Expression Recognition Based on Multi-Channel Attention Residual燦etwork

For the problems of complex model structure and too many training parameters in facial expression recognition algorithms, we proposed a residual network structure with a multi-headed channel attention (MCA) module. The migration learning algorithm is used to pre-train the convolutional layer parameters and mitigate the overfitting caused by the insufficient number of training samples. The designed MCA module is integrated into the ResNet18 backbone network. The attention mechanism highlights important information and suppresses irrelevant information by assigning different coefficients or weights, and the multi-head structure focuses more on the local features of the pictures, which improves the efficiency of facial expression recognition. Experimental results demonstrate that the model proposed in this paper achieves excellent recognition results in Fer2013, CK+ and Jaffe datasets, with accuracy rates of 72.7%, 98.8% and 93.33%, respectively.

Expression Recognition Algorithm Enhancements for Students Who Integrate Artificial Intelligence Games into Classrooms to Gain Sensitivity

This paper proposes an evaluation index system for students' learning performance in colleges and universities, which is composed of 16 indicators in four categories with the integration of artificial intelligence games into the classroom as the basic framework. In terms of functions and evaluation, the theoretical structure and analysis path have been improved. Guided by the improvement of residents' sense of acquisition, the index of students' sense of acquisition is selected. Combining the three-dimensional learning state space with the emotional dimension theory, by analyzing the students' expressive states in the pleasant dimension, combined with the intensity of arousal and valence, an effective classroom state division method is realized, based on behavior theory.

Research on Ensemble Convolutional Neural Networks in Digital Image Course experiments

In various digital image processing courses, the accuracy of traditional expression recognition algorithms cannot meet practical needs. This paper studies facial expression recognition using a convolutional neural network and ensemble learning. We develop three subnetwork models based on the classical convolutional neural network, using the BN layer to increase the network convergence rate and regularization is used to prevent subnetwork overfitting. The optimization idea used in this paper is applicable to most experimental improvements and raises the standard of the experiments.

FER-PCVT: Facial Expression Recognition with Patch-Convolutional Vision Transformer for Stroke Patients.

Early rehabilitation with the right intensity contributes to the physical recovery of stroke survivors. In clinical practice, physicians determine whether the training intensity is suitable for rehabilitation based on patients' narratives, training scores, and evaluation scales, which puts tremendous pressure on medical resources. In this study, a lightweight facial expression recognition algorithm is proposed to diagnose stroke patients' training motivations automatically. First, the properties of convolution are introduced into the Vision Transformer's structure, allowing the model to extract both local and global features of facial expressions. Second, the pyramid-shaped feature output mode in Convolutional Neural Networks is also introduced to reduce the model's parameters and calculation costs significantly. Moreover, a classifier that can better classify facial expressions of stroke patients is designed to improve performance further. We verified the proposed algorithm on the Real-world Affective Faces Database (RAF-DB), the Face Expression Recognition Plus Dataset (FER+), and a private dataset for stroke patients. Experiments show that the backbone network of the proposed algorithm achieves better performance than Pyramid Vision Transformer (PvT) and Convolutional Vision Transformer (CvT) with fewer parameters and Floating-point Operations Per Second (FLOPs). In addition, the algorithm reaches an 89.44% accuracy on the RAF-DB dataset, which is higher than other recent studies. In particular, it obtains an accuracy of 99.81% on the private dataset, with only 4.10M parameters.

RETRACTED ARTICLE: Expression recognition algorithm based on MDS-HOG feature optimization and differential weights

RETRACTED ARTICLE: Expression recognition algorithm based on MDS-HOG feature optimization and differential weights