Expression Recognition Method Research Articles

Eye Strain Expression Classification using Attention Capsule Network for Adapting Screen Vision

Beside the conventional facial expression recognition methods, the research focuses on developing a system for recognizing various eye expressions under different screen conditions. This research deals with the use of Capsule Network (a recent Deep Learning algorithm) to enhance facial expression recognition capabilities and to develop adaptive screen technologies aimed at mitigating digital eye strain. The main objective of this research is to engineer a sophisticated system that employs the capabilities of Capsule Nets to recognize the various expressions that user makes and based on the recognized expression, dynamically modify screen settings, ensuring optimal user visual comfort. The research primarily concentrates on the exploration and application of various Capsule Net architectures designed for the recognition of expressions related to eye strain. The baseline model utilized elementary convolutional layers which feed into subsequent fully connected layers for the task of classification. The model has since been refined by incorporating advanced techniques such as attention mechanisms and more sophisticated network architectures where the classification is done by Capsule Network. Results have demonstrated a modest enhancement in the Capsule Net’s predictive performance, attributed to its superior spatial and hierarchical processing of facial features, in comparison to conventional deep learning approaches. The final model has an accuracy of 82.27%. As a final system the model has been deployed to an application to process frames from video camera in the device and make prediction to prompt the notifications or recommendations.

XTime: A general rule-based method for time expression recognition and normalization

Time expression (a.k.a., timex) recognition and normalization (TERN) is a crucial task for downstream research. However, previous studies have overlooked the critical characteristics of timexes that significantly impact the task. To gain deeper insights, we conduct an analysis across four diverse English datasets to examine the key attributes of timex constituents. Our analysis reveals several noteworthy observations, such as: timexes tend to very short; the majority of timexes contain time tokens; there exist strong mapping relationships between time tokens and timex types; there exists a priority relationship among timex types; and timex values exhibit only some standard formats. Based on these insights, we propose a novel general rule-based method termed XTime11Source codes and datasets are available at https://github.com/xszhong/XTime. to recognize timexes from free text and normalize them into standard formats. Notably, XTime’s rules are designed in a general and heuristic manner, enabling its independence of diverse domains and text types. Experimental evaluations conducted on both in-domain and out-of-domain English datasets demonstrate that XTime consistently outperforms or performs comparably to representative state-of-the-art methods.

POST: Prototype‐oriented similarity transfer framework for cross‐domain facial expression recognition

AbstractFacial expression recognition (FER) is one of the popular research topics in computer vision. Most deep learning expression recognition methods perform well on a single dataset, but may struggle in cross‐domain FER applications when applied to different datasets. FER under cross‐dataset also suffers from difficulties such as feature distribution deviation and discriminator degradation. To address these issues, we propose a prototype‐oriented similarity transfer framework (POST) for cross‐domain FER. The bidirectional cross‐attention Swin Transformer (BCS Transformer) module is designed to aggregate local facial feature similarities across different domains, enabling the extraction of relevant cross‐domain features. The dual learnable category prototypes is designed to represent potential space samples for both source and target domains, ensuring enhanced domain alignment by leveraging both cross‐domain and specific domain features. We further introduce the self‐training resampling (STR) strategy to enhance similarity transfer. The experimental results with the RAF‐DB dataset as the source domain and the CK+, FER2013, JAFFE and SFEW 2.0 datasets as the target domains, show that our approach achieves much higher performance than the state‐of‐the‐art cross‐domain FER methods.

Lightweight facial expression estimation for mobile computing in portable device

AbstractFacial expression recognition has been studied for many years, especially with the development of deep learning. However, the existing researches still have the following two issues. Firstly, the intensity of facial expression is neglected. Secondly, the deep learning based approaches cannot be directly deployed in the devices with limited resources. In order to tackle these two issues, this paper proposes a lightweight facial expression estimation method using a shallow ordinal regression algorithm, which is deployed in a portable smart device for mobile computing in IoTs. Compared with classification based facial expression recognition methods, ordinal regression considers the intensity of facial expression to achieve better mean absolute error (MAE), which is validated by experiments on several public facial expression datasets. The simulation in portable device also demonstrates its effectiveness for mobile computing.

CSINet: Channel–Spatial Fusion Networks for Asymmetric Facial Expression Recognition

Occlusion or posture change of the face in natural scenes has typical asymmetry; however, an asymmetric face plays a key part in the lack of information available for facial expression recognition. To solve the problem of low accuracy of asymmetric facial expression recognition, this paper proposes a fusion of channel global features and a spatial local information expression recognition network called the “Channel–Spatial Integration Network” (CSINet). First, to extract the underlying detail information and deepen the network, the attention residual module with a redundant information filtering function is designed, and the backbone feature-extraction network is constituted by module stacking. Second, considering the loss of information in the local key area of face occlusion, the channel–spatial fusion structure is constructed, and the channel features and spatial features are combined to enhance the accuracy of occluded facial recognition. Finally, before the full connection layer, more local spatial information is embedded into the global channel information to capture the relationship between different channel–spatial targets, which improves the accuracy of feature expression. Experimental results on the natural scene facial expression data sets RAF-DB and FERPlus show that the recognition accuracies of the modeling approach proposed in this paper are 89.67% and 90.83%, which are 13.24% and 11.52% higher than that of the baseline network ResNet50, respectively. Compared with the latest facial expression recognition methods such as CVT, PACVT, etc., the method in this paper obtains better evaluation results of masked facial expression recognition, which provides certain theoretical and technical references for daily facial emotion analysis and human–computer interaction applications.

A novel deep learning model to improve the recognition of students’ facial expressions in online learning environments

With the fast development of artificial intelligence and emerging technologies, automatic recognition of students’ facial expressions has received increased attention. Facial expressions are a kind of external manifestation of emotional states. It is important for teachers to assess students’ emotional states and adjust teaching activities accordingly. However, existing methods for automatic facial expression recognition have the limitations of low accuracy of recognition and poor feature extraction. To address the problem, this study proposed a novel deep learning model called DenseNetX-CBAM to improve facial expression recognition by utilizing a variant of densely connected convolutional networks (DenseNet) to reduce unnecessary parameters and strengthen the reuse of expression features between networks; moreover, convolutional block attention module (CBAM) was integrated to allow the networks to focus on important special regions and important channels when representing features. The proposed model was tested using 217 video clips of 33 students in an online course. The results demonstrated promising effects of the method in improving the accuracy of facial expression recognition, which can help teachers to accurately recognize students’ emotions and provide real-time adjustment in online learning environments.

Studies Advanced in Development and Application of Facial Expression Recognition

Facial expression is an important way to convey people's inner emotional changes, as the basis of emotional understanding and the prerequisite for computers to understand human emotions, expression recognition has attracted more and more research attention from academia and industry in recent years. Most of the early expression recognition methods relied on manual features such as texture, geometry, and contour. Thanks to the powerful feature representation capability of convolutional neural networks, deep learning-based face recognition technology has made breakthroughs in recognition accuracy and speed. This paper details the latest research progress in the field of face expression recognition, including the design ideas, key steps, advantages and disadvantages of representative methods. This paper also introduces the common face expression recognition datasets and quantitatively compares the results of different methods on these datasets. Finally, this paper discusses the problems in the face expression recognition research field and look forward to the future development.

Improving recognition accuracy for facial expressions using scattering wavelet

One of the most evident and meaningful feedback about people’s emotions is through facial expressions. Facial expression recognition is helpful in social networks, marketing, and intelligent education systems. The use of Deep Learning based methods in facial expression identification is widespread, but challenges such as computational complexity and low recognition rate plague these methods. Scatter Wavelet is a type of Deep Learning that extracts features from Gabor filters in a structure similar to convolutional neural networks. This paper presents a new facial expression recognition method based on wavelet scattering that identifies six states: anger, disgust, fear, happiness, sadness, and surprise. The proposed method is simulated using the JAFFE and CK+ databases. The recognition rate of the proposed method is 99.7%, which indicates the superiority of the proposed method in recognizing facial expressions.

An efficient semi-dynamic ensemble pruning method for facial expression recognition

An efficient semi-dynamic ensemble pruning method for facial expression recognition

Correction to: Deep Learning Method of Facial Expression Recognition Based on Gabor Filter Bank Combined with PCNN

Correction to: Deep Learning Method of Facial Expression Recognition Based on Gabor Filter Bank Combined with PCNN

A method for facial expression recognition of image sequence based on spatial features

Facial expression recognition is the foundation of human emotion recognition, which has become a hot topic in the field of artificial intelligence in recent years. To some extent, facial expressions can be regarded as the process of facial muscle changes. Image sequence contains richer expression contents compared with a single image. So the expression recognition based on image sequence can yield more accurate results. A new method for facial expression recognition is proposed in this paper, which is based on spatial feature for image sequence. The method consists of four steps. Firstly, Siamese neural network is used to construct an evaluation model for changes of expression intensity, which extracts an appropriate image sequence from the video. Secondly, a convolutional neural network with attention mechanism is designed and trained, which is used to extract spatial features from each image in the sequence. Then, the spatial features of multiple images are fused. Finally, the fusion results are put into a convolutional neural network to recognize the facial expressions. This method is validated on CK+ dataset and the experimental results show that it’s more accurate than several other methods.

Auto Diagnosis of Parkinson's Disease Via a Deep Learning Model Based on Mixed Emotional Facial Expressions.

Parkinson's disease (PD) is a common degenerative disease of the nervous system in the elderly. The early diagnosis of PD is very important for potential patients to receive prompt treatment and avoid the aggravation of the disease. Recent studies have found that PD patients always suffer from emotional expression disorder, thus forming the characteristics of "masked faces". Based on this, we thus propose an auto PD diagnosis method based on mixed emotional facial expressions in the paper. Specifically, the proposed method is cast into four steps: Firstly, we synthesize virtual face images containing six basic expressions (i.e., anger, disgust, fear, happiness, sadness, and surprise) via generative adversarial learning, in order to approximate the premorbid expressions of PD patients; Secondly, we design an effective screening scheme to assess the quality of the above synthesized facial expression images and then shortlist the high-quality ones; Thirdly, we train a deep feature extractor accompanied with a facial expression classifier based on the mixture of the original facial expression images of the PD patients, the high-quality synthesized facial expression images of PD patients, and the normal facial expression images from other public face datasets; Finally, with the well-trained deep feature extractor, we thus adopt it to extract the latent expression features for six facial expression images of a potential PD patient to conduct PD/non-PD prediction. To show real-world impacts, we also collected a new facial expression dataset of PD patients in collaboration with a hospital. Extensive experiments are conducted to validate the effectiveness of the proposed method for PD diagnosis and facial expression recognition.

Unconstrained Facial Expression Recognition With No-Reference De-Elements Learning

Most unconstrained facial expression recognition (FER) methods take original facial images as inputs to learn discriminative features by well-designed loss functions, which cannot reflect important visual information in faces. Although existing methods have explored the visual information of constrained facial expressions, there is no explicit modeling of what visual information is important for unconstrained FER. To find out valuable information of unconstrained facial expressions, we pose a new problem of no-reference de-elements learning: we decompose any unconstrained facial image into the facial expression element and a neutral face without the reference of corresponding neutral faces. Importantly, the element provides visualization results to understand important facial expression information and improves the discriminative power of features. Moreover, we propose a simple yet effective <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> e- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> lements <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Net</b> work (DENet) to learn the element and introduce appropriate constraints to overcome no ground truth of corresponding neutral faces during the de-elements learning. We extensively evaluate the proposed method on in-the-wild FER datasets including RAF-DB, AffectNet, SFEW and FERPlus. The comparable results show that our method is promising to improve classification performance and achieves equivalent performance compared with state-of-the-art methods. Also, we demonstrate the strong generalization performance on realistic occlusion and pose variation datasets and the cross-dataset evaluation.

A Novel Attention Residual Network Expression Recognition Method

A Novel Attention Residual Network Expression Recognition Method

Adaptive semi-supervised facial expression recognition method based on improved ResNet50

Adaptive semi-supervised facial expression recognition method based on improved ResNet50

Accurate facial expression recognition method based on perceptual hash algorithm

Accurate facial expression recognition method based on perceptual hash algorithm

Facial expression recognition method based on multi-level feature fusion of high-resolution images

Facial expression recognition method based on multi-level feature fusion of high-resolution images

Expression Recognition Method Based on Convolutional Neural Network and Capsule Neural Network

Expression Recognition Method Based on Convolutional Neural Network and Capsule Neural Network

Accurate facial expression recognition method based on perceptual hash algorithm

Accurate facial expression recognition method based on perceptual hash algorithm

Optimization of Psychological Characteristics and Education Management Path of Poor College Students in Colleges and Universities Based on Improved PCA

Abstract Aiming at the mental health problems of poor college students, this study proposes an expression recognition method based on the optimization of PCA algorithm and applies it to the mental health test framework. By comparing the differences between poor college students and norms, and using multiple linear regression and typical correlation analysis to explore the impact of educational management paths. The study also addressed the effect of different types of colleges and universities on the level of management of academic support for poor students, and found that 985 colleges and universities significantly outperformed the norm in this area, i.e. P &gt; |t| = 0.031. This provides valuable insights into improving educational and psychological support for poor college students.