Expression Recognition Accuracy Research Articles

Online classroom student engagement analysis based on facial expression recognition using enhanced YOLOv5 for mitigating cyberbullying

Abstract The rapid expansion of online education has heightened concerns about cyberbullying in virtual classrooms. This paper presents a comprehensive approach for detecting cyberbullying by analyzing students' engagement and emotional responses in online classrooms. Due to the influence of camera resolution and surrounding light in online classrooms, students' facial expressions are often blurry, and the changes in facial features may not be significant. Moreover, most current recognition algorithms utilize larger models, which may not be suitable for real-time detection in online environments. To address these challenges, this study introduces a student facial expression recognition (SFER) method based on an enhanced YOLOv5 (you only look once version 5) model, termed SFER-YOLOv5. Firstly, the improved Soft-NMS is employed to replace the original non-maximum suppression (NMS), effectively enhancing training efficiency. Then, the coordinate attention (CA) module is incorporated into the backbone network to improve detection accuracy, particularly in classroom settings with multiple students or when students are at a considerable distance from the camera. Next, the efficient intersection over union (EIoU) loss function is utilized. EIoU calculates width and height losses separately based on complete IoU (CIoU), replacing the aspect ratio. Finally, Focal Loss is introduced to address sample imbalance issues. The comparative results show that SFER-YOLOv5 achieves an mAP@0.5 of 78.4% on the FER-2013 dataset, 98.1% on the CK+ dataset, and 88.9% on our self-constructed dataset (SFEC). These results underscore the effectiveness of SFER-YOLOv5 in enhancing the accuracy of student facial expression recognition. The proposed method detects reduced engagement, offering a preventive strategy for mitigating cyberbullying in virtual learning environments.

Facial expression recognition based on convolutional network attention mechanism

Aiming at the problems of large number of parameters and weak recognition ability in expression recognition, a method of facial expression recognition based on convolutional network is proposed. An improved residual module is introduced to reduce the number of parameters while enhancing the attention to the expression area; the channel-space attention mechanism is used to realize the weight distribution of different dimensions and positions of the expression area extracted by the network, focusing on the subtle difference feature information in the key points of the expression; the detail module is used to further extract the deep feature information. In order to obtain higher accuracy, a joint loss function is introduced to increase the out-of-class distance and reduce the intra-class distance to improve the accuracy of expression recognition. This paper applies this network to the datasets FER2013 and CK+. The experimental results show that the average recognition rate of this algorithm is respectively and 63.91%，97.98%the number of parameters is 11.34 M. Compared with VGG network, residual network, etc., this model not only improves the recognition rate but also reduces the redundant parameters.

Observe finer to select better: Learning key frame extraction via semantic coherence for dynamic facial expression recognition in the wild

In dynamic facial expression sequences captured in natural environments, the number and distribution of expression peak frames (key frames) show significant variation within video sequences. Efficient extraction and utilization of these key expression frames are crucial for improving the accuracy of expression recognition. To address the above challenges, this paper proposed the Key Frame Extraction via Semantic Coherence (KFE-SC) method. The method explores the spatial-temporal emotional semantic information within sequences, employing a coarse-to-fine approach to extract key expression frames, thus facilitating classification tasks. By introduce a Spatial Semantic Coherence (SSC) module and designing a Global-Local Feature Aggregation (GLFA) unit, the method explores the spatial semantic correlations at global-local levels in images. Through calculating weight proportions, the method achieves a coarse-grained extraction of key frames in the spatial dimension. Considering that video sequences contain both spatial and temporal semantic relevance, KFE-SC further introduces a Temporal Semantic Coherence (TSC) module, which include the Temporal Alignment Aggregation (TA2) unit to align short and long-term temporal information and explore temporal semantic relevance within the sequences, enabling fine-grained extraction of key frames in the temporal dimension through weight calculations.

A facial expression recognition network based on attention double branch enhanced fusion.

The facial expression reflects a person's emotion, cognition, and even physiological or mental state to a large extent. It has important application value in medical treatment, business, criminal investigation, education, and human-computer interaction. Automatic facial expression recognition technology has become an important research topic in computer vision. To solve the problems of insufficient feature extraction, loss of local key information, and low accuracy in facial expression recognition, this article proposes a facial expression recognition network based on attention double branch enhanced fusion. Two parallel branches are used to capture global enhancement features and local attention semantics respectively, and the fusion and complementarity of global and local information is realized through decision-level fusion. The experimental results show that the features extracted by the network are made more complete by fusing and enhancing the global and local features. The proposed method achieves 89.41% and 88.84% expression recognition accuracy on the natural scene face expression datasets RAF-DB and FERPlus, respectively, which is an excellent performance compared with many current methods and demonstrates the effectiveness and superiority of the proposed network model.

HSAW: A Half-Face Self-Attention Weighted Approach for Facial Expression Recognition

Facial expression recognition plays an increasingly important role in daily life, and it is used in several areas of human–computer interaction, such as robotics, assisted driving, and intelligent tutoring systems. However, the current mainstream methods are based on the whole face, and do not consider the existence of expression asymmetry between the left and right half-face. Hence, the accuracy of facial expression recognition needs to be improved. In this paper, we propose a half-face self-attention weighted approach called HSAW. Using statistical analysis and computer vision techniques, we found that the left half-face contains richer expression features than the right half-face. Specifically, we employed a self-attention mechanism to assign different weights to the left and right halves of the face. These weights are combined with convolutional neural network features for improved facial expression recognition. Furthermore, to attack the presence of uncertain categories in the dataset, we introduce adaptive re-labeling module, which can improve the recognition accuracy. Extensive experiments conducted on the FER2013 and RAF datasets have verified the effectiveness of the proposed method, which utilizes fewer parameters.

Discriminative dictionary learning for nonnegative representation based classification

Representation based classification (RC) algorithms have been successfully applied to pattern classification. However, most existing RC algorithms are not robust to bad training samples, since they ignore learning more discriminative dictionary atoms and instead directly use training samples as dictionary atoms. In addition, in order to improve expression ability and recognition accuracy, RC algorithms often need to expand the number of dictionary atoms, resulting in a surge in storage and computing costs. To obtain a more discriminative and compact dictionary, this study proposes discriminative dictionary learning for nonnegative representation based classification (DDLNRC). Specifically, in the DDLNRC, this paper utilizes nonnegative constraint to obtain a nonnegative representation for each training sample on the dictionary. In the dictionary learning stage, for a training sample, the DDLNRC minimizes the intra-class reconstruction error of the training sample and simultaneously enlarges the distance between the training sample and the atom that has the most influence on inter-class reconstruction error. Experiments demonstrate the effectiveness of the DDLNRC. Combined with the deep neural network features, it also can achieve higher accuracy than using the Softmax.

CSINet: Channel–Spatial Fusion Networks for Asymmetric Facial Expression Recognition

Occlusion or posture change of the face in natural scenes has typical asymmetry; however, an asymmetric face plays a key part in the lack of information available for facial expression recognition. To solve the problem of low accuracy of asymmetric facial expression recognition, this paper proposes a fusion of channel global features and a spatial local information expression recognition network called the “Channel–Spatial Integration Network” (CSINet). First, to extract the underlying detail information and deepen the network, the attention residual module with a redundant information filtering function is designed, and the backbone feature-extraction network is constituted by module stacking. Second, considering the loss of information in the local key area of face occlusion, the channel–spatial fusion structure is constructed, and the channel features and spatial features are combined to enhance the accuracy of occluded facial recognition. Finally, before the full connection layer, more local spatial information is embedded into the global channel information to capture the relationship between different channel–spatial targets, which improves the accuracy of feature expression. Experimental results on the natural scene facial expression data sets RAF-DB and FERPlus show that the recognition accuracies of the modeling approach proposed in this paper are 89.67% and 90.83%, which are 13.24% and 11.52% higher than that of the baseline network ResNet50, respectively. Compared with the latest facial expression recognition methods such as CVT, PACVT, etc., the method in this paper obtains better evaluation results of masked facial expression recognition, which provides certain theoretical and technical references for daily facial emotion analysis and human–computer interaction applications.

Action capture and VR interactive system for online experimental teaching

The development of technology has intensified people’s interest in virtual reality technology. Virtual reality technology is also widely used in fields such as teaching and healthcare. To improve the real-time and operability of virtual reality interaction systems, a motion capture and virtual reality interaction system design for online experimental teaching is proposed. The active apparent model is used for facial feature point localization, and the joint capture method is improved by combining threshold segmentation and forward kinematics algorithms. Experimental data confirms that compared to Kinect method, the improved hand joint detection method has higher joint positioning accuracy, with a loss rate of less than 40 % for hand joints. The average judgment accuracy of the six facial expression tests is 82 %, 81 %, 79 %, 78 %, 80 %, and 81 %, respectively. The comprehensive recognition accuracy of facial expression recognition is 80.17 %. The read, write, and update times for each frame of the virtual reality interaction system are 102.6 ms and 427 ms, respectively. The system memory usage is 920.5 M, and the CPU and GPU usage rates are 25.2 % and 4.4 %, respectively. The system can run smoothly. The virtual reality system can run continuously for 24 h, with strong operability and low difficulty in expanding system functions, achieving design goals.

Application of support vector machine system introducing multiple submodels in data mining

As the information age develops, the scale and form of data become more diverse and diverse. Therefore, people need to use effective means to process information. For large-scale data mining problems, a clustering-based kernel matrix inner product filtering method is introduced to decompose the original quadratic programming problem into multiple sub-problems to support parallel training. And a Spark-based multiple submodels parallel support vector machine is proposed. By introducing open-source tools such as OpenCV, image feature extraction can be performed on large-scale video data. Finally, combined with the designed parallel support vector machine algorithm, video facial and expression recognition is carried out. These experiments confirmed that the research method achieved a maximum acceleration ratio of 2090 times when processing Covtype datasets. The research model could achieve an accuracy of over 99 %. Under the maximum data scale experiment, the research model improved prediction accuracy by 21 percentage points with an acceptable additional time cost of about 4 min only. Task parallel processing could more fully utilize cluster performance, increasing by approximately 3.5 m/s2 from 30 to 150 cores. The research model had the highest recognition accuracy for facial expressions, further demonstrating the effectiveness and superiority of this method. The research method has improved the efficiency of big data analysis and mining, and is of great significance in parallel analysis of video data.

A novel deep learning model to improve the recognition of students’ facial expressions in online learning environments

With the fast development of artificial intelligence and emerging technologies, automatic recognition of students’ facial expressions has received increased attention. Facial expressions are a kind of external manifestation of emotional states. It is important for teachers to assess students’ emotional states and adjust teaching activities accordingly. However, existing methods for automatic facial expression recognition have the limitations of low accuracy of recognition and poor feature extraction. To address the problem, this study proposed a novel deep learning model called DenseNetX-CBAM to improve facial expression recognition by utilizing a variant of densely connected convolutional networks (DenseNet) to reduce unnecessary parameters and strengthen the reuse of expression features between networks; moreover, convolutional block attention module (CBAM) was integrated to allow the networks to focus on important special regions and important channels when representing features. The proposed model was tested using 217 video clips of 33 students in an online course. The results demonstrated promising effects of the method in improving the accuracy of facial expression recognition, which can help teachers to accurately recognize students’ emotions and provide real-time adjustment in online learning environments.

Improving recognition accuracy for facial expressions using scattering wavelet

One of the most evident and meaningful feedback about people’s emotions is through facial expressions. Facial expression recognition is helpful in social networks, marketing, and intelligent education systems. The use of Deep Learning based methods in facial expression identification is widespread, but challenges such as computational complexity and low recognition rate plague these methods. Scatter Wavelet is a type of Deep Learning that extracts features from Gabor filters in a structure similar to convolutional neural networks. This paper presents a new facial expression recognition method based on wavelet scattering that identifies six states: anger, disgust, fear, happiness, sadness, and surprise. The proposed method is simulated using the JAFFE and CK+ databases. The recognition rate of the proposed method is 99.7%, which indicates the superiority of the proposed method in recognizing facial expressions.

Deep learning strategies with CReToNeXt-YOLOv5 for advanced pig face emotion detection.

This study underscores the paramount importance of facial expressions in pigs, serving as a sophisticated mode of communication to gauge their emotions, physical well-being, and intentions. Given the inherent challenges in deciphering such expressions due to pigs' rudimentary facial muscle structure, we introduced an avant-garde pig facial expression recognition model named CReToNeXt-YOLOv5. The proposed model encompasses several refinements tailored for heightened accuracy and adeptness in detection. Primarily, the transition from the CIOU to the EIOU loss function optimized the training dynamics, leading to precision-driven regression outcomes. Furthermore, the incorporation of the Coordinate Attention mechanism accentuated the model's sensitivity to intricate expression features. A significant innovation was the integration of the CReToNeXt module, fortifying the model's prowess in discerning nuanced expressions. Efficacy trials revealed that CReToNeXt-YOLOv5 clinched a mean average precision (mAP) of 89.4%, marking a substantial enhancement by 6.7% relative to the foundational YOLOv5. Crucially, this advancement holds profound implications for animal welfare monitoring and research, as our findings underscore the model's capacity to revolutionize the accuracy of pig facial expression recognition, paving the way for more humane and informed livestock management practices.

A novel parameter dense three-dimensional convolution residual network method and its application in classroom teaching.

Improving the rationality and accuracy of classroom quality analysis is crucial in modern education. Traditional methods, such as questionnaires and manual recordings, are resource-intensive and subjective, leading to inconsistent results. As a solution, computer vision (CV) technologies have emerged as powerful tools for real-time classroom monitoring. This study proposes a novel Dense 3D Convolutional Residual Network (D3DCNN_ResNet) to recognize students' expressions and behaviors in English classrooms. The proposed method combines Single Shot Multibox Detector (SSD) for target detection with an improved D3DCNN_ResNet model. The network applies 3D convolution in both spatial and temporal domains, with shortcut connections from residual blocks to increase network depth. Dense connections are introduced to enhance the flow of high- and low-level features. The model was tested on two datasets: the CK+ dataset for expression recognition and the KTH dataset for behavior recognition. The experiments show that the proposed method is highly efficient in optimizing model training and improving recognition accuracy. On the CK+ dataset, the model achieved an expression recognition accuracy of 97.94%, while on the KTH dataset, the behavior recognition accuracy reached 98.86%. The combination of residual blocks and dense connections reduced feature redundancy and improved gradient flow, leading to better model performance. The results demonstrate that the D3DCNN_ResNet is well-suited for classroom quality analysis and has the potential to enhance teaching strategies by providing real-time feedback on student engagement.

Improved CycleGAN-based feature recognition in young children and preschool education research

Abstract Pre-school education is an integral part of primary education, and with the development of digital technology, the integration of pre-school education and information technology development is deepening. The study first adds category constraints to the CycleGAN network to improve the efficiency of algorithm operations. The facial features of young children are extracted using the LBP method and recognized by improving the CycleGAN network after they are classified using the SOC classification method. On this basis, the improved model is applied to early childhood preschool education to explore the recognition of young children’s characteristics and educational effects. In this paper, the recognition accuracy of the improved CycleGAN network model is improved by 0.02~0.14 in the expression recognition accuracy of seven basic expressions. in early childhood education, the third round of test scores of the experimental group of young children is 93 compared to the first round of test improved by 34 points, compared with the control group scores improved by 15 points. This study provides practical and feasible development directions for constructing and reforming information technology in preschool education.

NA

NA

Trainee psychotherapists’ emotion recognition accuracy during 1.5 years of psychotherapy education compared to a control group: no improvement after psychotherapy training

The ability to recognize and work with patients’ emotions is considered an important part of most psychotherapy approaches. Surprisingly, there is little systematic research on psychotherapists’ ability to recognize other people’s emotional expressions. In this study, we compared trainee psychotherapists’ nonverbal emotion recognition accuracy to a control group of undergraduate students at two time points: at the beginning and at the end of one and a half years of theoretical and practical psychotherapy training. Emotion recognition accuracy (ERA) was assessed using two standardized computer tasks, one for recognition of dynamic multimodal (facial, bodily, vocal) expressions and one for recognition of facial micro expressions. Initially, 154 participants enrolled in the study, 72 also took part in the follow-up. The trainee psychotherapists were moderately better at recognizing multimodal expressions, and slightly better at recognizing facial micro expressions, than the control group at the first test occasion. However, mixed multilevel modeling indicated that the ERA change trajectories for the two groups differed significantly. While the control group improved in their ability to recognize multimodal emotional expressions from pretest to follow-up, the trainee psychotherapists did not. Both groups improved their micro expression recognition accuracy, but the slope for the control group was significantly steeper than the trainee psychotherapists’. These results suggest that psychotherapy education and clinical training do not always contribute to improved emotion recognition accuracy beyond what could be expected due to time or other factors. Possible reasons for that finding as well as implications for the psychotherapy education are discussed.

Analysis and test of influence of memristor non-ideal characteristics on facial expression recognition accuracy

Analysis and test of influence of memristor non-ideal characteristics on facial expression recognition accuracy

Memristive patch attention neural network for facial expression recognition and edge computing.

Facial expression recognition has made a significant progress as a result of the advent of more and more convolutional neural networks (CNN). However, with the improvement of CNN, the models continues to get deeper and larger so as to a greater focus on the high-level features of the image and the low-level features tend to be lost. Because of the reason above, the dependence of low-level features between different areas of the face often cannot be summarized. In response to this problem, we propose a novel network based on the CNN model. To extract long-range dependencies of low-level features, multiple attention mechanisms has been introduced into the network. In this paper, the patch attention mechanism is designed to obtain the dependence between low-level features of facial expressions firstly. After fusion, the feature maps are input to the backbone network incorporating convolutional block attention module (CBAM) to enhance the feature extraction ability and improve the accuracy of facial expression recognition, and achieve competitive results on three datasets CK+(98.10%), JAFFE (95.12%) and FER2013 (73.50%). Further, according to the PA Net designed in this paper,a hardware friendly implementation scheme is designed based on memristor crossbars, which is expected to provide a software and hardware co-design scheme for edge computing of personal and wearable electronic products.

Expression Recognition of Multiple Faces Using a Convolution Neural Network Combining the Haar Cascade Classifier

Facial expression serves as the primary means for humans to convey emotions and communicate social signals. In recent years, facial expression recognition has become a viable application within medical systems because of the rapid development of artificial intelligence and computer vision. However, traditional facial expression recognition faces several challenges. The approach is designed to investigate the processing of facial expressions in real-time systems involving multiple individuals. These factors impact the accuracy and robustness of the model. In this paper, we adopted the Haar cascade classifier to extract facial features and utilized convolutional neural networks (CNNs) as the backbone model to achieve an efficient system. The proposed approach achieved an accuracy of approximately 70% on the FER-2013 dataset in the experiment. This result represents an improvement of 7.83% compared to that of the baseline system. This significant enhancement improves the accuracy of facial expression recognition. Herein, the proposed approach also extended to multiple face expression recognition; the module was further experimented with and obtained promising results. The outcomes of this research will establish a solid foundation for real-time monitoring and prevention of conditions such as depression through an emotion alert system.

Supraphysiological testosterone levels from anabolic steroid use and reduced sensitivity to negative facial expressions in men

RationaleAnabolic-androgenic steroids (AAS) are used to improve physical performance and appearance, but have been associated with deficits in social cognitive functioning. Approximately 30% of people who use AAS develop a dependence, increasing the risk for undesired effects.ObjectivesTo assess the relationship between AAS use (current/previous), AAS dependence, and the ability to recognize emotional facial expressions, and investigate the potential mediating role of hormone levels.MethodsIn total 156 male weightlifters, including those with current (n = 45) or previous (n = 34) AAS use and never-using controls (n = 77), completed a facial Emotion Recognition Task (ERT). Participants were presented with faces expressing one out of six emotions (sadness, happiness, fear, anger, disgust, and surprise) and were instructed to indicate which of the six emotions each face displayed. ERT accuracy and response time were recorded and evaluated for association with AAS use status, AAS dependence, and serum reproductive hormone levels. Mediation models were used to evaluate the mediating role of androgens in the relationship between AAS use and ERT performance.ResultsCompared to never-using controls, men currently using AAS exhibited lower recognition accuracy for facial emotional expressions, particularly anger (Cohen’s d = −0.57, pFDR = 0.03) and disgust (d = −0.51, pFDR = 0.05). Those with AAS dependence (n = 47) demonstrated worse recognition of fear relative to men without dependence (d = 0.58, p = 0.03). Recognition of disgust was negatively correlated with serum free testosterone index (FTI); however, FTI did not significantly mediate the association between AAS use and recognition of disgust.ConclusionsOur findings demonstrate impaired facial emotion recognition among men currently using AAS compared to controls. While further studies are needed to investigate potential mechanisms, our analysis did not support a simple mediation effect of serum FTI.