Emotion Recognition Studies Research Articles

Music-induced emotion flow modeling by ENMI Network.

The relation between emotions and music is substantial because music as an art can evoke emotions. Music emotion recognition (MER) studies the emotions that music brings in the effort to map musical features to the affective dimensions. This study conceptualizes the mapping of music and emotion as a multivariate time series regression problem, with the aim of capturing the emotion flow in the Arousal-Valence emotional space. The Efficient Net-Music Informer (ENMI) Network was introduced to address this phenomenon. The ENMI was used to extract Mel-spectrogram features, complementing the time series data. Moreover, the Music Informer model was adopted to train on both time series music features and Mel-spectrogram features to predict emotional sequences. In our regression task, the model achieved a root mean square error (RMSE) of 0.0440 and 0.0352 in the arousal and valence dimensions, respectively, in the DEAM dataset. A comprehensive analysis of the effects of different hyperparameters tuning was conducted. Furthermore, different sequence lengths were predicted for the regression accuracy of the ENMI Network on three different datasets, namely the DEAM dataset, the Emomusic dataset, and the augmented Emomusic dataset. Additionally, a feature ablation on the Mel-spectrogram features and an analysis of the importance of the various musical features in the regression results were performed, establishing the effectiveness of the model presented herein.

EEG Dataset for the Recognition of Different Emotions Induced in Voice-User Interaction

Electroencephalography (EEG)-based open-access datasets are available for emotion recognition studies, where external auditory/visual stimuli are used to artificially evoke pre-defined emotions. In this study, we provide a novel EEG dataset containing the emotional information induced during a realistic human-computer interaction (HCI) using a voice user interface system that mimics natural human-to-human communication. To validate our dataset via neurophysiological investigation and binary emotion classification, we applied a series of signal processing and machine learning methods to the EEG data. The maximum classification accuracy ranged from 43.3% to 90.8% over 38 subjects and classification features could be interpreted neurophysiologically. Our EEG data could be used to develop a reliable HCI system because they were acquired in a natural HCI environment. In addition, auxiliary physiological data measured simultaneously with the EEG data also showed plausible results, i.e., electrocardiogram, photoplethysmogram, galvanic skin response, and facial images, which could be utilized for automatic emotion discrimination independently from, as well as together with the EEG data via the fusion of multi-modal physiological datasets.

Hcnnxgboost: A Hybrid Cnn-Xgboost Approach for Effective Emotion Detection in Textual Data

in recent years, emotional analysis has become a key focus in computational studies, driven by the need to understand societal sentiments. this exploration is motivated by its many promising applications, such as community well-being evaluation, human-computer interaction, suicide prevention, and personalized recommendations. even with progress in other areas like identifying expressions from facial cues and speech, the study of text-based emotion recognition remains a fascinating field of research because machines struggle to interpret context, especially compared to human capabilities. our research addresses this by introducing a multiclass text-based emotion detection system that combines a cnn architecture with xgboost for improved classification in natural language processing. we pre-process publicly available datasets and use glove pre-trained word embeddings for better text representation. a major contribution of our work is enhancing the feature space by combining cnn probabilities with the original text data. the proposed hcnnxgboost model outperforms all other machine learning and deep learning algorithms across the emoint, isear, and crowdflower datasets, achieving f-scores of 90.1%, 87.4%, and 62.2%, respectively. experimental evaluations on benchmark datasets show better f-scores, confirming the effectiveness of our approach. comparisons with other classifiers highlight the enhanced performance and effectiveness of our hybrid cnnxgboost (hcnnxgboost) model, making it one of the best solutions for emotion classification in natural language processing tasks.

AVaTER: Fusing Audio, Visual, and Textual Modalities Using Cross-Modal Attention for Emotion Recognition.

Multimodal emotion classification (MEC) involves analyzing and identifying human emotions by integrating data from multiple sources, such as audio, video, and text. This approach leverages the complementary strengths of each modality to enhance the accuracy and robustness of emotion recognition systems. However, one significant challenge is effectively integrating these diverse data sources, each with unique characteristics and levels of noise. Additionally, the scarcity of large, annotated multimodal datasets in Bangla limits the training and evaluation of models. In this work, we unveiled a pioneering multimodal Bangla dataset, MAViT-Bangla (Multimodal Audio Video Text Bangla dataset). This dataset, comprising 1002 samples across audio, video, and text modalities, is a unique resource for emotion recognition studies in the Bangla language. It features emotional categories such as anger, fear, joy, and sadness, providing a comprehensive platform for research. Additionally, we developed a framework for audio, video and textual emotion recognition (i.e., AVaTER) that employs a cross-modal attention mechanism among unimodal features. This mechanism fosters the interaction and fusion of features from different modalities, enhancing the model's ability to capture nuanced emotional cues. The effectiveness of this approach was demonstrated by achieving an F1-score of 0.64, a significant improvement over unimodal methods.

Research Trend and Impact on Student Learning using Artificial Intelligence-Based Emotion Recognation: Systematic Bibliometric Analysis

Emotions have a significant function in the learning process. Good emotions such as enjoyment and curiosity can increase attention and comprehension, whereas negative emotions such as fear and boredom might impair academic achievement. Therefore, identifying and managing students' emotions is a critical step in developing an effective learning environment. In recent years, there has been a substantial progress in the application of artificial intelligence to detect emotions through facial expression analysis. Systematic literature reviews based on bibliometric analysis of artificial intelligence-based emotion recognition studies are still difficult to locate. This paper aims to undertake bibliometric and visualization analyses with VOSviewer for artificial intelligence-based emotion recognition. The evaluation includes 230 scholarly publications on emotion detection investigations indexed by Scopus quartiles Q1–Q4 from the Scopus database in the recent decade, namely 2014–2024. This bibliometric analysis has major implications for increasing academic, practical, and policy-related elements of artificial intelligence-based emotion recognition in education. By recognizing trends and major contributions in this subject, researchers, practitioners, and policymakers may work together to develop a more flexible and supportive learning environment, thereby boosting the overall quality of education.

A Research on Emotion Recognition of the Elderly Based on Transformer and Physiological Signals

Aiming at problems such as the difficulty of recognizing emotions in the elderly and the inability of traditional machine-learning models to effectively capture the nonlinear relationship between physiological signal data, a Recursive Map (RM) combined with a Vision Transformer (ViT) is proposed to recognize the emotions of the elderly based on Electroencephalogram (EEG), Electrodermal Activity (EDA), and Heart Rate Variability (HRV) signals. The Dung Beetle Optimizer (DBO) is used to optimize the variational modal decomposition of EEG, EDA, and HRV signals. The optimized decomposed time series signals are converted into two-dimensional images using RM, and then the converted image signals are applied to the ViT for the study of emotion recognition of the elderly. The pre-trained weights of ViT on the ImageNet-22k dataset are loaded into the model and retrained with the two-dimensional image data. The model is validated and compared using the test set. The research results show that the recognition accuracy of the proposed method on EEG, EDA, and HRV signals is 99.35%, 86.96%, and 97.20%, respectively. This indicates that EEG signals can better reflect the emotional problems of the elderly, followed by HRV signals, while EDA signals have poorer effects. Compared with Support Vector Machine (SVM), Naive Bayes (NB), and K-Nearest Neighbors (KNN), the recognition accuracy of the proposed method is increased by at least 9.4%, 11.13%, and 12.61%, respectively. Compared with ResNet34, EfficientNet-B0, and VGG16, it is increased by at least 1.14%, 0.54%, and 3.34%, respectively. This proves the superiority of the proposed method in emotion recognition for the elderly.

Deficits of Facial Emotion Recognition in Elderly Individuals with Mild Cognitive Impairment.

The study of facial emotion recognition is under-explored in subjects with mild cognitive impairment (MCI). We investigated whether deficits in facial emotion recognition are present in patients with MCI. We also analyzed the relationship between facial emotion recognition and different domains of cognitive function. This study included 300 participants aged 60 years or older with cognitive decline. We evaluated 181 MCI and 119 non-MCI subjects using the Seoul Neuropsychological Screening Battery-Core (SNSB-C) and facial emotion recognition task using six facial expressions (anger, disgust, fear, happiness, sadness and surprise). A Generalized Linear Model (GLM) was used to assess the association between cognitive performance and accuracy of facial emotion recognition and to compare facial emotion recognition in the MCI group based on the impairment of five different domains of cognitive function. The model was adjusted for age, sex, years of education, and depressive symptoms. Patients with MCI had a lower score for accurately recognizing total facial emotion (0.48 vs. 0.53; ρ = 0.0003) and surprise (0.73 vs. 0.81; ρ = 0.0215) when compared to cognitively healthy subjects. We also discovered that frontal/executive function domain (Digit Symbol Coding [DSC, 0.38 vs. 0.49; p < 0.0001], Controlled Oral Word Association Test [COWAT, 0.42 vs. 0.49; p = 0.0001], Korean-Trail Making Test [K-TMT, 0.37 vs. 0.48; p = 0.0073], Korean-Color Word Stroop Test [K-CWST, 0.43 vs. 0.49; p = 0.0219]) and language domain (Korean-Boston Naming Test [S-K-BNT, 0.46 vs. 0.47; p = 0.003]) were statistically associated with the deficits of facial emotion recognition in patients with MCI. We observed a significant association between deficits in facial emotion recognition and cognitive impairment in elderly individuals.

The Effects of Sensor and Feature Level Fusion Methods in Multimodal Emotion Analysis

Fusion-based multimodal emotion recognition (MER) studies are very popular nowadays. In this study, EEG signals and face images are fused with sensor level fusion (SLF) and feature level fusion (FLF) methods for multimodal emotion recognition. The general flow of the study is as follows. Firstly, EEG signals are converted into angle amplitude graph (AAG) images. Secondly, the most unique ones are automatically identified from all face images obtained from video recordings. Then, these modalities are fused separately using SLF and FLF methods. The fusion approaches were used to combine the obtained data and perform classification on the integrated data. The experiments were performed with the publicly available DEAP dataset. The highest accuracy was 82.14% with 5.26 standard deviations for SLF and 87.62% with 6.74 standard deviations for FLF. These results show that this study makes a significant contribution to the field of emotion recognition by providing an effective method.

Research on Recognition Method of Social Robot Based on T-A-GCNIIT in the Metaverse

Social robots are used in intelligent customer service, intelligent chat, intelligent shopping guides, and more because of emotion recognition studies in cognitive psychology. However, determining the user's purpose quickly and precisely has proved challenging. Domestic researchers proposed the A-GCNII model to address missing feature information; however, it needs a lot of math. This research offers a social robot recognition approach using the T-A-GCNIIT model and cognitive psychology to optimize computing complexity and performance. The T-A-GCNIIT algorithm processes social network data, and the Viola–Jones algorithm improves social robot intelligence to represent social robots in the meta-universe. The model performs well in node classification, link prediction, community discovery, and other tasks, with enhanced accuracy, recall, F1 score value, and other metrics. The model can also better comprehend the user's emotional state using cognitive psychology to better recognize their purpose and propose a fresh notion for enhancing social robots' cognitive psychology.

Predicting game-induced emotions using EEG, data mining and machine learning

BackgroundEmotion is a complex phenomenon that greatly affects human behavior and thinking in daily life. Electroencephalography (EEG), one of the human physiological signals, has been emphasized by most researchers in emotion recognition as its specific properties are closely associated with human emotion. However, the number of human emotion recognition studies using computer games as stimuli is still insufficient as there were no relevant publicly available datasets provided in the past decades. Most of the recent studies using the Gameemo public dataset have not clarified the relationship between the EEG signal’s changes and the emotion elicited using computer games. Thus, this paper is proposed to introduce the use of data mining techniques in investigating the relationships between the frequency changes of EEG signals and the human emotion elicited when playing different kinds of computer games. The data acquisition stage, data pre-processing, data annotation and feature extraction stage were designed and conducted in this paper to obtain and extract the EEG features from the Gameemo dataset. The cross-subject and subject-based experiments were conducted to evaluate the classifiers’ performance. The top 10 association rules generated by the RCAR classifier will be examined to determine the possible relationship between the EEG signal's frequency changes and game-induced emotions.ResultsThe RCAR classifier constructed for cross-subject experiment achieved highest accuracy, precision, recall and F1-score evaluated with over 90% in classifying the HAPV, HANV and LANV game-induced emotions. The 20 experiment cases’ results from subject-based experiments supported that the SVM classifier could accurately classify the 4 emotion states with a kappa value over 0.62, demonstrating the SVM-based algorithm’s capabilities in precisely determining the emotion label for each participant’s EEG features’ instance.ConclusionThe findings in this study fill the existing gap of game-induced emotion recognition field by providing an in-depth evaluation on the ruleset algorithm’s performance and feasibility of applying the generated rules on the game-induced EEG data for justifying the emotional state prediction result.

Cross-modal credibility modelling for EEG-based multimodal emotion recognition

Objective. The study of emotion recognition through electroencephalography (EEG) has garnered significant attention recently. Integrating EEG with other peripheral physiological signals may greatly enhance performance in emotion recognition. Nonetheless, existing approaches still suffer from two predominant challenges: modality heterogeneity, stemming from the diverse mechanisms across modalities, and fusion credibility, which arises when one or multiple modalities fail to provide highly credible signals. Approach. In this paper, we introduce a novel multimodal physiological signal fusion model that incorporates both intra-inter modality reconstruction and sequential pattern consistency, thereby ensuring a computable and credible EEG-based multimodal emotion recognition. For the modality heterogeneity issue, we first implement a local self-attention transformer to obtain intra-modal features for each respective modality. Subsequently, we devise a pairwise cross-attention transformer to reveal the inter-modal correlations among different modalities, thereby rendering different modalities compatible and diminishing the heterogeneity concern. For the fusion credibility issue, we introduce the concept of sequential pattern consistency to measure whether different modalities evolve in a consistent way. Specifically, we propose to measure the varying trends of different modalities, and compute the inter-modality consistency scores to ascertain fusion credibility. Main results. We conduct extensive experiments on two benchmarked datasets (DEAP and MAHNOB-HCI) with the subject-dependent paradigm. For the DEAP dataset, our method improves the accuracy by 4.58%, and the F1 score by 0.63%, compared to the state-of-the-art baseline. Similarly, for the MAHNOB-HCI dataset, our method improves the accuracy by 3.97%, and the F1 score by 4.21%. In addition, we gain much insight into the proposed framework through significance test, ablation experiments, confusion matrices and hyperparameter analysis. Consequently, we demonstrate the effectiveness of the proposed credibility modelling through statistical analysis and carefully designed experiments. Significance. All experimental results demonstrate the effectiveness of our proposed architecture and indicate that credibility modelling is essential for multimodal emotion recognition.

DC-BVM: Dual-channel information fusion network based on voting mechanism

Emotion recognition in conversations (ERC) has been challenging due to the dynamics and complexity of emotions in conversations. Most current emotion recognition studies have focused on modeling temporal dimensions, such as context-sensitive dependencies while ignoring the spatial dimensional relationships between discourses. In this paper, we propose a dual-channel information fusion network based on a voting mechanism, in which we model modal information in two dimensions: distance dependence is constructed in the spatial dimension, and context dependence is constructed in the temporal dimension. We aim to extract more comprehensive and accurate sentiment features to recognize verbal emotions from limited data. In addition, we discard the traditional modal fusion method and propose a fusion strategy based on the voting mechanism, which significantly accelerates the model convergence speed and recognition performance. We conducted experiments on two benchmark datasets, IEMOCAP and MELD, respectively, and the experimental results demonstrate the superiority of the designed network in emotion recognition.

Subject-Independent Emotion Recognition Based on EEG Frequency Band Features and Self-Adaptive Graph Construction.

Emotion is one of the most important higher cognitive functions of the human brain and plays an important role in transaction processing and decisions. In traditional emotion recognition studies, the frequency band features in EEG signals have been shown to have a high correlation with emotion production. However, traditional emotion recognition methods cannot satisfactorily solve the problem of individual differences in subjects and data heterogeneity in EEG, and subject-independent emotion recognition based on EEG signals has attracted extensive attention from researchers. In this paper, we propose a subject-independent emotion recognition model based on adaptive extraction of layer structure based on frequency bands (BFE-Net), which is adaptive in extracting EEG map features through the multi-graphic layer construction module to obtain a frequency band-based multi-graphic layer emotion representation. To evaluate the performance of the model in subject-independent emotion recognition studies, extensive experiments are conducted on two public datasets including SEED and SEED-IV. The experimental results show that in most experimental settings, our model has a more advanced performance than the existing studies of the same type. In addition, the visualization of brain connectivity patterns reveals that some of the findings are consistent with previous neuroscientific validations, further validating the model in subject-independent emotion recognition studies.

Speech Emotion Recognition Using Deep Learning Transfer Models and Explainable Techniques

This study aims to establish a greater reliability compared to conventional speech emotion recognition (SER) studies. This is achieved through preprocessing techniques that reduce uncertainty elements, models that combine the structural features of each model, and the application of various explanatory techniques. The ability to interpret can be made more accurate by reducing uncertain learning data, applying data in different environments, and applying techniques that explain the reasoning behind the results. We designed a generalized model using three different datasets, and each speech was converted into a spectrogram image through STFT preprocessing. The spectrogram was divided into the time domain with overlapping to match the input size of the model. Each divided section is expressed as a Gaussian distribution, and the quality of the data is investigated by the correlation coefficient between distributions. As a result, the scale of the data is reduced, and uncertainty is minimized. VGGish and YAMNet are the most representative pretrained deep learning networks frequently used in conjunction with speech processing. In dealing with speech signal processing, it is frequently advantageous to use these pretrained models synergistically rather than exclusively, resulting in the construction of ensemble deep networks. And finally, various explainable models (Grad CAM, LIME, occlusion sensitivity) are used in analyzing classified results. The model exhibits adaptability to voices in various environments, yielding a classification accuracy of 87%, surpassing that of individual models. Additionally, output results are confirmed by an explainable model to extract essential emotional areas, converted into audio files for auditory analysis using Grad CAM in the time domain. Through this study, we enhance the uncertainty of activation areas that are generated by Grad CAM. We achieve this by applying the interpretable ability from previous studies, along with effective preprocessing and fusion models. We can analyze it from a more diverse perspective through other explainable techniques.

EEG Emotion Classification Based on Graph Convolutional Network

EEG-based emotion recognition is a task that uses scalp-EEG data to classify the emotion states of humans. The study of EEG-based emotion recognition can contribute to a large spectrum of application fields including healthcare and human–computer interaction. Recent studies in neuroscience reveal that the brain regions and their interactions play an essential role in the processing of different stimuli and the generation of corresponding emotional states. Nevertheless, such regional interactions, which have been proven to be critical in recognizing emotions in neuroscience, are largely overlooked in existing machine learning or deep learning models, which focus on individual channels in brain signals. Motivated by this, in this paper, we present RGNet, a model that is designed to learn the regional level representation of EEG signal for accurate emotion recognition. Specifically, after applying preprocessing and feature extraction techniques on raw signals, RGNet adopts a novel region-wise encoder to extract the features of channels located within each region as input to compute the regional level features, enabling the model to effectively explore the regional functionality. A graph is then constructed by considering each region as a node and connections between regions as edges, upon which a graph convolutional network is designed with spectral filtering and learned adjacency matrix. Instead of focusing on only the spatial proximity, it allows the model to capture more complex functional relationships. We conducted experiments from the perspective of region division strategies, region encoders and input feature types. Our model has achieved 98.64% and 99.33% for Deap and Dreamer datasets, respectively. The comparison studies show that RGNet outperforms the majority of the existing models for emotion recognition from EEG signals.

Embedded EEG Feature Selection for Multi-Dimension Emotion Recognition via Local and Global Label Relevance.

Due to the problem of a small amount of EEG samples and relatively high dimensionality of electroencephalogram (EEG) features, feature selection plays an essential role in EEG-based emotion recognition. However, current EEG-based emotion recognition studies utilize a problem transformation approach to transform multi-dimension emotional labels into single-dimension labels, and then implement commonly used single-label feature selection methods to search feature subsets, which ignores the relations between different emotional dimensions. To tackle the problem, we propose an efficient EEG feature selection method for multi-dimension emotion recognition (EFSMDER) via local and global label relevance. First, to capture the local label correlations, EFSMDER implements orthogonal regression to map the original EEG feature space into a low-dimension space. Then, it employs the global label correlations in the original multi-dimension emotional label space to effectively construct the label information in the low-dimension space. With the aid of local and global relevance information, EFSMDER can conduct representational EEG feature subset selection. Three EEG emotional databases with multi-dimension emotional labels were used for performance comparison between EFSMDER and fourteen state-of-the-art methods, and the EFSMDER method achieves the best multi-dimension classification accuracies of 86.43, 84.80, and 97.86 percent on the DREAMER, DEAP, and HDED datasets, respectively.

Image-Evoked Emotion Recognition for Hearing-Impaired Subjects with EEG Signals

In recent years, there has been a growing interest in the study of emotion recognition through electroencephalogram (EEG) signals. One particular group of interest are individuals with hearing impairments, who may have a bias towards certain types of information when communicating with those in their environment. To address this, our study collected EEG signals from both hearing-impaired and non-hearing-impaired subjects while they viewed pictures of emotional faces for emotion recognition. Four kinds of feature matrices, symmetry difference, and symmetry quotient based on original signal and differential entropy (DE) were constructed, respectively, to extract the spatial domain information. The multi-axis self-attention classification model was proposed, which consists of local attention and global attention, combining the attention model with convolution through a novel architectural element for feature classification. Three-classification (positive, neutral, negative) and five-classification (happy, neutral, sad, angry, fearful) tasks of emotion recognition were carried out. The experimental results show that the proposed method is superior to the original feature method, and the multi-feature fusion achieved a good effect in both hearing-impaired and non-hearing-impaired subjects. The average classification accuracy for hearing-impaired subjects and non-hearing-impaired subjects was 70.2% (three-classification) and 50.15% (five-classification), and 72.05% (three-classification) and 51.53% (five-classification), respectively. In addition, by exploring the brain topography of different emotions, we found that the discriminative brain regions of the hearing-impaired subjects were also distributed in the parietal lobe, unlike those of the non-hearing-impaired subjects.

Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

The recognition of emotions is one of the most challenging issues in human-computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in emotion recognition studies. Most studies, however, use other methods to extract handcrafted features, such as Pearson correlation coefficient (PCC), Principal Component Analysis, Higuchi Fractal Dimension (HFD), etc., even though DNN is capable of generating meaningful features. Furthermore, most earlier studies largely ignored spatial information between the different channels, focusing mainly on time domain and frequency domain representations. This study utilizes a pre-trained 3D-CNN MobileNet model with transfer learning on the spatio-temporal representation of EEG signals to extract features for emotion recognition. In addition to fully connected layers, hybrid models were explored using other decision layers such as multilayer perceptron (MLP), k-nearest neighbor (KNN), extreme learning machine (ELM), XGBoost (XGB), random forest (RF), and support vector machine (SVM). Additionally, this study investigates the effects of post-processing or filtering output labels. Extensive experiments were conducted on the SJTU Emotion EEG Dataset (SEED) (three classes) and SEED-IV (four classes) datasets, and the results obtained were comparable to the state-of-the-art. Based on the conventional 3D-CNN with ELM classifier, SEED and SEED-IV datasets showed a maximum accuracy of 89.18% and 81.60%, respectively. Post-filtering improved the emotional classification performance in the hybrid 3D-CNN with ELM model for SEED and SEED-IV datasets to 90.85% and 83.71%, respectively. Accordingly, spatial-temporal features extracted from the EEG, along with ensemble classifiers, were found to be the most effective in recognizing emotions compared to state-of-the-art methods.

Extracting a Novel Emotional EEG Topographic Map Based on a Stacked Autoencoder Network.

Emotion recognition based on brain signals has increasingly become attractive to evaluate human's internal emotional states. Conventional emotion recognition studies focus on developing machine learning and classifiers. However, most of these methods do not provide information on the involvement of different areas of the brain in emotions. Brain mapping is considered as one of the most distinguishing methods of showing the involvement of different areas of the brain in performing an activity. Most mapping techniques rely on projection and visualization of only one of the electroencephalogram (EEG) subband features onto brain regions. The present study aims to develop a new EEG-based brain mapping, which combines several features to provide more complete and useful information on a single map instead of common maps. In this study, the optimal combination of EEG features for each channel was extracted using a stacked autoencoder (SAE) network and visualizing a topographic map. Based on the research hypothesis, autoencoders can extract optimal features for quantitative EEG (QEEG) brain mapping. The DEAP EEG database was employed to extract topographic maps. The accuracy of image classifiers using the convolutional neural network (CNN) was used as a criterion for evaluating the distinction of the obtained maps from a stacked autoencoder topographic map (SAETM) method for different emotions. The average classification accuracy was obtained 0.8173 and 0.8037 in the valence and arousal dimensions, respectively. The extracted maps were also ranked by a team of experts compared to common maps. The results of quantitative and qualitative evaluation showed that the obtained map by SAETM has more information than conventional maps.

A retrospective case-control study of facial emotion recognition in male veterans with chronic schizophrenia and its correlation with interpersonal communication

ObjectiveTo understand the facial emotion recognition of male veterans with chronic schizophrenia and the relationship between facial emotion recognition and interpersonal communication to provide a reference for designing social skills training programmes.MethodFifty-six eligible male patients with chronic schizophrenia who were admitted to our hospital from October 2020 to April 2021 were selected, and 24 healthy people were selected as controls. Facial emotion recognition, social communication skills and self-perceived interpersonal disturbance were assessed using a facial emotion recognition stimulus manual, the Social Skills Checklist (SSC) and the Interpersonal Relationship Integrative Diagnostic Scale (IRIDS). Disease status was assessed using the Positive and Negative Syndrome Scale.ResultsBoth the control group and the patient group had the highest recognition accuracy for neutral faces. The recognition rate for neutral expression was higher in the control group than in the patient group (p = 0.008). The rate of neutral expressions identified as happiness was higher in the patient group than in the control group (p = 0.001). The identification of anger as happiness was higher in the control group than in the patient group (p = 0.026), and the pattern of misidentification was similar between the control group and the patient group. The accuracy of facial emotion recognition was negatively associated with the age of onset (p < 0.05). The recognition accuracy for happiness was negatively associated with negative symptoms, general pathological symptoms and total scale scores (p < 0.05). The total score for expression recognition was negatively associated with the negative symptom subscale scores (p < 0.05), and there was no correlation between expression recognition and positive symptoms (p > 0.05). The recognition accuracy for happiness was negatively correlated with the IRIDS conversation factor (p < 0.05). The recognition accuracy for happiness and anger and the total scores for facial emotion recognition were negatively correlated with the SSC subscale score and the total score (p < 0.05 and p < 0.01, respectively). The main influencing factors on facial emotion recognition were the SSC total score (p < 0.001) and the positive factor score (p = 0.039).ConclusionVeterans with chronic schizophrenia have facial emotion recognition impairments affected by negative symptoms. There is a correlation between facial emotion recognition and interpersonal communication.Highlights1. There are extensive facial expression recognition disorders in schizophrenia.2. The pattern of misidentification was similar in both the control group and the patient group, with the tendency for happiness to be identified as a neutral emotion, anger as happiness, and fear as neutral emotion and anger.3. Based on the comprehensive assessment of social cognitive impairment in patients with schizophrenia, prospective studies of standardised interventions are designed to provide support for clinical practice.