Emotion Recognition Performance Research Articles

Multimodal Dataset Construction and Validation for Driving-Related Anger: A Wearable Physiological Conduction and Vehicle Driving Data Approach

Anger impairs a driver’s control and risk assessment abilities, heightening traffic accident risks. Constructing a multimodal dataset during driving tasks is crucial for accurate anger recognition. This study developed a multimodal physiological -vehicle driving dataset (DPV-MFD) based on drivers’ self-reported anger during simulated driving tasks. In Experiment 1, responses from 624 participants to anger-inducing videos and driving scenarios were collected via questionnaires to select appropriate materials. In Experiments 2 and 3, multimodal dynamic data and self-reported SAM emotion ratings were collected during simulated and real-vehicle tasks, capturing physiological and vehicle responses in neutral and anger states. Spearman’s correlation coefficient analysis validated the DPV-MFD’s effectiveness and explored the relationships between multimodal data and emotional dimensions. The CNN-LSTM deep learning network was used to assess the emotion recognition performance of the DPV-MFD across different time windows, and its applicability in real-world driving scenarios was validated. Compared to using EEG data alone, integrating multimodal data significantly improved anger recognition accuracy, with accuracy and F1 scores rising by 4.49% and 9.14%, respectively. Additionally, real-vehicle data closely matched simulated data, confirming the dataset’s effectiveness for real-world applications. This research is pivotal for advancing emotion-aware human–machine- interaction and intelligent transportation systems.

Introducing a novel dataset for facial emotion recognition and demonstrating significant enhancements in deep learning performance through pre-processing techniques

Facial expression recognition (FER) plays a pivotal role in various applications, ranging from human-computer interaction to psychoanalysis. To improve the accuracy of facial emotion recognition (FER) models, this study focuses on enhancing and augmenting FER datasets. It comprehensively analyzes the Facial Emotion Recognition dataset (FER13) to identify defects and correct misclassifications. The FER13 dataset represents a crucial resource for researchers developing Deep Learning (DL) models aimed at recognizing emotions based on facial features. Subsequently, this article develops a new facial dataset by expanding upon the original FER13 dataset. Similar to the FER + dataset, the expanded dataset incorporates a wider range of emotions while maintaining data accuracy. To further improve the dataset, it will be integrated with the extended Cohn-Kanade (CK+) dataset.This paper investigates the application of modern DL models to enhance emotion recognition in human faces. By training a new dataset, the study demonstrates significant performance gains compared with its counterparts. Furthermore, the article examines recent advances in FER technology and identifies critical requirements for DL models to overcome the inherent challenges of this task effectively. The study explores several DL architectures for emotion recognition in facial image datasets, with a particular focus on convolutional neural networks (CNNs). Our findings indicate that complex architecture, such as EfficientNetB7, outperforms other DL architectures, achieving a test accuracy of 78.9 %. Notably, the model surpassed the EfficientNet-XGBoost model, especially when used with the new dataset. Our approach leverages EfficientNetB7 as a backbone to build a model capable of efficiently recognizing emotions from facial images. Our proposed model, EfficientNetB7-CNN, achieved a peak accuracy of 81 % on the test set despite facing challenges such as GPU memory limitations. This demonstrates the model's robustness in handling complex facial expressions. Furthermore, to enhance feature extraction and attention mechanisms, we propose a new hybrid model, CBAM-4CNN, which integrates the convolutional block attention module (CBAM) with a custom 4-layer CNN architecture. The results showed that the CBAM-4CNN model outperformed existing models, achieving higher accuracy, precision, and recall metrics across multiple emotion classes. The results highlight the critical role of comprehensive and diverse data in enhancing model performance for facial emotion recognition.

Multi-view brain functional connectivity and hierarchical fusion for EEG-based emotion recognition

Emotional states evolve gradually from their onset to full manifestation, reflected in changes in brain functional connectivity. Existing research often focuses only on the final emotional state, missing the rich, dynamic information of emotional evolution. This paper proposes a novel network framework leveraging global and local brain functional connectivity through a multi-view approach. We introduce a global and local connectivity modeling method using electroencephalogram signals to simulate brain connectivity changes across time and specific moments. A residual-based local detail feature extractor combined with a coordinate attention module captures long-range dependencies and maintains positional accuracy. A global feature extractor is also designed for the effective extraction of emotional information. Finally, a hierarchical multi-scale cross-attention feature fusion module integrates features across different levels to enhance emotion recognition performance and robustness. Extensive experiments on three publicly available datasets demonstrate the framework’s superiority and generalization capability.

EAV: EEG-Audio-Video Dataset for Emotion Recognition in Conversational Contexts

Understanding emotional states is pivotal for the development of next-generation human-machine interfaces. Human behaviors in social interactions have resulted in psycho-physiological processes influenced by perceptual inputs. Therefore, efforts to comprehend brain functions and human behavior could potentially catalyze the development of AI models with human-like attributes. In this study, we introduce a multimodal emotion dataset comprising data from 30-channel electroencephalography (EEG), audio, and video recordings from 42 participants. Each participant engaged in a cue-based conversation scenario, eliciting five distinct emotions: neutral, anger, happiness, sadness, and calmness. Throughout the experiment, each participant contributed 200 interactions, which encompassed both listening and speaking. This resulted in a cumulative total of 8,400 interactions across all participants. We evaluated the baseline performance of emotion recognition for each modality using established deep neural network (DNN) methods. The Emotion in EEG-Audio-Visual (EAV) dataset represents the first public dataset to incorporate three primary modalities for emotion recognition within a conversational context. We anticipate that this dataset will make significant contributions to the modeling of the human emotional process, encompassing both fundamental neuroscience and machine learning viewpoints.

Hybrid CNN-BiLSTM architecture with multiple attention mechanisms to enhance speech emotion recognition

During recent years, the concept of attention in deep learning has been increasingly used to boost the performance of Speech Emotion Recognition (SER) models. However, these models for SER exhibit shortcomings in jointly emphasizing the time-frequency and dynamic sequential variations, often under-utilizing the rich contextual emotion-related information. We propose a hybrid deep learning model for SER using Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory Networks (BiLSTM) with multiple attention mechanisms. Our model utilizes features from the speech waveform viz. Mel spectrograms and Mel Frequency Cepstral Coefficients (MFCC), along with their time derivatives as input to the CNN and BiLSTM modules, respectively. A Time–Frequency Attention (TFA) mechanism, optimally incorporated into CNN, helps to selectively focus on emotion-related energy–time–frequency variations in Mel spectrograms. Attention-based BiLSTM uses MFCC and its time derivatives to identify the positional information of emotion for addressing the dynamic sequential variations. Finally, we fuse the attention-learned features from the CNN and BiLSTM modules and feed them to a Deep Neural Network (DNN) for SER. The experiments were carried out using three different datasets: Emo-DB and IEMOCAP, which are public datasets, and Amritaemo_Arabic; a private dataset. The hybrid model exhibited superior performance on both the public and private datasets, generating an average SER accuracy of 94.62%, 67.85%, and 95.80% with Emo-DB, IEMOCAP, and Amritaemo_Arabic datasets, respectively, effectively outperforming several state-of-the-art models.

Advancing emotion recognition in facial expressions through PCA, RFE, and MLP Integration

Emotion recognition through facial expressions is vital for enhancing human-computer interaction, making systems more intuitive and responsive to user needs. This study introduces an innovative approach to emotion detection, leveraging Principal Component Analysis (PCA) and Recursive Feature Elimination (RFE) for feature extraction and optimization. The methodology focuses on refining facial feature representations to improve classification accuracy, which is critical for accurately detecting emotions like happiness, sadness, fear, and surprise. The approach was applied to two widely recognized facial emotion datasets: the Cohn-Kanade (CK) and the Japanese Female Facial Expression (JAFFE) datasets. By integrating PCA and RFE, the model efficiently selects the most relevant features, enhancing the overall performance of emotion recognition. A comparative analysis with existing deep learning models highlights the advantages of the proposed method. The effectiveness of this approach is further supported by the results, where the model achieves an accuracy of 98.49% on the CK dataset and 96.03% on the JAFFE dataset. These results demonstrate a significant improvement over recent methods, indicating the model's potential for real-world applications.

SFT-SGAT: A semi-supervised fine-tuning self-supervised graph attention network for emotion recognition and consciousness detection

Emotional recognition is highly important in the field of brain-computer interfaces (BCIs). However, due to the individual variability in electroencephalogram (EEG) signals and the challenges in obtaining accurate emotional labels, traditional methods have shown poor performance in cross-subject emotion recognition. In this study, we propose a cross-subject EEG emotion recognition method based on a semi-supervised fine-tuning self-supervised graph attention network (SFT-SGAT). First, we model multi-channel EEG signals by constructing a graph structure that dynamically captures the spatiotemporal topological features of EEG signals. Second, we employ a self-supervised graph attention neural network to facilitate model training, mitigating the impact of signal noise on the model. Finally, a semi-supervised approach is used to fine-tune the model, enhancing its generalization ability in cross-subject classification. By combining supervised and unsupervised learning techniques, the SFT-SGAT maximizes the utility of limited labeled data in EEG emotion recognition tasks, thereby enhancing the model’s performance. Experiments based on leave-one-subject-out cross-validation demonstrate that SFT-SGAT achieves state-of-the-art cross-subject emotion recognition performance on the SEED and SEED-IV datasets, with accuracies of 92.04% and 82.76%, respectively. Furthermore, experiments conducted on a self-collected dataset comprising ten healthy subjects and eight patients with disorders of consciousness (DOCs) revealed that the SFT-SGAT attains high classification performance in healthy subjects (maximum accuracy of 95.84%) and was successfully applied to DOC patients, with four patients achieving emotion recognition accuracies exceeding 60%. The experiments demonstrate the effectiveness of the proposed SFT-SGAT model in cross-subject EEG emotion recognition and its potential for assessing levels of consciousness in patients with DOC.

Multimodal insights into granger causality connectivity: Integrating physiological signals and gated eye-tracking data for emotion recognition using convolutional neural network

This study introduces a groundbreaking method to enhance the accuracy and reliability of emotion recognition systems by combining electrocardiogram (ECG) with electroencephalogram (EEG) data, using an eye-tracking gated strategy. Initially, we propose a technique to filter out irrelevant portions of emotional data by employing pupil diameter metrics from eye-tracking data. Subsequently, we introduce an innovative approach for estimating effective connectivity to capture the dynamic interaction between the brain and the heart during emotional states of happiness and sadness. Granger causality (GC) is estimated and utilized to optimize input for a highly effective pre-trained convolutional neural network (CNN), specifically ResNet-18. To assess this methodology, we employed EEG and ECG data from the publicly available MAHNOB-HCI database, using a 5-fold cross-validation approach. Our method achieved an impressive average accuracy and area under the curve (AUC) of 91.00 % and 0.97, respectively, for GC-EEG-ECG images processed with ResNet-18. Comparative analysis with state-of-the-art studies clearly shows that augmenting ECG with EEG and refining data with an eye-tracking strategy significantly enhances emotion recognition performance across various emotions.

Deficits of Facial Emotion Recognition in Elderly Individuals with Mild Cognitive Impairment.

The study of facial emotion recognition is under-explored in subjects with mild cognitive impairment (MCI). We investigated whether deficits in facial emotion recognition are present in patients with MCI. We also analyzed the relationship between facial emotion recognition and different domains of cognitive function. This study included 300 participants aged 60 years or older with cognitive decline. We evaluated 181 MCI and 119 non-MCI subjects using the Seoul Neuropsychological Screening Battery-Core (SNSB-C) and facial emotion recognition task using six facial expressions (anger, disgust, fear, happiness, sadness and surprise). A Generalized Linear Model (GLM) was used to assess the association between cognitive performance and accuracy of facial emotion recognition and to compare facial emotion recognition in the MCI group based on the impairment of five different domains of cognitive function. The model was adjusted for age, sex, years of education, and depressive symptoms. Patients with MCI had a lower score for accurately recognizing total facial emotion (0.48 vs. 0.53; ρ = 0.0003) and surprise (0.73 vs. 0.81; ρ = 0.0215) when compared to cognitively healthy subjects. We also discovered that frontal/executive function domain (Digit Symbol Coding [DSC, 0.38 vs. 0.49; p < 0.0001], Controlled Oral Word Association Test [COWAT, 0.42 vs. 0.49; p = 0.0001], Korean-Trail Making Test [K-TMT, 0.37 vs. 0.48; p = 0.0073], Korean-Color Word Stroop Test [K-CWST, 0.43 vs. 0.49; p = 0.0219]) and language domain (Korean-Boston Naming Test [S-K-BNT, 0.46 vs. 0.47; p = 0.003]) were statistically associated with the deficits of facial emotion recognition in patients with MCI. We observed a significant association between deficits in facial emotion recognition and cognitive impairment in elderly individuals.

EEG emotion recognition using EEG-SWTNS neural network through EEG spectral image

The rapid development in deep learning models provide considerable advancements in emotion recognition by electroencephalogram (EEG). However, existing approaches primarily focus on temporal and frequency domain features of EEG signals, neglecting spatial domain features in model information integration. Therefore, this study proposed a novel EEG input format, called EEG spectral image (ESI), which integrates spatial domain features using Azimuthal Equidistant Projection (AEP) and frequency domain features through differential entropy (DE). To better utilize this fine-grained data format, we propose the EEG Swin Transformer (EEG-SWTNS), which combines the window attention mechanism with shifted window partitioning. Window attention mechanism can concentrate on affective information within various regions and use shifted window partitioning to disrupt aggregated emotional representations for more granular features. Different from traditional graph neural networks, this method leverages the spatial locality of region information, leading to enhanced performance in EEG-based emotion recognition. Experiments conducted on SEED and SEED IV datasets demonstrate superior performance compared to baseline methods and the state-of-the-art models. Relative improvements of 0.6% and 0.08% are observed in subject-dependent experiments, while accuracies of 80.07% and 66.72% are achieved in subject-independent experiments without utilizing transfer learning techniques.

Evolutionary Ensemble Learning for EEG-Based Cross-Subject Emotion Recognition.

Electroencephalogram (EEG) has been widely utilized in emotion recognition due to its high temporal resolution and reliability. However, the individual differences and non-stationary characteristics of EEG, along with the complexity and variability of emotions, pose challenges in generalizing emotion recognition models across subjects. In this paper, an end-to-end framework is proposed to improve the performance of cross-subject emotion recognition. A novel evolutionary programming (EP)-based optimization strategy with neural network (NN) as the base classifier termed NN ensemble with EP (EPNNE) is designed for cross-subject emotion recognition. The effectiveness of the proposed method is evaluated on the publicly available DEAP, FACED, SEED, and SEED-IV datasets. Numerical results demonstrate that the proposed method is superior to state-of-the-art cross-subject emotion recognition methods. The proposed end-to-end framework for cross-subject emotion recognition aids biomedical researchers in effectively assessing individual emotional states, thereby enabling efficient treatment and interventions.

Optimizing Emotional Insight through Unimodal and Multimodal Long Short-term Memory Models

The field of multimodal emotion recognition is increasingly gaining popularity as a research area. It involves analyzing human emotions across multiple modalities, such as acoustic, visual, and language. Emotion recognition is more effective as a multimodal learning task than relying on a single modality. In this paper, we present an unimodal and multimodal long short-term memory model with a class weight parameter technique for emotion recognition on the CMU-Multimodal Opinion Sentiment and Emotion Intensity dataset. In addition, a critical challenge lies in selecting the most effective fusion method for integrating multiple modalities. To address this, we applied four different fusion techniques: Early fusion, late fusion, deep fusion, and tensor fusion. These fusion methods improved the performance of multimodal emotion recognition compared to unimodal approaches. With the highly imbalanced number of samples per emotion class in the MOSEI dataset, adding a class weight parameter technique leads our model to outperform the state of the art on all three modalities — acoustic, visual, and language — as well as on all the fusion models. The challenges of class imbalance, which can lead to biased model performance, and using an effective fusion method for integrating multiple modalities often result in decreased accuracy in recognizing less frequent emotion classes. Our proposed model shows 2–3% performance improvement in the unimodal and 2% in the multimodal over the state-of-the-art achieved results.

Facial emotion recognition and judgment of affective scenes in Parkinson's disease

Emotional dysfunctions in Parkinson's disease (PD) remain a controversial issue. While previous investigations showed compromised recognition of expressive faces in PD, no studies evaluated potential deficits in recognizing the emotional valence of affective scenes. This study aimed to investigate both facial emotion recognition performance and the ability to judge affective scenes in PD patients.Forty PD patients (mean age ± SD: 64.50 ± 8.19 years; 27 men) and forty healthy subjects (64.95 ± 8.25 years; 27 men) were included. Exclusion criteria were previous psychiatric disorders, previous Deep Brain Stimulation, and cognitive impairment. Participants were evaluated through the Ekman 60-Faces test and the International Affective Picture System. The accuracy in recognizing the emotional valence of facial expressions and affective scenes was compared between groups using linear mixed models. Pearson's correlation was performed to test the association between accuracy measures.The groups did not differ in sex, age, education, and Mini-Mental State Examination scores. Patients showed a lower recognition accuracy of facial expressions (68.54 % ± 15.83 %) than healthy participants (78.67 % ± 12.04 %; p < 0.001). Specifically, the PD group was characterized by lower recognition of faces expressing fear, sadness, and anger than the control group (all p < 0.020). No difference was detected for faces expressing disgust, surprise, and happiness (all p ≥ 0.25). Furthermore, patients showed lower accuracy in recognizing the emotional valence of affective scenes (66.75 % ± 14.59 %) than healthy subjects (74.83 % ± 12.65 %; p = 0.010). Pearson's correlations indicated that higher accuracy in recognizing the emotional facial expressions was associated with higher accuracy in classifying the valence of affective scenes in patients (r = 0.57, p < 0.001) and control participants (r = 0.57, p < 0.001).Our study suggested maladaptive affective processing in PD, leading patients to misinterpret both facial expressions and the emotional valence of complex evocative scenes.

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

Facial emotion recognition function and white matter microstructural alterations in drug-naive, comorbidity-free autism.

Individuals with autism spectrum disorder have deficits in facial emotion recognition and white matter microstructural alterations. Nonetheless, most previous studies were confounded by different variables, such as psychiatric comorbidities and psychotropic medications used by ASD participants. Also, it remains unclear how exactly FER deficits are related to white matter microstructural alterations in ASD. Accordingly, we aimed to investigate the FER functions, white matter microstructure, and their relationship in drug-naive and comorbidity-free ASD individuals. 59 ASD individuals and 59 typically developed individuals were included, where 46 ASD and 50 TD individuals completed FER tasks. Covariance analysis showed scores were lower in both basic and complex FER tasks in the ASD group. Tract-Based Spatial Statistics showed FA values in widespread white matter fibers were lower in the ASD group than in the TD group, including forceps major and forceps minor of the corpus callosum, anterior thalamic radiation, corticospinal tract, cingulum, inferior frontal-occipital fasciculus, inferior longitudinal fasciculus, superior longitudinal fasciculus. Moreover, in the TD group but not the ASD group, the performance in the complex FER task was negatively correlated with the FA value in some white matter fibers, including forceps major of the corpus callosum, ATR, CT, cingulum, IFOF, ILF, SLF. Our study suggests children with ASD may experience deficits in facial emotion recognition and exhibit alterations in white matter microstructure. More importantly, our study indicates that white matter microstructural alterations may be involved in FER deficits in children with ASD.

Joint enhancement and classification constraints for noisy speech emotion recognition

In the natural environment, the received speech signal is often interfered by noise, which reduces the performance of speech emotion recognition (SER) system. To this end, a noisy SER method based on joint constraints, including enhancement constraint and arousal-valence classification constraint (EC-AVCC), is proposed. This method introduces speech enhancement features and constraints into emotion recognition for the first time, which helps to improve the performance of noisy SER. It does not require denoising pre-processing for noisy speech, thus saving testing time. Specifically, it uses multi-domain statistical features (MDSF) composed of emotional and enhanced features as the input of the SER model based on convolution neural network (CNN) and long short-term memory-attention (ALSTM). In addition, the model is constrained jointly by speech enhancement and arousal-valence classification to obtain the robust and discriminative deep emotion features. Furthermore, in the auxiliary speech enhancement task, a joint loss function that simultaneously constrains the error of the ideal ratio mask and the error of the corresponding MDSF is introduced to obtain more robust features. The experimental results on the CASIA Chinese emotional dataset and the EMO-DB German emotional database show that compared with the CNN baseline, the proposed method improves the accuracy of SER in white noise and babble noise by 4.7%-9.9%.

Detecting emotions through EEG signals based on modified convolutional fuzzy neural network

Emotion is a human sense that can influence an individual’s life quality in both positive and negative ways. The ability to distinguish different types of emotion can lead researchers to estimate the current situation of patients or the probability of future disease. Recognizing emotions from images have problems concealing their feeling by modifying their facial expressions. This led researchers to consider Electroencephalography (EEG) signals for more accurate emotion detection. However, the complexity of EEG recordings and data analysis using conventional machine learning algorithms caused inconsistent emotion recognition. Therefore, utilizing hybrid deep learning models and other techniques has become common due to their ability to analyze complicated data and achieve higher performance by integrating diverse features of the models. However, researchers prioritize models with fewer parameters to achieve the highest average accuracy. This study improves the Convolutional Fuzzy Neural Network (CFNN) for emotion recognition using EEG signals to achieve a reliable detection system. Initially, the pre-processing and feature extraction phases are implemented to obtain noiseless and informative data. Then, the CFNN with modified architecture is trained to classify emotions. Several parametric and comparative experiments are performed. The proposed model achieved reliable performance for emotion recognition with an average accuracy of 98.21% and 98.08% for valence (pleasantness) and arousal (intensity), respectively, and outperformed state-of-the-art methods.

Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism.

Modeling correlations between multimodal physiological signals [e.g., canonical correlation analysis (CCA)] for emotion recognition has attracted much attention. However, existing studies rarely consider the neural nature of emotional responses within physiological signals. Furthermore, during fusion space construction, the CCA method maximizes only the correlations between different modalities and neglects the discriminative information of different emotional states. Most importantly, temporal mismatches between different neural activities are often ignored; therefore, the theoretical assumptions that multimodal data should be aligned in time and space before fusion are not fulfilled. To address these issues, we propose a discriminative correlation fusion method coupled with a temporal alignment mechanism for multimodal physiological signals. We first use neural signal analysis techniques to construct neural representations of the central nervous system (CNS) and autonomic nervous system (ANS). respectively. Then, emotion class labels are introduced in CCA to obtain more discriminative fusion representations from multimodal neural responses, and the temporal alignment between the CNS and ANS is jointly optimized with a fusion procedure that applies the Bayesian algorithm. The experimental results demonstrate that our method significantly improves the emotion recognition performance. Additionally, we show that this fusion method can model the underlying mechanisms in human nervous systems during emotional responses, and our results are consistent with prior findings. This study may guide a new approach for exploring human cognitive function based on physiological signals at different time scales and promote the development of computational intelligence and harmonious human-computer interactions.

EEG-eye movement based subject dependence, cross-subject, and cross-session emotion recognition with multidimensional homogeneous encoding space alignment

The joint learning of multimodal is helpful to extract the general information cross-modality in improving the performance of multimodal emotion recognition. However, focusing on a single common pattern can cause multimodal data to deviate from its original distribution and fail to fully capture the potential representation of the data. Therefore, we propose a multi-dimensional homogenous encoding spatial alignment (MHESA) method, which consists of two parts: multi-modal joint learning and modal knowledge transfer. To obtain a common projection space of EEG-EM features, we use a multimodal joint space encoder to learn the homogeneous joint space of EEG-Eye Movement (EM). To obtain a homogeneous encoding space based on modal knowledge, the knowledge transfer module learns the spatial distribution of EM features while retaining the original EEG features. The output of each module is used to construct a multidimensional homogeneous encoding space. The weight and multi-task loss function of the multi-dimensional homogeneous encoding space are dynamically adjusted by the Multi-task Joint Optimization Strategy (MJOS). By analyzing the effect of multi-task optimization, we found that compared with the subject dependence scene, the cross-subject scene has an advantage in the construction of joint encoding space, and the modal knowledge transfer feature has a higher contribution degree in cross-session. The experimental results show the MHESA method can make the model achieve more stable performance in three emotion recognition scenes.

Self-Rated Confidence in Vocal Emotion Recognition Ability: The Role of Gender.

We studied the role of gender in metacognition of voice emotion recognition ability (ERA), reflected by self-rated confidence (SRC). To this end, we guided our study in two approaches: first, by examining the role of gender in voice ERA and SRC independently and second, by looking for gender effects on the ERA association with SRC. We asked 100 participants (50 men, 50 women) to interpret a set of vocal expressions portrayed by 30 actors (16 men, 14 women) as defined by their emotional meaning. Targets were 180 repetitive lexical sentences articulated in congruent emotional voices (anger, sadness, surprise, happiness, fear) and neutral expressions. Trial by trial, the participants were assigned retrospective SRC based on their emotional recognition performance. A binomial generalized linear mixed model (GLMM) estimating ERA accuracy revealed a significant gender effect, with women encoders (speakers) yielding higher accuracy levels than men. There was no significant effect of the decoder's (listener's) gender. A second GLMM estimating SRC found a significant effect of encoder and decoder genders, with women outperforming men. Gamma correlations were significantly greater than zero for women and men decoders. In spite of varying interpretations of gender in each independent rating (ERA and SRC), our results suggest that both men and women decoders were accurate in their metacognition regarding voice emotion recognition. Further research is needed to study how individuals of both genders use metacognitive knowledge in their emotional recognition and whether and how such knowledge contributes to effective social communication.