Emotion Recognition Performance Research Articles

Improvement of Multimodal Emotion Recognition Based on Temporal-Aware Bi-Direction Multi-Scale Network and Multi-Head Attention Mechanisms

Emotion recognition is a crucial research area in natural language processing (NLP), aiming to identify emotional states such as happiness, anger, and sadness from various sources like speech, text, and facial expressions. In this paper, we propose an improved MMER (multimodal emotion recognition) method using TIM-Net (Temporal-Aware Bi-Direction Multi-Scale Network) and attention mechanisms. Firstly, we introduce the methods for extracting and fusing the multimodal features. Then, we present the TIM-Net and attention mechanisms, which are utilized to enhance the MMER algorithm. We evaluate our approach on the IEMOCAP and MELD datasets, and compared to existing methods, our approach demonstrates superior performance. The weighted accuracy recall (WAR) on the IEMOCAP dataset is 83.9%, and the weighted accuracy recall rate on the MELD dataset is 62.7%. Finally, the impact of the TIM-Net model and the attention mechanism on the emotion recognition performance is further investigated through ablation experiments.

A multimodal shared network with a cross-modal distribution constraint for continuous emotion recognition

Continuous emotion recognition has been a compelling topic in affective computing because it can interpret human emotions subtly and continuously. Existing studies have achieved advanced emotion recognition performance using multimodal knowledge. However, these studies generally ignore the circumstances where some particular modalities are missing in the inference phase and thus become sensitive to the absence of modalities. To resolve this issue, we propose a novel multimodal shared network with a cross-modal distribution constraint, i.e. the DS-Net, which aims to improve the robustness of the model to missing modalities. The training process of the proposed network generally includes two components: multimodal shared space modeling and a cross-modal distribution matching constraint. The former utilizes the local and temporal information of multimodal signals for multimodal shared space modeling, while the latter further enhances the multimodal shared space via a loose constraint method. Coupled with the latter, the former can effectively exploit the complementarity between videos and peripheral physiological signals (PPSs), thus enhancing the discriminative capability of the shared space. Based on the shared space, the DS-Net works during the inference phase with only one modality input and can leverage multimodal knowledge to improve emotion recognition accuracy. Comprehensive experiments were conducted on two public datasets. Results demonstrate that the proposed method is competitive or superior to the current state-of-the-art methods. Further experiments indicate that the proposed method can be extended to handle other modalities and to deal with partially missing modalities, demonstrating its potential in real-world applications.

Enhancing cross-subject EEG emotion recognition through multi-source manifold metric transfer learning

Transfer learning (TL) has demonstrated its efficacy in addressing the cross-subject domain adaptation challenges in affective brain-computer interfaces (aBCI). However, previous TL methods usually use a stationary distance, such as Euclidean distance, to quantify the distribution dissimilarity between two domains, overlooking the inherent links among similar samples, potentially leading to suboptimal feature mapping. In this study, we introduced a novel algorithm called multi-source manifold metric transfer learning (MSMMTL) to enhance the efficacy of conventional TL. Specifically, we first selected the source domain based on Mahalanobis distance to enhance the quality of the source domains and then used manifold feature mapping approach to map the source and target domains on the Grassmann manifold to mitigate data drift between domains. In this newly established shared space, we optimized the Mahalanobis metric by maximizing the inter-class distances while minimizing the intra-class distances in the target domain. Recognizing that significant distribution discrepancies might persist across different domains even on the manifold, to ensure similar distributions between the source and target domains, we further imposed constraints on both domains under the Mahalanobis metric. This approach aims to reduce distributional disparities and enhance the electroencephalogram (EEG) emotion recognition performance. In cross-subject experiments, the MSMMTL model exhibits average classification accuracies of 88.83 % and 65.04 % for SEED and DEAP, respectively, underscoring the superiority of our proposed MSMMTL over other state-of-the-art methods. MSMMTL can effectively solve the problem of individual differences in EEG-based affective computing.

Cross-modal credibility modelling for EEG-based multimodal emotion recognition

Objective. The study of emotion recognition through electroencephalography (EEG) has garnered significant attention recently. Integrating EEG with other peripheral physiological signals may greatly enhance performance in emotion recognition. Nonetheless, existing approaches still suffer from two predominant challenges: modality heterogeneity, stemming from the diverse mechanisms across modalities, and fusion credibility, which arises when one or multiple modalities fail to provide highly credible signals. Approach. In this paper, we introduce a novel multimodal physiological signal fusion model that incorporates both intra-inter modality reconstruction and sequential pattern consistency, thereby ensuring a computable and credible EEG-based multimodal emotion recognition. For the modality heterogeneity issue, we first implement a local self-attention transformer to obtain intra-modal features for each respective modality. Subsequently, we devise a pairwise cross-attention transformer to reveal the inter-modal correlations among different modalities, thereby rendering different modalities compatible and diminishing the heterogeneity concern. For the fusion credibility issue, we introduce the concept of sequential pattern consistency to measure whether different modalities evolve in a consistent way. Specifically, we propose to measure the varying trends of different modalities, and compute the inter-modality consistency scores to ascertain fusion credibility. Main results. We conduct extensive experiments on two benchmarked datasets (DEAP and MAHNOB-HCI) with the subject-dependent paradigm. For the DEAP dataset, our method improves the accuracy by 4.58%, and the F1 score by 0.63%, compared to the state-of-the-art baseline. Similarly, for the MAHNOB-HCI dataset, our method improves the accuracy by 3.97%, and the F1 score by 4.21%. In addition, we gain much insight into the proposed framework through significance test, ablation experiments, confusion matrices and hyperparameter analysis. Consequently, we demonstrate the effectiveness of the proposed credibility modelling through statistical analysis and carefully designed experiments. Significance. All experimental results demonstrate the effectiveness of our proposed architecture and indicate that credibility modelling is essential for multimodal emotion recognition.

Facial emotion recognition using enhanced multi-verse optimizer method

In recent years, facial emotion recognition has gained significant improvement and attention. This technology utilizes advanced algorithms to analyze facial expressions, enabling computers to detect and interpret human emotions accurately. Its applications span over a wide range of fields, from improving customer service through sentiment analysis, to enhancing mental health support by monitoring emotional states. However, there are several challenges in facial emotion recognition, including variability in individual expressions, cultural differences in emotion display, and privacy concerns related to data collection and usage. Lighting conditions, occlusions, and the need for diverse datasets also impacts accuracy. To solve these issues, an enhanced multi-verse optimizer (EMVO) technique is proposed to improve the efficiency of recognizing emotions. Moreover, EMVO is used to improve the convergence speed, exploration-exploitation balance, solution quality, and the applicability in different types of optimization problems. Two datasets were used to collect the data, namely YouTube and surrey audio-visual expressed emotion (SAVEE) datasets. Then, the classification is done using the convolutional neural networks (CNN) to improve the performance of emotion recognition. When compared to the existing methods shuffled frog leaping algorithm-incremental wrapper-based subset selection (SFLA-IWSS), hierarchical deep neural network (H-DNN) and unique preference learning (UPL), the proposed method achieved better accuracies, measured at 98.65% and 98.76% on the YouTube and SAVEE datasets, respectively.

Fine-Grained Interpretability for EEG Emotion Recognition: Concat-Aided Grad-CAM and Systematic Brain Functional Network

EEG emotion recognition plays a significant role in various mental health services. Deep learning-based methods perform excellently, but still suffer from interpretability. Although methods such as Gradient-weighted Class Activation Mapping(Grad-CAM) can cope with the above problem, their coarse granularity cannot accurately reveal the mechanism to promote emotional intelligence. In this paper, fine-grained interpretability is proposed, called Concat-aided Grad-CAM. Specifically, the multi-level feature mapping before the fully connected layer is concatenated to obtain the gradients of the target concept so that the discriminant information can be directly located in the high-precision area. Unlike coarse-grained interpretability methods applied in EEG emotion recognition, it can accurately highlight the EEG channels related to emotion rather than an obscure area. In addition, a systematic brain functional network is proposed to reveal the relationship between those channels and to further improve emotion recognition performance. The channels with greater contributions are connected, and those connections are learned by dynamic graph convolutional networks, while the others are independent to eliminate interference. Experiments on two EEG emotion recognition datasets manifest that Concat-aided Grad-CAM can be interpreted by the fine-grained. In addition, it has been shown that the learned brain functional network can improve the performance of the baselines. Significantly, the experiment results achieve state-of-the-art performance in subject-dependent experiments.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects.

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Beyond Mimicking Under-Represented Emotions: Deep Data Augmentation with Emotional Subspace Constraints for EEG-Based Emotion Recognition

In recent years, using Electroencephalography (EEG) to recognize emotions has garnered considerable attention. Despite advancements, limited EEG data restricts its potential. Thus, Generative Adversarial Networks (GANs) are proposed to mimic the observed distributions and generate EEG data. However, for imbalanced datasets, GANs struggle to produce reliable augmentations for under-represented minority emotions by merely mimicking them. Thus, we introduce Emotional Subspace Constrained Generative Adversarial Networks (ESC-GAN) as an alternative to existing frameworks. We first propose the EEG editing paradigm, editing reference EEG signals from well-represented to under-represented emotional subspaces. Then, we introduce diversity-aware and boundary-aware losses to constrain the augmented subspace. Here, the diversity-aware loss encourages a diverse emotional subspace by enlarging the sample difference, while boundary-aware loss constrains the augmented subspace near the decision boundary where recognition models can be vulnerable. Experiments show ESC-GAN boosts emotion recognition performance on benchmark datasets, DEAP, AMIGOS, and SEED, while protecting against potential adversarial attacks. Finally, the proposed method opens new avenues for editing EEG signals under emotional subspace constraints, facilitating unbiased and secure EEG data augmentation.

Optimal Electrodermal Activity Segment for Enhanced Emotion Recognition Using Spectrogram-Based Feature Extraction and Machine Learning.

In clinical and scientific research on emotion recognition using physiological signals, selecting the appropriate segment is of utmost importance for enhanced results. In our study, we optimized the electrodermal activity (EDA) segment for an emotion recognition system. Initially, we obtained EDA signals from two publicly available datasets: the Continuously annotated signals of emotion (CASE) and Wearable stress and affect detection (WESAD) for 4-class dimensional and three-class categorical emotional classification, respectively. These signals were pre-processed, and decomposed into phasic signals using the 'convex optimization to EDA' method. Further, the phasic signals were segmented into two equal parts, each subsequently segmented into five nonoverlapping windows. Spectrograms were then generated using short-time Fourier transform and Mel-frequency cepstrum for each window, from which we extracted 85 features. We built four machine learning models for the first part, second part, and whole phasic signals to investigate their performance in emotion recognition. In the CASE dataset, we achieved the highest multi-class accuracy of 62.54% using the whole phasic and 61.75% with the second part phasic signals. Conversely, the WESAD dataset demonstrated superior performance in three-class emotions classification, attaining an accuracy of 96.44% for both whole phasic and second part phasic segments. As a result, the second part of EDA is strongly recommended for optimal outcomes.

GPT-4V with emotion: A zero-shot benchmark for Generalized Emotion Recognition

Recently, GPT-4 with Vision (GPT-4V) has demonstrated remarkable visual capabilities across various tasks, but its performance in emotion recognition has not been fully evaluated. To bridge this gap, we present the quantitative evaluation results of GPT-4V on 21 benchmark datasets covering 6 tasks: visual sentiment analysis, tweet sentiment analysis, micro-expression recognition, facial emotion recognition, dynamic facial emotion recognition, and multimodal emotion recognition. This paper collectively refers to these tasks as “Generalized Emotion Recognition (GER)”. Through experimental analysis, we observe that GPT-4V exhibits strong visual understanding capabilities in GER tasks. Meanwhile, GPT-4V shows the ability to integrate multimodal clues and exploit temporal information, which is also critical for emotion recognition. However, it is worth noting that GPT-4V is primarily designed for general domains and cannot recognize micro-expressions that require specialized knowledge. To the best of our knowledge, this paper provides the first quantitative assessment of GPT-4V for GER tasks. We have open-sourced the code and encourage subsequent researchers to broaden the evaluation scope by including more tasks and datasets. Our code and evaluation results are available at: https://github.com/zeroQiaoba/gpt4v-emotion.

Uncinate fasciculus microstructural organisation and emotion recognition in schizophrenia: controlling for hit rate bias.

Schizophrenia (SCZ) is a complex neurodevelopmental disorder characterised by functional and structural brain dysconnectivity and disturbances in perception, cognition, emotion, and social functioning. In the present study, we investigated whether the microstructural organisation of the uncinate fasciculus (UF) was associated with emotion recognition (ER) performance. Additionally, we investigated the usefulness of an unbiased hit rate (UHR) score to control for response biases (i.e., participant guessing) during an emotion recognition task (ERT). Fifty-eight individuals diagnosed with SCZ were included. The CANTAB ERT was used to measure social cognition. Specific ROI manual tract segmentation was completed using ExploreDTI and followed the protocol previously outlined by Coad et al. (2020). We found that the microstructural organisation of the UF was significantly correlated with physical neglect and ER outcomes. Furthermore, we found that the UHR score was more sensitive to ERT subscale emotion items than the standard HR score. Finally, given the association between childhood trauma (in particular childhood neglect) and social cognition in SCZ, a mediation analysis found evidence that microstructural alterations of the UF mediated an association between childhood trauma and social cognitive performance. The mediating role of microstructural alterations in the UF on the association between childhood trauma and social cognitive performance suggests that early life adversity impacts both brain development and social cognitive outcomes for people with SCZ. Limitations of the present study include the restricted ability of the tensor model to correctly assess multi-directionality at regions where fibre populations intersect.

Enhancing emotion recognition using multimodal fusion of physiological, environmental, personal data

Human emotion recognition, crucial for interpersonal relations and human-building interaction, identifies emotions from various behavioral signals to improve user interactions. To enhance the performance of emotion recognition, this study proposed a novel model that fuses physiological, environmental, and personal data. A unique dataset was created via experiments conducted in an environmental chamber, and an emotion recognition model was subsequently developed using a multimodal fusion approach. The model transforms physiological data into 2D images to capture time series and spatial features and uniquely incorporates metadata, including environmental and personal data. The model’s generalizability was validated using a leave-one-sample-out approach. The result showed 31.6% reduction of error with a predicted area when physiological, environmental, and personal data were fused in the emotion recognition model, suggesting that incorporating various contextual factors beyond physiological changes, such as the surrounding environment and inherent or acquired individual traits, can significantly enhance the model’s understanding of emotions. Furthermore, the model was to be robust to individual differences, offering consistent emotion recognition across different subjects. These findings suggest that the proposed model can serve as a potent tool for emotion recognition in built environmental applications.

Emotion recognition based on microstate analysis from temporal and spatial patterns of electroencephalogram.

Recently, the microstate analysis method has been widely used to investigate the temporal and spatial dynamics of electroencephalogram (EEG) signals. However, most studies have focused on EEG at resting state, and few use microstate analysis to study emotional EEG. This paper aims to investigate the temporal and spatial patterns of EEG in emotional states, and the specific neurophysiological significance of microstates during the emotion cognitive process, and further explore the feasibility and effectiveness of applying the microstate analysis to emotion recognition. We proposed a KLGEV-criterion-based microstate analysis method, which can automatically and adaptively identify the optimal number of microstates in emotional EEG. The extracted temporal and spatial microstate features then served as novel feature sets to improve the performance of EEG emotion recognition. We evaluated the proposed method on two publicly available emotional EEG datasets: the SJTU Emotion EEG Dataset (SEED) and the Database for Emotion Analysis using Physiological Signals (DEAP). For the SEED dataset, 10 microstates were identified using the proposed method. These temporal and spatial features were fed into AutoGluon, an open-source automatic machine learning model, yielding an average three-class accuracy of 70.38% (±8.03%) in subject-dependent emotion recognition. For the DEAP dataset, the method identified 9 microstates. The average accuracy in the arousal dimension was 74.33% (±5.17%) and 75.49% (±5.70%) in the valence dimension, which were competitive performance compared to some previous machine-learning-based studies. Based on these results, we further discussed the neurophysiological relationship between specific microstates and emotions, which broaden our knowledge of the interpretability of EEG microstates. In particular, we found that arousal ratings were positively correlated with the activity of microstate C (anterior regions of default mode network) and negatively correlated with the activity of microstate D (dorsal attention network), while valence ratings were positively correlated with the activity of microstate B (visual network) and negatively correlated with the activity of microstate D (dorsal attention network). In summary, the findings in this paper indicate that the proposed KLGEV-criterion-based method can be employed to research emotional EEG signals effectively, and the microstate features are promising feature sets for EEG-based emotion recognition.

EEG Emotion Recognition Based on Self-Distillation Convolutional Graph Attention Network

A convolution graph attention model based on self-distillation convolutional graph attention network (SDC-GAT) is proposed for multi-channel EEG emotion recognition. Firstly, two-dimensional feature matrix based on EEG time-domain features are constructed, and the matrix is fed into the graph attention neural network to learn the internal connections between electrical brain channels located in different brain regions. Meanwhile, the three-dimensional feature matrix is constructed according to the relative positions of the electrode channels, and the self-distillation network is employed to extract local high-level abstract features containing electrode spatial position information from the three-dimensional feature matrix. Finally, outputs of the two networks are integrated to determine the emotional states. Experiments were performed on the DEAP dataset. The experimental results show that the spatial domain information of the electrode channel and the internal connection relationship between different channels are beneficial for emotion recognition. In addition, the proposed model can effectively fuse these information to improve the performance of multi-channel EEG emotion recognition.

Hierarchical Context-Based Emotion Recognition With Scene Graphs.

For a better intention inference, we often try to figure out the emotional states of other people in social communications. Many studies on affective computing have been carried out to infer emotions through perceiving human states, i.e., facial expression and body posture. Such methods are skillful in a controlled environment. However, it often leads to misestimation due to the deficiency of effective inputs in unconstrained circumstances, that is, where context-aware emotion recognition appeared. We take inspiration from the advanced reasoning pattern of humans in perceived emotion recognition and propose the hierarchical context-based emotion recognition method with scene graphs. We propose to extract three contexts from the image, i.e., the entity context, the global context, and the scene context. The scene context contains abstract information about entity labels and their relationships. It is similar to the information processing of the human visual sensing mechanism. After that, these contexts are further fused to perform emotion recognition. We carried out a bunch of experiments on the widely used context-aware emotion datasets, i.e., CAER-S, EMOTIC, and BOdy Language Dataset (BoLD). We demonstrate that the hierarchical contexts can benefit emotion recognition by improving the accuracy of the SOTA score from 84.82% to 90.83% on CAER-S. The ablation experiments show that hierarchical contexts provide complementary information. Our method improves the F1 score of the SOTA result from 29.33% to 30.24% (C-F1) on EMOTIC. We also build the image-based emotion recognition task with BoLD-Img from BoLD and obtain a better emotion recognition score (ERS) score of 0.2153.

Effect of Phase Clustering Bias on Phase-Amplitude Coupling for Emotional EEG.

Emotions are thought to be related to distinct patterns of neural oscillations, but the interactions among multi-frequency neural oscillations during different emotional states lack full exploration. Phase-amplitude coupling is a promising tool for understanding the complexity of the neurophysiological system, thereby playing a crucial role in revealing the physiological mechanisms underlying emotional electroencephalogram (EEG). However, the non-sinusoidal characteristics of EEG lead to the non-uniform distribution of phase angles, which could potentially affect the analysis of phase-amplitude coupling. Removing phase clustering bias (PCB) can uniform the distribution of phase angles, but the effect of this approach is unknown on emotional EEG phase-amplitude coupling. This study aims to explore the effect of PCB on cross-frequency phase-amplitude coupling for emotional EEG. The technique of removing PCB was implemented on a publicly accessible emotional EEG dataset to calculate debiased phase-amplitude coupling. Statistical analysis and classification were conducted to compare the difference in emotional EEG phase-amplitude coupling prior to and post the removal of PCB. Emotional EEG phase-amplitude coupling values are overestimated due to PCB. Removing PCB enhances the difference in coupling strength between fear and happy emotions in the frontal lobe. Comparable emotion recognition performance was achieved with fewer features after removing PCB. These findings suggest that removing PCB enhances the difference in emotional EEG phase-amplitude coupling patterns and generates features that contain more emotional information. Removing PCB may be advantageous for analyzing emotional EEG phase-amplitude coupling and recognizing human emotions.

ERTNet: an interpretable transformer-based framework for EEG emotion recognition.

Emotion recognition using EEG signals enables clinicians to assess patients' emotional states with precision and immediacy. However, the complexity of EEG signal data poses challenges for traditional recognition methods. Deep learning techniques effectively capture the nuanced emotional cues within these signals by leveraging extensive data. Nonetheless, most deep learning techniques lack interpretability while maintaining accuracy. We developed an interpretable end-to-end EEG emotion recognition framework rooted in the hybrid CNN and transformer architecture. Specifically, temporal convolution isolates salient information from EEG signals while filtering out potential high-frequency noise. Spatial convolution discerns the topological connections between channels. Subsequently, the transformer module processes the feature maps to integrate high-level spatiotemporal features, enabling the identification of the prevailing emotional state. Experiments' results demonstrated that our model excels in diverse emotion classification, achieving an accuracy of 74.23% ± 2.59% on the dimensional model (DEAP) and 67.17% ± 1.70% on the discrete model (SEED-V). These results surpass the performances of both CNN and LSTM-based counterparts. Through interpretive analysis, we ascertained that the beta and gamma bands in the EEG signals exert the most significant impact on emotion recognition performance. Notably, our model can independently tailor a Gaussian-like convolution kernel, effectively filtering high-frequency noise from the input EEG data. Given its robust performance and interpretative capabilities, our proposed framework is a promising tool for EEG-driven emotion brain-computer interface.

Multi-Region and Multi-Band Electroencephalogram Emotion Recognition Based on Self-Attention and Capsule Network

Research on emotion recognition based on electroencephalogram (EEG) signals is important for human emotion detection and improvements in mental health. However, the importance of EEG signals from different brain regions and frequency bands for emotion recognition is different. For this problem, this paper proposes the Capsule–Transformer method for multi-region and multi-band EEG emotion recognition. First, the EEG features are extracted from different brain regions and frequency bands and combined into feature vectors which are input into the fully connected network for feature dimension alignment. Then, the feature vectors are inputted into the Transformer for calculating the self-attention of EEG features among different brain regions and frequency bands to obtain contextual information. Finally, utilizing capsule networks captures the intrinsic relationship between local and global features. It merges features from different brain regions and frequency bands, adaptively computing weights for each brain region and frequency band. Based on the DEAP dataset, experiments show that the Capsule–Transformer method achieves average classification accuracies of 96.75%, 96.88%, and 96.25% on the valence, arousal, and dominance dimensions, respectively. Furthermore, in emotion recognition experiments conducted on individual brain regions or frequency bands, it was observed that the frontal lobe exhibits the highest average classification accuracy, followed by the parietal, temporal, and occipital lobes. Additionally, emotion recognition performance is superior for high-frequency band EEG signals compared to low-frequency band signals.

Correction to: Convolutional neural network and ensemble machine learning model for optimizing performance of emotion recognition in wild

Correction to: Convolutional neural network and ensemble machine learning model for optimizing performance of emotion recognition in wild

Adaptive Spatial–Temporal Aware Graph Learning for EEG-Based Emotion Recognition

An intelligent emotion recognition system based on electroencephalography (EEG) signals shows considerable potential in various domains such as healthcare, entertainment, and education, thanks to its portability, high temporal resolution, and real-time capabilities. However, the existing research in this field faces limitations stemming from the nonstationary nature and individual variability of EEG signals. In this study, we present a novel EEG emotion recognition model, named GraphEmotionNet, designed to enhance the accuracy of EEG-based emotion recognition through the incorporation of a spatiotemporal attention mechanism and transfer learning. The proposed GraphEmotionNet model can effectively learn the intrinsic connections between EEG channels and construct an adaptive graph. This graph’s adaptive nature is crucial in optimizing spatial–temporal graph convolutions, which in turn enhances spatial–temporal feature characterization and contributes to the process of emotion classification. Moreover, an integration of domain adaptation aligns the extracted features across different domains, further alleviating the impact of individual EEG variability. We evaluate the model performance on two benchmark databases, employing two types of cross-validation protocols: within-subject cross-validation and cross-subject cross-validation. The experimental results affirm the model’s efficacy in extracting EEG features linked to emotional semantics and demonstrate its promising performance in emotion recognition.