EEG-based Emotion Recognition Research Articles

Advanced functional connectivity analysis with integrated EEG signal enhancement and GTW-EEG-GAN for emotion detection

Emotion recognition is crucial in human-computer interaction and psychological research, utilizing modalities such as facial expressions, voice intonations, and EEG signals. This research investigates AI-driven techniques by employing Graph Neural Networks (GNNs) on functional connectivity matrices derived from EEG data to advance emotion recognition. Our approach integrates sophisticated preprocessing methods, including Integrated EEG Signal Enhancement (IESE) and novel augmentation techniques such as Gaussian Time Warping Generative Adversarial Network (GTW-EEG-GAN). This integration aims at enhancing data quality and diversity. Signal decomposition using EMD-Wavelet Hybrid Decomposition further refines feature extraction, enabling robust analysis of EEG signals. Functional connectivity metrics capture critical neural interactions intended for precise emotion characterization. The GNN architecture effectively processes these features, achieving significant accuracy improvements. Evaluations on the DEAP dataset demonstrate promising results, attaining 94.5% accuracy before hyperparameter tuning and further improving to 98.6% after tuning, highlighting the system's efficacy. The methodology showcases the integration of advanced signal processing techniques with AI, offering a comprehensive framework for future studies. The findings advocate potential applications in areas such as personalized mental health monitoring, adaptive learning environments, and responsive gaming experiences, demonstrating the broad impact and versatility of AI-driven EEG-based emotion recognition.

EEGER – A Model for Recognition of Human Emotion Using Brain Signal

Emotions significantly impact human thinking, judgment, health, and communication. EEG-based emotion detection has advanced with the use of Brain-Computer Interface (BCI) technology, proving more effective than other physiological data. Despite progress in affective computing, emotion recognition remains a challenge. However, it's increasingly common in brain–machine interfaces, and research shows EEG brain waves are valuable in identifying emotional states. This research introduces a novel automated system for emotion recognition using deep learning techniques on EEG data collected from the GAMEEMO dataset, where participants played emotional assessment games. The system is designed to identify four emotions experienced during gameplay. The proposed model, called EEGER, was trained exclusively on EEG signals and demonstrated a 99.99% accuracy with minimal computational time. Key to its efficiency is the use of LSTM (Long Short-Term Memory) classifiers, which simplify the process by automatically extracting relevant features. The system was tested across different learning rates and epoch values, showing that 10 epochs with a learning rate of 0.0001 were sufficient to achieve the best accuracy. EEGER was also compared with other methods like K-Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), and Adaptive Boosting (AdaBoost), outperforming them in both accuracy and efficiency. These findings suggest that EEGER offers a promising new approach to EEG-based emotion recognition, optimizing performance with lower complexity and computation time.

Multi-source domain transfer network based on subdomain adaptation and minimum class confusion for EEG emotion recognition

Electroencephalogram (EEG) signals, which objectively reflect the state of the brain, are widely favored in emotion recognition research. However, the presence of cross-session and cross-subject variation in EEG signals has hindered the practical implementation of EEG-based emotion recognition technologies. In this article, we propose a multi-source domain transfer method based on subdomain adaptation and minimum class confusion (MS-SAMCC) in response to the addressed issue. First, we introduce the mix-up data augmentation technique to generate augmented samples. Next, we propose a minimum class confusion subdomain adaptation method (MCCSA) as a sub-module of the multi-source domain adaptation module. This approach enables global alignment between each source domain and the target domain, while also achieving alignment among individual subdomains within them. Additionally, we employ minimum class confusion (MCC) as a regularizer for this sub-module. We performed experiments on SEED, SEED IV, and FACED datasets. In the cross-subject experiments, our method achieved mean classification accuracies of 87.14% on SEED, 63.24% on SEED IV, and 42.07% on FACED. In the cross-session experiments, our approach obtained average classification accuracies of 94.20% on SEED and 71.66% on SEED IV. These results demonstrate that the MS-SAMCC approach proposed in this study can effectively address EEG-based emotion recognition tasks.

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition.

EEG-based emotion recognition has gradually become a new research direction, known as affective Brain-Computer Interface (aBCI), which has huge application potential in human-computer interaction and neuroscience. However, how to extract spatio-temporal fusion features from complex EEG signals and build learning method with high recognition accuracy and strong interpretability is still challenging. In this paper, we propose a hybrid attention spatio-temporal feature fusion network for EEG-based emotion recognition. First, we designed a spatial attention feature extractor capable of merging shallow and deep features to extract spatial information and adaptively select crucial features under different emotional states. Then, the temporal feature extractor based on the multi-head attention mechanism is integrated to perform spatio-temporal feature fusion to achieve emotion recognition. Finally, we visualize the extracted spatial attention features using feature maps, further analyzing key channels corresponding to different emotions and subjects. Our method outperforms the current state-of-the-art methods on two public datasets, SEED and DEAP. The recognition accuracy are 99.12% ± 1.25% (SEED), 98.93% ± 1.45% (DEAP-arousal), and 98.57% ± 2.60% (DEAP-valence). We also conduct ablation experiments, using statistical methods to analyze the impact of each module on the final result. The spatial attention features reveal that emotion-related neural patterns indeed exist, which is consistent with conclusions in the field of neurology. The experimental results show that our method can effectively extract and fuse spatial and temporal information. It has excellent recognition performance, and also possesses strong robustness, performing stably across different datasets and experimental environments for emotion recognition.

MAS-DGAT-Net: A dynamic graph attention network with multibranch feature extraction and staged fusion for EEG emotion recognition

In recent years, with the rise of deep learning technologies, EEG-based emotion recognition has garnered significant attention. However, most existing methods tend to focus on the spatiotemporal information of EEG signals while overlooking the potential topological information of brain regions. To address this issue, this paper proposes a dynamic graph attention network with multi-branch feature extraction and staged fusion (MAS-DGAT-Net), which integrates graph convolutional neural networks (GCN) for EEG emotion recognition. Specifically, the differential entropy (DE) features of EEG signals are first reconstructed into a correlation matrix using the Spearman correlation coefficient. Then, the brain-region connectivity-feature extraction (BCFE) module is employed to capture the brain connectivity features associated with emotional activation states. Meanwhile, this paper introduces a dual-branch cross-fusion feature extraction (CFFE) module, which consists of an attention-based cross-fusion feature extraction branch (A-CFFEB) and a cross-fusion feature extraction branch (CFFEB). A-CFFEB efficiently extracts key channel-frequency information from EEG features by using an attention mechanism and then fuses it with the output features from the BCFE. The fused features are subsequently input into the proposed dynamic graph attention module with a broad learning system (DGAT-BLS) to mine the brain connectivity feature information further. Finally, the deep features output by DGAT-BLS and CFFEB are combined for emotion classification. The proposed algorithm has been experimentally validated on SEED, SEED-IV, and DEAP datasets in subject-dependent and subject-independent settings, with the results confirming the model's effectiveness. The source code is publicly available at: https://github.com/cvmdsp/MAS-DGAT-Net

Multi-view brain functional connectivity and hierarchical fusion for EEG-based emotion recognition

Emotional states evolve gradually from their onset to full manifestation, reflected in changes in brain functional connectivity. Existing research often focuses only on the final emotional state, missing the rich, dynamic information of emotional evolution. This paper proposes a novel network framework leveraging global and local brain functional connectivity through a multi-view approach. We introduce a global and local connectivity modeling method using electroencephalogram signals to simulate brain connectivity changes across time and specific moments. A residual-based local detail feature extractor combined with a coordinate attention module captures long-range dependencies and maintains positional accuracy. A global feature extractor is also designed for the effective extraction of emotional information. Finally, a hierarchical multi-scale cross-attention feature fusion module integrates features across different levels to enhance emotion recognition performance and robustness. Extensive experiments on three publicly available datasets demonstrate the framework’s superiority and generalization capability.

CSA-SA-CRTNN: A Dual-Stream Adaptive Convolutional Cyclic Hybrid Network Combining Attention Mechanisms for EEG Emotion Recognition.

In recent years, EEG-based emotion recognition technology has made progress, but there are still problems of low model efficiency and loss of emotional information, and there is still room for improvement in recognition accuracy. To fully utilize EEG's emotional information and improve recognition accuracy while reducing computational costs, this paper proposes a Convolutional-Recurrent Hybrid Network with a dual-stream adaptive approach and an attention mechanism (CSA-SA-CRTNN). Firstly, the model utilizes a CSAM module to assign corresponding weights to EEG channels. Then, an adaptive dual-stream convolutional-recurrent network (SA-CRNN and MHSA-CRNN) is applied to extract local spatial-temporal features. After that, the extracted local features are concatenated and fed into a temporal convolutional network with a multi-head self-attention mechanism (MHSA-TCN) to capture global information. Finally, the extracted EEG information is used for emotion classification. We conducted binary and ternary classification experiments on the DEAP dataset, achieving 99.26% and 99.15% accuracy for arousal and valence in binary classification and 97.69% and 98.05% in ternary classification, and on the SEED dataset, we achieved an accuracy of 98.63%, surpassing relevant algorithms. Additionally, the model's efficiency is significantly higher than other models, achieving better accuracy with lower resource consumption.

Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods

A systematic review on machine-learning strategies for improving generalization in electroencephalography-based emotion classification was realized. In particular, cross-subject and cross-session generalization was focused. In this context, the non-stationarity of electroencephalographic (EEG) signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. In this review, 449 papers were retrieved from the Scopus, IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 79 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject or cross-session validation strategy, or making use of other biosignals as support were excluded. On the basis of the selected papers’ analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion of the different ML approaches involved. The studies reporting the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances.

Self-Supervised EEG Representation Learning for Robust Emotion Recognition

Emotion recognition based on electroencephalography (EEG) is becoming a growing concern of researchers due to its various applications and portable devices. Existing methods are mainly dedicated to EEG feature representation and have made impressive progress. However, the problem of scarce labels restricts their further promotion. In light of this, we propose a self-supervised framework with contrastive learning for robust EEG-based emotion recognition, which can effectively leverage both readily available unlabeled EEG signals and labeled ones to learn highly discriminative EEG features. Firstly, we construct a specific pretext task according to the sequential non-stationarity of emotional EEG signals for contrastive learning, which aims at extracting pseudo-label information from all EEG data. Meanwhile, we propose a novel negative segment selection algorithm to reduce the noise of unlabeled data during the contrastive learning process. Secondly, to mitigate the overfitting issue induced by a small number of labeled samples during learning, we originate a loss function with label smoothing regularization that can guide the model to learn generalizable features. Extensive experiments over three benchmark datasets demonstrate the effectiveness and superiority of our model on EEG-based emotion recognition task. Besides, the generalization and robustness of the model have also been proved through sufficient experiments.

Enhancing local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) signals is crucial in understanding human affective states. Current research has limitations in extracting local features. The representation capabilities of local features are limited, making it difficult to comprehensively capture emotional information. In this study, a novel approach is proposed to enhance local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition. By leveraging the functional connectivity of brain regions, EEG signals are divided into global embeddings that represent comprehensive brain connectivity patterns throughout the entire process and local embeddings that reflect dynamic interactions within specific brain functional networks at particular moments. Firstly, a convolutional feature extraction branch based on the residual network is designed to extract local features from the global embedding. To further improve the representation ability and accuracy of local features, a multidimensional collaborative attention (MCA) module is introduced. Secondly, the local features and patch embedded local embeddings are integrated into the feature coupling module (FCM), which utilizes hierarchical connections and enhanced cross-attention to couple region-level features, thereby enhancing local representation learning. Experimental results on three public datasets show that compared with other methods, this method improves accuracy by 4.92% on the DEAP, by 1.11% on the SEED, and by 7.76% on the SEED-IV, demonstrating its superior performance in emotion recognition tasks.

EEG emotion recognition using EEG-SWTNS neural network through EEG spectral image

The rapid development in deep learning models provide considerable advancements in emotion recognition by electroencephalogram (EEG). However, existing approaches primarily focus on temporal and frequency domain features of EEG signals, neglecting spatial domain features in model information integration. Therefore, this study proposed a novel EEG input format, called EEG spectral image (ESI), which integrates spatial domain features using Azimuthal Equidistant Projection (AEP) and frequency domain features through differential entropy (DE). To better utilize this fine-grained data format, we propose the EEG Swin Transformer (EEG-SWTNS), which combines the window attention mechanism with shifted window partitioning. Window attention mechanism can concentrate on affective information within various regions and use shifted window partitioning to disrupt aggregated emotional representations for more granular features. Different from traditional graph neural networks, this method leverages the spatial locality of region information, leading to enhanced performance in EEG-based emotion recognition. Experiments conducted on SEED and SEED IV datasets demonstrate superior performance compared to baseline methods and the state-of-the-art models. Relative improvements of 0.6% and 0.08% are observed in subject-dependent experiments, while accuracies of 80.07% and 66.72% are achieved in subject-independent experiments without utilizing transfer learning techniques.

A novel semi-supervised deep learning method for enhancing discriminability and diversity in EEG-based emotion recognition task

Numerous deep learning models have been introduced for EEG-based Emotion recognition tasks. Nevertheless, the majority of these models are fully supervised, demanding substantial amounts of labeled EEG signals. The labeling process of EEG signals is both time-intensive and costly, involving numerous trials and meticulous analysis by experts. Recently, some advanced semi-supervised algorithms that can achieve a competitive performance with fully-supervised methods by using only a small set of labeled data have been presented. However, these algorithms are primarily developed for the image data type, and naïve adaptation of them for EEG applications results in unsatisfactory performance. To address this issue, we present a robust semi-supervised EEG-based method that exploits the best techniques from advanced semi-supervised algorithms in the computer vision domain enriched with novel regularization terms for unlabeled signals. The proposed regularization terms improve both the discriminability and diversity of the model’s predictions and effectively leverage prior knowledge about the class distributions, thereby achieving a superior performance compared to the distribution alignment techniques in state-of-the-art methods. We evaluate our method on the DEAP dataset for cross-subject valence/arousal emotion recognition tasks, and on the SEED in a cross-session setting. The results indicate that the proposed method consistently surpasses the peer methods at different numbers of labeled data by a large margin.

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Emotion recognition based on Electroencephalogram (EEG) has been applied in various fields, including human–computer interaction and healthcare. However, for the popular Valence-Arousal-Dominance emotion model, researchers often classify the dimensions into high and low categories, which cannot reflect subtle changes in emotion. Furthermore, there are issues with the design of EEG features and the efficiency of transformer. To address these issues, we have designed TPRO-NET, a neural network that takes differential entropy and enhanced differential entropy features as input and outputs emotion categories through convolutional layers and improved transformer encoders. For our experiments, we categorized the emotions in the DEAP dataset into 8 classes and those in the DREAMER dataset into 5 classes. On the DEAP and the DREAMER datasets, TPRO-NET achieved average accuracy rates of 97.63%/97.47%/97.88% and 98.18%/98.37%/98.40%, respectively, on the Valence/Arousal/Dominance dimension for the subject-dependent experiments. Compared to other advanced methods, TPRO-NET demonstrates superior performance.

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

A comprehensive review of deep learning in EEG-based emotion recognition: classifications, trends, and practical implications

Emotion recognition utilizing EEG signals has emerged as a pivotal component of human–computer interaction. In recent years, with the relentless advancement of deep learning techniques, using deep learning for analyzing EEG signals has assumed a prominent role in emotion recognition. Applying deep learning in the context of EEG-based emotion recognition carries profound practical implications. Although many model approaches and some review articles have scrutinized this domain, they have yet to undergo a comprehensive and precise classification and summarization process. The existing classifications are somewhat coarse, with insufficient attention given to the potential applications within this domain. Therefore, this article systematically classifies recent developments in EEG-based emotion recognition, providing researchers with a lucid understanding of this field’s various trajectories and methodologies. Additionally, it elucidates why distinct directions necessitate distinct modeling approaches. In conclusion, this article synthesizes and dissects the practical significance of EEG signals in emotion recognition, emphasizing its promising avenues for future application.

Data-Segmentation Verification and a Target Generative Adversarial Network: EEG-Based Emotion Recognition

Data-Segmentation Verification and a Target Generative Adversarial Network: EEG-Based Emotion Recognition

TSANN-TG: Temporal-Spatial Attention Neural Networks with Task-Specific Graph for EEG Emotion Recognition.

Electroencephalography (EEG)-based emotion recognition is increasingly pivotal in the realm of affective brain-computer interfaces. In this paper, we propose TSANN-TG (temporal-spatial attention neural network with a task-specific graph), a novel neural network architecture tailored for enhancing feature extraction and effectively integrating temporal-spatial features. TSANN-TG comprises three primary components: a node-feature-encoding-and-adjacency-matrices-construction block, a graph-aggregation block, and a graph-feature-fusion-and-classification block. Leveraging the distinct temporal scales of features from EEG signals, TSANN-TG incorporates attention mechanisms for efficient feature extraction. By constructing task-specific adjacency matrices, the graph convolutional network with an attention mechanism captures the dynamic changes in dependency information between EEG channels. Additionally, TSANN-TG emphasizes feature integration at multiple levels, leading to improved performance in emotion-recognition tasks. Our proposed TSANN-TG is applied to both our FTEHD dataset and the publicly available DEAP dataset. Comparative experiments and ablation studies highlight the excellent recognition results achieved. Compared to the baseline algorithms, TSANN-TG demonstrates significant enhancements in accuracy and F1 score on the two benchmark datasets for four types of cognitive tasks. These results underscore the significant potential of the TSANN-TG method to advance EEG-based emotion recognition.

Semi-supervised bipartite graph construction with active EEG sample selection for emotion recognition.

Electroencephalogram (EEG) signals are derived from the central nervous system and inherently difficult to camouflage, leading to the recent popularity of EEG-based emotion recognition. However, due to the non-stationary nature of EEG, inter-subject variabilities become obstacles for recognition models to well adapt to different subjects. In this paper, we propose a novel approach called semi-supervised bipartite graph construction with active EEG sample selection (SBGASS) for cross-subject emotion recognition, which offers two significant advantages. Firstly, SBGASS adaptively learns a bipartite graph to characterize the underlying relationships between labeled and unlabeled EEG samples, effectively implementing the semantic connection for samples from different subjects. Secondly, we employ active sample selection technique in this paper to reduce the impact of negative samples (outliers or noise in the data) on bipartite graph construction. Drawing from the experimental results with the SEED-IV data set, we have gained the following three insights. (1) SBGASS actively rejects negative labeled samples, which helps mitigate the impact of negative samples when constructing the optimal bipartite graph and improves the model performance. (2) Through the learned optimal bipartite graph in SBGASS, the transferability of labeled EEG samples is quantitatively analyzed, which exhibits a decreasing tendency as the distance between each labeled sample and the corresponding class centroid increases. (3) Besides the improved recognition accuracy, the spatial-frequency patterns in emotion recognition are investigated by the acquired projection matrix.

Optimal channel dynamic selection for Constructing lightweight Data EEG-based emotion recognition

At present, most methods to improve the accuracy of emotion recognition based on electroencephalogram (EEG) are achieved by means of increasing the number of channels and feature types. This is to use the big data to train the classification model but it also increases the code complexity and consumes a large amount of computer time. We propose a method of Ant Colony Optimization with Convolutional Neural Networks and Long Short-Term Memory (ACO–CNN–LSTM) which can attain the dynamic optimal channels for lightweight data. First, transform the time-domain EEG signal to the frequency domain by Fast Fourier Transform (FFT), and the Differential Entropy (DE) of the three frequency bands (α, β and γ) are extracted as the feature data; Then, based on the DE feature dataset, ACO is employed to plan the path where the electrodes are located in the brain map. The classification accuracy of CNN-LSTM is used as the objective function for path determination, and the electrodes on the optimal path are used as the optimal channels; Next, the initial learning rate and batchsize parameters are exactly matched the data characteristics, which can obtain the best initial learning rate and batchsize; Finally, the SJTU Emotion EEG Dataset (SEED) dataset is used for emotion recognition based on the ACO–CNN–LSTM. From the experimental results, it can be seen that: the average accuracy of three-classification (positive, neutral, negative) can achieve 96.59 %, which is based on the lightweight data by means of ACO–CNN–LSTM proposed in the paper. Meanwhile, the computer time consumed is reduced. The computational efficiency is increased by 15.85 % compared with the traditional CNN-LSTM method. The accuracy can achieve more than 90 % when the data volume is reduced to 50 %. In summary, the proposed method of ACO–CNN–LSTM in the paper can get higher efficiency and accuracy.

Unveiling Emotional Stress: Analyzing EEG Signals With Deep Learning Insights

This study focuses on a comprehensive exploration of EEG-based emotion recognition, with a particular focus on leveraging recurrent neural network (RNN) architectures, including long short-term memory (LSTM) and gated recurrent unit (GRU). The investigation is rooted in a publicly available EEG Brainwave Dataset, meticulously curated to capture human responses to emotional stimuli. The dataset features signals obtained from frontal and temporal brain lobes, categorized into distinct emotional states encompassing positive, neutral, and negative emotions. The analysis spans four distinct scenarios, each representing different combinations of features and preprocessing strategies. Notably primary emphasis is on LSTM and GRU architectures due to their proven effectiveness in sequential data processing tasks. In addition to evaluating the performance of LSTM and GRU models a thorough examination of all the four cases to assess their efficacy in accurately predicting emotional states from EEG signals. The study also emphasis on the importance of meticulous feature selection and preprocessing in optimizing model accuracy and robustness, highlighting the intricate interplay between data representation, model architecture, and task-specific requirements. Ultimately, the research findings contribute to advanced understanding of EEG-based emotion recognition and pave the way for further research in this exciting interdisciplinary field. Key Words: Electroencephalography(EEG), Long short-term memory (LSTM), Gated recurrent unit (GRU), Feature extraction, Preprocessing techniques, Statistical metrics, Frequency domain analysis.