EEG-based Emotion Recognition Research Articles

CSA-SA-CRTNN: A Dual-Stream Adaptive Convolutional Cyclic Hybrid Network Combining Attention Mechanisms for EEG Emotion Recognition.

In recent years, EEG-based emotion recognition technology has made progress, but there are still problems of low model efficiency and loss of emotional information, and there is still room for improvement in recognition accuracy. To fully utilize EEG's emotional information and improve recognition accuracy while reducing computational costs, this paper proposes a Convolutional-Recurrent Hybrid Network with a dual-stream adaptive approach and an attention mechanism (CSA-SA-CRTNN). Firstly, the model utilizes a CSAM module to assign corresponding weights to EEG channels. Then, an adaptive dual-stream convolutional-recurrent network (SA-CRNN and MHSA-CRNN) is applied to extract local spatial-temporal features. After that, the extracted local features are concatenated and fed into a temporal convolutional network with a multi-head self-attention mechanism (MHSA-TCN) to capture global information. Finally, the extracted EEG information is used for emotion classification. We conducted binary and ternary classification experiments on the DEAP dataset, achieving 99.26% and 99.15% accuracy for arousal and valence in binary classification and 97.69% and 98.05% in ternary classification, and on the SEED dataset, we achieved an accuracy of 98.63%, surpassing relevant algorithms. Additionally, the model's efficiency is significantly higher than other models, achieving better accuracy with lower resource consumption.

Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods

A systematic review on machine-learning strategies for improving generalization in electroencephalography-based emotion classification was realized. In particular, cross-subject and cross-session generalization was focused. In this context, the non-stationarity of electroencephalographic (EEG) signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. In this review, 449 papers were retrieved from the Scopus, IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 79 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject or cross-session validation strategy, or making use of other biosignals as support were excluded. On the basis of the selected papers’ analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion of the different ML approaches involved. The studies reporting the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances.

Self-Supervised EEG Representation Learning for Robust Emotion Recognition

Emotion recognition based on electroencephalography (EEG) is becoming a growing concern of researchers due to its various applications and portable devices. Existing methods are mainly dedicated to EEG feature representation and have made impressive progress. However, the problem of scarce labels restricts their further promotion. In light of this, we propose a self-supervised framework with contrastive learning for robust EEG-based emotion recognition, which can effectively leverage both readily available unlabeled EEG signals and labeled ones to learn highly discriminative EEG features. Firstly, we construct a specific pretext task according to the sequential non-stationarity of emotional EEG signals for contrastive learning, which aims at extracting pseudo-label information from all EEG data. Meanwhile, we propose a novel negative segment selection algorithm to reduce the noise of unlabeled data during the contrastive learning process. Secondly, to mitigate the overfitting issue induced by a small number of labeled samples during learning, we originate a loss function with label smoothing regularization that can guide the model to learn generalizable features. Extensive experiments over three benchmark datasets demonstrate the effectiveness and superiority of our model on EEG-based emotion recognition task. Besides, the generalization and robustness of the model have also been proved through sufficient experiments.

Enhancing local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) signals is crucial in understanding human affective states. Current research has limitations in extracting local features. The representation capabilities of local features are limited, making it difficult to comprehensively capture emotional information. In this study, a novel approach is proposed to enhance local representation learning through global–local integration with functional connectivity for EEG-based emotion recognition. By leveraging the functional connectivity of brain regions, EEG signals are divided into global embeddings that represent comprehensive brain connectivity patterns throughout the entire process and local embeddings that reflect dynamic interactions within specific brain functional networks at particular moments. Firstly, a convolutional feature extraction branch based on the residual network is designed to extract local features from the global embedding. To further improve the representation ability and accuracy of local features, a multidimensional collaborative attention (MCA) module is introduced. Secondly, the local features and patch embedded local embeddings are integrated into the feature coupling module (FCM), which utilizes hierarchical connections and enhanced cross-attention to couple region-level features, thereby enhancing local representation learning. Experimental results on three public datasets show that compared with other methods, this method improves accuracy by 4.92% on the DEAP, by 1.11% on the SEED, and by 7.76% on the SEED-IV, demonstrating its superior performance in emotion recognition tasks.

EEG emotion recognition using EEG-SWTNS neural network through EEG spectral image

The rapid development in deep learning models provide considerable advancements in emotion recognition by electroencephalogram (EEG). However, existing approaches primarily focus on temporal and frequency domain features of EEG signals, neglecting spatial domain features in model information integration. Therefore, this study proposed a novel EEG input format, called EEG spectral image (ESI), which integrates spatial domain features using Azimuthal Equidistant Projection (AEP) and frequency domain features through differential entropy (DE). To better utilize this fine-grained data format, we propose the EEG Swin Transformer (EEG-SWTNS), which combines the window attention mechanism with shifted window partitioning. Window attention mechanism can concentrate on affective information within various regions and use shifted window partitioning to disrupt aggregated emotional representations for more granular features. Different from traditional graph neural networks, this method leverages the spatial locality of region information, leading to enhanced performance in EEG-based emotion recognition. Experiments conducted on SEED and SEED IV datasets demonstrate superior performance compared to baseline methods and the state-of-the-art models. Relative improvements of 0.6% and 0.08% are observed in subject-dependent experiments, while accuracies of 80.07% and 66.72% are achieved in subject-independent experiments without utilizing transfer learning techniques.

A novel semi-supervised deep learning method for enhancing discriminability and diversity in EEG-based emotion recognition task

Numerous deep learning models have been introduced for EEG-based Emotion recognition tasks. Nevertheless, the majority of these models are fully supervised, demanding substantial amounts of labeled EEG signals. The labeling process of EEG signals is both time-intensive and costly, involving numerous trials and meticulous analysis by experts. Recently, some advanced semi-supervised algorithms that can achieve a competitive performance with fully-supervised methods by using only a small set of labeled data have been presented. However, these algorithms are primarily developed for the image data type, and naïve adaptation of them for EEG applications results in unsatisfactory performance. To address this issue, we present a robust semi-supervised EEG-based method that exploits the best techniques from advanced semi-supervised algorithms in the computer vision domain enriched with novel regularization terms for unlabeled signals. The proposed regularization terms improve both the discriminability and diversity of the model’s predictions and effectively leverage prior knowledge about the class distributions, thereby achieving a superior performance compared to the distribution alignment techniques in state-of-the-art methods. We evaluate our method on the DEAP dataset for cross-subject valence/arousal emotion recognition tasks, and on the SEED in a cross-session setting. The results indicate that the proposed method consistently surpasses the peer methods at different numbers of labeled data by a large margin.

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Emotion recognition based on Electroencephalogram (EEG) has been applied in various fields, including human–computer interaction and healthcare. However, for the popular Valence-Arousal-Dominance emotion model, researchers often classify the dimensions into high and low categories, which cannot reflect subtle changes in emotion. Furthermore, there are issues with the design of EEG features and the efficiency of transformer. To address these issues, we have designed TPRO-NET, a neural network that takes differential entropy and enhanced differential entropy features as input and outputs emotion categories through convolutional layers and improved transformer encoders. For our experiments, we categorized the emotions in the DEAP dataset into 8 classes and those in the DREAMER dataset into 5 classes. On the DEAP and the DREAMER datasets, TPRO-NET achieved average accuracy rates of 97.63%/97.47%/97.88% and 98.18%/98.37%/98.40%, respectively, on the Valence/Arousal/Dominance dimension for the subject-dependent experiments. Compared to other advanced methods, TPRO-NET demonstrates superior performance.

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

ConTL: CNN, Transformer 및 LSTM의 결합을 통한 EEG 기반 감정인식 성능 개선

A comprehensive review of deep learning in EEG-based emotion recognition: classifications, trends, and practical implications

Emotion recognition utilizing EEG signals has emerged as a pivotal component of human–computer interaction. In recent years, with the relentless advancement of deep learning techniques, using deep learning for analyzing EEG signals has assumed a prominent role in emotion recognition. Applying deep learning in the context of EEG-based emotion recognition carries profound practical implications. Although many model approaches and some review articles have scrutinized this domain, they have yet to undergo a comprehensive and precise classification and summarization process. The existing classifications are somewhat coarse, with insufficient attention given to the potential applications within this domain. Therefore, this article systematically classifies recent developments in EEG-based emotion recognition, providing researchers with a lucid understanding of this field’s various trajectories and methodologies. Additionally, it elucidates why distinct directions necessitate distinct modeling approaches. In conclusion, this article synthesizes and dissects the practical significance of EEG signals in emotion recognition, emphasizing its promising avenues for future application.

Data-Segmentation Verification and a Target Generative Adversarial Network: EEG-Based Emotion Recognition

Data-Segmentation Verification and a Target Generative Adversarial Network: EEG-Based Emotion Recognition

TSANN-TG: Temporal-Spatial Attention Neural Networks with Task-Specific Graph for EEG Emotion Recognition.

Electroencephalography (EEG)-based emotion recognition is increasingly pivotal in the realm of affective brain-computer interfaces. In this paper, we propose TSANN-TG (temporal-spatial attention neural network with a task-specific graph), a novel neural network architecture tailored for enhancing feature extraction and effectively integrating temporal-spatial features. TSANN-TG comprises three primary components: a node-feature-encoding-and-adjacency-matrices-construction block, a graph-aggregation block, and a graph-feature-fusion-and-classification block. Leveraging the distinct temporal scales of features from EEG signals, TSANN-TG incorporates attention mechanisms for efficient feature extraction. By constructing task-specific adjacency matrices, the graph convolutional network with an attention mechanism captures the dynamic changes in dependency information between EEG channels. Additionally, TSANN-TG emphasizes feature integration at multiple levels, leading to improved performance in emotion-recognition tasks. Our proposed TSANN-TG is applied to both our FTEHD dataset and the publicly available DEAP dataset. Comparative experiments and ablation studies highlight the excellent recognition results achieved. Compared to the baseline algorithms, TSANN-TG demonstrates significant enhancements in accuracy and F1 score on the two benchmark datasets for four types of cognitive tasks. These results underscore the significant potential of the TSANN-TG method to advance EEG-based emotion recognition.

Semi-supervised bipartite graph construction with active EEG sample selection for emotion recognition.

Electroencephalogram (EEG) signals are derived from the central nervous system and inherently difficult to camouflage, leading to the recent popularity of EEG-based emotion recognition. However, due to the non-stationary nature of EEG, inter-subject variabilities become obstacles for recognition models to well adapt to different subjects. In this paper, we propose a novel approach called semi-supervised bipartite graph construction with active EEG sample selection (SBGASS) for cross-subject emotion recognition, which offers two significant advantages. Firstly, SBGASS adaptively learns a bipartite graph to characterize the underlying relationships between labeled and unlabeled EEG samples, effectively implementing the semantic connection for samples from different subjects. Secondly, we employ active sample selection technique in this paper to reduce the impact of negative samples (outliers or noise in the data) on bipartite graph construction. Drawing from the experimental results with the SEED-IV data set, we have gained the following three insights. (1) SBGASS actively rejects negative labeled samples, which helps mitigate the impact of negative samples when constructing the optimal bipartite graph and improves the model performance. (2) Through the learned optimal bipartite graph in SBGASS, the transferability of labeled EEG samples is quantitatively analyzed, which exhibits a decreasing tendency as the distance between each labeled sample and the corresponding class centroid increases. (3) Besides the improved recognition accuracy, the spatial-frequency patterns in emotion recognition are investigated by the acquired projection matrix.

Optimal channel dynamic selection for Constructing lightweight Data EEG-based emotion recognition

At present, most methods to improve the accuracy of emotion recognition based on electroencephalogram (EEG) are achieved by means of increasing the number of channels and feature types. This is to use the big data to train the classification model but it also increases the code complexity and consumes a large amount of computer time. We propose a method of Ant Colony Optimization with Convolutional Neural Networks and Long Short-Term Memory (ACO–CNN–LSTM) which can attain the dynamic optimal channels for lightweight data. First, transform the time-domain EEG signal to the frequency domain by Fast Fourier Transform (FFT), and the Differential Entropy (DE) of the three frequency bands (α, β and γ) are extracted as the feature data; Then, based on the DE feature dataset, ACO is employed to plan the path where the electrodes are located in the brain map. The classification accuracy of CNN-LSTM is used as the objective function for path determination, and the electrodes on the optimal path are used as the optimal channels; Next, the initial learning rate and batchsize parameters are exactly matched the data characteristics, which can obtain the best initial learning rate and batchsize; Finally, the SJTU Emotion EEG Dataset (SEED) dataset is used for emotion recognition based on the ACO–CNN–LSTM. From the experimental results, it can be seen that: the average accuracy of three-classification (positive, neutral, negative) can achieve 96.59 %, which is based on the lightweight data by means of ACO–CNN–LSTM proposed in the paper. Meanwhile, the computer time consumed is reduced. The computational efficiency is increased by 15.85 % compared with the traditional CNN-LSTM method. The accuracy can achieve more than 90 % when the data volume is reduced to 50 %. In summary, the proposed method of ACO–CNN–LSTM in the paper can get higher efficiency and accuracy.

Unveiling Emotional Stress: Analyzing EEG Signals With Deep Learning Insights

This study focuses on a comprehensive exploration of EEG-based emotion recognition, with a particular focus on leveraging recurrent neural network (RNN) architectures, including long short-term memory (LSTM) and gated recurrent unit (GRU). The investigation is rooted in a publicly available EEG Brainwave Dataset, meticulously curated to capture human responses to emotional stimuli. The dataset features signals obtained from frontal and temporal brain lobes, categorized into distinct emotional states encompassing positive, neutral, and negative emotions. The analysis spans four distinct scenarios, each representing different combinations of features and preprocessing strategies. Notably primary emphasis is on LSTM and GRU architectures due to their proven effectiveness in sequential data processing tasks. In addition to evaluating the performance of LSTM and GRU models a thorough examination of all the four cases to assess their efficacy in accurately predicting emotional states from EEG signals. The study also emphasis on the importance of meticulous feature selection and preprocessing in optimizing model accuracy and robustness, highlighting the intricate interplay between data representation, model architecture, and task-specific requirements. Ultimately, the research findings contribute to advanced understanding of EEG-based emotion recognition and pave the way for further research in this exciting interdisciplinary field. Key Words: Electroencephalography(EEG), Long short-term memory (LSTM), Gated recurrent unit (GRU), Feature extraction, Preprocessing techniques, Statistical metrics, Frequency domain analysis.

EEG-based Emotion Recognition: Assessing Misclassification Rates Using Machine Learning Techniques

Everyday human interaction heavily relies on emotions, which play a pivotal role in fostering genuine human-machine interaction. Emotional equilibrium is crucial for individual well-being, with regular meditation being a widely acknowledged means to achieve it. This study delves into the effect of meditation on emotional responses using Electroencephalography (EEG) technology. EEG can establish correlations between mental attributes and emotional states, with four emotions (Thrilled, Angry, Sad, and Relax) used as categorized stimuli based on valence-stimulation.
 Functional connectivity in EEG brainwave activity is compared during these four emotions both before and after meditation sessions. Results demonstrate that meditation promotes more cohesive emotional experiences, albeit with a reduction in classification accuracy observed after an 8-week meditation regimen. This research utilizes EEG data to develop a classification system for discerning familiarity levels within EEG signals, employing the Hjorth Descriptor to condense signal characteristics into three distinct criteria. A Multilayer Perceptron classifier, leveraging input parameters, achieves a peak accuracy of 96% through a combined application of three functions.

Review of EEG Affective Recognition with a Neuroscience Perspective.

Emotions are a series of subconscious, fleeting, and sometimes elusive manifestations of the human innate system. They play crucial roles in everyday life-influencing the way we evaluate ourselves, our surroundings, and how we interact with our world. To date, there has been an abundance of research on the domains of neuroscience and affective computing, with experimental evidence and neural network models, respectively, to elucidate the neural circuitry involved in and neural correlates for emotion recognition. Recent advances in affective computing neural network models often relate closely to evidence and perspectives gathered from neuroscience to explain the models. Specifically, there has been growing interest in the area of EEG-based emotion recognition to adopt models based on the neural underpinnings of the processing, generation, and subsequent collection of EEG data. In this respect, our review focuses on providing neuroscientific evidence and perspectives to discuss how emotions potentially come forth as the product of neural activities occurring at the level of subcortical structures within the brain's emotional circuitry and the association with current affective computing models in recognizing emotions. Furthermore, we discuss whether such biologically inspired modeling is the solution to advance the field in EEG-based emotion recognition and beyond.

EEG emotion recognition based on differential entropy feature matrix through 2D-CNN-LSTM network

Emotion recognition research has attracted great interest in various research fields, and electroencephalography (EEG) is considered a promising tool for extracting emotion-related information. However, traditional EEG-based emotion recognition methods ignore the spatial correlation between electrodes. To address this problem, this paper proposes an EEG-based emotion recognition method combining differential entropy feature matrix (DEFM) and 2D-CNN-LSTM. In this work, first, the one-dimensional EEG vector sequence is converted into a two-dimensional grid matrix sequence, which corresponds to the distribution of brain regions of the EEG electrode positions, and can better characterize the spatial correlation between the EEG signals of multiple adjacent electrodes. Then, the EEG signal is divided into equal time windows, and the differential entropy (DE) of each electrode in this time window is calculated, it is combined with a two-dimensional grid matrix and differential entropy to obtain a new data representation that can capture the spatiotemporal correlation of the EEG signal, which is called DEFM. Secondly, we use 2D-CNN-LSTM to accurately identify the emotional categories contained in the EEG signals and finally classify them through the fully connected layer. Experiments are conducted on the widely used DEAP dataset. Experimental results show that the method achieves an average classification accuracy of 91.92% and 92.31% for valence and arousal, respectively. The method performs outstandingly in emotion recognition. This method effectively combines the temporal and spatial correlation of EEG signals, improves the accuracy and robustness of EEG emotion recognition, and has broad application prospects in the field of emotion classification and recognition based on EEG signals.

Discriminative Joint Knowledge Transfer With Online Updating Mechanism for EEG-Based Emotion Recognition

Discriminative Joint Knowledge Transfer With Online Updating Mechanism for EEG-Based Emotion Recognition

MES-CTNet: A Novel Capsule Transformer Network Base on a Multi-Domain Feature Map for Electroencephalogram-Based Emotion Recognition.

Emotion recognition using the electroencephalogram (EEG) has garnered significant attention within the realm of human-computer interaction due to the wealth of genuine emotional data stored in EEG signals. However, traditional emotion recognition methods are deficient in mining the connection between multi-domain features and fitting their advantages. In this paper, we propose a novel capsule Transformer network based on a multi-domain feature for EEG-based emotion recognition, referred to as MES-CTNet. The model's core consists of a multichannel capsule neural network(CapsNet) embedded with ECA (Efficient Channel Attention) and SE (Squeeze and Excitation) blocks and a Transformer-based temporal coding layer. Firstly, a multi-domain feature map is constructed by combining the space-frequency-time characteristics of the multi-domain features as inputs to the model. Then, the local emotion features are extracted from the multi-domain feature maps by the improved CapsNet. Finally, the Transformer-based temporal coding layer is utilized to globally perceive the emotion feature information of the continuous time slices to obtain a final emotion state. The paper fully experimented on two standard datasets with different emotion labels, the DEAP and SEED datasets. On the DEAP dataset, MES-CTNet achieved an average accuracy of 98.31% in the valence dimension and 98.28% in the arousal dimension; it achieved 94.91% for the cross-session task on the SEED dataset, demonstrating superior performance compared to traditional EEG emotion recognition methods. The MES-CTNet method, utilizing a multi-domain feature map as proposed herein, offers a broader observation perspective for EEG-based emotion recognition. It significantly enhances the classification recognition rate, thereby holding considerable theoretical and practical value in the EEG emotion recognition domain.

AC-CfC: An attention-based convolutional closed-form continuous-time neural network for raw multi-channel EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) is a critical task in the field of affective brain-computer interfaces. However, due to the non-stationarity and individual variability of EEG, hand-designed features cannot adequately capture the nonlinear and high-dimensional properties of raw EEG. Currently, spatial–temporal models have been verified to effectively capture the spatial–temporal information for EEG. However, these models are characterized by complex structures, large parameter counts, and the need for extensive training data. To overcome these disadvantages, this paper propose an end-to-end spatial–temporal model for emotion recognition based on raw EEG, called attention-based convolutional closed-form continuous-time neural network (AC-CfC). This model employs a channel attention mechanism to weight and encode EEG, capturing channel dependencies. Subsequently, one-dimensional convolutional neural networks and closed-form continuous-time neural networks are used to extract deep spatial–temporal features. Additionally, an adaptive loss-controlling mechanism is designed to enhance the model's decision-making ability between classes that are easily confused. To verify the effectiveness of the proposed model, experiments are conducted on the DEAP and DREAMER datasets. The average accuracies of the proposed model reach to 94.76% and 93.01% for valence and arousal in subject-independent experiments on the DEAP dataset, with an improvement of 11.8% and 8.73% respectively compared with the state-of-art model ACRNN. On the DREAMER dataset, the average accuracies reach to 81.83%, 81.22%, and 80.63% for valence, arousal and dominance, with an improvement of 2.55%, 6.64%, and 6.98% respectively over ACRNN. These results show that our proposed model exhibits better performance than some state-of-art models in the same category.