SEED Dataset Research Articles

Classification of Trifolium Seeds by Computer Vision Methods

Traditional machine learning methods have been extensively used in computer vision applications. However, recent improvements in computer technology have changed this trend. The dominance of deep learning methods in the field is observed when state-of-the-art studies are examined. This study employs traditional computer vision methods and deep learning to classify five different types of Trifolium seeds. Trifolium, the leading food for nutritious dairy products, plays an essential role in livestock in some parts of the world. First, an image data set consisting of 1903 images belonging to five different species of Trifolium was created. Descriptive and quantitative morphological features of each species are extracted using image-processing techniques. Then a feature matrix was created using eight different features. After feature selection and transformation, unnecessary and irrelevant features were removed from the data set to build more accurate and robust classification models. Four common and frequently applied classification algorithms created a prediction model in the seed data set. In addition, the same dataset was trained using VGG19, a convolutional neural network. Finally, the performance metrics of each classifier were computed and evaluated. The decision tree has the worst accuracy among the four traditional methods, 92.07%. On the other hand, Artificial Neural Network has the highest accuracy with 94.59%. As expected, VGG19 outperforms all traditional methods with 96.29% accuracy. However, as the results show, traditional methods can also produce results close to the deep learning methods.

Deep time-frequency features and semi-supervised dimension reduction for subject-independent emotion recognition from multi-channel EEG signals

In recent years, human emotion recognition has received great attention since it plays an essential role in human-computer interactions. Traditional methods focused on electroencephalogram(EEG) analysis in either time or frequency domain are unsuitable since EEG signals are nonlinear. This paper proposes subject-independent human emotion recognition from multi-channel EEG signals. The proposed method first obtains the time-frequency content of each channel using the modified Stockwell transform and then extracts the deep features from each time-frequency content by a deep convolutional neural network (CNN). Since there is a huge number of deep features, semi-supervised dimension reduction (SSDR) is utilized to reduce them, and reduced features of all channels are fused to construct the final feature vector. Several CNNs and classifiers are examined respectively for deep feature extraction and classification. The classification considering two-class and four-class scenarios on the DEAP dataset and three-class scenario on the SEED dataset show that the Inception-V3 CNN and support vector machine (SVM) classifier yields the highest accuracy. We present the extensive simulation to present the efficiency of the proposed method. Also, performance comparison with current methods based on time-frequency analysis demonstrates that the proposed method outperforms the others.

Multi-domain Encoding of Spatiotemporal Dynamics in EEG for Emotion Recognition.

The common goal of the studies is to map any emotional states encoded from electroencephalogram (EEG) into 2-dimensional arousal-valance scores. It is still challenging due to each emotion having its specific spatial structure and dynamic dependence over the distinct time segments among EEG signals. This paper aims to model human dynamic emotional behavior by considering the location connectivity and context dependency of brain electrodes. Thus, we designed a hybrid EEG modeling method that mainly adopts the attention mechanism, combining a multi-domain spatial transformer (MST) module and a dynamic temporal transformer (DTT) module, named MSDTTs. Specifically, the MST module extracts single-domain and cross-domain features from different brain regions and fuses them into multi-domain spatial features. Meanwhile, the temporal dynamic excitation (TDE) is inserted into the multi-head convolutional transformer to form the DTT module. These two blocks work together to activate and extract the emotion-related dynamic temporal features within the DTT module. Furthermore, we place the convolutional mapping into the transformer structure to mine the static context features among the keyframes. Overall results show that high classification accuracy of 98.91%/0.14% was obtained by the β frequency band of the DEAP dataset, and 97.52%/0.12% and 96.70%/0.26% were obtained by the γ frequency band of SEED and SEED-IV datasets. Empirical experiments indicate that our proposed method can achieve remarkable results in comparison with state-of-the-art algorithms.

Emotion Classification from Multi-Band Electroencephalogram Data Using Dynamic Simplifying Graph Convolutional Network and Channel Style Recalibration Module.

Because of its ability to objectively reflect people's emotional states, electroencephalogram (EEG) has been attracting increasing research attention for emotion classification. The classification method based on spatial-domain analysis is one of the research hotspots. However, most previous studies ignored the complementarity of information between different frequency bands, and the information in a single frequency band is not fully mined, which increases the computational time and the difficulty of improving classification accuracy. To address the above problems, this study proposes an emotion classification method based on dynamic simplifying graph convolutional (SGC) networks and a style recalibration module (SRM) for channels, termed SGC-SRM, with multi-band EEG data as input. Specifically, first, the graph structure is constructed using the differential entropy characteristics of each sub-band and the internal relationship between different channels is dynamically learned through SGC networks. Second, a convolution layer based on the SRM is introduced to recalibrate channel features to extract more emotion-related features. Third, the extracted sub-band features are fused at the feature level and classified. In addition, to reduce the redundant information between EEG channels and the computational time, (1) we adopt only 12 channels that are suitable for emotion classification to optimize the recognition algorithm, which can save approximately 90.5% of the time cost compared with using all channels; (2) we adopt information in the θ, α, β, and γ bands, consequently saving 23.3% of the time consumed compared with that in the full bands while maintaining almost the same level of classification accuracy. Finally, a subject-independent experiment is conducted on the public SEED dataset using the leave-one-subject-out cross-validation strategy. According to experimental results, SGC-SRM improves classification accuracy by 5.51-15.43% compared with existing methods.

Emotion recognition using spatial-temporal EEG features through convolutional graph attention network

Objective. Constructing an efficient human emotion recognition model based on electroencephalogram (EEG) signals is significant for realizing emotional brain–computer interaction and improving machine intelligence. Approach. In this paper, we present a spatial-temporal feature fused convolutional graph attention network (STFCGAT) model based on multi-channel EEG signals for human emotion recognition. First, we combined the single-channel differential entropy (DE) feature with the cross-channel functional connectivity (FC) feature to extract both the temporal variation and spatial topological information of EEG. After that, a novel convolutional graph attention network was used to fuse the DE and FC features and further extract higher-level graph structural information with sufficient expressive power for emotion recognition. Furthermore, we introduced a multi-headed attention mechanism in graph neural networks to improve the generalization ability of the model. Main results. We evaluated the emotion recognition performance of our proposed model on the public SEED and DEAP datasets, which achieved a classification accuracy of 99.11% ± 0.83% and 94.83% ± 3.41% in the subject-dependent and subject-independent experiments on the SEED dataset, and achieved an accuracy of 91.19% ± 1.24% and 92.03% ± 4.57% for discrimination of arousal and valence in subject-independent experiments on DEAP dataset. Notably, our model achieved state-of-the-art performance on cross-subject emotion recognition tasks for both datasets. In addition, we gained insight into the proposed frame through both the ablation experiments and the analysis of spatial patterns of FC and DE features. Significance. All these results prove the effectiveness of the STFCGAT architecture for emotion recognition and also indicate that there are significant differences in the spatial-temporal characteristics of the brain under different emotional states.

EEG emotion recognition using improved graph neural network with channel selection

Background and objective: Emotion classification tasks based on electroencephalography (EEG) are an essential part of artificial intelligence, with promising applications in healthcare areas such as autism research and emotion detection in pregnant women. However, the complex data acquisition environment provides a variable number of EEG channels, which interferes with the model to simulate the process of information transfer in the human brain. Therefore, this paper proposes an improved graph convolution model with dynamic channel selection.Methods: The proposed model combines the advantages of 1D convolution and graph convolution to capture the intra- and inter-channel EEG features, respectively. We add functional connectivity in the graph structure that helps to simulate the relationship between brain regions further. In addition, an adjustable scale of channel selection can be performed based on the attention distribution in the graph structure.Results: We conducted various experiments on the DEAP-Twente, DEAP-Geneva, and SEED datasets and achieved average accuracies of 90.74%, 91%, and 90.22%, respectively, which exceeded most existing models. Meanwhile, with only 20% of the EEG channels retained, the models achieved average accuracies of 82.78%, 84%, and 83.93% on the above three datasets, respectively.Conclusions: The experimental results show that the proposed model can achieve effective emotion classification in complex dataset environments. Also, the proposed channel selection method is informative for reducing the cost of affective computing.

SEER-net: Simple EEG-based Recognition network

This paper presents a Simple ElectroEncephalographic-based Recognition Network (SEER-net) utilizing normalized EEG signals directly as input for the purpose of end-to-end classification tasks. SEER-net utilizes a forked design with kernels in each fork tailored to operate along different directions on extracted features from temporal convolution over inputs. With this design, SEER-net is able to train with significantly fewer parameters than comparable networks while preserving classification accuracy comparable to state-of-the-art. In our experiments, the proposed SEER-net achieves 90.73% mean test accuracy with 3485 parameters in the subject-dependent emotion classification tasks on SEED dataset. Additional ablation studies are also performed in order to compare the effects three variations within the SEER-net architecture have on performance. These include: (1) comparing the forked design against the single branch approaches, (2) investigating changes in the prediction accuracy when inputs are differently filtered, and (3) comparing SEER-net’s performance when a wavelet kernel is used for the first temporal convolution. Finally, several visualization techniques are adopted to explore different representations and patterns present within the trained SEER-net in order to extract deeper physical insight and potential human-relatable interpretations.

Emotional State Classification from MUSIC-Based Features of Multichannel EEG Signals.

Electroencephalogram (EEG)-based emotion recognition is a computationally challenging issue in the field of medical data science that has interesting applications in cognitive state disclosure. Generally, EEG signals are classified from frequency-based features that are often extracted using non-parametric models such as Welch's power spectral density (PSD). These non-parametric methods are not computationally sound due to having complexity and extended run time. The main purpose of this work is to apply the multiple signal classification (MUSIC) model, a parametric-based frequency-spectrum-estimation technique to extract features from multichannel EEG signals for emotional state classification from the SEED dataset. The main challenge of using MUSIC in EEG feature extraction is to tune its parameters for getting the discriminative features from different classes, which is a significant contribution of this work. Another contribution is to show some flaws of this dataset for the first time that contributed to achieving high classification accuracy in previous research works. This work used MUSIC features to classify three emotional states and achieve 97% accuracy on average using an artificial neural network. The proposed MUSIC model optimizes a 95-96% run time compared with the conventional classical non-parametric technique (Welch's PSD) for feature extraction.

Multisource Associate Domain Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition

Emotion recognition is important in the application of brain-computer interface (BCI). Building a robust emotion recognition model across subjects and sessions is critical in emotion based BCI systems. Electroencephalogram (EEG) is a widely used tool to recognize different emotion states. However, EEG has disadvantages such as small amplitude, low signal-to-noise ratio, and non-stationary properties, resulting in large differences across subjects. To solve these problems, this paper proposes a new emotion recognition method based on a multi-source associate domain adaptation network, considering both domain invariant and domain-specific features. First, separate branches were constructed for multiple source domains, assuming that different EEG data shared the same low-level features. Secondly, the domain specific features were extracted by using the one-to-one associate domain adaptation. Then, the weighted scores of specific sources were obtained according to the distribution distance, and multiple source classifiers were deduced with the corresponding weighted scores. Finally, EEG emotion recognition experiments were conducted on different datasets, including SEED, DEAP, and SEED-IV dataset. Results indicated that, in the cross-subject experiment, the average accuracy in SEED dataset was 86.16%, DEAP dataset was 65.59%, and SEED-IV was 59.29%. In the cross-session experiment, the accuracies of SEED and SEED-IV datasets were 91.10% and 66.68%, respectively. Our proposed method has achieved better classification results compared to state-of-the-art domain adaptation methods.

CDBA: a novel multi-branch feature fusion model for EEG-based emotion recognition.

EEG-based emotion recognition through artificial intelligence is one of the major areas of biomedical and machine learning, which plays a key role in understanding brain activity and developing decision-making systems. However, the traditional EEG-based emotion recognition is a single feature input mode, which cannot obtain multiple feature information, and cannot meet the requirements of intelligent and high real-time brain computer interface. And because the EEG signal is nonlinear, the traditional methods of time domain or frequency domain are not suitable. In this paper, a CNN-DSC-Bi-LSTM-Attention (CDBA) model based on EEG signals for automatic emotion recognition is presented, which contains three feature-extracted channels. The normalized EEG signals are used as an input, the feature of which is extracted by multi-branching and then concatenated, and each channel feature weight is assigned through the attention mechanism layer. Finally, Softmax was used to classify EEG signals. To evaluate the performance of the proposed CDBA model, experiments were performed on SEED and DREAMER datasets, separately. The validation experimental results show that the proposed CDBA model is effective in classifying EEG emotions. For triple-category (positive, neutral and negative) and four-category (happiness, sadness, fear and neutrality), the classification accuracies were respectively 99.44% and 99.99% on SEED datasets. For five classification (Valence 1-Valence 5) on DREAMER datasets, the accuracy is 84.49%. To further verify and evaluate the model accuracy and credibility, the multi-classification experiments based on ten-fold cross-validation were conducted, the elevation indexes of which are all higher than other models. The results show that the multi-branch feature fusion deep learning model based on attention mechanism has strong fitting and generalization ability and can solve nonlinear modeling problems, so it is an effective emotion recognition method. Therefore, it is helpful to the diagnosis and treatment of nervous system diseases, and it is expected to be applied to emotion-based brain computer interface systems.

Dual-encoder VAE-GAN with Spatiotemporal Features for Emotional EEG Data Augmentation.

The current data scarcity problem in EEG-based emotion recognition tasks leads to difficulty in building high-precision models using existing deep learning methods. To tackle this problem, a dual encoder variational autoencoder-generative adversarial network (DEVAE-GAN) incorporating spatiotemporal features is proposed to generate high-quality artificial samples. First, EEG data for different emotions are preprocessed as differential entropy features under five frequency bands and divided into segments with a 5s time window. Secondly, each feature segment is processed in two forms: the temporal morphology data and the spatial morphology data distributed according to the electrode position. Finally, the proposed dual encoder is trained to extract information from these two features, concatenate the two pieces of information as latent variables, and feed them into the decoder to generate artificial samples. To evaluate the effectiveness, a systematic experimental study was conducted in this work on the SEED dataset. First, the original training dataset is augmented with different numbers of generated samples; then, the augmented training datasets are used to train the deep neural network to construct the sentiment model. The results show that the augmented datasets generated by the proposed method have an average accuracy of 97.21% on all subjects, which is a 5% improvement compared to the original dataset, and the similarity between the generated data and the original data distribution is proved. These results demonstrate that our proposed model can effectively learn the distribution of raw data to generate high-quality artificial samples, which can effectively train a high-precision affective model.

Bi-Branch Vision Transformer Network for EEG Emotion Recognition

Electroencephalogram (EEG) signals have emerged as an important tool for emotion research due to their objective reflection of real emotional states. Deep learning-based EEG emotion classification algorithms have made preliminary progress, but existing models struggle with capturing long-range dependence and integrating temporal, frequency, and spatial domain features to limit their classification ability. To address these challenges, this study proposes a Bi-branch Vision Transformer-based EEG emotion recognition model, Bi-ViTNet, that integrates spatial-temporal and spatial-frequency feature representations. Specifically, Bi-ViTNet is composed of spatial-frequency feature extraction branch and spatial-temporal feature extraction branch, which can fuse spatial-frequency-temporal features in a unified framework. Each branch is composed of Linear Embedding and Transformer Encoder, which is used to extract spatial-frequency features and spatial-temporal features. Finally, fusion and classification are performed by the Fusion and Classification layer. Experiments on SEED and SEED-IV datasets demonstrate that Bi-ViTNet outperforms state-of-the-art baselines.

EEG-based Emotion Recognition using Transfer Learning Based Feature Extraction and Convolutional Neural Network

In this paper, a novel method for EEG(Electroencephalography) based emotion recognition is introduced. This method uses transfer learning to extract features from multichannel EEG signals, these features are then arranged in an 8×9 map to represent their spatial location on scalp and then we introduce a CNN model which takes in the spatial feature map and extracts spatial relations between EEG channel and finally classify the emotions. First, EEG signals are converted to spectrogram and passed through a pre-trained image classification model to get a feature vector from spectrogram of EEG. Then, feature vectors of different channels are rearranged and are presented as input to a CNN model which extracts spatial features or dependencies of channels as part of training. Finally, CNN outputs are flattened and passed through dense layer to classify between emotion classes. In this study, SEED, SEED-IV and SEED-V EEG emotion data-sets are used for classification and our method achieves best classification accuracy of 97.09% on SEED, 89.81% on SEED-IV and 88.23% on SEED-V data-set with fivefold cross validation.

An innovative EEG-based emotion recognition using a single channel-specific feature from the brain rhythm code method.

Efficiently recognizing emotions is a critical pursuit in brain-computer interface (BCI), as it has many applications for intelligent healthcare services. In this work, an innovative approach inspired by the genetic code in bioinformatics, which utilizes brain rhythm code features consisting of δ, θ, α, β, or γ, is proposed for electroencephalography (EEG)-based emotion recognition. These features are first extracted from the sequencing technique. After evaluating them using four conventional machine learning classifiers, an optimal channel-specific feature that produces the highest accuracy in each emotional case is identified, so emotion recognition through minimal data is realized. By doing so, the complexity of emotion recognition can be significantly reduced, making it more achievable for practical hardware setups. The best classification accuracies achieved for the DEAP and MAHNOB datasets range from 83-92%, and for the SEED dataset, it is 78%. The experimental results are impressive, considering the minimal data employed. Further investigation of the optimal features shows that their representative channels are primarily on the frontal region, and associated rhythmic characteristics are typical of multiple kinds. Additionally, individual differences are found, as the optimal feature varies with subjects. Compared to previous studies, this work provides insights into designing portable devices, as only one electrode is appropriate to generate satisfactory performances. Consequently, it would advance the understanding of brain rhythms, which offers an innovative solution for classifying EEG signals in diverse BCI applications, including emotion recognition.

Multidimensional Feature in Emotion Recognition Based on Multi-Channel EEG Signals.

As a major daily task for the popularization of artificial intelligence technology, more and more attention has been paid to the scientific research of mental state electroencephalogram (EEG) in recent years. To retain the spatial information of EEG signals and fully mine the EEG timing-related information, this paper proposes a novel EEG emotion recognition method. First, to obtain the frequency, spatial, and temporal information of multichannel EEG signals more comprehensively, we choose the multidimensional feature structure as the input of the artificial neural network. Then, a neural network model based on depthwise separable convolution is proposed, extracting the input structure's frequency and spatial features. The network can effectively reduce the computational parameters. Finally, we modeled using the ordered neuronal long short-term memory (ON-LSTM) network, which can automatically learn hierarchical information to extract deep emotional features hidden in EEG time series. The experimental results show that the proposed model can reasonably learn the correlation and temporal dimension information content between EEG multi-channel and improve emotion classification performance. We performed the experimental validation of this paper in two publicly available EEG emotional datasets. In the experiments on the DEAP dataset (a dataset for emotion analysis using EEG, physiological, and video signals), the mean accuracy of emotion recognition for arousal and valence is 95.02% and 94.61%, respectively. In the experiments on the SEED dataset (a dataset collection for various purposes using EEG signals), the average accuracy of emotion recognition is 95.49%.

Comparison of the representational ability in individual difference analysis using 2-D time-series image and time-series feature patterns

Physiological signals, as crucial indicators for measuring physical health, must be acquired by model learning, but the modeling and analysis performance is strongly affected by different representation patterns and feature extraction methods, even for the same data. This work investigated the capacity of the 2-D time-series image patterns on the weakening of individual differences and compared them with the time-series feature patterns. First, the time-series features of electrocardiograph (ECG) were extracted using the permutation conditional mutual information (PCMI), principal component analysis (PCA), power spectral density (PSD), and compactly supported sym-5 orthogonal wavelets with approximate symmetry (CSOWAS); the time-series coupling features of electroencephalogram (EEG) and SEED were extracted using PCMI. Then, the features of ECG, EEG, and SEED were reconstructed into 2-dimensional (2-D) time-series images using Gradient Angle Difference Field (GADF) algorithm. Finally, the classification performance of the 2-D time-series images and time-series features on the ECG, EEG, and SEED data sets are analyzed using the backpropagation (BP) neural network, LeNet 5, convolutional neural network (CNN), Deep Residual Shrinkage Networks (DRSN), ResNet 18, and VGG 16 models and compared to each other. The results show that 1) the representation ability of the 2-D time-series image pattern is better than that of the time-series feature pattern in physiological data (ECG, EEG, and SEED) analysis; 2) the Gamma band is a stabler in the classification task of individual differences (best accuracy: 100%), which can be used as the first choice for the analysis of similarity features between individuals; 3) the classification performance of the 2-D time-series images in Theta, Alpha 1, Beta 1, and Beta 2 bands on EEG datasets are greatly improved. This work shows that the 2-D time-series image patterns of the ECG, EEG, and SEED are an effective method for distinguishing individual differences.

Dynamic differential entropy and brain connectivity features based EEG emotion recognition

Emotion recognition has become a research focus in the brain–computer interface and cognitive neuroscience. Electroencephalogram (EEG) is employed for its advantages as accurate, objective, and noninvasive nature. However, many existing research only focus on extracting the time and frequency domain features of the EEG signals while failing to utilize the dynamic temporal changes and the positional relationships between different electrode channels. To fill this gap, we develop the dynamic differential entropy and brain connectivity features based EEG emotion recognition using linear graph convolutional network named DDELGCN. First, the dynamic differential entropy feature which represents the frequency domain feature as well as time domain feature is extracted based on the traditional differential entropy feature. Second, brain connectivity matrices are constructed by calculating the Pearson correlation coefficient, phase-locked value and transfer entropy, and then are used to denote the connectivity features of all electrode combinations. Finally, a linear graph convolutional network is customized and applied to aggregate the features from total electrode combinations and then classifies the emotional states, which consists of five layers, namely, an input layer, two linear graph convolutional layers, a fully connected layer, and a softmax layer. Extensive experiments show that the accuracies in the valence and arousal dimensions reach 90.88% and 91.13%, and the precision reaches 96.66% and 97.02% on the DEAP dataset, respectively. On the SEED dataset, the accuracy and precision reach 91.56% and 97.38%, respectively.

Multi-Source Domain Transfer Discriminative Dictionary Learning Modeling for Electroencephalogram-Based Emotion Recognition

Cognitive computing is dedicated to researching a computing principle and method that can simulate the intelligence ability of human brain. Human emotion is the basic component of human cognitive activities. Electroencephalogram (EEG) computer signals obtained from a brain computer interface are difficult to conceal, and using machine learning methods to analyze EEG emotion is a hot topic in artificial intelligence. However, the EEG signal is non-stationary, making it difficult to select sufficient data from the same person to train a classifier for a subject. To promote the performance of emotion recognition methods, a multi-source domain transfer discriminative dictionary learning modeling (MDTDDL) is proposed in this study. The method integrates transfer learning and dictionary learning in a learning model, including the concepts of subspace learning, manifold smoothness, margin-based discriminant embedding, and large margin. The domain-specific transformation matrix projects EEG signals from various domains into the transfer subspace. The domain-invariant dictionary can find potential connections between multiple source domains and target domain. The manifold smoothness and margin-based discriminant embedding term further improve the model’s learning ability. The alternating optimization technique is used in model solving to efficiently compute model parameters. Experiments on the SEED and DEAP datasets demonstrate the effectiveness of MDTDDL.

Emotion recognition with residual network driven by spatial-frequency characteristics of EEG recorded from hearing-impaired adults in response to video clips

In recent years, emotion recognition based on electroencephalography (EEG) signals has attracted plenty of attention. Most of the existing works focused on normal or depressed people. Due to the lack of hearing ability, it is difficult for hearing-impaired people to express their emotions through language in their social activities. In this work, we collected the EEG signals of hearing-impaired subjects when they were watching six kinds of emotional video clips (happiness, inspiration, neutral, anger, fear, and sadness) for emotion recognition. The biharmonic spline interpolation method was utilized to convert the traditional frequency domain features, Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Entropy (WE) into the spatial domain. The patch embedding (PE) method was used to segment the feature map into the same patch to obtain the differences in the distribution of emotional information among brain regions. For feature classification, a compact residual network with Depthwise convolution (DC) and Pointwise convolution (PC) is proposed to separate spatial and channel mixing dimensions to better extract information between channels. Dependent subject experiments based on 70% training sets and 30% testing sets were performed. The results showed that the average classification accuracies by PE (DE), PE (PSD), and PE (WE) were 91.75%, 85.53%, and 75.68%, respectively which were improved by 11.77%, 23.54%, and 16.61% compared with DE, PSD, and WE. Moreover, the comparison experiments were carried out on the SEED and DEAP datasets with PE (DE), which achieved average accuracies of 90.04% (positive, neutral, and negative) and 88.75% (high valence and low valence). By exploring the emotional brain regions, we found that the frontal, parietal, and temporal lobes of hearing-impaired people were associated with emotional activity compared to normal people whose main emotional brain area was the frontal lobe.

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism.

The dynamic uncertain relationship among each brain region is a necessary factor that limits EEG-based emotion recognition. It is a thought-provoking problem to availably employ time-varying spatial and temporal characteristics from multi-channel electroencephalogram (EEG) signals. Although deep learning has made remarkable achievements in emotion recognition, the biological topological information among brain regions does not fully exploit, which is vital for EEG-based emotion recognition. In response to this problem, we design a hybrid model called ST-GCLSTM, which comprises a spatial-graph convolutional network (SGCN) module and an attention-enhanced bi-directional Long Short-Term Memory (LSTM) module. The main advantage of ST-GCLSTM is that it can consider the biological topology information of each brain region to extract representative spatial-temporal features from multiple EEG channels. Specifically, we construct two layers SGCN by introducing adjacency matrices to adaptively learn the intrinsic connection among different EEG channels. Moreover, an attention-enhanced mechanism is placed into a bi-directional LSTM module to extract the crucial spatial-temporal features from sequential EEG data, and then these features serve as the input layer of the classifier to learn discriminative emotion-related features. Extensive experiments on the DEAP, SEED, and SEED-IV datasets demonstrate the effectiveness of the proposed ST-GCLSTM model, revealing that our model had an absolute performance improvement over state-of-the-art strategies.