DEAP Database Research Articles

Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction

Electroencephalography (EEG) is a physiological signal directly generated by the central nervous system. Brain rhythm is closely related to a person’s emotional state and is widely used for EEG emotion recognition. In previous studies, the rhythm specificity between different brain channels was seldom explored. In this paper, the rhythm specificity of brain channels is studied to improve the accuracy of EEG emotion recognition. Variational mode decomposition is used to decompose rhythm signals and enhance features, and two kinds of information entropy, i.e., differential entropy (DE) and dispersion entropy (DispEn) are extracted. The rhythm being used to get the best result of single channel emotion recognition is selected as the representative rhythm, and the remove one method is employed to obtain rhythm information entropy feature. In the experiment, the DEAP database was used for EEG emotion recognition in valence-arousal space. The results showed that the best result of rhythm DE feature classification in the valence dimension is 77.04%, and the best result of rhythm DispEn feature classification in the arousal dimension is 79.25%.

Assessment of Valance Emotional State Using EEG-EDA Coupling and Explainable Classifiers.

Emotion influences human life and impacts daily life activities. During emotional processes, physiological signals interact with each other instead of functioning separately. Although unimodal and multimodal approaches have been explored for emotion classification, there is a lack of inclusion of central and peripheral nervous system signal interaction-based approaches. In this study, an attempt has been made to characterize valance emotional states using Electroencephalogram (EEG)- Electrodermal activity (EDA) based coupling. For this, multimodal signals are obtained from the publicly available DEAP database (n=32 subjects). The EEG signals are decomposed into θ, α, β, and bands and EDA signals are decomposed into phasic and tonic components. Then two EEG, three EDA, and two EEG-EDA coupling-based features are extracted and applied to three classifiers namely Random Forest (RF), Linear discriminant analysis, and Adaptive boosting. In addition, SHAP analysis is performed to explain classifiers' performance with respect to features. The result shows that the proposed approach is able to classify valence emotional states. The feature combination of EEG, EDA, and EEG-EDA coupling-based features with an RF classifier performs best with an F1-score of 68.21%. SHAP analysis in frontal electrodes with γ band obtained better discrimination among different valance states. This study underscores the significance of the coupling studies of EEG with EDA in classifying emotion. Therefore, the proposed approach can be extended to emotional state assessment in clinical settings.

Analysis of EEG Fluctuation Patterns Using Nonlinear Phase-Based Functional Connectivity Measures for Emotion Recognition

Automated emotion recognition is crucial in identifying and monitoring psychological disorders. Although several electroencephalography (EEG)-based methods have been explored for emotion recognition, capturing the subtle oscillations within EEG signals associated with distinct emotional states remains a persistent challenge. Nonlinear phase-based functional connectivity (FC) can capture the intricate time-varying patterns of brain activity during the processing of emotions. In this work, an attempt has been made to characterize the EEG-based emotional states using nonlinear phase-based FC techniques. For this, the EEG signals are obtained from the publicly available DEAP database and decomposed into four frequency bands: Theta (4–7[Formula: see text]Hz), alpha (8–12[Formula: see text]Hz), beta (13–30[Formula: see text]Hz) and gamma (30–45[Formula: see text]Hz). Three nonlinear phase-based FC measures, namely phase lag index (PLI), phase locking value (PLV) and Rho index, are extracted from individual frequency bands. Two types of features, namely network features and FC indices, are fed to three classifiers, namely random forest (RF), extreme gradient boosting (XGB) and K-Nearest Neighbors (KNN). The results reveal that the proposed approach can capture EEG dynamics to characterize emotional states. The gamma band-based Rho index demonstrated prominence in discriminating arousal and valence emotional states. The utilization of the Rho index-based FC feature effectively reveals interactions among cortical brain regions in response to audio-visual stimuli. Thus, the proposed approach could be extended to classifying various emotional states in clinical settings.

Multi-scale 3D-CRU for EEG emotion recognition* *This research has been supported by the National Natural Science Foundation of China Joint Fund Key Project (U2003207), National Natural Science Foundation of China (6220100261902064), Natural Science Foundation of Anhui Province (No. 1908085MF209), Key Projects of Natural Science Foundation of Anhui Province Universities (No. KJ2018A0018).

In this paper, we propose a novel multi-scale 3D-CRU model, with the goal of extracting more discriminative emotion feature from EEG signals. By concurrently exploiting the relative electrode locations and different frequency subbands of EEG signals, a three-dimensional feature representation is reconstructed wherein the Delta (δ) frequency pattern is included. We employ a multi-scale approach, termed 3D-CRU, to concurrently extract frequency and spatial features at varying levels of granularity within each time segment. In the proposed 3D-CRU, we introduce a multi-scale 3D Convolutional Neural Network (3D-CNN) to effectively capture discriminative information embedded within the 3D feature representation. To model the temporal dynamics across consecutive time segments, we incorporate a Gated Recurrent Unit (GRU) module to extract temporal representations from the time series of combined frequency-spatial features. Ultimately, the 3D-CRU model yields a global feature representation, encompassing comprehensive information across time, frequency, and spatial domains. Numerous experimental assessments conducted on publicly available DEAP and SEED databases provide empirical evidence supporting the enhanced performance of our proposed model in the domain of emotion recognition. These findings underscore the efficacy of the features extracted by the proposed multi-scale 3D-GRU model, particularly with the incorporation of the Delta (δ) frequency pattern. Specifically, on the DEAP dataset, the accuracy of Valence and Arousal are 93.12% and 94.31%, respectively, while on the SEED dataset, the accuracy is 92.25%.

FC-TFS-CGRU: A Temporal-Frequency-Spatial Electroencephalography Emotion Recognition Model Based on Functional Connectivity and a Convolutional Gated Recurrent Unit Hybrid Architecture.

The gated recurrent unit (GRU) network can effectively capture temporal information for 1D signals, such as electroencephalography and event-related brain potential, and it has been widely used in the field of EEG emotion recognition. However, multi-domain features, including the spatial, frequency, and temporal features of EEG signals, contribute to emotion recognition, while GRUs show some limitations in capturing frequency-spatial features. Thus, we proposed a hybrid architecture of convolutional neural networks and GRUs (CGRU) to effectively capture the complementary temporal features and spatial-frequency features hidden in signal channels. In addition, to investigate the interactions among different brain regions during emotional information processing, we considered the functional connectivity relationship of the brain by introducing a phase-locking value to calculate the phase difference between the EEG channels to gain spatial information based on functional connectivity. Then, in the classification module, we incorporated attention constraints to address the issue of the uneven recognition contribution of EEG signal features. Finally, we conducted experiments on the DEAP and DREAMER databases. The results demonstrated that our model outperforms the other models with remarkable recognition accuracy of 99.51%, 99.60%, and 99.59% (58.67%, 65.74%, and 67.05%) on DEAP and 98.63%, 98.7%, and 98.71% (75.65%, 75.89%, and 71.71%) on DREAMER in a subject-dependent experiment (subject-independent experiment) for arousal, valence, and dominance.

Cerebral asymmetry representation learning-based deep subdomain adaptation network for electroencephalogram-based emotion recognition.

Objective.Extracting discriminative spatial information from multiple electrodes is a crucial and challenging problem for electroencephalogram (EEG)-based emotion recognition. Additionally, the domain shift caused by the individual differences degrades the performance of cross-subject EEG classification.Approach.To deal with the above problems, we propose the cerebral asymmetry representation learning-based deep subdomain adaptation network (CARL-DSAN) to enhance cross-subject EEG-based emotion recognition. Specifically, the CARL module is inspired by the neuroscience findings that asymmetrical activations of the left and right brain hemispheres occur during cognitive and affective processes. In the CARL module, we introduce a novel two-step strategy for extracting discriminative features through intra-hemisphere spatial learning and asymmetry representation learning. Moreover, the transformer encoders within the CARL module can emphasize the contributive electrodes and electrode pairs. Subsequently, the DSAN module, known for its superior performance over global domain adaptation, is adopted to mitigate domain shift and further improve the cross-subject performance by aligning relevant subdomains that share the same class samples.Main Results.To validate the effectiveness of the CARL-DSAN, we conduct subject-independent experiments on the DEAP database, achieving accuracies of 68.67% and 67.11% for arousal and valence classification, respectively, and corresponding accuracies of 67.70% and 67.18% on the MAHNOB-HCI database.Significance.The results demonstrate that CARL-DSAN can achieve an outstanding cross-subject performance in both arousal and valence classification.

A hybrid sequential forward channel selection method for enhancing EEG-Based emotion recognition

ABSTRACT In recent times, EEG-based emotion recognition has gained significant attention in affective computing. One of the major challenges in designing an efficient EEG-based emotion-recognition framework lies in handling the high dimensionality of the data recorded from a large number of EEG electrodes. This paper develops a hybrid sequential forward channel selection (HSFCSER) based emotion-recognition method, which aims to identify the most relevant subset of EEG channels and thus reduces the dimensionality of data. In HSFCSER, the Fisher score is used as the filter method, and the wrapper method includes Support Vector Machine (SVM). Wavelet-based features are extracted from the selected optimal EEG channels, followed by the feature selection using a multi-objective genetic algorithm. The proposed method is evaluated on the DEAP database. The methodology identifies 1) two classes of emotions viz. Low/High Valence with an average accuracy of 91.25 ± 5.48% for subject-dependent and 89.38 ± 4.76% for subject-independent, and Low/High Arousal with an average accuracy of 89.92 ± 5.81% for subject-dependent and 86.32 ± 5.34% for subject-independent, and 2) four classes of emotions viz. High Valence-Low Arousal (HVLA), High Valence-High Arousal, Low Valence-Low Arousal, and Low Valence-High Arousal with 75.94 ± 11.07% for subject-dependent and 73.13 ± 8.17% for subject-independent. The efficient electrode positions for emotion recognition are also depicted. The reported results are better compared to the existing results in the literature. The source code of the proposed work is made available at https://github.com/shyammarjit/HSFCS.

An emotion recognition method based on EWT-3D–CNN–BiLSTM-GRU-AT model

This has become a significant study area in recent years because of its use in brain-machine interaction (BMI). The robustness problem of emotion classification is one of the most basic approaches for improving the quality of emotion recognition systems. One of the two main branches of these approaches deals with the problem by extracting the features using manual engineering and the other is the famous artificial intelligence approach, which infers features of EEG data. This study proposes a novel method that considers the characteristic behavior of EEG recordings and based on the artificial intelligence method. The EEG signal is a noisy signal with a non-stationary and non-linear form. Using the Empirical Wavelet Transform (EWT) signal decomposition method, the signal's frequency components are obtained. Then, frequency-based features, linear and non-linear features are extracted. The resulting frequency-based, linear, and nonlinear features are mapped to the 2-D axis according to the positions of the EEG electrodes. By merging this 2-D images, 3-D images are constructed. In this way, the multichannel brain frequency of EEG recordings, spatial and temporal relationship are combined. Lastly, 3-D deep learning framework was constructed, which was combined with convolutional neural network (CNN), bidirectional long-short term memory (BiLSTM) and gated recurrent unit (GRU) with self-attention (AT). This model is named EWT-3D–CNN–BiLSTM-GRU-AT. As a result, we have created framework comprising handcrafted features generated and cascaded from state-of-the-art deep learning models. The framework is evaluated on the DEAP recordings based on the person-independent approach. The experimental findings demonstrate that the developed model can achieve classification accuracies of 90.57 % and 90.59 % for valence and arousal axes, respectively, for the DEAP database. Compared with existing cutting-edge emotion classification models, the proposed framework exhibits superior results for classifying human emotions.

Emotion recognition with multi-modal peripheral physiological signals

IntroductionHealthcare wearables allow researchers to develop various system approaches that recognize and understand the human emotional experience. Previous research has indicated that machine learning classifiers, such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Tree (DT), can improve the accuracy of physiological signal analysis and emotion recognition. However, various emotions can have distinct effects on physiological signal alterations. Therefore, solely relying on a single type of physiological signal analysis is insufficient for accurately recognizing and understanding human emotional experiences.MethodsResearch on multi-modal emotion recognition systems (ERS) has commonly gathered physiological signals using expensive devices, which required participants to remain in fixed positions in the lab setting. This limitation restricts the potential for generalizing the ERS technology for peripheral use in daily life. Therefore, considering the convenience of data collection from everyday devices, we propose a multi-modal physiological signals-based ERS based on peripheral signals, utilizing the DEAP database. The physiological signals selected for analysis include photoplethysmography (PPG), galvanic skin response (GSR), and skin temperature (SKT). Signal features were extracted using the “Toolbox for Emotional Feature Extraction from Physiological Signals” (TEAP) library and further analyzed with three classifiers: SVM, KNN, and DT.ResultsThe results showed improved accuracy in the proposed system compared to a single-modal ERS application, which also outperformed current DEAP multi-modal ERS applications.DiscussionThis study sheds light on the potential of combining multi-modal peripheral physiological signals in ERS for ubiquitous applications in daily life, conveniently captured using smart devices.

Categorization of Emotion Based on Causality

Background: Emotions come in all shapes and forms. Some of them can be external, visible, and clearly comprehensible, while others can seemingly be coming out of thin air. Knowing what causes an emotion is crucial for better therapy and mental health. Therefore, in this manuscript, we address the problem of emotions causality. Methods: We propose a comparison of three traditional clustering models: Gaussian mixture model, HDBSCAN, and fuzzy c-means, to categorize each emotion described in the DEAP database. It contains over 1700 points, and has no prior label as to which type of stressor the subject’s emotion is generated from. This labelling task has been conducted by a psychiatrist. Results: The fuzzy c-means yields the highest results, with an accuracy of 57.13%, followed by the Gaussian mixture model at 39.49% and the HDBSCAN method with only 18.86%. Another score computed is the mutual information score which shows how homogenous the clusters are for each model. Conclusion: The data from DEAP is very heterogeneous and its density is stable, which may indicate that classification would be the better option, in terms of accuracy and homogeneity of the clusters.

Design of subject independent 3D VAD emotion detection system using EEG signals and machine learning algorithms

This work aims to develop a subject-independent Emotion Detection System (EDS) based on EEG signals and the 3D Valence-Arousal-Dominance (VAD) model. The DEAP database physiological signals are considered for the system design. A multi-domain feature extraction is performed using the Wavelet-based Atomic Function time–frequency domain technique; and various time and frequency domain feature extraction techniques. Further, principal component analysis reduces the data dimensionality and redundancy in the obtained feature set. The minimal feature set is analysed using machine learning classifiers, i.e., gradient boosting, decision tree, and random forest. Additionally, the hyperparameters of machine learning algorithms are tuned using Optuna to improve the performance of the proposed model. Three EDS are designed in this work; EDS1 considers the 3D VAD model for three class classifications. 2D VA model is used in EDS2 to determine 9 discrete emotions, and EDS3 detects 12 discrete emotions using the 3D VAD model. Results reveal the highest classification accuracy of about 99% with EDS1, whereas an average accuracy of 99.82% and 98.44% is obtained with EDS2 and EDS3, respectively. The results reveal that both EDS1 and EDS2 show improvement as compared to the existing literature. Also, the proposed EDS3 provides the classification of the highest number of discrete emotions, which may facilitate the design of an efficient human–computer interface system.

Heartbeat detector from ECG and PPG signals based on wavelet transform and upper envelopes

The analysis of cardiac activity is one of the most common elements for evaluating the state of a subject, either to control possible health risks, sports performance, stress levels, etc. This activity can be recorded using different techniques, with electrocardiogram and photoplethysmogram being the most common. Both techniques make significantly different waveforms, however the first derivative of the photoplethysmographic data produces a signal structurally similar to the electrocardiogram, so any technique focusing on detecting QRS complexes, and thus heartbeats in electrocardiogram, is potentially applicable to photoplethysmogram. In this paper, we develop a technique based on the wavelet transform and envelopes to detect heartbeats in both electrocardiogram and photoplethysmogram. The wavelet transform is used to enhance QRS complexes with respect to other signal elements, while the envelopes are used as an adaptive threshold to determine their temporal location. We compared our approach with three other techniques using electrocardiogram signals from the Physionet database and photoplethysmographic signals from the DEAP database. Our proposal showed better performances when compared to others. When the electrocardiographic signal was considered, the method had an accuracy greater than 99.94%, a true positive rate of 99.96%, and positive prediction value of 99.76%. When photoplethysmographic signals were investigated, an accuracy greater than 99.27%, a true positive rate of 99.98% and positive prediction value of 99.50% were obtained. These results indicate that our proposal can be adapted better to the recording technology.

A Two-Step Framework to Recognize Emotion Using the Combinations of Adjacent Frequency Bands of EEG

Electroencephalography (EEG)-based emotion recognition technologies can effectively help robots to perceive human behavior, which have attracted extensive attention in human–machine interaction (HMI). Due to the complexity of EEG data, current researchers tend to extract different types of hand-crafted features and connect all frequency bands for further study. However, this may result in the loss of some discriminative information of frequency band combinations and make the classification models unable to obtain the best results. In order to recognize emotions accurately, this paper designs a novel EEG-based emotion recognition framework using complementary information of frequency bands. First, after the features of the preprocessed EEG data are extracted, the combinations of all the adjacent frequency bands in different scales are obtained through permutation and reorganization. Subsequently, the improved classification method, homogeneous-collaboration-representation-based classification, is used to obtain the classification results of each combination. Finally, the circular multi-grained ensemble learning method is put forward to re-exact the characteristics of each result and merge the machine learning methods and simple majority voting for the decision fusion. In the experiment, the classification accuracies of our framework in arousal and valence on the DEAP database are 95.09% and 94.38% respectively, and that in the four classification problems on the SEED IV database is 96.37%.

An AI-Inspired Spatio-Temporal Neural Network for EEG-Based Emotional Status.

The accurate identification of the human emotional status is crucial for an efficient human-robot interaction (HRI). As such, we have witnessed extensive research efforts made in developing robust and accurate brain-computer interfacing models based on diverse biosignals. In particular, previous research has shown that an Electroencephalogram (EEG) can provide deep insight into the state of emotion. Recently, various handcrafted and deep neural network (DNN) models were proposed by researchers for extracting emotion-relevant features, which offer limited robustness to noise that leads to reduced precision and increased computational complexity. The DNN models developed to date were shown to be efficient in extracting robust features relevant to emotion classification; however, their massive feature dimensionality problem leads to a high computational load. In this paper, we propose a bag-of-hybrid-deep-features (BoHDF) extraction model for classifying EEG signals into their respective emotion class. The invariance and robustness of the BoHDF is further enhanced by transforming EEG signals into 2D spectrograms before the feature extraction stage. Such a time-frequency representation fits well with the time-varying behavior of EEG patterns. Here, we propose to combine the deep features from the GoogLeNet fully connected layer (one of the simplest DNN models) together with the OMTLBP_SMC texture-based features, which we recently developed, followed by a K-nearest neighbor (KNN) clustering algorithm. The proposed model, when evaluated on the DEAP and SEED databases, achieves a 93.83 and 96.95% recognition accuracy, respectively. The experimental results using the proposed BoHDF-based algorithm show an improved performance in comparison to previously reported works with similar setups.

A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

Emotion Recognition Using Continuous Wavelet Transform and Ensemble of Convolutional Neural Networks through Transfer Learning from Electroencephalogram Signal

Purpose: Emotions are integral brain states that can influence our behavior, decision-making, and functions. Electroencephalogram (EEG) is an appropriate modality for emotion recognition since it has high temporal resolution and is a non-invasive and cheap technique.
 Materials and Methods: A novel approach based on Ensemble pre-trained Convolutional Neural Networks (ECNNs) is proposed to recognize four emotional classes from EEG channels of individuals watching music video clips. First, scalograms are built from one-dimensional EEG signals by applying the Continuous Wavelet Transform (CWT) method. Then, these images are used to re-train five CNNs: AlexNet, VGG-19, Inception-v1, ResNet-18, and Inception-v3. Then, the majority voting method is applied to make the final decision about emotional classes. The 10-fold cross-validation method is used to evaluate the performance of the proposed method on EEG signals of 32 subjects from the DEAP database.
 Results:.The experiments showed that applying the proposed ensemble approach in combinations of scalograms of frontal and parietal regions improved results. The best accuracy, sensitivity, precision, and F-score to recognize four emotional states achieved 96.90% ± 0.52, 97.30 ± 0.55, 96.97 ± 0.62, and 96.74 ± 0.56, respectively.
 Conclusion: So, the newly proposed model from EEG signals improves recognition of the four emotional states in the DEAP database.

Automated accurate emotion classification using Clefia pattern-based features with EEG signals

ABSTRACT Background: The electroencephalogram (EEG) emotion classification/recognition is one of the popular issues for advanced signal classification. However, it is difficult to manually screen the EEG signals as they are highly nonlinear and non-stationary. Methods: This paper introduces a novel nonlinear and multileveled features-based automatic EEG emotion classification method. Our presented EEG classification model uses feature vector creation deploying an S-Box-based local pattern with a decomposition (tunable q-factor wavelet transform is utilized), the most significant features chosen, classification a shallow machine learning method, and hard majority voting. The novel side of this research is the presented feature extractor since a component of the Clefia cipher has been considered to create a local feature extractor. Results: We have obtained an accuracy of 100.0%, 98.02%, 99.33%, and for valence, arousal, and dominance cases using the DEAP database. Also, we achieved 99.69%, 98.98%, and 99.66% accuracies for valence, dominance, and arousal cases with the DREAMER database. Our proposed model is able to classify arousal, dominance, and valence cases with an accuracy of more than 98% using both databases. Conclusions: The results show that the clefia pattern can perform automatic emotion classification with low computational complexity and high accuracy.

Subject-independent EEG emotion recognition with hybrid spatio-temporal GRU-Conv architecture.

Recently, various deep learning frameworks have shown excellent performance in decoding electroencephalogram (EEG) signals, especially in human emotion recognition. However, most of them just focus on temporal features and ignore the features based on spatial dimensions. Traditional gated recurrent unit (GRU) model performs well in processing time series data, and convolutional neural network (CNN) can obtain spatial characteristics from input data. Therefore, this paper introduces a hybrid GRU and CNN deep learning framework named GRU-Conv to fully leverage the advantages of both. Nevertheless, contrary to most previous GRU architectures, we retain the output information of all GRU units. So, the GRU-Conv model could extract crucial spatio-temporal features from EEG data. And more especially, the proposed model acquires the multi-dimensional features of multi-units after temporal processing in GRU and then uses CNN to extract spatial information from the temporal features. In this way, the EEG signals with different characteristics could be classified more accurately. Finally, the subject-independent experiment shows that our model has good performance on SEED and DEAP databases. The average accuracy of the former is 87.04%. The mean accuracy of the latter is 70.07% for arousal and 67.36% for valence.

Multi-Channel EEG Emotion Recognition Based on Parallel Transformer and 3D-Convolutional Neural Network

Due to its covert and real-time properties, electroencephalography (EEG) has long been the medium of choice for emotion identification research. Currently, EEG-based emotion recognition focuses on exploiting temporal, spatial, and spatiotemporal EEG data for emotion recognition. Due to the lack of consideration of both spatial and temporal aspects of EEG data, the accuracy of EEG emotion detection algorithms employing solely spatial or temporal variables is low. In addition, approaches that use spatiotemporal properties of EEG for emotion recognition take temporal and spatial characteristics of EEG into account; however, these methods extract temporal and spatial information directly from EEG data. Since there is no reconstruction of the EEG data format, the temporal and spatial properties of the EEG data cannot be extracted efficiently. To address the aforementioned issues, this research proposes a multi-channel EEG emotion identification model based on the parallel transformer and three-dimensional convolutional neural networks (3D-CNN). First, parallel channel EEG data and position reconstruction EEG sequence data are created separately. The temporal and spatial characteristics of EEG are then retrieved using transformer and 3D-CNN models. Finally, the features of the two parallel modules are combined to form the final features for emotion recognition. On the DEAP, Dreamer, and SEED databases, the technique achieved greater accuracy in emotion recognition than other methods. It demonstrates the efficiency of the strategy described in this paper.

MEMD-HHT based Emotion Detection from EEG using 3D CNN.

In this study, the Multivariate Empirical Mode Decomposition (MEMD) is applied to multichannel EEG to obtain scale-aligned intrinsic mode functions (IMFs) as input features for emotion detection. The IMFs capture local signal variation related to emotion changes. Among the extracted IMFs, the high oscillatory ones are found to be significant for the intended task. The Marginal Hilbert spectrum (MHS) is computed from the selected IMFs. A 3D convolutional neural network (CNN) is adopted to perform emotion detection with spatial-temporal-spectral feature representations that are constructed by stacking the multi-channel MHS over consecutive signal segments. The proposed approach is evaluated on the publicly available DEAP database. On binary classification of valence and arousal level (high versus low), the attained accuracies are 89.25% and 86.23% respectively, which significantly outperform previously reported systems with 2D CNN and/or conventional temporal and spectral features.