EEG-based Emotion Classification Research Articles

EEG 기반 감정 분류에서 MSP를 사용한 OOD 검출 적용

EEG 기반 감정 분류에서 MSP를 사용한 OOD 검출 적용

FetchEEG: a hybrid approach combining feature extraction and temporal-channel joint attention for EEG-based emotion classification

Objective. Electroencephalogram (EEG) analysis has always been an important tool in neural engineering, and the recognition and classification of human emotions are one of the important tasks in neural engineering. EEG data, obtained from electrodes placed on the scalp, represent a valuable resource of information for brain activity analysis and emotion recognition. Feature extraction methods have shown promising results, but recent trends have shifted toward end-to-end methods based on deep learning. However, these approaches often overlook channel representations, and their complex structures pose certain challenges to model fitting. Approach. To address these challenges, this paper proposes a hybrid approach named FetchEEG that combines feature extraction and temporal-channel joint attention. Leveraging the advantages of both traditional feature extraction and deep learning, the FetchEEG adopts a multi-head self-attention mechanism to extract representations between different time moments and channels simultaneously. The joint representations are then concatenated and classified using fully-connected layers for emotion recognition. The performance of the FetchEEG is verified by comparison experiments on a self-developed dataset and two public datasets. Main results. In both subject-dependent and subject-independent experiments, the FetchEEG demonstrates better performance and stronger generalization ability than the state-of-the-art methods on all datasets. Moreover, the performance of the FetchEEG is analyzed for different sliding window sizes and overlap rates in the feature extraction module. The sensitivity of emotion recognition is investigated for three- and five-frequency-band scenarios. Significance. FetchEEG is a novel hybrid method based on EEG for emotion classification, which combines EEG feature extraction with Transformer neural networks. It has achieved state-of-the-art performance on both self-developed datasets and multiple public datasets, with significantly higher training efficiency compared to end-to-end methods, demonstrating its effectiveness and feasibility.

EEG-based Emotion Classification using Deep Learning: Approaches, Trends and Bibliometrics

Emotion classification has emerged as a critical area of research, holding immense significance in the understanding of human behaviour, mental health, and social interactions. The increasing recognition of emotional well-being's crucial role in various domains, such as healthcare, psychology, and human-computer interaction, has driven substantial attention toward accurately classifying and analysing emotions. In this study, we conducted a comprehensive bibliometric analysis to unravel the scientific production and temporal evolution of research related to emotion classification. Leveraging the extensive Scopus database, we meticulously collected and meticulously analysed a diverse range of 440 articles on emotion classification from its inception to the present day. The application of advanced bibliometric measures has yielded vital insights into current trends, patterns, and characteristics in this field of study. Our data indicated an unexpected trend: an increase in research activity, especially after 2018. The understanding of how emotions impact human experiences and behaviour has advanced significantly. Researchers from several fields have emphasised the need of better understanding and describing emotions, resulting in a large rise in study output. However, there is still need for improvement in terms of agreement on emotion categorization assessment approaches and standardisation processes. It is difficult to compare and duplicate study findings due to a lack of precise assessment criteria. To address this problem, it's crucial for researchers to collaborate and develop a common knowledge. The aim of the paper is to widen our knowledge of emotions so that it can eventually result in policies being formed to improve our overall health. This knowledge could be implemented in psychological counselling and health promotion resulting in the development of closer social bonds.

EEG-Based Emotion Classification: A Biologically Informed Channel Selection Approach

EEG-Based Emotion Classification: A Biologically Informed Channel Selection Approach

Attention with kernels for EEG-based emotion classification

Attention with kernels for EEG-based emotion classification

MI-EEG: Generalized model based on mutual information for EEG emotion recognition without adversarial training

EEG-based emotion classification is a vital aspect of human–machine interfaces. However, inter-subject variability poses a challenge for accurate domain-agnostic EEG emotion recognition, often requiring individual model calibration with a robust base model for fine-tuning. To overcome this limitation and develop a generalized model, we propose a Generalized Model based on Mutual Information for EEG Emotion Recognition without Adversarial Training (MI-EEG). The MI-EEG model leverages disentanglement to extract shared features, wherein it separates EEG features into domain-invariant class-relevant features and other features. To avoid adversarial training, mutual information minimization is applied during the decoupling process. Additionally, mutual information maximization is used to enrich the features by strengthening the relationship between domain-invariant class-relevant features and emotion labels. Furthermore, the transformer-based feature extractor, which utilizes a multi-headed attention mechanism and pooling operations, enhances the feature quality in the time dimension. The experimental evaluation on two emotional EEG datasets demonstrates the superior performance of the proposed EEG-MI model compared to existing state-of-the-art methods.

Cross-subject emotion EEG signal recognition based on source microstate analysis

Electroencephalogram (EEG) signals are very weak and have low spatial resolution, which has led to less satisfactory accuracy in cross-subject EEG-based emotion classification studies. Microstate analyses of EEG sources can be performed to determine the important spatiotemporal characteristics of EEG signals. Such analyses can be used to cluster rapidly changing EEG signals into multiple brain prototype topographies, fully utilizing the spatial information contained in the EEG signals and providing a neural representation for emotional dynamics. To better utilize the spatial information of brain signals, source localization analysis on the EEG signals was first conducted. Then, a microstate analysis on the source-reconstructed EEG signals is conducted to extract the microstate features of the data. We conducted source microstate analysis on the participant data from the odor-video physiological signal database (OVPD-II) dataset. The experimental results show that the source microstate feature topologies of different participants under the same emotion exhibited a high degree of correlation, which was proven by the analysis of microstate feature topographic maps and the comparison of two-dimensional feature visualization maps of the differential entropy (DE) and power spectral density (PSD). The microstate features represent more abstract emotional information and are more robust. The extracted microstate features were then used with the style transfer mapping method to transfer the feature data from the source domain to the target domain and were then used in support vector machines (SVMs) and convolutional neural networks (CNNs) for emotion recognition. The experimental results show that the cross-subject classification accuracies of the microstate features in SVMs were 84.90 ± 8.24% and 87.43 ± 7.54%, which were 7.19 and 6.95% higher than those obtained with the PSD and 0.51 and 1.79% higher than those obtained with the DE features. In CNN, the average cross-subject classification accuracies of the microstate features were 86.44 and 91.49%, which were 7.71 and 19.41% higher than those obtained with the PSD and 2.7 and 11.76% higher than those obtained with the DE features.

Tagging Continuous Labels for EEG-based Emotion Classification.

EEG-based emotion classification has long been a critical task in the field of affective brain-computer interface (aBCI). The majority of leading researches construct supervised learning models based on labeled datasets. Several datasets have been released, including different kinds of emotions while utilizing various forms of stimulus materials. However, they adopt discrete labeling methods, in which the EEG data collected during the same stimulus material are given a same label. These methods neglect the fact that emotion changes continuously, and mislabeled data possibly exist. The imprecision of discrete labels may hinder the progress of emotion classification in concerned works. Therefore, we develop an efficient system in this paper to support continuous labeling by giving each sample a unique label, and construct a continuously labeled EEG emotion dataset. Using our dataset with continuous labels, we demonstrate the superiority of continuous labeling in emotion classification through experiments on several classification models. We further utilize the continuous labels to identify the EEG features under induced and non-induced emotions in both our dataset and a public dataset. Our experimental results reveal the learnability and generality of the relation between the EEG features and their continuous labels.

The Efficacy and Utility of Lower-Dimensional Riemannian Geometry for EEG-Based Emotion Classification

Electroencephalography (EEG) signals have diverse applications in brain-computer interfaces (BCIs), neurological condition diagnoses, and emotion recognition across healthcare, education, and entertainment domains. This paper presents a robust method that leverages Riemannian geometry to enhance the accuracy of EEG-based emotion classification. The proposed approach involves adaptive feature extraction using principal component analysis (PCA) in the Euclidean space to capture relevant signal characteristics and improve classification performance. Covariance matrices are derived from the extracted features and projected onto the Riemannian manifold. Emotion classification is performed using the minimum distance to Riemannian mean (MDRM) classifier. The effectiveness of the method was evaluated through experiments on four datasets, DEAP, DREAMER, MAHNOB, and SEED, demonstrating its generalizability and consistent accuracy improvement across different scenarios. The classification accuracy and robustness were compared with several state-of-the-art classification methods, which supports the validity and efficacy of using Riemannian geometry for enhancing the accuracy of EEG-based emotion classification.

From gram to attention matrices: a monotonicity constrained method for eeg-based emotion classification

From gram to attention matrices: a monotonicity constrained method for eeg-based emotion classification

EEG-Based Emotion Classification in Financial Trading Using Deep Learning: Effects of Risk Control Measures

Day traders in the financial markets are under constant pressure to make rapid decisions and limit capital losses in response to fluctuating market prices. As such, their emotional state can greatly influence their decision-making, leading to suboptimal outcomes in volatile market conditions. Despite the use of risk control measures such as stop loss and limit orders, it is unclear if these strategies have a substantial impact on the emotional state of traders. In this paper, we aim to determine if the use of limit orders and stop loss has a significant impact on the emotional state of traders compared to when these risk control measures are not applied. The paper provides a technical framework for valence-arousal classification in financial trading using EEG data and deep learning algorithms. We conducted two experiments: the first experiment employed predetermined stop loss and limit orders to lock in profit and risk objectives, while the second experiment did not employ limit orders or stop losses. We also proposed a novel hybrid neural architecture that integrates a Conditional Random Field with a CNN-BiLSTM model and employs Bayesian Optimization to systematically determine the optimal hyperparameters. The best model in the framework obtained classification accuracies of 85.65% and 85.05% in the two experiments, outperforming previous studies. Results indicate that the emotions associated with Low Valence and High Arousal, such as fear and worry, were more prevalent in the second experiment. The emotions associated with High Valence and High Arousal, such as hope, were more prevalent in the first experiment employing limit orders and stop loss. In contrast, High Valence and Low Arousal (calmness) emotions were most prominent in the control group which did not engage in trading activities. Our results demonstrate the efficacy of our proposed framework for emotion classification in financial trading and aid in the risk-related decision-making abilities of day traders. Further, we present the limitations of the current work and directions for future research.

Online Learning for Wearable EEG-Based Emotion Classification

Giving emotional intelligence to machines can facilitate the early detection and prediction of mental diseases and symptoms. Electroencephalography (EEG)-based emotion recognition is widely applied because it measures electrical correlates directly from the brain rather than indirect measurement of other physiological responses initiated by the brain. Therefore, we used non-invasive and portable EEG sensors to develop a real-time emotion classification pipeline. The pipeline trains different binary classifiers for Valence and Arousal dimensions from an incoming EEG data stream achieving a 23.9% (Arousal) and 25.8% (Valence) higher F1-Score on the state-of-art AMIGOS dataset than previous work. Afterward, the pipeline was applied to the curated dataset from 15 participants using two consumer-grade EEG devices while watching 16 short emotional videos in a controlled environment. Mean F1-Scores of 87% (Arousal) and 82% (Valence) were achieved for an immediate label setting. Additionally, the pipeline proved to be fast enough to achieve predictions in real-time in a live scenario with delayed labels while continuously being updated. The significant discrepancy from the readily available labels on the classification scores leads to future work to include more data. Thereafter, the pipeline is ready to be used for real-time applications of emotion classification.

Self-supervised EEG emotion recognition models based on CNN.

Emotion plays crucial roles in human life. Recently, emotion classification from electroencephalogram (EEG) signal has attracted attention by researchers due to the rapid development of brain computer interface (BCI) techniques and machine learning algorithms. However, recent studies on emotion classification show resource utilization because they use the fully-supervised learning methods. Therefore, in this study, we applied the self-supervised learning methods to improve the efficiency of resources usage. We employed a self-supervised approach to train deep multi-task convolutional neural network (CNN) for EEG-based emotion classification. First, six signal transformations were performed on unlabeled EEG data to construct the pretext task. Second, a multi-task CNN was used to perform signal transformation recognition on the transformed signals together with the original signals. After the signal transformation recognition network was trained, the convolutional layer network was frozen and the fully connected layer was reconstructed as emotion recognition network. Finally, the EEG data with affective labels were used to train the emotion recognition network to clarify the emotion. In this paper, we conduct extensive experiments from the data scaling perspective using the SEED, DEAP affective dataset. Results showed that the self-supervised learning methods can learn the internal representation of data and save computation time compared to the fully-supervised learning methods. In conclusion, our study suggests that the self-supervised machine learning model can improve the performance of emotion classification compared to the conventional fully supervised model.

Multi-Kernel Temporal and Spatial Convolution for EEG-Based Emotion Classification.

Deep learning using an end-to-end convolutional neural network (ConvNet) has been applied to several electroencephalography (EEG)-based brain-computer interface tasks to extract feature maps and classify the target output. However, the EEG analysis remains challenging since it requires consideration of various architectural design components that influence the representational ability of extracted features. This study proposes an EEG-based emotion classification model called the multi-kernel temporal and spatial convolution network (MultiT-S ConvNet). The multi-scale kernel is used in the model to learn various time resolutions, and separable convolutions are applied to find related spatial patterns. In addition, we enhanced both the temporal and spatial filters with a lightweight gating mechanism. To validate the performance and classification accuracy of MultiT-S ConvNet, we conduct subject-dependent and subject-independent experiments on EEG-based emotion datasets: DEAP and SEED. Compared with existing methods, MultiT-S ConvNet outperforms with higher accuracy results and a few trainable parameters. Moreover, the proposed multi-scale module in temporal filtering enables extracting a wide range of EEG representations, covering short- to long-wavelength components. This module could be further implemented in any model of EEG-based convolution networks, and its ability potentially improves the model's learning capacity.

EEG-Based Emotion Classification Using Improved Cross-Connected Convolutional Neural Network.

The use of electroencephalography to recognize human emotions is a key technology for advancing human–computer interactions. This study proposes an improved deep convolutional neural network model for emotion classification using a non-end-to-end training method that combines bottom-, middle-, and top-layer convolution features. Four sets of experiments using 4500 samples were conducted to verify model performance. Simultaneously, feature visualization technology was used to extract the three-layer features obtained by the model, and a scatterplot analysis was performed. The proposed model achieved a very high accuracy of 93.7%, and the extracted features exhibited the best separability among the tested models. We found that adding redundant layers did not improve model performance, and removing the data of specific channels did not significantly reduce the classification effect of the model. These results indicate that the proposed model allows for emotion recognition with a higher accuracy and speed than the previously reported models. We believe that our approach can be implemented in various applications that require the quick and accurate identification of human emotions.

Exploiting Multiple EEG Data Domains with Adversarial Learning.

Electroencephalography (EEG) is shown to be a valuable data source for evaluating subjects' mental states. However, the interpretation of multi-modal EEG signals is challenging, as they suffer from poor signal-to-noise-ratio, are highly subject-dependent, and are bound to the equipment and experimental setup used, (i.e. domain). This leads to machine learning models often suffer from poor generalization ability, where they perform significantly worse on real-world data than on the exploited training data. Recent research heavily focuses on cross-subject and cross-session transfer learning frameworks to reduce domain calibration efforts for EEG signals. We argue that multi-source learning via learning domain-invariant representations from multiple data-sources is a viable alternative, as the available data from different EEG data-source domains (e.g., subjects, sessions, experimental setups) grow massively. We propose an adversarial inference approach to learn data-source invariant representations in this context, enabling multi-source learning for EEG-based brain- computer interfaces. We unify EEG recordings from different source domains (i.e., emotion recognition datasets SEED, SEED-IV, DEAP, DREAMER), and demonstrate the feasibility of our invariant representation learning approach in suppressing data- source-relevant information leakage by 35% while still achieving stable EEG-based emotion classification performance.

EmoPercept: EEG-based emotion classification through perceiver

EmoPercept: EEG-based emotion classification through perceiver

A Novel Spatio-Temporal Field for Emotion Recognition Based on EEG Signals

Electroencephalogram (EEG) sensor data contain rich information about human emotionality. Emotion recognition based on EEG signals has attracted growing attention of researchers, especially with the fast progress of intelligent sensing technology. Numerous methods for the issue of EEG-based emotion classification have been presented in recent years. Although these methods have promoted the research development of this issue, the performance enhancement seems very slow, because the useful emotional information is quite weak compared with severe noise interferences and serious data variations. To capture the weak emotional information from the EEG signals that are distorted by various disturbances, this paper proposes a novel and effective approach, “Spatio-Temporal Field (STF)”. This method extracts the Rational Asymmetry of Spectral Power features from the EEG signals at first, and then divides the feature space into the local field via the set-based discriminative measure, and finally employs the Bidirectional Long Short-Term Memory in the local field to exploit the local dynamic information for emotion classification. Experimental results have demonstrated the advantage of STF in EEG-based emotion classification by the public challenging databases, DEAP and DREAMER. STF can be regarded as an initial attempt to deal with the issue of EEG-based emotion classification from the local field perspective. We expect that the proposed method not only can provide inspirations for the further research on this issue, but may also enrich the field methodology for more general signal classification topics.

EEG-based emotion classification using LSTM under new paradigm

Deep learning has gained much popularity in solving challenging machine learning problems related to image, speech classification, etc. Research has been conducted to apply deep learning models in emotion classification based on physiological signals such as EEG. Most of the research works have based their model on the spatial aspects of the EEG. However, the emotion features in EEG are spread across the time domain during an emotional episode. Therefore, in this work, the emotion classification problem is modelled as a sequence classification problem. The power band frequency based features of every time segment of EEG sequences generated from 32-channel EEG data are used to train three different models of Long Short-Term Memory (LSTM1, LSTM2, and LSTM3). Four class (HVHA, HVLA, LVHA, and LVLA) classification experiments were performed based on the valence and arousal emotion models. The LSTM3 model with 128 memory cells achieved the highest classification accuracy of 90%, whereas LSTM1 (32 cells) and LSTM2 (64 cells) yielded classification accuracies of 85% and 89% respectively. Further, the impact of segment size on classification accuracy was also investigated in this work. Results obtained indicate that a smaller segment size leads to higher classification accuracy using LSTM models.

EEG-Based Emotion Classification for Verifying the Korean Emotional Movie Clips with Support Vector Machine (SVM)

Emotion plays a crucial role in understanding each other under natural communication in daily life. Electroencephalogram (EEG), based on emotion classification, has been widely utilized in the fields of interdisciplinary studies because of emotion representation’s objectiveness. In this paper, it aimed to introduce the Korean continuous emotional database and investigate brain activity during emotional processing. Moreover, we selected emotion-related channels for verifying the generated database using the Support Vector Machine (SVM). First, we recorded EEG signals, collected from 28 subjects, to investigate the brain activity across brain areas while watching movie clips by five emotions (anger, excitement, fear, sadness, and happiness) and a neutral state. We analyzed EEG raw signals to investigate the emotion-related brain area and select suitable emotion-related channels using spectral power across frequency bands, i.e., alpha and beta bands. As a result, we select the eight-channel set, namely, AF3-AF4, F3-F4, F7-F8, and P7-P8, from statistical and brain topography analysis. We perform the classification using SVM and achieve the best accuracy of 94.27% when utilizing the selected channels set with five emotions. In conclusion, we provide a fundamental emotional database reflecting Korean feelings and the evidence of different emotions for application to broaden area.