EEG-based Emotion Recognition System Research Articles

Comprehensive Analysis of Feature Extraction Methods for Emotion Recognition from Multichannel EEG Recordings

Advances in signal processing and machine learning have expedited electroencephalogram (EEG)-based emotion recognition research, and numerous EEG signal features have been investigated to detect or characterize human emotions. However, most studies in this area have used relatively small monocentric data and focused on a limited range of EEG features, making it difficult to compare the utility of different sets of EEG features for emotion recognition. This study addressed that by comparing the classification accuracy (performance) of a comprehensive range of EEG feature sets for identifying emotional states, in terms of valence and arousal. The classification accuracy of five EEG feature sets were investigated, including statistical features, fractal dimension (FD), Hjorth parameters, higher order spectra (HOS), and those derived using wavelet analysis. Performance was evaluated using two classifier methods, support vector machine (SVM) and classification and regression tree (CART), across five independent and publicly available datasets linking EEG to emotional states: MAHNOB-HCI, DEAP, SEED, AMIGOS, and DREAMER. The FD-CART feature-classification method attained the best mean classification accuracy for valence (85.06%) and arousal (84.55%) across the five datasets. The stability of these findings across the five different datasets also indicate that FD features derived from EEG data are reliable for emotion recognition. The results may lead to the possible development of an online feature extraction framework, thereby enabling the development of an EEG-based emotion recognition system in real time.

EEG Emotion Recognition Based on Self-attention Dynamic Graph Neural Networks.

In recent years, due to the fundamental role played by the central nervous system in emotion expression, electroencephalogram (EEG) signals have emerged as the most robust signals for use in emotion recognition and inference. Current emotion recognition methods mainly employ deep learning technology to learn the spatial or temporal representation of each channel, then obtain complementary information from different EEG channels by adopting a multi-modal fusion strategy. However, emotional expression is usually accompanied by the dynamic spatio-temporal evolution of functional connections in the brain. Therefore, the effective learning of more robust long-term dynamic representations for the brain's functional connection networks is a key to improving the EEG-based emotion recognition system. To address these issues, we propose a brain network representation learning method that employs self-attention dynamic graph neural networks to obtain the spatial structure information and temporal evolution characteristics of brain networks. Experimental results on the AMIGOS dataset show that the proposed method is superior to the state-of-the-art methods.

Deep BiLSTM neural network model for emotion detection using cross-dataset approach

The purpose of this research is to use a cross-dataset approach to construct an EEG-based emotion recognition system. So far, numerous modeling strategies for emotion recognition have been revealed using the same dataset and subject-dependent and independent criteria. We propose EEG-based cross-dataset emotion classification in this study, where the datasets for training and testing are completely distinct. The research is carried out using two benchmark datasets, DEAP and SEED, as well as our own IDEA dataset. The three datasets differ in a variety of technical factors, including electroencephalography (EEG) devices, stimuli, methodology, subject, country, culture, and so on. Multilayer perceptron (MLP), support vector machine (SVM), k-nearest neighbors (k-NN), and deep RNNs model based on bidirectional long short-term memory (BiLSTM) network were trained in this study using features namely PSD, Hjorth parameters, DE, and LF-DE. When the DEAP dataset is used to train the model and the SEED dataset is used to test it, the recognition accuracy improves by 8.2 %, and when the model is the SEED dataset-trained and the DEAP dataset-tested, the recognition accuracy improves by 1.5 % when compared to the previous result. It has been revealed that LF-DE with BiLSTM outperforms other features and classifiers for the same input data. A deep neural network-BiLSTM gives deep features from the lowest level to the highest level from large datasets. The results of the experiments reveal that the optimization of deep neural network parameters can improve the performance of the emotion recognition system to a positive extent.

Emotion recognition by distinguishing appropriate EEG segments based on random matrix theory

This work proposes an emotion recognition technique by distinguishing appropriate electroencephalogram (EEG) segments from acquired signal for target emotions. Generally, EEG based emotion recognition system works considering entire acquired signal from different channels, which is improper as emotional states do not remain steady for entire duration of signal acquisition. Therefore, in this work, appropriate EEG signal segments (L) from diverse channels have been distinguished using random matrix theory (RMT). Next, classification tasks have been performed considering two features of the selected EEG segments — power spectral density (PSD) and continuous wavelet transform (CWT), in three classifiers — k-nearest neighbour (k-NN), support vector machine (SVM) and random forest (RF). Experimental validations have been performed on two EEG datasets for emotion analysis — SEED (distinct emotions) and DEAP (dimensional emotions). The proposed method achieved accuracies upto 95% (L = 60 s) and 86% (L = 2 s) for SEED (positive vs negative) and DEAP (high vs low arousal) respectively, when considering segments from entire EEG signals. Meanwhile, the EEG sub-bands, α, β and γ, displayed the highest accuracy upto 98% (L = 60–80 s) and 94% (L = 2–15 s) for SEED and DEAP respectively. For both cases, k-NN classifier with CWT feature and channels from frontal, temporal and occipital brain regions have been found very suitable. Additionally, the selected EEG segments having low coefficient-of-variation of entropy achieves the highest classification accuracy. In a comparative study, this method reveals its superiority by achieving the highest classification performance by considering automatically selected short length EEG signal segments from distinct channels.

EEG based mental state analysis

This work proposes a method for mental state analysis from the Electroencephalogram (EEG) signal data using Convolutional Neural Network (CNN). In recent years, deep learning techniques, particularly CNN have become a very popular topic. Emotions also play a very crucial role in our daily life. Emotions affect a person’s cognition, behaviour and decision-making. In this paper, we analyse the EEG signals and classify them into various emotions. The result would be a real-time EEG-based emotion recognition system. The DEAP dataset, which is a multimodal dataset, has been used in this study for analysis of human mental states.

Emotion Recognition Based on DEAP Database using EEG Time-Frequency Features and Machine Learning Methods

In recent years, research of the human emotional state is becoming importance, especially in its application for patient monitoring and in the treatment management system of that patient. In this paper, an EEG based emotion recognition system is developed that consists of a feature extraction subsystem and a classifier subsystem. As better performance of the feature extraction subsystem may produce higher recognition accuracy, nine features derived from the time and frequency domain from the EEG signal is used and analyzed. We have utilized support vector machine and Random Forest methods for classifying the emotional state of the subject, and compare its results with other machine learning methods. Using two-fold data validation model, the experiment result shows that the highest recognition accuracy is produced by using Random Forest method, i.e., 62.58%.

EEG-based emotion recognition using simple recurrent units network and ensemble learning

The purpose of this research is to develop an EEG-based emotion recognition system for identification of three emotions: positive, neutral and negative. Up to now, various modeling approaches for automatic emotion recognition have been reported. However, the time dependency property during emotion process has not been fully considered. In order to grasp the temporal information of EEG, we adopt deep Simple Recurrent Units (SRU) network which is not only capable of processing sequence data but also has the ability to solve the problem of long-term dependencies occurrence in normal Recurrent Neural Network (RNN). Before training the emotion models, Dual-tree Complex Wavelet Transform (DT-CWT) was applied to decompose the original EEG into five constituent sub-bands, from which features were then extracted using time, frequency and nonlinear analysis. Next, deep SRU models were established using four different features over five frequency bands and favorable results were found to be related to higher frequency bands. Finally, three ensemble strategies were employed to integrate base SRU models to get more desirable classification performance. We evaluate and compare the performance of shallow models, deep models and ensemble models. Our experimental results demonstrated that the proposed emotion recognition system based on SRU network and ensemble learning could achieve satisfactory identification performance with relatively economic computational cost.

Development of Filtered Bispectrum for EEG Signal Feature Extraction in Automatic Emotion Recognition Using Artificial Neural Networks

The development of automatic emotion detection systems has recently gained significant attention due to the growing possibility of their implementation in several applications, including affective computing and various fields within biomedical engineering. Use of the electroencephalograph (EEG) signal is preferred over facial expression, as people cannot control the EEG signal generated by their brain; the EEG ensures a stronger reliability in the psychological signal. However, because of its uniqueness between individuals and its vulnerability to noise, use of EEG signals can be rather complicated. In this paper, we propose a methodology to conduct EEG-based emotion recognition by using a filtered bispectrum as the feature extraction subsystem and an artificial neural network (ANN) as the classifier. The bispectrum is theoretically superior to the power spectrum because it can identify phase coupling between the nonlinear process components of the EEG signal. In the feature extraction process, to extract the information contained in the bispectrum matrices, a 3D pyramid filter is used for sampling and quantifying the bispectrum value. Experiment results show that the mean percentage of the bispectrum value from 5 × 5 non-overlapped 3D pyramid filters produces the highest recognition rate. We found that reducing the number of EEG channels down to only eight in the frontal area of the brain does not significantly affect the recognition rate, and the number of data samples used in the training process is then increased to improve the recognition rate of the system. We have also utilized a probabilistic neural network (PNN) as another classifier and compared its recognition rate with that of the back-propagation neural network (BPNN), and the results show that the PNN produces a comparable recognition rate and lower computational costs. Our research shows that the extracted bispectrum values of an EEG signal using 3D filtering as a feature extraction method is suitable for use in an EEG-based emotion recognition system.

EEG Based Emotion Recognition System Induced by Video Music Using a Wavelet Feature Vectors and an Artificial Neural Networks

EEG Based Emotion Recognition System Induced by Video Music Using a Wavelet Feature Vectors and an Artificial Neural Networks

ReliefF-Based EEG Sensor Selection Methods for Emotion Recognition.

Electroencephalogram (EEG) signals recorded from sensor electrodes on the scalp can directly detect the brain dynamics in response to different emotional states. Emotion recognition from EEG signals has attracted broad attention, partly due to the rapid development of wearable computing and the needs of a more immersive human-computer interface (HCI) environment. To improve the recognition performance, multi-channel EEG signals are usually used. A large set of EEG sensor channels will add to the computational complexity and cause users inconvenience. ReliefF-based channel selection methods were systematically investigated for EEG-based emotion recognition on a database for emotion analysis using physiological signals (DEAP). Three strategies were employed to select the best channels in classifying four emotional states (joy, fear, sadness and relaxation). Furthermore, support vector machine (SVM) was used as a classifier to validate the performance of the channel selection results. The experimental results showed the effectiveness of our methods and the comparison with the similar strategies, based on the F-score, was given. Strategies to evaluate a channel as a unity gave better performance in channel reduction with an acceptable loss of accuracy. In the third strategy, after adjusting channels’ weights according to their contribution to the classification accuracy, the number of channels was reduced to eight with a slight loss of accuracy (58.51% ± 10.05% versus the best classification accuracy 59.13% ± 11.00% using 19 channels). In addition, the study of selecting subject-independent channels, related to emotion processing, was also implemented. The sensors, selected subject-independently from frontal, parietal lobes, have been identified to provide more discriminative information associated with emotion processing, and are distributed symmetrically over the scalp, which is consistent with the existing literature. The results will make a contribution to the realization of a practical EEG-based emotion recognition system.

Development of EEG based Emotion Recognition System using Song Induced Activity

In this study we build a mood recognition system using EEG signal of Song Induced activity. In this we have analyzed alpha EEG powers related to left hemisphere, right hemisphere regions of brains. This has given the significance of different brain region related to emotions. This study successfully achieves the goal to design a system which offers offline mood recognition system. In this study we show that it is possible to recognize the different moods of person using EEG signal. We observe the different brain locations as Left Hemisphere and Right Hemisphere to recognize the significance according to different moods. The alpha powers are more alert during National, Happy, Romantic mood as compared to Sad mood. So it is possible to distinguish these different moods using alpha power values. The distance matrices also shows that it is possible to differentiate the emotions of persons using alpha power values.

A Review on the Computational Methods for Emotional State Estimation from the Human EEG

A growing number of affective computing researches recently developed a computer system that can recognize an emotional state of the human user to establish affective human-computer interactions. Various measures have been used to estimate emotional states, including self-report, startle response, behavioral response, autonomic measurement, and neurophysiologic measurement. Among them, inferring emotional states from electroencephalography (EEG) has received considerable attention as EEG could directly reflect emotional states with relatively low costs and simplicity. Yet, EEG-based emotional state estimation requires well-designed computational methods to extract information from complex and noisy multichannel EEG data. In this paper, we review the computational methods that have been developed to deduct EEG indices of emotion, to extract emotion-related features, or to classify EEG signals into one of many emotional states. We also propose using sequential Bayesian inference to estimate the continuous emotional state in real time. We present current challenges for building an EEG-based emotion recognition system and suggest some future directions.