Wavelet based Emotion Detection from Multi-channel EEG using a Hybrid CNN-LSTM Model

Abstract

In this study, the continuous wavelet transform (CWT) is applied on multi-channel EEG signal to obtain wavelet coefficients in the time-frequency domain. The coefficients are considered to yield the spectral energy as input feature for detection of emotional states. Scalogram of each EEG signal covers a range of frequency components which indicates the percentage of total energy of the signal carried by each of those component. The useful spectral features obtained by wavelet scalogram of each EEG channel are aggregated to constitute the input frame of a combined neural network model comprising convolutional neural network (CNN) along with Long-short-term memory (LSTM). The spatial and temporal sequence based features are extracted simultaneously by the combined two dimensional (2D) CNN-LSTM network. This hybrid deep neural network showed notable performance on emotion detection for both binary and multi-class classification with the modeled spatial-temporal-spectral features. Performance of our proposed approach is evaluated on the publicly available DEAP and SEED dataset. On binary classification of valence and arousal state (high versus low level), the obtained accuracies are 94.36 % and 94.07 % respectively, meanwhile accuracy of 94.41 % is attained on 4-class classification with DEAP dataset. Additionally, our model achieved 97.40 % accuracy for multi-class emotion detection with SEED dataset, which significantly outperform the reported state-of-the-art systems with CNN/LSTM and/or conventional temporal and spectral features.