Leveraging spatial-temporal convolutional features for EEG-based emotion recognition

Abstract

The electroencephalogram (EEG) signal is a medium to realize a brain–computer interface (BCI) system due to its zero clinical risk and portable acquisition devices. As deep learning technology has been considered to obtain a great success towards solving various vision-based research problems such as affective computing. Therefore, in the present paper, a novel framework for EEG-based emotion recognition is proposed. The framework consists of two modules. The first module is deep convolutional neural network (DCNN) architecture, which can represent the inter-channel correlation among physically adjacent EEG signals by converting the chain-like EEG sequence into 2D frame sequences. The second module is ConvLSTM, which can represent the sequence information of the EEG data samples. After that, the features of DCNN and ConvLSTM are concatenated and represented by attention mechanism for final emotion recognition. Extensive experiments conducted on the DEAP database demonstrate that: (1) The proposed framework effectively improves the accuracies of both emotion classification, with arousal dimension up to 87.69%, which is higher than the most of the state-of-the-art methods. (2) The dimension of valence also obtains comparable emotion recognition performance with the accuracy of 87.84%, which surpass the most of the state-of-the-art methods.