Recognizing Emotions Evoked by Music Using CNN-LSTM Networks on EEG Signals

Abstract

Emotion is considered to be critical for the actual interpretation of actions and relationships. Recognizing emotions from EEG signals is also becoming an important computer-aided method for diagnosing emotional disorders in neurology and psychiatry. Another advantage of this approach is recognizing emotions without clinical and medical examination, which plays a major role in completing the Brain-Computer Interface (BCI) structure. Emotions recognition ability, without traditional utilization strategies such as self-assessment tests, is of paramount importance. EEG signals are considered the most reliable technique for emotions recognition because of the non-invasive nature. Manual analysis of EEG signals is impossible for emotions recognition, so an automatic method of EEG signals should be provided for emotions recognition. One problem with automatic emotions recognition is the extraction and selection of discriminative features that generally lead to high computational complexity. This paper was design to prepare a new approach to automatic two-stage classification (negative and positive) and three-stage classification (negative, positive, and neutral) of emotions from EEG signals. In the proposed method, directly apply the raw EEG signal to the convolutional neural network and long short-term memory network (CNN-LSTM), without involving feature extraction/selection. In prior literature, this is a challenging method. The suggested deep neural network architecture includes 10-convolutional layers with 3-LSTM layers followed by 2-fully connected layers. The LSTM network in a fusion of the CNN network has been used to increase stability and reduce oscillation. In the present research, we also recorded the EEG signals of 14 subjects with music stimulation for the process. The simulation results of the proposed algorithm for two-stage classification (negative and positive) and three-stage classification (negative, neutral and positive) of emotion for 12 active channels showed 97.42% and 96.78% accuracy and Kappa coefficient of 0.94 and 0.93 respectively. We also compared our proposed LSTM-CNN network (end-to-end) with other hand-crafted methods based on MLP and DBM classifiers and achieved promising results in comparison with similar approaches. According to the high accuracy of the proposed method, it can be used to develop the human-computer interface system.