Continuous emotion recognition during music listening using EEG signals: A fuzzy parallel cascades model

Abstract

A controversial issue in artificial intelligence is human emotion recognition. This paper presents a fuzzy parallel cascades (FPC) model for predicting the continuous subjective emotional appraisal of music by time-varying spectral content of electroencephalogram (EEG) signals. The EEG, along with an emotional appraisal of 15 subjects, was recorded during listening to seven musical excerpts. The emotional appraisement was recorded along the valence and arousal emotional axes as a continuous signal. The FPC model was composed of parallel cascades with each cascade containing a fuzzy logic-based system. The FPC model performance was evaluated using linear regression (LR), support vector regression (SVR), and Long–Short-Term-Memory recurrent neural network (LSTM-RNN) models by 4 fold cross-validation. The root mean square error (RMSE) of the FPC was lower than other models in the estimation of both valence and arousal of all musical excerpts. The lowest obtained RMSE was 0.082, which was acquired by the FPC model. The analysis of mutual information of frontal EEG with the valence confirms the role of frontal channels in the theta frequency band in emotion recognition. Considering the dynamic variations of musical features during songs, employing a modeling approach to predict dynamic variations of the emotional appraisal can be a plausible substitute for the classification of musical excerpts into predefined labels.