DRS-Net: A spatial–temporal affective computing model based on multichannel EEG data

Abstract

Affective computing based on electroencephalography (EEG) is a promising field that highly integrates research and technology. A critical challenge is effectively extracting and integrating the temporal and spatial information to form a better representation for multichannel EEG data. Most existing studies use hand-selected features from each channel, which neglect high-dimensional dynamic temporal features and interplay of data from different electrodes. This study proposed a Dynamic Reservoir State Network (DRS-Net) to recognize the subject’s emotional states. The novel end-to-end model constructs a dynamic reservoir state encoder to extract multi-channel EEG data’s dynamic high dimension non-linear spatial–temporal information with high speed and low complexity. Then, a Long-Short Term Memory-dense decoder model is devised to detect emotional states. The effectiveness of the proposed DRS-Net model was evaluated on SEED, SEED-IV, and DEAP datasets. To validate the performance of the proposed method, we first combined the hand-selected features (differential entropy, power spectra density, fractal dimension, and statistics features) and classic machine learning classifiers methods (support vector machine, random forest, and k-nearest neighbor). Then, we compare them with the proposed method and other state-of-the-art deep learning methods. The experimental results generated by our method outperform all other methods in terms of accuracy and F1 score.