EEG-ConvNet: Convolutional networks for EEG-based subject-dependent emotion recognition

Abstract

Biomedical researchers face a significant challenge in identifying emotions from electroencephalogram (EEG) signals due to their intricate and dynamic nature. The deep learning (DL) models, particularly the convolutional neural networks (CNNs), have shown significant potential in identifying the emotions for the EEG signals. However, most current DL models require complex feature engineering implicated in increased computational complexities. This research introduces a new CNN, i.e., the EEG-ConvNet, to overcome these limitations and challenges. The proposed EEG-ConvNet comprises five convolutional layers with batch normalization and max pooling. In addition, fine-tuning techniques improve the validation of pre-trained models. The study also employs the Short-time Fourier transform (STFT) and Mel spectrograms involving EEG signals from the SEED dataset. The suggested approaches effectively extract and organize emotion-related information from simple 2D spectrograms derived from 1D EEG data. The pre-trained GoogLeNet and ResNet-34 models are fine-tuned on these simple spectrograms to discover relevant features. For interpretability, the study employs explainable artificial intelligence (XAI) methods, specifically Gradient class activation mapping (Grad-CAM) and integrated gradients (IG). The STFT-based GoogLeNet and ResNet-34 models achieve accuracies of 99.97% and 99.95%, respectively. The Mel spectrogram-based GoogLeNet and ResNet-34 models achieve accuracies of 99.49% and 99.31%, respectively. The suggested EEG-ConvNet achieves an accuracy of 99.03% on STFT spectrograms. The EEG-ConvNet has a prediction time of only 6.5 ms, paving the way for real-time emotion recognition. While comparing with the previously published DL models, the proposed classification models exhibit better classification performances on the common SEED dataset.