ICA-Evolution Based Data Augmentation with Ensemble Deep Neural Networks Using Time and Frequency Kernels for Emotion Recognition from EEG-Data

Abstract

The aim of this study is to recognize human emotions from electroencephalographic (EEG) signals using deep neural networks. Large training data is an important prerequisite for successful implementation of deep neural networks. In this view, we propose an independent component analysis (ICA) - evolution based data augmentation method. This method performs ICA to extract and accumulate clean independent components (ICs) of each class. The new ICs are generated by selection which uses a fitness function such as mutual information (MI) and crossover in component space. Data augmentation is done by performing mutation, and crossover on generated data in sensor space. Since EEG signals are non-stationary, with time-varying frequency contents, emotional patterns associated with EEG are detected in the time-frequency (TF) domain using a spectrogram. To extract emotion related features from a spectrogram, we train an ensemble convolutional neural networks (CNNs) with convolutional kernels in time and frequency axes. The information integrated over both the axes is concatenated and fed to long short-term memory (LSTM). We used the benchmark DEAP dataset for emotion classification to evaluate our approach. The results highlight the potential of proposed ICA-evolution based data augmentation and an ensemble CNNs with LSTM model for emotion recognition.