Learning visual stimulus-evoked EEG manifold for neural image classification

Abstract

Visual neural decoding, namely the ability to interpret external visual stimuli from patterns of brain activity, is a challenging task in neuroscience research. Recent studies have focused on characterizing patterns of activity across multiple neurons that can be described in terms of population-level features. In this study, we combine spatial, spectral, and temporal features to achieve neural manifold classification capable to characterize visual perception and to simulate the working memory activity in the human brain. We treat spatio-temporal and spectral information separately by means of custom deep learning architectures based on Riemann manifold and the two-dimensional EEG spectrogram representation. In addition, a CNN-based classification model is used to classify visual stimulus-evoked EEG signals while viewing the 11-class (i.e., all-black plus 0-9 digit images) MindBigData Visual MNIST dataset. The effectiveness of the proposed integration strategy is evaluated on the stimulus-evoked EEG signal classification task, achieving an overall accuracy of 86%, comparable to state-of-the-art benchmarks.