An Interpretable Convolutional Neural Network for P300 Detection: Analysis of Time Frequency Features for Limited Data

Abstract

In this study, a new deep learning-based methodology is developed for P300 detection in brain computer interface (BCI) systems based on time-frequency (TF) features of EEG signals coupled to deep learning. The TF distributions can transform EEG signals to the TF images by simultaneous representation of time and frequency properties of the signal. However, they do not display the energy distribution of signals at different scales identically and their advantages may be greatly incorporated by using them together. Here, four TF images of single-trail EEG signal are computed and the concatenation of the TF (cTF) images of each signal is developed to be used as training data for a simple and lightweight deep learning-based classifier. The applied TF distributions are spectrogram, Wigner–Ville distribution, Morlet-scalogram, and Bertrand distribution. Performance of method is evaluated over limited data acquired from the normal and amyotrophic lateral sclerosis (ALS) datasets and accuracy of 96.56%, and 96.84% are achieved respectively, which is superior to the other comparing algorithms. Moreover, results of cross-subject classification indicate the promising ability of the method in eliminating calibration in BCI systems. Furthermore, the heat maps of the P300 and non-P300 classes are produced to explain important regions of cTF image for classifier decision and investigate which TF may help better classification. Results revealed the efficiency of cTF images for accurate P300 detection in simple structure classifiers having the advantage of fewer data and less memory requirement. This method can be employed in P300 speller BCI systems to improve the character recognition performance in.