Abstract
Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end‐to‐end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end‐to‐end EEG analysis, but a better understanding of how to design and train ConvNets for end‐to‐end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task‐related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG‐based brain mapping. Hum Brain Mapp 38:5391–5420, 2017. © 2017 Wiley Periodicals, Inc.
Highlights
1 IntroductionMachine-learning techniques allow to extract information from electroencephalographic (EEG) recordings of brain activity and play a crucial role in several important EEG-based research and application areas
We show for the first time that end-to-end deep ConvNets can reach accuracies at least in the same range as filter bank common spatial patterns (FBCSP) for decoding movement-related information from EEG
We first provide basic definitions with respect to brain-signal decoding as a supervised classification problem used in the remaining Methods section. This is followed by the principles of both filter bank common spatial patterns (FBCSP), the established baseline decoding method referred to throughout the present study, and of convolutional neural networks (ConvNets)
Summary
1 IntroductionMachine-learning techniques allow to extract information from electroencephalographic (EEG) recordings of brain activity and play a crucial role in several important EEG-based research and application areas. Machine-learning techniques are a central component of many EEGbased brain-computer interface (BCI) systems for clinical applications. Such systems already allowed, for example, persons with severe paralysis to communicate (Nijboer et al, 2008), to draw pictures (Munßinger et al, 2010), and to control telepresence robots (Tonin et al, 2011). For example, persons with severe paralysis to communicate (Nijboer et al, 2008), to draw pictures (Munßinger et al, 2010), and to control telepresence robots (Tonin et al, 2011) Such systems may facilitate stroke rehabilitation (Ramos-Murguialday et al, 2013) and may be used in the treatment of epilepsy (Gadhoumi et al, 2016) (for more examples of potential clinical applications, see Moghimi et al (2013)). Several important methodological questions on EEG analysis with ConvNets remain, as detailed below and addressed in the present study
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
