Decoding Motor Imagery Research Articles

Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals.

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from “BCI Competition IV”. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

Exploiting the heightened phase synchrony in patients with neuromuscular disease for the establishment of efficient motor imagery BCIs

BackgroundPhase synchrony has extensively been studied for understanding neural coordination in health and disease. There are a few studies concerning the implications in the context of BCIs, but its potential for establishing a communication channel in patients suffering from neuromuscular disorders remains totally unexplored. We investigate, here, this possibility by estimating the time-resolved phase connectivity patterns induced during a motor imagery (MI) task and adopting a supervised learning scheme to recover the subject’s intention from the streaming data.MethodsElectroencephalographic activity from six patients suffering from neuromuscular disease (NMD) and six healthy individuals was recorded during two randomly alternating, externally cued, MI tasks (clenching either left or right fist) and a rest condition. The metric of Phase locking value (PLV) was used to describe the functional coupling between all recording sites. The functional connectivity patterns and the associate network organization was first compared between the two cohorts. Next, working at the level of individual patients, we trained support vector machines (SVMs) to discriminate between “left” and “right” based on different instantiations of connectivity patterns (depending on the encountered brain rhythm and the temporal interval). Finally, we designed and realized a novel brain decoding scheme that could interpret the intention from streaming connectivity patterns, based on an ensemble of SVMs.ResultsThe group-level analysis revealed increased phase synchrony and richer network organization in patients. This trend was also seen in the performance of the employed classifiers. Time-resolved connectivity led to superior performance, with distinct SVMs acting as local experts, specialized in the patterning emerged within specific temporal windows (defined with respect to the external trigger). This empirical finding was further exploited in implementing a decoding scheme that can be activated without the need of the precise timing of a trigger.ConclusionThe increased phase synchrony in NMD patients can turn to a valuable tool for MI decoding. Considering the fast implementation for the PLV pattern computation in multichannel signals, we can envision the development of efficient personalized BCI systems in assistance of these patients.

Decoding Motor Imagery through Common Spatial Pattern Filters at the EEG Source Space.

Brain-Computer Interface (BCI) is a rapidly developing technology that aims to support individuals suffering from various disabilities and, ultimately, improve everyday quality of life. Sensorimotor rhythm-based BCIs have demonstrated remarkable results in controlling virtual or physical external devices but they still face a number of challenges and limitations. Main challenges include multiple degrees-of-freedom control, accuracy, and robustness. In this work, we develop a multiclass BCI decoding algorithm that uses electroencephalography (EEG) source imaging, a technique that maps scalp potentials to cortical activations, to compensate for low spatial resolution of EEG. Spatial features were extracted using Common Spatial Pattern (CSP) filters in the cortical source space from a number of selected Regions of Interest (ROIs). Classification was performed through an ensemble model, based on individual ROI classification models. The evaluation was performed on the BCI Competition IV dataset 2a, which features 4 motor imagery classes from 9 participants. Our results revealed a mean accuracy increase of 5.6% with respect to the conventional application method of CSP on sensors. Neuroanatomical constraints and prior neurophysiological knowledge play an important role in developing source space-based BCI algorithms. Feature selection and classifier characteristics of our implementation will be explored to raise performance to current state-of-the-art.

Across-subject offline decoding of motor imagery from MEG and EEG

Long calibration time hinders the feasibility of brain-computer interfaces (BCI). If other subjects’ data were used for training the classifier, BCI-based neurofeedback practice could start without the initial calibration. Here, we compare methods for inter-subject decoding of left- vs. right-hand motor imagery (MI) from MEG and EEG. Six methods were tested on data involving MEG and EEG measurements of healthy participants. Inter-subject decoders were trained on subjects showing good within-subject accuracy, and tested on all subjects, including poor performers. Three methods were based on Common Spatial Patterns (CSP), and three others on logistic regression with l1 - or l2,1 -norm regularization. The decoding accuracy was evaluated using (1) MI and (2) passive movements (PM) for training, separately for MEG and EEG. With MI training, the best accuracies across subjects (mean 70.6% for MEG, 67.7% for EEG) were obtained using multi-task learning (MTL) with logistic regression and l2,1-norm regularization. MEG yielded slightly better average accuracies than EEG. With PM training, none of the inter-subject methods yielded above chance level (58.7%) accuracy. In conclusion, MTL and training with other subject’s MI is efficient for inter-subject decoding of MI. Passive movements of other subjects are likely suboptimal for training the MI classifiers.

A Self-Adaptive Online Brain–Machine Interface of a Humanoid Robot Through a General Type-2 Fuzzy Inference System

This paper presents a self-adaptive autonomous online learning through a general type-2 fuzzy system (GT2 FS) for the motor imagery (MI) decoding of a brain-machine interface (BMI) and navigation of a bipedal humanoid robot in a real experiment, using electroencephalography (EEG) brain recordings only. GT2 FSs are applied to BMI for the first time in this study. We also account for several constraints commonly associated with BMI in real practice: 1) the maximum number of EEG channels is limited and fixed; 2) no possibility of performing repeated user training sessions; and 3) desirable use of unsupervised and low-complexity feature extraction methods. The novel online learning method presented in this paper consists of a self-adaptive GT2 FS that can autonomously self-adapt both its parameters and structure via creation, fusion, and scaling of the fuzzy system rules in an online BMI experiment with a real robot. The structure identification is based on an online GT2 Gath–Geva algorithm where every MI decoding class can be represented by multiple fuzzy rules (models), which are learnt in a continous (trial-by-trial) non-iterative basis. The effectiveness of the proposed method is demonstrated in a detailed BMI experiment, in which 15 untrained users were able to accurately interface with a humanoid robot, in a single session, using signals from six EEG electrodes only.

Brain-actuated gait trainer with visual and proprioceptive feedback

Objective. Brain-machine interfaces (BMIs) have been proposed in closed-loop applications for neuromodulation and neurorehabilitation. This study describes the impact of different feedback modalities on the performance of an EEG-based BMI that decodes motor imagery (MI) of leg flexion and extension. Approach. We executed experiments in a lower-limb gait trainer (the legoPress) where nine able-bodied subjects participated in three consecutive sessions based on a crossover design. A random forest classifier was trained from the offline session and tested online with visual and proprioceptive feedback, respectively. Post-hoc classification was conducted to assess the impact of feedback modalities and learning effect (an improvement over time) on the simulated trial-based performance. Finally, we performed feature analysis to investigate the discriminant power and brain pattern modulations across the subjects. Main results. (i) For real-time classification, the average accuracy was % and % for the two online sessions. The results were significantly higher than chance level, demonstrating the feasibility to distinguish between MI of leg extension and flexion. (ii) For post-hoc classification, the performance with proprioceptive feedback (%) was significantly better than with visual feedback (%), while there was no significant learning effect. (iii) We reported individual discriminate features and brain patterns associated to each feedback modality, which exhibited differences between the two modalities although no general conclusion can be drawn. Significance. The study reported a closed-loop brain-controlled gait trainer, as a proof of concept for neurorehabilitation devices. We reported the feasibility of decoding lower-limb movement in an intuitive and natural way. As far as we know, this is the first online study discussing the role of feedback modalities in lower-limb MI decoding. Our results suggest that proprioceptive feedback has an advantage over visual feedback, which could be used to improve robot-assisted strategies for motor training and functional recovery.

Reject option to reduce false prediction rates for EEG-motor imagery based BCI.

The Filter Bank Common Spatial Pattern (FBCSP) algorithm had been shown to be effective in performing multi-class Electroencephalogram (EEG) decoding of motor imagery using the one-versus-the-rest approach on the BCI Competition IV Dataset IIa. In this paper, we propose a method to reduce false detection rates of decoding through a rejection option based on the difference in the posterior probability computed by the Naïve Bayesian classifier. We applied the proposed approach on the BCI Competition IV Dataset IIa, and the results showed a decrease in the false detection rates from 34.6 % to 6.9%, while average decoded trials decreased from 93.2% to 34.2% using a rejection threshold between 0.1 and 0.9. We subsequently formulated a method to optimize the rejection threshold based on the maximum F0.5 score. The optimal rejection threshold yielded an average decrease in false detection rate to 19.1% with an average of 67.5% of trials decoded. The results showed the feasibility of decreasing false detection rates at a cost of rejection. Nevertheless, the results suggest that the use of reject option (RO) may be used as a training feedback system to train subjects' overt and covert EEG control strategies for better (dexterity and safety) continuous control of external device.

Optimized Bi-Objective EEG Channel Selection and Cross-Subject Generalization With Brain–Computer Interfaces

Electroencephalography (EEG) signal processing to decode motor imagery (MI) involves high-dimensional features, which increases the computational complexity. To reduce this computational burden due to the large number of channels, an iterative multiobjective optimization for channel selection (IMOCS) is proposed in this paper. For a given MI classification task, the proposed method initializes a reference candidate solution and subsequently finds a set of the most relevant channels in an iterative manner by exploiting both the anatomical and functional relevance of EEG channels. The proposed approach is evaluated on the Wadsworth dataset for the right fist versus left fist MI tasks, while considering the cross-validation accuracy as the performance evaluation criteria. Furthermore, 12 other dimension reduction and channel selection algorithms are used for benchmarking. The proposed approach (IMOCS) achieved an average classification accuracy of about 80% when evaluated using 35 best-performing subjects. One-way analysis of variance revealed the statistical significance of the proposed approach with at least 7% improvement over other benchmarking algorithms. Furthermore, a cross-subject generalization of channel selection on untrained subjects shows that the subject-independent channels perform as good as using all channels achieving an average classification accuracy of 61%. These results are promising for the online brain–computer interface (BCI) paradigm that requires low computational complexity and also for reducing the preparation time while conducting multiple session BCI experiments for a larger pool of subjects.

A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching

Objective. In order to increase the number of states classified by a brain–computer interface (BCI), we utilized a motor imagery task where subjects imagined both force and speed of hand clenching. Approach. The BCI utilized simultaneously recorded electroencephalographic (EEG) and functional near-infrared spectroscopy (fNIRS) signals. The time-phase-frequency feature was extracted from EEG, whereas the HbD [the difference of oxy-hemoglobin (HbO) and deoxy-hemoglobin (Hb)] feature was used to improve the classification accuracy of fNIRS. The EEG and fNIRS features were combined and optimized using the joint mutual information (JMI) feature selection criterion; then the extracted features were classified with the extreme learning machines (ELMs). Main results. In this study, the averaged classification accuracy of EEG signals achieved by the time-phase-frequency feature improved by 7%, to 18%, more than the single-type feature, and improved by 15% more than common spatial pattern (CSP) feature. The HbD feature of fNIRS signals improved the accuracy by 1%, to 4%, more than Hb, HbO, or HbT (total hemoglobin). The EEG–fNIRS feature for decoding motor imagery of both force and speed of hand clenching achieved an accuracy of 89% ± 2%, and improved the accuracy by 1% to 5% more than the sole EEG or fNIRS feature. Significance. Our novel motor imagery paradigm improves BCI performance by increasing the number of extracted commands. Both the time-phase-frequency and the HbD feature improve the classification accuracy of EEG and fNIRS signals, respectively, and the hybrid EEG–fNIRS technique achieves a higher decoding accuracy for two-class motor imagery, which may provide the framework for future multi-modal online BCI systems.