EEG-Based Analysis of Motor Imagery and Multi-Speed Passive Pedaling: Implications for Brain–Computer Interfaces

BackgroundOne of the most promising applications for electroencephalogram (EEG)-based brain computer interface is for stroke rehabilitation. Implemented as a standalone motor imagery (MI) training system or as part of a rehabilitation robotic system, many studies have shown benefits of using them to restore motor control in stroke patients. Hand movements have widely been chosen as MI tasks. Although potentially more challenging to analyze, wrist and forearm movement such as wrist flexion/extension and forearm pronation/supination should also be considered for MI tasks, because these movements are part of the main exercises given to patients in conventional stroke rehabilitation. This paper will evaluate the effectiveness of such movements for MI tasks.MethodsThree hand and wrist movement tasks which were hand opening/closing, wrist flexion/extension and forearm pronation/supination were chosen as motor imagery tasks for both hands. Eleven subjects participated in the experiment. All of them completed hand opening/closing task session. Ten subjects completed two MI task sessions which were hand opening/closing and wrist flexion/extension. Five subjects completed all three MI tasks sessions. Each MI task comprised 8 sessions spanning a 4 weeks period. For classification, feature extraction based on common spatial pattern (CSP) algorithm was used. Two types were implemented, one with conventional CSP (termed WB) and one with an increase number of features achieved by filtering EEG data into five bands (termed FB). Classification was done by linear discriminant analysis (LDA) and support vector machine (SVM).ResultsEight-fold cross validation was applied on EEG data. LDA and SVM gave comparable classification accuracy. FB achieved significantly higher classification accuracy compared to WB. The accuracy of classifying wrist flexion/extension task were higher than that of classifying hand opening/closing task in all subjects. Classifying forearm pronation/supination task achieved higher accuracy than classifying hand opening/closing task in most subjects but achieved lower accuracy than classifying wrist flexion/extension task in all subjects. Significant improvements of classification accuracy were found in nine subjects when considering individual sessions of experiments of all MI tasks. The results of classifying hand opening/closing task and wrist flexion/extension task were comparable to the results of classifying hand opening/closing task and forearm pronation/supination task. Classification accuracy of wrist flexion/extension task and forearm pronation/supination task was lower than those of hand movement tasks and wrist movement tasks.ConclusionHigh classification accuracy of the three MI tasks support the possibility of using EEG-based stroke rehabilitation system with these movements. Either LDA or SVM can equally be chosen as a classifier since the difference of their accuracies is not statistically significant. Significantly higher classification accuracy made FB more suitable for classifying MI task compared to WB. More training sessions could potentially lead to better accuracy as evident in most subjects in this experiment.

Objective. Motor-imagery (MI) classification base on electroencephalography (EEG) has been long studied in neuroscience and more recently widely used in healthcare applications such as mobile assistive robots and neurorehabilitation. In particular, EEG-based MI classification methods that rely on convolutional neural networks (CNNs) have achieved relatively high classification accuracy. However, naively training CNNs to classify raw EEG data from all channels, especially for high-density EEG, is computationally demanding and requires huge training sets. It often also introduces many irrelevant input features, making it difficult for the CNN to extract the informative ones. This problem is compounded by a dearth of training data, which is particularly acute for MI tasks, because these are cognitively demanding and thus fatigue inducing. Approach. To address these issues, we proposed an end-to-end CNN-based neural network with attentional mechanism together with different data augmentation (DA) techniques. We tested it on two benchmark MI datasets, brain–computer interface (BCI) competition IV 2a and 2b. In addition, we collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks, which we share with the community. Main results. Our proposed neural-network architecture outperformed all state-of-the-art methods that we found in the literature, with and without DA, reaching an average classification accuracy of 93.6% and 87.83% on BCI 2a and 2b, respectively. We also directly compare decoding of MI and ME tasks. Focusing on MI classification, we find optimal channel configurations and the best DA techniques as well as investigate combining data across participants and the role of transfer learning. Significance. Our proposed approach improves the classification accuracy for MI in the benchmark datasets. In addition, collecting our own dataset enables us to compare MI and ME and investigate various aspects of EEG decoding critical for neuroscience and BCI.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

In recent years, the decoding of motor imagery (MI) from electroencephalography (EEG) signals has become a focus of research for brain-machine interfaces (BMIs) and neurorehabilitation. However, EEG signals present challenges due to their non-stationarity and the substantial presence of noise commonly found in recordings, making it difficult to design highly effective decoding algorithms. These algorithms are vital for controlling devices in neurorehabilitation tasks, as they activate the patient's motor cortex and contribute to their recovery. This study proposes a novel approach for decoding MI during pedalling tasks using EEG signals. A widespread approach is based on feature extraction using Common Spatial Patterns (CSP) followed by a linear discriminant analysis (LDA) as a classifier. The first approach covered in this work aims to investigate the efficacy of a task-discriminative feature extraction method based on CSP filter and LDA classifier. Additionally, the second alternative hypothesis explores the potential of a spectro-spatial Convolutional Neural Network (CNN) to further enhance the performance of the first approach. The proposed CNN architecture combines a preprocessing pipeline based on filter banks in the frequency domain with a convolutional neural network for spectro-temporal and spectro-spatial feature extraction. To evaluate the approaches and their advantages and disadvantages, EEG data has been recorded from several able-bodied users while pedalling in a cycle ergometer in order to train motor imagery decoding models. The results show levels of accuracy up to 80% in some cases. The CNN approach shows greater accuracy despite higher instability.

Brain activity is composed of oscillatory and broadband arrhythmic components; however, there is more focus on oscillatory sensorimotor rhythms to study movement, but temporal dynamics of broadband arrhythmic electroencephalography (EEG) remain unexplored. We have previously demonstrated that broadband arrhythmic EEG contains both short- and long-range temporal correlations that change significantly during movement. In this study, we build upon our previous work to gain a deeper understanding of these changes in the long-range temporal correlation (LRTC) in broadband EEG and contrast them with the well-known LRTC in alpha oscillation amplitude typically found in the literature. We investigate and validate changes in LRTCs during five different types of movements and motor imagery tasks using two independent EEG datasets recorded with two different paradigms—our finger tapping dataset with single self-initiated asynchronous finger taps and publicly available EEG dataset containing cued continuous movement and motor imagery of fists and feet. We quantified instantaneous changes in broadband LRTCs by detrended fluctuation analysis on single trial 2 s EEG sliding windows. The broadband LRTC increased significantly (p < 0.05) during all motor tasks as compared to the resting state. In contrast, the alpha oscillation LRTC, which had to be computed on longer stitched EEG segments, decreased significantly (p < 0.05) consistently with the literature. This suggests the complementarity of underlying fast and slow neuronal scale-free dynamics during movement and motor imagery. The single trial broadband LRTC gave high average binary classification accuracy in the range of 70.54±10.03% to 76.07±6.40% for all motor execution and imagery tasks and hence can be used in brain–computer interface (BCI). Thus, we demonstrate generalizability, robustness, and reproducibility of novel motor neural correlate, the single trial broadband LRTC, during different motor execution and imagery tasks in single asynchronous and cued continuous motor-BCI paradigms and its contrasting behavior with LRTC in alpha oscillation amplitude.

Common spatial patterns (CSP) is one of the most prevalent feature extraction approaches that has been used in Brain-computer interfaces (BCI) due to its simplicity and efficiency. Nevertheless, CSP suffers from the problems of sensitivity to noise and overfitting. To overcome these issues, the regularized CSP (RCSP) has been proposed recently. In addition, CSP was originally designed for two-class classification. However, a practical BCI usually needs four-class commands to be able to operate. Thus, there is a high demand for increasing the performance of multi-class BCI. In this paper, we provide a complete study of classification accuracy in multi-class BCI using regularization theory, and compare it with the standard CSP to determine the suitable method for feature extraction in BCI learning. Besides CSP, linear discriminant analysis (LDA) has shown its robust and widespread use for machine learning in BCI. LDA estimates covariance matrices from extracted features. But for high-dimensional features with only a small amount of training data given, the estimation may become imprecise. In the attempt of clarifying the regularizing effects in BCI, this paper also provides the classification results of the regularized LDA (RLDA). The performance evaluation of this work was taken on data from 9 subjects, from BCI competition datasets. Results show that the combination of standard CSP and LDA has a slightly better accuracy than the regularizing methods.

Common Spatial Pattern (CSP) has been recognized as a standard and powerful method for the identification of Electroencephalography (EEG)-based Motor Imagery (MI) tasks when implementing brain-computer interface (BCI) systems towards the motor rehabilitation of lost movements. The combination of BCI systems with robotic systems, such as upper limb exoskeletons, has proven to be a reliable tool for neuromotor rehabilitation. Therefore, in this study, the effects of temporal and frequency segmentation combined with layer increase for spatial filtering were evaluated, using three variations of the CSP method for the identification of passive movement vs. MI+passive movement. The passive movements were generated using a left upper-limb exoskeleton to assist flexion/extension tasks at two speeds (high-85 rpm and low-30 rpm). Ten healthy subjects were evaluated in two recording sessions using Linear Discriminant Analysis (LDA) as a classifier, and accuracy (ACC) and False Positive Rate (FPR) as metrics. The results allow concluding that the use of temporal, frequency or spatial selective information does not significantly ( 0.05) improve task identification performance. Furthermore, dynamic temporal segmentation strategies may perform better than static segmentation tasks. The findings of this study are a starting point for the exploration of complex MI tasks and their application to neurorehabilitation, as well as the study of brain effects during exoskeleton-assisted MI tasks.

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.

Brain-computer interface (BCI) systems having the ability to classify brain waves with greater accuracy are highly desirable. To this end, a number of techniques have been proposed aiming to be able to classify brain waves with high accuracy. However, the ability to classify brain waves and its implementation in real-time is still limited. In this study, we introduce a novel scheme for classifying motor imagery (MI) tasks using electroencephalography (EEG) signal that can be implemented in real-time having high classification accuracy between different MI tasks. We propose a new predictor, OPTICAL, that uses a combination of common spatial pattern (CSP) and long short-term memory (LSTM) network for obtaining improved MI EEG signal classification. A sliding window approach is proposed to obtain the time-series input from the spatially filtered data, which becomes input to the LSTM network. Moreover, instead of using LSTM directly for classification, we use regression based output of the LSTM network as one of the features for classification. On the other hand, linear discriminant analysis (LDA) is used to reduce the dimensionality of the CSP variance based features. The features in the reduced dimensional plane after performing LDA are used as input to the support vector machine (SVM) classifier together with the regression based feature obtained from the LSTM network. The regression based feature further boosts the performance of the proposed OPTICAL predictor. OPTICAL showed significant improvement in the ability to accurately classify left and right-hand MI tasks on two publically available datasets. The improvements in the average misclassification rates are 3.09% and 2.07% for BCI Competition IV Dataset I and GigaDB dataset, respectively. The Matlab code is available at https://github.com/ShiuKumar/OPTICAL.

Towards real-world BCI: CCSPNet, a compact subject-independent motor imagery framework

Brain-computer interfaces (BCIs) have been gaining momentum in making human-computer interaction more natural, especially for people with neuro-muscular disabilities. Among the existing solutions the systems relying on electroencephalograms (EEG) occupy the most prominent place due to their non-invasiveness. In this work, we provide a review of various existing techniques for the identification of motor imagery (MI) tasks. More specifically, we perform a comparison between Common Spatial Patterns (CSP) related features and features based on Power Spectral Density (PSD) techniques. Furthermore, for the identification of MI tasks, two well-known classifiers are used, the Linear Discriminant Analysis (LDA) and the Support Vector Machines (SVM). Our results confirm that PSD features demonstrate the most consistent robustness and effectiveness in extracting patterns for accurately discriminating between left and right MI tasks.

Motor imagery (MI) tasks of different body parts have been successfully decoded by conventional classifiers, such as LDA and SVM. On the other hand, decoding MI tasks within the same limb is a challenging problem with these classifiers; however, it would provide more options to control robotic devices. This work proposes to improve the hand MI tasks decoding within the same limb in a brain-computer interface using convolutional neural networks (CNNs); the CNN EEGNet, LDA, and SVM classifiers were evaluated for two (flexion/extension) and three (flexion/extension/grasping) MI tasks. Our approach is the first attempt to apply CNNs for solving this problem to our best knowledge. In addition, visual and electrotactile stimulation were included as BCI training reinforcement after the MI task similar to feedback sessions; then, they were compared. The EEGNet achieved maximum mean accuracies of 78.46% (±12.50%) and 76.72% (±11.67%) for two and three classes, respectively. Outperforming conventional classifiers with results around 60% and 48%, and similar works with results lower than 67% and 75%, respectively. Moreover, the electrical stimulation over the visual stimulus was not significant during the calibration session. The deep learning scheme enhanced the decoding of MI tasks within the same limb against the conventional framework.

Analyzing electroencephalography (EEG) signals with machine learning approaches has become an attractive research domain for linking the brain to the outside world to establish communication in the name of the Brain-Computer Interface (BCI). Many researchers have been working on developing successful motor imagery (MI)-based BCI systems. However, they still face challenges in producing better performance with them because of the irrelevant features and high computational complexity. Selecting discriminative and relevant features to overcome the existing issues is crucial. In our proposed work, different feature selection algorithms have been studied to reduce the dimension of multiband feature space to improve MI task classification performance. In the procedure, we first decomposed the MI-based EEG signal into four sets of the narrowband signal. Then a common spatial pattern (CSP) approach was employed for each narrowband to extract and combine effective features, producing a high-dimensional feature vector. Three feature selection approaches, named correlation-based feature selection (CFS), minimum redundancy and maximum relevance (mRMR), and multi-subspace randomization and collaboration-based unsupervised feature selection (SRCFS), were used in this study to select the relevant and effective features for improving classification accuracy. Among them, the SRCFS feature selection approach demonstrated outstanding performance for MI classification compared to other schemes. The SRCFS is based on the multiple k-nearest neighbour graphs method for learning feature weight based on the Laplacian score and then discarding the irrelevant features based on the weight value, reducing the feature dimension. Finally, the selected features are fed into the support vector machines (SVM), linear discriminative analysis (LDA), and multi-layer perceptron (MLP) for classification. The proposed model is evaluated with two benchmark datasets, namely BCI Competition III dataset IVA and dataset IIIB, which are publicly available and mainly used to recognize the MI tasks. The LDA classifier with the SRCFS feature selection algorithm exhibits better performance. It proves the superiority of our proposed study compared to the other state-of-the-art BCI-based MI task classification systems.

Objective. This work proposes a method for two calibration schemes based on sensory feedback to extract reliable motor imagery (MI) features, and provide classification outputs more correlated to the user’s intention. Method. After filtering the raw electroencephalogram (EEG), a two-step method for spatial feature extraction by using the Riemannian covariance matrices (RCM) method and common spatial patterns is proposed here. It uses EEG data from trials providing feedback, in an intermediate step composed of both kth nearest neighbors and probability analyses, to find periods of time in which the user probably performed well the MI task without feedback. These periods are then used to extract features with better separability, and train a classifier for MI recognition. For evaluation, an in-house dataset with eight healthy volunteers and two post-stroke patients that performed lower-limb MI, and consequently received passive movements as feedback was used. Other popular public EEG datasets (such as BCI Competition IV dataset IIb, among others) from healthy subjects that executed upper-and lower-limbs MI tasks under continuous visual sensory feedback were further used. Results. The proposed system based on the Riemannian geometry method in two-steps (RCM–RCM) outperformed significantly baseline methods, reaching average accuracy up to 82.29%. These findings show that EEG data on periods providing passive movement can be used to contribute greatly during MI feature extraction. Significance. Unconscious brain responses elicited over the sensorimotor areas may be avoided or greatly reduced by applying our approach in MI-based brain–computer interfaces (BCIs). Therefore, BCI’s outputs more correlated to the user’s intention can be obtained.

Decoding motor imagery (MI) signals accurately is important for Brain-Computer Interface (BCI) systems for healthcare applications. Electroencephalography (EEG) decoding is a challenging task because of its complexity, and dynamic nature. By improving EEG signal classification, the performance of MI-based BCI can be enhanced. In this paper, five features (Band Power (BP), Approximate Entropy (ApEn), statistical features, wavelet-based features, and Common Spatial Pattern (CSP)) are extracted from EEG signals. For classification, Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Artificial Neural Network (ANN) are used. These methods are tested on a publicly available Physionet motor imagery database. The EEG signals are recorded from 64 channels for 50 subjects, while the subject is performing four different MI tasks. The proposed method achieved an accuracy of 98.53% for left and right hands MI tasks with ApEn feature (overlapping ratio~ 0.8) and SVM classifier. Hence the proposed method shows better results than several EEG MI classification methods proposed in the literature.