Limb Motor Imagery Research Articles

Subband Cascaded CSP-Based Deep Transfer Learning for Cross-Subject Lower Limb Motor Imagery Classification

Subband Cascaded CSP-Based Deep Transfer Learning for Cross-Subject Lower Limb Motor Imagery Classification

Motor imagery decoding using source optimized transfer learning based on multi-loss fusion CNN.

Transfer learning is increasingly used to decode multi-class motor imagery tasks. Previous transfer learning ignored the optimizability of the source model, weakened the adaptability to the target domain and limited the performance. This paper first proposes the multi-loss fusion convolutional neural network (MF-CNN) to make an optimizable source model. Then we propose a novel source optimized transfer learning (SOTL), which optimizes the source model to make it more in line with the target domain's features to improve the target model's performance. We transfer the model trained from 16 healthy subjects to 16 stroke patients. The average classification accuracy achieves 51.2 ± 0.17% in the four types of unilateral upper limb motor imagery tasks, which is significantly higher than the classification accuracy of deep learning (p < 0.001) and transfer learning (p < 0.05). In this paper, an MI model from the data of healthy subjects can be used for the classification of stroke patients and can demonstrate good classification results, which provides experiential support for the study of transfer learning and the modeling of stroke rehabilitation training.

Unilateral movement decoding of upper and lower limbs using magnetoencephalography

Problem: The effectiveness of distinguishing unilateral lower limb motor execution (ME) or imagery (MI) based on electroencephalogram (EEG) is limited by its low spatial resolution, since the somatotopies of bilateral lower limbs are close to each other. Aim: This research focused on differentiating unilateral lower limb tasks using magnetoencephalography (MEG) signals. Methods: MEG signals were recorded during unilateral upper and lower limb movements. Channel selection and extraction of band power features were performed at both the conventional sensor level and the proposed source level, reconstructed using beamforming. The classification performance at both levels was compared. Results: The theta band changes exhibited the most significant differences and lateralization across tasks. Task-related features were more prominent at the source than at the sensor level. The classification results indicate that all methods achieved an average accuracy of over 98.33% in classifying the upper limb task. For the classification of lower limb tasks, the average accuracies achieved at the source level (96.97% and 95.86%) were significantly higher than those obtained at the sensor level (90.00% and 92.57%). Conclusion and Significance: Our results demonstrated the feasibility of MEG in classifying unilateral lower limb movements, providing a potentially effective method for brain-computer interfaces (BCIs) to enhance control commands. Furthermore, signal processing at the source level can effectively enhance inter-task differences to achieve higher classification performance. Meanwhile, the theta band is highly effective in movement classification and may play an important role in motor function.

TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI.

The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% ± 0.043 on our dataset and 49.7% ± 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.

Cross-Subject EEG Channel Selection Method for Lower Limb Brain-Computer Interface

Article Cross-Subject EEG Channel Selection Method for Lower Limb Brain-Computer Interface Mingnan Wei 1,2, Mengjie Huang 3,*, and Jiaying Ni 3 1 School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China 2 Department of Computer Science, University of Liverpool, Liverpool, L69 3BX, United Kingdom 3 Design School, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China * Correspondence: Mengjie.Huang@xjtlu.edu.cn Received: 27 April 2023 Accepted: 30 June 2023 Published: 26 September 2023 Abstract: Lower limb motor imagery (MI) classification is a challenging research topic in the area of brain-computer interfaces (BCIs), and entails numerous signal channels to provide sufficient information about the background neural activity. However, practical applications often lack the environment to accommodate excessive channels due to the time-consuming setup process, inconvenient movement, and restricted application scenarios. The existing channel selection algorithms (designed for the individual subject) place a great deal of focus on the classified performance comparisons, whereas the significance of actual locations and neural functions of brain regions is disregarded. Although these algorithms require significant computation resources, their selected solutions cannot be re-used for other subjects to realize the cross-subject channel selection and improve the reusability of model due to poor interpretability and inapplicability. To date, there have been no investigations about the cross-subject channel selection problem for the lower limb MI stepping tasks. This study proposes an optimal cross-subject lower limb channel selection that selectively retains significant channels, narrows the computation scope of the selection, and obtains the optimal selection solutions. Through stepping-based MI experiments, the proposed optimal channel selection enables effective recognition in low-channel settings, thereby contributing a lot to the development of generic and convenient lower limb BCI systems. Additionally, statistical analysis reveals a significant difference in energy spectrum between left and right stepping-based MI tasks in the and bands of the frontal lobe channels, providing new evidence that the frontal lobe dramatically affects lower limb MI tasks.

EEG motor imagery classification using deep learning approaches in naïve BCI users

Motor Imagery (MI)-Brain Computer-Interfaces (BCI) illiteracy defines that not all subjects can achieve a good performance in MI-BCI systems due to different factors related to the fatigue, substance consumption, concentration, and experience in the use. To reduce the effects of lack of experience in the use of BCI systems (naïve users), this paper presents the implementation of three Deep Learning (DL) methods with the hypothesis that the performance of BCI systems could be improved compared with baseline methods in the evaluation of naïve BCI users. The methods proposed here are based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM)/Bidirectional Long Short-Term Memory (BiLSTM), and a combination of CNN and LSTM used for upper limb MI signal discrimination on a dataset of 25 naïve BCI users. The results were compared with three widely used baseline methods based on the Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP), and Filter Bank Common Spatial-Spectral Pattern (FBCSSP), in different temporal window configurations. As results, the LSTM-BiLSTM-based approach presented the best performance, according to the evaluation metrics of Accuracy, F-score, Recall, Specificity, Precision, and ITR, with a mean performance of 80% (maximum 95%) and ITR of 10 bits/min using a temporal window of 1.5 s. The DL Methods represent a significant increase of 32% compared with the baseline methods (p < 0.05). Thus, with the outcomes of this study, it is expected to increase the controllability, usability, and reliability of the use of robotic devices in naïve BCI users.

Evaluation of the Effectiveness of Control Using a Brain–Computer Interface in Training to Upper and Lower Limb Motor Imagery

Evaluation of the Effectiveness of Control Using a Brain–Computer Interface in Training to Upper and Lower Limb Motor Imagery

Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network.

Motor imagery-based brain-computer interfaces (MI-BCI) have important application values in the field of neurorehabilitation and robot control. At present, MI-BCI mostly use bilateral upper limb motor tasks, but there are relatively few studies on single upper limb MI tasks. In this work, we conducted studies on the recognition of motor imagery EEG signals of the right upper limb and proposed a multi-branch fusion convolutional neural network (MF-CNN) for learning the features of the raw EEG signals as well as the two-dimensional time-frequency maps at the same time. The dataset used in this study contained three types of motor imagery tasks: extending the arm, rotating the wrist, and grasping the object, 25 subjects were included. In the binary classification experiment between the grasping object and the arm-extending tasks, MF-CNN achieved an average classification accuracy of 78.52% and kappa value of 0.57. When all three tasks were used for classification, the accuracy and kappa value were 57.06% and 0.36, respectively. The comparison results showed that the classification performance of MF-CNN is higher than that of single CNN branch algorithms in both binary-class and three-class classification. In conclusion, MF-CNN makes full use of the time-domain and frequency-domain features of EEG, can improve the decoding accuracy of single limb motor imagery tasks, and it contributes to the application of MI-BCI in motor function rehabilitation training after stroke.

Decoding the Continuous Motion Imagery Trajectories of Upper Limb Skeleton Points for EEG-Based Brain–Computer Interface

In the field of brain–computer interface (BCI), brain decoding using electroencephalography (EEG) is an essential direction, and motion imagery EEG-based BCI can not only help rehabilitation of patients with physical disabilities, but also enhance the endurance and power of people. Most of the existing MI-based BCI studies are limited to discrete EEG classification or 3-D directional limb trajectory reconstruction. To suitable for the requirements of BCI systems in practical applications, here, we explored the decoding of trajectories of continuous nondirectional motion imagination in 3-D space based on Chinese sign language. We propose a motion imagery trajectory reconstruction Transformer (MITRT) model to decode the EEG signals of the subjects performing motion imagery, and obtain the positional changes in the 3-D space of the shoulder, elbow, and wrist skeleton points in the neural activity. We add the geometric constraint features of upper limb skeleton points to the model, and the MITRT decoding model can obtain prior knowledge to improve the reconstruction accuracy of spatial positions. To verify the decoding performance of our proposed model, we collected motor imagery (MI) EEG signals of 20 subjects based on Chinese sign language for experiments. The experimental results show that the average Pearson correlation coefficient of the six skeleton points was 0.975, which was significantly higher than the contrast models. This study is the first attempt to reconstruct multidirectional continuous nondirectional upper limb MI trajectories based on Chinese sign language. The experimental results show that it is feasible to decode and reconstruct imagined 3-D trajectories of human upper limb skeleton points from scalp EEG.

TDLNet: Transfer Data Learning Network for Cross-Subject Classification Based on Multiclass Upper Limb Motor Imagery EEG.

The limited number of brain-computer interface based on motor imagery (MI-BCI) instruction sets for different movements of single limbs makes it difficult to meet practical application requirements. Therefore, designing a single-limb, multi-category motor imagery (MI) paradigm and effectively decoding it is one of the important research directions in the future development of MI-BCI. Furthermore, one of the major challenges in MI-BCI is the difficulty of classifying brain activity across different individuals. In this article, the transfer data learning network (TDLNet) is proposed to achieve the cross-subject intention recognition for multiclass upper limb motor imagery. In TDLNet, the Transfer Data Module (TDM) is used to process cross-subject electroencephalogram (EEG) signals in groups and then fuse cross-subject channel features through two one-dimensional convolutions. The Residual Attention Mechanism Module (RAMM) assigns weights to each EEG signal channel and dynamically focuses on the EEG signal channels most relevant to a specific task. Additionally, a feature visualization algorithm based on occlusion signal frequency is proposed to qualitatively analyze the proposed TDLNet. The experimental results show that TDLNet achieves the best classification results on two datasets compared to CNN-based reference methods and transfer learning method. In the 6-class scenario, TDLNet obtained an accuracy of 65%±0.05 on the UML6 dataset and 63%±0.06 on the GRAZ dataset. The visualization results demonstrate that the proposed framework can produce distinct classifier patterns for multiple categories of upper limb motor imagery through signals of different frequencies. The ULM6 dataset is available at https://dx.doi.org/10.21227/8qw6-f578.

Multibranch convolutional neural network with contrastive representation learning for decoding same limb motor imagery tasks.

Emerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks. The present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification. The experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively. Despite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

Interference of unilateral lower limb amputation on motor imagery rhythm and remodeling of sensorimotor areas.

The effect of sensorimotor stripping on neuroplasticity and motor imagery capacity is unknown, and the physiological mechanisms of post-amputation phantom limb pain (PLP) illness remain to be investigated. In this study, an electroencephalogram (EEG)-based event-related (de)synchronization (ERD/ERS) analysis was conducted using a bilateral lower limb motor imagery (MI) paradigm. The differences in the execution of motor imagery tasks between left lower limb amputations and healthy controls were explored, and a correlation analysis was calculated between level of phantom limb pain and ERD/ERS. The multiple frequency bands showed a significant ERD phenomenon when the healthy control group performed the motor imagery task, whereas amputees showed significant ERS phenomena in mu band. Phantom limb pain in amputees was negatively correlated with bilateral sensorimotor areas electrode powers. Sensorimotor abnormalities reduce neural activity in the sensorimotor cortex, while the motor imagination of the intact limb is diminished. In addition, phantom limb pain may lead to over-activation of sensorimotor areas, affecting bilateral sensorimotor area remodeling.

Enhancement of lower limb motor imagery ability via dual-level multimodal stimulation and sparse spatial pattern decoding method.

Recently, motor imagery brain-computer interfaces (MI-BCIs) with stimulation systems have been developed in the field of motor function assistance and rehabilitation engineering. An efficient stimulation paradigm and Electroencephalogram (EEG) decoding method have been designed to enhance the performance of MI-BCI systems. Therefore, in this study, a multimodal dual-level stimulation paradigm is designed for lower-limb rehabilitation training, whereby visual and auditory stimulations act on the sensory organ while proprioceptive and functional electrical stimulations are provided to the lower limb. In addition, upper triangle filter bank sparse spatial pattern (UTFB-SSP) is proposed to automatically select the optimal frequency sub-bands related to desynchronization rhythm during enhanced imaginary movement to improve the decoding performance. The effectiveness of the proposed MI-BCI system is demonstrated on an the in-house experimental dataset and the BCI competition IV IIa dataset. The experimental results show that the proposed system can effectively enhance the MI performance by inducing the α, β and γ rhythms in lower-limb movement imagery tasks.

The unilateral upper limb classification from fMRI-weighted EEG signals using convolutional neural network

BackgroundUnilateral upper limb multitasking brings essential improvements to stroke rehabilitation and prosthetic control. However, the influence and recognition of multiple tasks are core issues in the Motor Imagery (MI) system. MethodsFirst, we design the unilateral upper limb MI experimental paradigm and acquire asynchronous functional Magnetic Resonance Imaging (fMRI) and Electroencephalogram based on MI (MI-EEG) data. Then, Brain activation areas for each task are statistically analyzed by fMRI data. A novel fMRI-weighted Convolutional Neural Network (CNN) is designed to reassign each channel’s weight based on brain activation areas to improve classification accuracy. Finally, the EEG data is classified by the fMRI-weighted CNN. ResultsThe four classes of MI tasks reflect significant activation in brain motor areas. The average classification accuracy of fMRI-weighted CNN is 47.0%. The visualization results show the similarity between fMRI and EEG activation areas in the same task. ConclusionsThe distinguishability of multiple tasks and the influence on motor areas of the brain are confirmed by fMRI experiments. The classification accuracy of multitasks is improved according to the fMRI-weighted CNN. This paper provides a reference for further research into asynchronous fMRI-EEG modeling and multitask MI classification.

Embodiment Comfort Levels During Motor Imagery Training Combined With Immersive Virtual Reality in a Spinal Cord Injury Patient.

Brain–machine interfaces combining visual, auditory, and tactile feedback have been previously used to generate embodiment experiences during spinal cord injury (SCI) rehabilitation. It is not known if adding temperature to these modalities can result in discomfort with embodiment experiences. Here, comfort levels with the embodiment experiences were investigated in an intervention that required a chronic pain SCI patient to generate lower limb motor imagery commands in an immersive environment combining visual (virtual reality -VR), auditory, tactile, and thermal feedback. Assessments were made pre-/ post-, throughout the intervention (Weeks 0–5), and at 7 weeks follow up. Overall, high levels of embodiment in the adapted three-domain scale of embodiment were found throughout the sessions. No significant adverse effects of VR were reported. Although sessions induced only a modest reduction in pain levels, an overall reduction occurred in all pain scales (Faces, Intensity, and Verbal) at follow up. A high degree of comfort in the comfort scale for the thermal-tactile sleeve, in both the thermal and tactile feedback components of the sleeve was reported. This study supports the feasibility of combining multimodal stimulation involving visual (VR), auditory, tactile, and thermal feedback to generate embodiment experiences in neurorehabilitation programs.

Influence of EEG channel reduction on lower limb motor imagery during electrical stimulation in healthy and paraplegic subjects

Influence of EEG channel reduction on lower limb motor imagery during electrical stimulation in healthy and paraplegic subjects

Classification of lower limb motor imagery based on iterative EEG source localization and feature fusion

Motor imagery (MI) brain–computer interface (BCI) systems have broad application prospects in rehabilitation and other fields. However, to achieve accurate and practical MI-BCI applications, there are still several critical issues, such as channel selection, electroencephalogram (EEG) feature extraction and EEG classification, needed to be better resolved. In this paper, these issues are studied for lower limb MI which is more difficult and less studied than upper limb MI. First, a novel iterative EEG source localization method is proposed for channel selection. Channels FC1, FC2, C1, C2 and Cz, instead of the commonly used traditional channel set (TCS) C3, C4 and Cz, are selected as the optimal channel set (OCS). Then, a multi-domain feature (MDF) extraction algorithm is presented to fuse single-domain features into multi-domain features. Finally, a particle swarm optimization based support vector machine (SVM) method is utilized to classify the EEG data collected by the lower limb MI experiment designed by us. The results show that the classification accuracy is 88.43%, 3.35–5.41% higher than those of using traditional SVM to classify single-domain features on the TCS, which proves that the combination of OCS and MDF can not only reduce the amount of data processing, but also retain more feature information to improve the accuracy of EEG classification.

Home-based transcranial direct current stimulation (tDCS) and motor imagery for phantom limb pain using statistical learning to predict treatment response: an open-label study protocol.

Phantom limb pain (PLP) management has been a challenge due to its response heterogeneity and lack of treatment access. This study will evaluate the feasibility of a remotely home-based M1 anodal tDCS combined with motor imagery in phantom limb patients and assess the preliminary efficacy, safety, and predictors of response of this therapy. This is a pilot, single-arm, open-label trial in which we will recruit 10 subjects with phantom limb pain. The study will include 20 sessions. All participants will receive active anodal M1 tDCS combined with phantom limb motor imagery training. Our primary outcome will be the acceptability and feasibility of this combined intervention. Moreover, we will assess preliminary clinical (pain intensity) and physiological (motor inhibition tasks and heart rate variability) changes after treatment. Finally, we will implement a supervised statistical learning (SL) model to identify predictors of treatment response (to tDCS and phantom limb motor imagery) in PLP patients. We will also use data from our previous clinical trial (total observations=224 [n=112 x timepoints = 2)) for our statistical learning algorithms. The new prospective data from this open-label study will be used as an independent test dataset. This protocol proposes to assess the feasibility of a novel, neuromodulatory combined intervention that will allow the design of larger remote clinical trials, thus increasing access to safe and effective treatments for PLP patients. Moreover, this study will allow us to identify possible predictors of pain response and PLP clinical endotypes.

Home-based transcranial direct current stimulation (tDCS) and motor imagery for phantom limb pain using statistical learning to predict treatment response: an open-label study protocol

Background: Phantom limb pain (PLP) management has been a challenge due to its response heterogeneity and lack of treatment access. This study will evaluate the feasibility of a remotely home-based M1 anodal tDCS combined with motor imagery in phantom limb patients and assess the preliminary efficacy, safety, and predictors of response of this therapy. Methods: This is a pilot, single-arm, open-label trial in which we will recruit 10 subjects with phantom limb pain. The study will include 20 sessions. All participants will receive active anodal M1 tDCS combined with phantom limb motor imagery training. Our primary outcome will be the acceptability and feasibility of this combined intervention. Moreover, we will assess preliminary clinical (pain intensity) and physiological (motor inhibition tasks and heart rate variability) changes after treatment. Finally, we will implement a supervised statistical learning (SL) model to identify predictors of treatment response (to tDCS and phantom limb motor imagery) in PLP patients. We will also use data from our previous clinical trial (total observations=224 [n=112 x timepoints = 2)) for our statistical learning algorithms. The new prospective data from this open-label study will be used as an independent test dataset. Discussion: This protocol proposes to assess the feasibility of a novel, neuromodulatory combined intervention that will allow the design of larger remote clinical trials, thus increasing access to safe and effective treatments for PLP patients. Moreover, this study will allow us to identify possible predictors of pain response and PLP clinical endotypes. Keywords: phantom limb pain, home-based tDCS, motor imagery, statistical learning, remote trial.

A new 2-class unilateral upper limb motor imagery tasks for stroke rehabilitation training

The rehabilitation training based on motor imagery brain-computer interface (MI-BCI) is considered to be an effective method. We designed a new 2-class unilateral upper limb motor imagery tasks. Data from 15 healthy subjects and 10 stoke patients are collected in the study. The results of event-related desynchronization/synchronization (ERD/ERS) and power spectral density (PSD) analysis showed the significant different features on health subjects and stroke patients. The improved 2-Conv-FBCNET is used to classify Electroencephalogram (EEG) signals and the accuracy is (health: 61.0%, stroke: 59.4%). The new two types of tasks provide a new training method for MI-BCI rehabilitation training system.