Decoding Motor Imagery Research Articles

Diffusion models-based motor imagery EEG sample augmentation via mixup strategy

Deep representation learning has been widely explored for decoding motor imagery electroencephalogram (MI-EEG) to build EEG-tailored brain-computer interfaces. Due to the labor-intensive and time-consuming recording to MI-EEG, recently BCIs were suffered from sample scarcity, especially for the limitation of sample diversity. To solve this issue, we proposed a novel sample augmentation method, Diffusion models-based EEG Sample Augmentation method via Mixup strategy (DESAM), to solve sample scarcity and improve sample diversity for both multivariate time-series and time–frequency images forms. To reduce the characteristics conflicts brought by the augmented samples, a mixup strategy with data weighting was proposed for both forms. To validate the efficacy of sample augmentation by DESAM, we conducted comparative experiments on three publicly available MI-EEG datasets, and reported the average decoding accuracies and Kappa values on augmented samples by various deep learning models. For the BCI Competition IV datasets 2a, 2b, and 1, the two forms of DESAM sample augmentations achieved improvements for both accuracies and Kappa values. Ablation studies have demonstrated the necessity and significance of our DESAM method, and it makes a possible manner to solve sample scarcity issue of MI-EEG.

E-SAT: an extreme learning machine based self attention approach for decoding motor imagery EEG in subject-specific tasks

The advancements in Brain-Computer Interface (BCI) have substantially evolved people's lives by enabling direct communication between the human brain and external peripheral devices. In recent years, the integration of machine larning (ML) and deep learning (DL) models have considerably imrpoved the performances of BCIs for decoding the motor imagery (MI) tasks. However, there still exist several limitations, e.g., extensive training time and high sensitivity to noises or outliers with those existing models, which largely hinder the rapid developments of BCIs. To address such issues, this paper proposes a novel extreme learning machine (ELM) based self-attention (E-SAT) mechanism to enhance subject-specific classification performances. Specifically, for E-SAT, ELM is employed both to imrpove self-attention module generalization ability for feature extraction and to optimize the model's parameter initialization process. Meanwhile, the extracted features are also classified using ELM, and the end-to-end ELM based setup is used to evaluate E-SAT performances on different MI EEG signals. Extensive experiments with different datasets, such as BCI Competition III Dataset IV-a, IV-b and BCI Competition IV Datasets 1,2a,2b,3, are conducted to verify the effectiveness of proposed E-SAT strategy. Results show that E-SAT outperforms several state-of-the-art (SOTA) existing methods in subject-specific classification on all the datasets, with an average classification accuracy of 99.8%,99.1%,98.9%,75.8%, 90.8%, and 95.4%, being achieved for each datasets, respectively. The experimental results not only show outstanding performance of E-SAT in feature extractions, but also demonstrate that it helps achieves the best results among nine other robust ones. In addition, results in this study also demonstrate that E-SAT achieves exceptional performance in both binary and multi-class classification tasks, as well as for noisy and non-noisy datatsets.&#xD.

Cross-Channel Specific-Mutual Feature Transfer Learning for Motor Imagery EEG Signals Decoding.

In recent years, with the rapid development of deep learning, various deep learning frameworks have been widely used in brain-computer interface (BCI) research for decoding motor imagery (MI) electroencephalogram (EEG) signals to understand brain activity accurately. The electrodes, however, record the mixed activities of neurons. If different features are directly embedded in the same feature space, the specific and mutual features of different neuron regions are not considered, which will reduce the expression ability of the feature itself. We propose a cross-channel specific-mutual feature transfer learning (CCSM-FT) network model to solve this problem. The multibranch network extracts the specific and mutual features of brain's multiregion signals. Effective training tricks are used to maximize the distinction between the two kinds of features. Suitable training tricks can also improve the effectiveness of the algorithm compared with novel models. Finally, we transfer two kinds of features to explore the potential of mutual and specific features to enhance the expressive power of the feature and use the auxiliary set to improve identification performance. The experimental results show that the network has a better classification effect in the BCI Competition IV-2a and the HGD datasets.

Decoding motor imagery loaded on steady-state somatosensory evoked potential based on complex task-related component analysis

Background and objectiveMotor Imagery (MI) recognition is one of the most critical decoding problems in brain- computer interface field. Combined with the steady-state somatosensory evoked potential (MI-SSSEP), this new paradigm can achieve higher recognition accuracy than the traditional MI paradigm. Typical algorithms do not fully consider the characteristics of MI-SSSEP signals. Developing an algorithm that fully captures the paradigm's characteristics to reduce false triggering rate is the new step in improving performance. MethodsThe idea to use complex signal task-related component analysis (cTRCA) algorithm for spatial filtering processing has been proposed in this paper according to the features of SSSEP signal. In this research, it's proved from the analysis of simulation signals that task-related component analysis (TRCA) as typical method is affected when the response between stimuli has reduced correlation and the proposed algorithm can effectively overcome this problem. The experimental data under the MI-SSSEP paradigm have been used to identify right-handed target tasks and three unique interference tasks are used to test the false triggering rate. cTRCA demonstrates superior performance as confirmed by the Wilcoxon signed-rank test. ResultsThe recognition algorithm of cTRCA combined with mutual information-based best individual feature (MIBIF) and minimum distance to mean (MDM) can obtain AUC value up to 0.89, which is much higher than traditional algorithm common spatial pattern (CSP) combined with support vector machine (SVM) (the average AUC value is 0.77, p < 0.05). Compared to CSP+SVM, this algorithm model reduced the false triggering rate from 38.69 % to 20.74 % (p < 0.001). ConclusionsThe research prove that TRCA is influenced by MI-SSSEP signals. The results further prove that the motor imagery task in the new paradigm MI-SSSEP causes the phase change in evoked potential. and the cTRCA algorithm based on such phase change is more suitable for this hybrid paradigm and more conducive to decoding the motor imagery task and reducing false triggering rate.

Efficient Multi-View Graph Convolutional Network with Self-Attention for Multi-Class Motor Imagery Decoding.

Research on electroencephalogram-based motor imagery (MI-EEG) can identify the limbs of subjects that generate motor imagination by decoding EEG signals, which is an important issue in the field of brain-computer interface (BCI). Existing deep-learning-based classification methods have not been able to entirely employ the topological information among brain regions, and thus, the classification performance needs further improving. In this paper, we propose a multi-view graph convolutional attention network (MGCANet) with residual learning structure for multi-class MI decoding. Specifically, we design a multi-view graph convolution spatial feature extraction method based on the topological relationship of brain regions to achieve more comprehensive information aggregation. During the modeling, we build an adaptive weight fusion (Awf) module to adaptively merge feature from different brain views to improve classification accuracy. In addition, the self-attention mechanism is introduced for feature selection to expand the receptive field of EEG signals to global dependence and enhance the expression of important features. The proposed model is experimentally evaluated on two public MI datasets and achieved a mean accuracy of 78.26% (BCIC IV 2a dataset) and 73.68% (OpenBMI dataset), which significantly outperforms representative comparative methods in classification accuracy. Comprehensive experiment results verify the effectiveness of our proposed method, which can provide novel perspectives for MI decoding.

Deep learning in motor imagery EEG signal decoding: A Systematic Review

Thanks to the fast evolution of electroencephalography (EEG)-based brain–computer interfaces (BCIs) and computing technologies, as well as the availability of large EEG datasets, decoding motor imagery (MI) EEG signals is rapidly shifting from traditional machine learning (ML) to deep learning (DL) approaches. Furthermore, real-world MI-EEG BCI applications are progressively requiring higher generalization capabilities, which can be achieved by leveraging publicly available MI-EEG datasets and high-performance decoding models. Within this context, this paper provides a systematic review of DL approaches for MI-EEG decoding, focusing on studies that work on publicly available EEG-MI datasets. This review paper firstly provides a clear overview of these datasets that can be used for DL model training and testing. Afterwards, considering each dataset, related DL studies are discussed with respect to the four decoding paradigms identified in the literature, i.e., subject-dependent, subject-independent, transfer learning, and global decoding paradigms. Having analyzed the reviewed studies, the current trends and strategies, popular architectures, baseline models that are used for comprehensive analysis, and techniques to ensure reproducibility of the results in DL-based MI-EEG decoding are also identified and discussed. The selection and screening of the studies included in this review follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, leading to a comprehensive analysis of 394 papers published between January 1, 2017, and January 23, 2023.

A novel dual-step transfer framework based on domain selection and feature alignment for motor imagery decoding

A novel dual-step transfer framework based on domain selection and feature alignment for motor imagery decoding

EEG motor imagery decoding: a framework for comparative analysis with channel attention mechanisms

Objective. The objective of this study is to investigate the application of various channel attention mechanisms within the domain of brain-computer interface (BCI) for motor imagery decoding. Channel attention mechanisms can be seen as a powerful evolution of spatial filters traditionally used for motor imagery decoding. This study systematically compares such mechanisms by integrating them into a lightweight architecture framework to evaluate their impact. Approach. We carefully construct a straightforward and lightweight baseline architecture designed to seamlessly integrate different channel attention mechanisms. This approach is contrary to previous works which only investigate one attention mechanism and usually build a very complex, sometimes nested architecture. Our framework allows us to evaluate and compare the impact of different attention mechanisms under the same circumstances. The easy integration of different channel attention mechanisms as well as the low computational complexity enables us to conduct a wide range of experiments on four datasets to thoroughly assess the effectiveness of the baseline model and the attention mechanisms. Results. Our experiments demonstrate the strength and generalizability of our architecture framework as well as how channel attention mechanisms can improve the performance while maintaining the small memory footprint and low computational complexity of our baseline architecture. Significance. Our architecture emphasizes simplicity, offering easy integration of channel attention mechanisms, while maintaining a high degree of generalizability across datasets, making it a versatile and efficient solution for electroencephalogram motor imagery decoding within BCIs.

Manifold attention-enhanced multi-domain convolutional network for decoding motor imagery intention

Research in brain-computer interface (BCI), particularly in brain intention decoding, has made significant strides relying on the remarkable capabilities of deep learning (DL). However, many current DL framework-based models struggle to effectively utilize complementary information present in various domains of electroencephalographic (EEG) signals. This limitation may lead to suboptimal performance, particularly exacerbated by the presence of noise and artifacts. Therefore, this study proposed a manifold attention-based multi-domain convolutional neural network, known as MAMCNet, to benefit from complementary strengths across different domains of EEG signals. Specifically, the EEG signals were divided into multi-band data representations based on frequency. Multiple spatiotemporal convolutional blocks were then devised to simultaneously extract the spatiotemporal feature maps from these sub-band components. To enhance the model's robustness, these maps were mapped onto the Riemannian manifold. Additionally, manifold attention was utilized to emphasize salient spatiotemporal feature maps with attention focused on the target. A frequency convolutional block was specifically designed to merge optimized feature maps from different bands, extracting temporospatiofrequency features in EEG signals comprehensively. The experimental results on two datasets demonstrated that the proposed model outperformed the mainstream DL architecture-based model. These results suggest that the proposed model could offer an alternative scheme for facilitating BCI applications.

Motor imagery decoding using source optimized transfer learning based on multi-loss fusion CNN

Motor imagery decoding using source optimized transfer learning based on multi-loss fusion CNN

Self-supervised contrastive learning for EEG-based cross-subject motor imagery recognition.

Objective. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals.Approach. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network's ability to extract deep features from original signals without relying on the true labels of the data.Main results. To evaluate our framework's efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32%, 82.34%, and 81.13%on the three datasets, demonstrating superior performance compared to existing methods.Significance. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.

On decoding of rapid motor imagery in a diverse population using a high-density NIRS device.

Functional near-infrared spectroscopy (fNIRS) aims to infer cognitive states such as the type of movement imagined by a study participant in a given trial using an optical method that can differentiate between oxygenation states of blood in the brain and thereby indirectly between neuronal activity levels. We present findings from an fNIRS study that aimed to test the applicability of a high-density (>3000 channels) NIRS device for use in short-duration (2 s) left/right hand motor imagery decoding in a diverse, but not explicitly balanced, subject population. A side aim was to assess relationships between data quality, self-reported demographic characteristics, and brain-computer interface (BCI) performance, with no subjects rejected from recruitment or analysis. BCI performance was quantified using several published methods, including subject-specific and subject-independent approaches, along with a high-density fNIRS decoder previously validated in a separate study. We found that decoding of motor imagery on this population proved extremely challenging across all tested methods. Overall accuracy of the best-performing method (the high-density decoder) was 59.1 +/- 6.7% after excluding subjects where almost no optode-scalp contact was made over motor cortex and 54.7 +/- 7.6% when all recorded sessions were included. Deeper investigation revealed that signal quality, hemodynamic responses, and BCI performance were all strongly impacted by the hair phenotypical and demographic factors under investigation, with over half of variance in signal quality explained by demographic factors alone. Our results contribute to the literature reporting on challenges in using current-generation NIRS devices on subjects with long, dense, dark, and less pliable hair types along with the resulting potential for bias. Our findings confirm the need for increased focus on these populations, accurate reporting of data rejection choices across subject intake, curation, and final analysis in general, and signal a need for NIRS optode designs better optimized for the general population to facilitate more robust and inclusive research outcomes.

Exploring EEG-based motor imagery decoding: a dual approach using spatial features and spectro-spatial Deep Learning model IFNet.

In recent years, the decoding of motor imagery (MI) from electroencephalography (EEG) signals has become a focus of research for brain-machine interfaces (BMIs) and neurorehabilitation. However, EEG signals present challenges due to their non-stationarity and the substantial presence of noise commonly found in recordings, making it difficult to design highly effective decoding algorithms. These algorithms are vital for controlling devices in neurorehabilitation tasks, as they activate the patient's motor cortex and contribute to their recovery. This study proposes a novel approach for decoding MI during pedalling tasks using EEG signals. A widespread approach is based on feature extraction using Common Spatial Patterns (CSP) followed by a linear discriminant analysis (LDA) as a classifier. The first approach covered in this work aims to investigate the efficacy of a task-discriminative feature extraction method based on CSP filter and LDA classifier. Additionally, the second alternative hypothesis explores the potential of a spectro-spatial Convolutional Neural Network (CNN) to further enhance the performance of the first approach. The proposed CNN architecture combines a preprocessing pipeline based on filter banks in the frequency domain with a convolutional neural network for spectro-temporal and spectro-spatial feature extraction. To evaluate the approaches and their advantages and disadvantages, EEG data has been recorded from several able-bodied users while pedalling in a cycle ergometer in order to train motor imagery decoding models. The results show levels of accuracy up to 80% in some cases. The CNN approach shows greater accuracy despite higher instability.

Adaptive filter of frequency bands based coordinate attention network for EEG-based motor imagery classification.

In the brain-computer interface (BCI), motor imagery (MI) could be defined as the Electroencephalogram (EEG) signals through imagined movements, and ultimately enabling individuals to control external devices. However, the low signal-to-noise ratio, multiple channels and non-linearity are the essential challenges of accurate MI classification. To tackle these issues, we investigate the role of adaptive frequency bands selection and spatial-temporal feature learning in decoding motor imagery. We propose an Adaptive Filter of Frequency Bands based Coordinate Attention Network (AFFB-CAN) to improve the performance of MI classification. Specifically, we design the AFFB to adaptively obtain the upper and lower limits of frequency bands in order to alleviate information loss caused by manual selection. Next, we propose the CAN-based network to emphasize the key brain regions and temporal segments. And we design a multi-scale module to enhance temporal context learning. The conducted experiments on the BCI Competition IV-2a and 2b datasets reveal that our approach achieves an outstanding average accuracy, kappa values, and Macro F1-Score with 0.7825, 0.7104, and 0.7486 respectively. Similarly, for the BCI Competition IV-2b dataset, the average accuracy, kappa values, and F1-Score obtained are 0.8879, 0.7427, and 0.8734, respectively. The proposed AFFB-CAN method improves the performance of MI classification. In addition, our study confirms previous findings that motor imagery is mainly associated with µ and β rhythms. Moreover, we also find that γ rhythms also play an important role in MI classification.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

M-FANet: Multi-Feature Attention Convolutional Neural Network for Motor Imagery Decoding.

Motor imagery (MI) decoding methods are pivotal in advancing rehabilitation and motor control research. Effective extraction of spectral-spatial-temporal features is crucial for MI decoding from limited and low signal-to-noise ratio electroencephalogram (EEG) signal samples based on brain-computer interface (BCI). In this paper, we propose a lightweight Multi-Feature Attention Neural Network (M-FANet) for feature extraction and selection of multi-feature data. M-FANet employs several unique attention modules to eliminate redundant information in the frequency domain, enhance local spatial feature extraction and calibrate feature maps. We introduce a training method called Regularized Dropout (R-Drop) to address training-inference inconsistency caused by dropout and improve the model's generalization capability. We conduct extensive experiments on the BCI Competition IV 2a (BCIC-IV-2a) dataset and the 2019 World robot conference contest-BCI Robot Contest MI (WBCIC-MI) dataset. M-FANet achieves superior performance compared to state-of-the-art MI decoding methods, with 79.28% 4-class classification accuracy (kappa: 0.7259) on the BCIC-IV-2a dataset and 77.86% 3-class classification accuracy (kappa: 0.6650) on the WBCIC-MI dataset. The application of multi-feature attention modules and R-Drop in our lightweight model significantly enhances its performance, validated through comprehensive ablation experiments and visualizations.

Cortical ROI importance improves MI decoding from EEG using fused light neural network.

Decoding motor imagery (MI) using deep learning in cortical level has potential in brain computer interface based intelligent rehabilitation. However, a mass of dipoles is inconvenient to extract the personalized features and requires a more complex neural network. In consideration of the structural and functional similarity of the neurons in a neuroanatomical region, i.e., a region of interest (ROI), we propose that the comprehensive performance of each ROI may be reflected by a specific representative dipole (RD), and the time-frequency spectrums of all RDs are applied simultaneously to Random Forest algorithm to give a quantitative metric of each ROI importance (RI). Then, the more divided sub-band spectral powers are reinforced by RI, and they are interpolated to a 2-dimensional (2D) plane transformed from 3D space of all RDs, yielding an ensemble representation of RD feature image sequences (ERDFIS). Furthermore, a lightweight network, including 2D separable convolution and gated recurrent unit (2DSCG), is developed to extract and classify the frequency-spatial and temporal features from ERDFIS, forming a novel MI decoding method in cortical level (called ERDFIS-2DSCG). Based on two public datasets, the decoding accuracies of ten-fold cross-validation are 89.89% and 94.35%, respectively. The results suggest that RD can embody the overall property of ROI in time-frequency-space domains, and ROI importance is helpful to highlight the subject-based characteristics of MI-EEG. Meanwhile, 2DSCG is matched well with ERDFIS, jointly improving the decoding performance.

A Strong and Simple Deep Learning Baseline for BCI Motor Imagery Decoding.

We propose EEG-SimpleConv, a straightforward 1D convolutional neural network for Motor Imagery decoding in BCI. Our main motivation is to propose a simple and performing baseline that achieves high classification accuracy, using only standard ingredients from the literature, to serve as a standard for comparison. The proposed architecture is composed of standard layers, including 1D convolutions, batch normalisations, ReLU activation functions and pooling functions. EEG-SimpleConv architecture is accompanied by a straightforward and tailored training routine, which is subjected to an extensive ablation study to quantify the influence of its components. We evaluate its performance on four EEG Motor Imagery datasets, including simulated online setups, and compare it to recent Deep Learning and Machine Learning approaches. EEG-SimpleConv is at least as good or far more efficient than other approaches, showing strong knowledge-transfer capabilities across subjects, at the cost of a low inference time. We believe that using standard components and ingredients can significantly help the adoption of Deep Learning approaches for BCI. We make the code of the models and the experiments accessible.

MSVTNet: Multi-Scale Vision Transformer Neural Network for EEG-Based Motor Imagery Decoding.

Transformer-based neural networks have been applied to the electroencephalography (EEG) decoding for motor imagery (MI). However, most networks focus on applying the self-attention mechanism to extract global temporal information, while the cross-frequency coupling features between different frequencies have been neglected. Additionally, effectively integrating different neural networks poses challenges for the advanced design of decoding algorithms. This study proposes a novel end-to-end Multi-Scale Vision Transformer Neural Network (MSVTNet) for MI-EEG classification. MSVTNet first extracts local spatio-temporal features at different filtered scales through convolutional neural networks (CNNs). Then, these features are concatenated along the feature dimension to form local multi-scale spatio-temporal feature tokens. Finally, Transformers are utilized to capture cross-scale interaction information and global temporal correlations, providing more distinguishable feature embeddings for classification. Moreover, auxiliary branch loss is leveraged for intermediate supervision to ensure the effective integration of CNNs and Transformers. The performance of MSVTNet was assessed through subject-dependent (session-dependent and session-independent) and subject-independent experiments on three MI datasets, i.e., the BCI competition IV 2a, 2b and OpenBMI datasets. The experimental results demonstrate that MSVTNet achieves state-of-the-art performance in all analyses. MSVTNet shows superiority and robustness in enhancing MI decoding performance. The source code for MSVTNet is available at https://github.com/SheepTAO/MSVTNet.

Motor Imagery Classification for Asynchronous EEG-Based Brain-Computer Interfaces.

Motor imagery (MI) based brain-computer interfaces (BCIs) enable the direct control of external devices through the imagined movements of various body parts. Unlike previous systems that used fixed-length EEG trials for MI decoding, asynchronous BCIs aim to detect the user's MI without explicit triggers. They are challenging to implement, because the algorithm needs to first distinguish between resting-states and MI trials, and then classify the MI trials into the correct task, all without any triggers. This paper proposes a sliding window prescreening and classification (SWPC) approach for MI-based asynchronous BCIs, which consists of two modules: a prescreening module to screen MI trials out of the resting-state, and a classification module for MI classification. Both modules are trained with supervised learning followed by self-supervised learning, which refines the feature extractors. Within-subject and cross-subject asynchronous MI classifications on four different EEG datasets validated the effectiveness of SWPC, i.e., it always achieved the highest average classification accuracy, and outperformed the best state-of-the-art baseline on each dataset by about 2%.