Motor Imagery Electroencephalography Signals Research Articles

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Neurophysiologically Meaningful Motor Imagery EEG Simulation With Applications to Data Augmentation.

Motor imagery-based Brain-Computer Interfaces (MI-BCIs) have gained a lot of attention due to their potential usability in neurorehabilitation and neuroprosthetics. However, the accurate recognition of MI patterns in electroencephalography signals (EEG) is hindered by several data-related limitations, which restrict the practical utilization of these systems. Moreover, leveraging deep learning (DL) models for MI decoding is challenged by the difficulty of accessing user-specific MI-EEG data on large scales. Simulated MI-EEG signals can be useful to address these issues, providing well-defined data for the validation of decoding models and serving as a data augmentation approach to improve the training of DL models. While substantial efforts have been dedicated to implementing effective data augmentation strategies and model-based EEG signal generation, the simulation of neurophysiologically plausible EEG-like signals has not yet been exploited in the context of data augmentation. Furthermore, none of the existing approaches have integrated user-specific neurophysiological information during the data generation process. Here, we present PySimMIBCI, a framework for generating realistic MI-EEG signals by integrating neurophysiologically meaningful activity into biophysical forward models. By means of PySimMIBCI, different user capabilities to control an MI-BCI can be simulated and fatigue effects can be included in the generated EEG. Results show that our simulated data closely resemble real data. Moreover, a proposed data augmentation strategy based on our simulated user-specific data significantly outperforms other state-of-the-art augmentation approaches, enhancing DL models performance by up to 15%.

Riemannian geometric and ensemble learning for decoding cross-session motor imagery electroencephalography signals

Objective. Brain–computer interfaces (BCIs) enable a direct communication pathway between the human brain and external devices, without relying on the traditional peripheral nervous and musculoskeletal systems. Motor imagery (MI)-based BCIs have attracted significant interest for their potential in motor rehabilitation. However, current algorithms fail to account for the cross-session variability of electroencephalography signals, limiting their practical application. Approach. We proposed a Riemannian geometry-based adaptive boosting and voting ensemble (RAVE) algorithm to address this issue. Our approach segmented the MI period into multiple sub-datasets using a sliding window approach and extracted features from each sub-dataset using Riemannian geometry. We then trained adaptive boosting (AdaBoost) ensemble learning classifiers for each sub-dataset, with the final BCI output determined by majority voting of all classifiers. We tested our proposed RAVE algorithm and eight other competing algorithms on four datasets (Pan2023, BNCI001-2014, BNCI001-2015, BNCI004-2015). Main results. Our results showed that, in the cross-session scenario, the RAVE algorithm outperformed the eight other competing algorithms significantly under different within-session training sample sizes. Compared to traditional algorithms that involved a large number of training samples, the RAVE algorithm achieved similar or even better classification performance on the datasets (Pan2023, BNCI001-2014, BNCI001-2015), even when it did not use or only used a small number of within-session training samples. Significance. These findings indicate that our cross-session decoding strategy could enable MI-BCI applications that require no or minimal training process.

Multi-scale feature extraction and classification of motor imagery electroencephalography based on time series data enhancement

The brain-computer interface (BCI) based on motor imagery electroencephalography (MI-EEG) enables direct information interaction between the human brain and external devices. In this paper, a multi-scale EEG feature extraction convolutional neural network model based on time series data enhancement is proposed for decoding MI-EEG signals. First, an EEG signals augmentation method was proposed that could increase the information content of training samples without changing the length of the time series, while retaining its original features completely. Then, multiple holistic and detailed features of the EEG data were adaptively extracted by multi-scale convolution module, and the features were fused and filtered by parallel residual module and channel attention. Finally, classification results were output by a fully connected network. The application experimental results on the BCI Competition IV 2a and 2b datasets showed that the proposed model achieved an average classification accuracy of 91.87% and 87.85% for the motor imagery task, respectively, which had high accuracy and strong robustness compared with existing baseline models. The proposed model does not require complex signals pre-processing operations and has the advantage of multi-scale feature extraction, which has high practical application value.

Classification of Motor Imagery EEG Signals Based on Data Augmentation and Convolutional Neural Networks.

In brain-computer interface (BCI) systems, motor imagery electroencephalography (MI-EEG) signals are commonly used to detect participant intent. Many factors, including low signal-to-noise ratios and few high-quality samples, make MI classification difficult. In order for BCI systems to function, MI-EEG signals must be studied. In pattern recognition and other fields, deep learning approaches have recently been successfully applied. In contrast, few effective deep learning algorithms have been applied to BCI systems, especially MI-based systems. In this paper, we address these problems from two aspects based on the characteristics of EEG signals: first, we proposed a combined time-frequency domain data enhancement method. This method guarantees that the size of the training data is effectively increased while maintaining the intrinsic composition of the data. Second, our design consists of a parallel CNN that takes both raw EEG images and images transformed through continuous wavelet transform (CWT) as inputs. We conducted classification experiments on a public data set to verify the effectiveness of the algorithm. According to experimental results based on the BCI Competition IV Dataset2a, the average classification accuracy is 97.61%. A comparison of the proposed algorithm with other algorithms shows that it performs better in classification. The algorithm can be used to improve the classification performance of MI-based BCIs and BCI systems created for people with disabilities.

Channels Selection for Pattern Recognition of Five Fingers Motor Imagery Electroencephalography Signals

Research about pattern recognition on electroencephalography (EEG) signal of finger motor imagery (MI) plays a critical role in Brain-Computer Interfaces (BCI) based hand prosthetics development. However, the previous research still used irrelevant channels to finger MI. This work proposed optimal EEG channels combination for five-finger MI. It is achieved by subject-dependence channel selection using One versus Rest Common Spatial Pattern (CSP-OVR) combined with sequential searching algorithms due to specific neural activation areas of MI. Optimal channels combinations are of great importance to reduce channels number. It supports the development of practical BCI-based hand prosthetics that can help hand handicapped to do daily activities easier. Experimental results show 4 out of 19 channels are relevant to five-finger MI with 0,6% accuracy degradation compared with EEG-MI pattern recognition using 19 channels. This result is better than the Principal Component Analysis (PCA) channel selection method that only selects 11 out of 19 channels with 1 % accuracy degradation.

A Pruned Deep Learning Approach for Classification of Motor Imagery Electroencephalography Signals.

The Deep Learning (DL) approach has been gaining much popularity in recent years in the development of electroencephalogram (EEG) based Motor Imagery (MI) Brain-Computer Interface (BCI) systems, aiming to improve the performance of existing stroke rehabilitation strategies. A complex deep neural network structure has lots of neurons with thousands of parameters to optimize, and a great deal of data is often required to train the network and the training process can take an extremely long time. High training costs and high model complexity not only have negative impacts on the performance of the BCI system but also on its applicability to meet the real-time requirement to support the rehabilitation exercises of patients. To tackle the challenge, a contribution-based neuron selection method is proposed in this paper. A Convolutional Neural Network (CNN) based motor imagery classification framework is implemented, and a neuron pruning approach is developed and applied. The temporal and spatial features of EEG signals are captured by the CNN layers, and then the fast recursive algorithm (FRA) is applied to prune redundant parameters in the fully connected layers which reduces the computation cost of the CNN model without affecting its performance. The experimental results show that the proposed method can achieve up to 50% model size reduction and 67.09% computation savings.

Transfer learning algorithm design for feature transfer problem in motor imagery brain-computer interface

The non-stationary of the motor imagery electroencephalography(MI-EEG) signal is one of the main limitations for the development of motor imagery brain-computer interfaces(MI-BCI). The non-stationary of the MI-EEG signal and the changes of the experimental environment make the feature distribution of the testing set and training set deviates, which reduces the classification accuracy of MI-BCI. In this paper, we propose a Kullback-Leibler divergence (KL)-based transfer learning algorithm to solve the problem of feature transfer, the proposed algorithm uses KL to measure the similarity between the training set and the testing set, adds support vector machine (SVM) classification probability to classify and weight the covariance, and discards the poorly performing samples. The results show that the proposed algorithm can significantly improve the classification accuracy of the testing set compared with the traditional algorithms, especially for subjects with medium classification accuracy. Moreover, the algorithm based on transfer learning has the potential to improve the consistency of feature distribution that the traditional algorithms do not have, which is significant for the application of MI-BCI.

A multiscale siamese convolutional neural network with cross-channel fusion for motor imagery decoding

BackgroundRecently, convolutional neural networks (CNN) are widely applied in motor imagery electroencephalography (MI-EEG) signal classification tasks. However, a simple CNN framework is challenging to satisfy the complex MI-EEG signal decoding. New methodIn this study, we propose a multiscale Siamese convolutional neural network with cross-channel fusion (MSCCF-Net) for MI-EEG classification tasks. The proposed network consists of three parts: Siamese cross-channel fusion streams, similarity module and classification module. Each Siamese cross-channel fusion stream contains multiple branches, and each branch is supplemented by cross-channel fusion modules to improve multiscale temporal feature representation capability. The similarity module is adopted to measure the feature similarity between multiple branches. At the same time, the classification module provides a strong constraint to classify the features from all Siamese cross-channel fusion streams. The combination of the similarity module and classification module constitutes a new joint training strategy to further optimize the network performance. ResultsThe experiment is conducted on the public BCI Competition IV 2a and 2b datasets, and the results show that the proposed network achieves an average accuracy of 87.36% and 87.33%, respectively. Comparison with existing methods and ConclusionsThe proposed network adopts cross-channel fusion to learn multiscale temporal characteristics and joint training strategy to optimize the training process. Therefore, the performance outperforms other state-of-the-art MI-EEG signal classification methods.

Classification of Motor Imagery Electroencephalography Signals Based on Image Processing Method.

In recent years, more and more frameworks have been applied to brain-computer interface technology, and electroencephalogram-based motor imagery (MI-EEG) is developing rapidly. However, it is still a challenge to improve the accuracy of MI-EEG classification. A deep learning framework termed IS-CBAM-convolutional neural network (CNN) is proposed to address the non-stationary nature, the temporal localization of excitation occurrence, and the frequency band distribution characteristics of the MI-EEG signal in this paper. First, according to the logically symmetrical relationship between the C3 and C4 channels, the result of the time-frequency image subtraction (IS) for the MI-EEG signal is used as the input of the classifier. It both reduces the redundancy and increases the feature differences of the input data. Second, the attention module is added to the classifier. A convolutional neural network is built as the base classifier, and information on the temporal location and frequency distribution of MI-EEG signal occurrences are adaptively extracted by introducing the Convolutional Block Attention Module (CBAM). This approach reduces irrelevant noise interference while increasing the robustness of the pattern. The performance of the framework was evaluated on BCI competition IV dataset 2b, where the mean accuracy reached 79.6%, and the average kappa value reached 0.592. The experimental results validate the feasibility of the framework and show the performance improvement of MI-EEG signal classification.

Electroencephalography-based motor imagery classification using temporal convolutional network fusion

Motor imagery electroencephalography (MI-EEG) signals are generated when a person imagines a task without actually performing it. In recent studies, MI-EEG has been used in the rehabilitation process of paralyzed patients, therefore, decoding MI-EEG signals accurately is an important task, and it is difficult task due to the low signal-to-noise ratio and the variation of brain waves between subjects. Deep learning techniques such as the convolution neural network (CNN) have shown an impact in extracting meaningful features to improve the accuracy of classification. In this paper, we propose TCNet-Fusion, a fixed hyperparameter-based CNN model that utilizes multiple techniques, such as temporal convolutional networks (TCNs), separable convolution, depth-wise convolution, and the fusion of layers. This model outperforms other fixed hyperparameter-based CNN models while remaining similar to those that utilize variable hyperparameter networks, which are networks that change their hyperparameters based on each subject, resulting in higher accuracy than fixed networks. It also uses less memory than variable networks. The EEG signal undergoes two successive 1D convolutions, first along with the time domain, then channel-wise. Then, we obtain an image-like representation, which is fed to the main TCN. During experimentation, the model achieved a classification accuracy of 83.73 % on the four-class MI of the BCI Competition IV-2a dataset, and an accuracy of 94.41 % on the High Gamma Dataset.

Key Band Image Sequences and A Hybrid Deep Neural Network for Recognition of Motor Imagery EEG

Deep neural network is a promising method to recognize motor imagery electroencephalography (MI-EEG), which is often used as the source signal of a rehabilitation system; and the core issues are the data representation and the matched neural networks. MI-EEG images is one of the main expressions, however, all the measured data of a trial are usually integrated into one image, causing information loss, especially in the time dimension; and the neural network architecture might not fully extract the features over the α and β frequency bands, which are closely related to MI. In this paper, we propose a key band imaging method (KBIM). A short time Fourier transform is applied to each electrode of the MI-EEG signal to generate a time-frequency image, and the parts corresponding to the α and β bands are intercepted, fused, and further arranged into the EEG electrode map by the nearest neighbor interpolation method, forming two key band image sequences. In addition, a hybrid deep neural network named the parallel multimodule convolutional neural network and long short-term memory network (PMMCL) is designed for the extraction and fusion of the spatial-spectral and temporal features of two key band image sequences to realize automatic classification of MI-EEG signals. Extensive experiments are conducted on two public datasets, and the accuracies after 10-fold cross-validation are 97.42% and 77.33%, respectively. Statistical analysis shows the superb discrimination ability for multiclass MI-EEG too. The results demonstrate that KBIM can preserve the integrity of the feature information, and they well match with PMMCL.

Classification of motor imagery electroencephalography signals using continuous small convolutional neural network

AbstractAs an important part of brain‐computer interface (BCI), the electroencephalography (EEG) technology of motor imagery (MI) has been gradually recognized for its great theoretical value and practical application. In this study, in view of the different MI tasks corresponding to active region of the EEG signals, we adopt a two‐dimensional form including time, frequency, and electrode location information, then we design a classification method containing continuous small convolutional neural network (CSCNN). This method is mainly used for feature extraction through continuous small convolutional kernels and one rectangle convolutional kernel, and the softmax classifier for classification. In the experiment, classification accuracy and kappa value are used as evaluation criteria to verify the effectiveness of the method proposed in this study. For classification accuracy, BCI competition IV data set 2b is used to compare with the other five classification methods (CNN, CNN‐SAE, stacked autoencoder [SAE], support vector machine [SVM], and one‐dimensional convolution combined with gated recurrent unit [1DCGRU]). The results demonstrate that the overall accuracy of CSCNN is higher than other methods, and CSCNN obtains an average accuracy of 82.8%. For kappa value, BCI competition IV data set 2b is used to compare with the other three methods (filter bank common spatial pattern [FBCSP], Twin SVM, and CNN‐SAE). The performance of CSCNN is better with an average value of 0.663. Overall, the results show that CSCNN maintains a small number of parameters and improves the classification accuracy.

Classification of Non-Discriminant ERD/ERS Comprising Motor Imagery Electroencephalography Signals

Classification of Motor Imagery (MI) Electroencephalography (EEG) signals has always been an important aspect of Brain Computer Interface (BCI) systems. Event Related Desynchronization (ERD)/ Event Related Synchronization (ERS) plays a significant role in finding discriminant features of MI EEG signals. ERD/ERS is one type and Evoked Potential (EP) is another type of brain response. This study focuses upon the classification of MI EEG signals by Removing Evoked Potential (REP) from non-discriminant MI EEG data in filter band selection, called REP. This optimization is done to enhance the classification performance. A comprehensive comparison of several pipelines is presented by using famous feature extraction methods, namely Common Spatial Pattern (CSP), XDawn. The effectiveness of REP is demonstrated on the PhysioNet dataset which is an online data resource. Comparison is done between the performance of pipelines including proposed one (Common Spatial Pattern (CSP) and Gaussian Process Classifier (GPC)) as well as before and after applying REP. It is observed that the REP approach has improved the classification accuracy of all the subjects used as well as all the pipelines, including state of the art algorithms, up to 20%.

A Novel Deep Learning Scheme for Motor Imagery EEG Decoding Based on Spatial Representation Fusion

Motor imagery electroencephalography (MI-EEG), which is an important subfield of active brain-computer interface (BCI) systems, can be applied to help disabled people to consciously and directly control prosthesis or external devices, aiding them in certain daily activities. However, the low signal-to-noise ratio and spatial resolution make MI-EEG decoding a challenging task. Recently, some deep neural approaches have shown good improvements over state-of-the-art BCI methods. In this study, an end-to-end scheme that includes a multi-layer convolution neural network is constructed for an accurate spatial representation of multi-channel grouped MI-EEG signals, which is employed to extract the useful information present in a multi-channel MI signal. Then the invariant spatial representations are captured from across-subjects training for enhancing the generalization capability through a stacked sparse autoencoder framework, which is inspired by representative deep learning models. Furthermore, a quantitative experimental analysis is conducted on our private dataset and on a public BCI competition dataset. The results show the effectiveness and significance of the proposed methodology.

A novel end‐to‐end deep learning scheme for classifying multi‐class motor imagery electroencephalography signals

AbstractAn important subfield of brain–computer interface is the classification of motor imagery (MI) signals where a presumed action, for example, imagining the hands' motions, is mentally simulated. The brain dynamics of MI is usually measured by electroencephalography (EEG) due to its noninvasiveness. The next generation of brain–computer interface systems can benefit from the generative deep learning (GDL) models by providing end‐to‐end (e2e) machine learning and increasing their accuracy. In this study, to exploit the e2e‐property of deep learning models, a novel GDL methodology is proposed where only minimal objective‐free preprocessing steps are needed. Furthermore, to deal with the complicated multi‐class MI–EEG signals, an innovative multilevel GDL‐based classifying scheme is proposed. The effectiveness of the proposed model and its robustness against noisy MI–EEG signals is evaluated using two different GDL models, that is, deep belief network and stacked sparse autoencoder in e2e manner. Experimental results demonstrate the effectiveness of the proposed methodology with improved accuracy compared with the widely used filter bank common spatial patterns algorithm.

Analysis of the Effect of Variation of Reference Channel on Neuronal Activity for Motor Imagery Electroencephalography Signal

Analysis of the Effect of Variation of Reference Channel on Neuronal Activity for Motor Imagery Electroencephalography Signal

A Synergetic Brain-Machine Interfacing Paradigm for Multi-DOF Robot Control

This paper proposes a novel brain-machine interfacing (BMI) paradigm for control of a multijoint redundant robot system. Here, the user would determine the direction of end-point movement of a 3-degrees of freedom (DOF) robot arm using motor imagery electroencephalography signal with co-adaptive decoder (adaptivity between the user and the decoder) while a synergetic motor learning algorithm manages a peripheral redundancy in multi-DOF joints toward energy optimality through tacit learning. As in human motor control, torque control paradigm is employed for a robot to be adaptive to the given physical environment. The dynamic condition of the robot arm is taken into consideration by the learning algorithm. Thus, the user needs to only think about the end-point movement of the robot arm, which allows simultaneous multijoints control by BMI. The support vector machine-based decoder designed in this paper is adaptive to the changing mental state of the user. Online experiments reveals that the users successfully reach their targets with an average decoder accuracy of over 75% in different end-point load conditions.

Analysis of the Effect of Variation of Reference Channel on Neuronal Activity for Motor Imagery Electroencephalography Signal

Objectives: Topographic map has been studied to observe the amount of neuronal activity at various electrode locations. Methods/Statistical Analysis: Electroencephalography (EEG) refers to recording of electrical activities (potential or current density) produced by neurons. A raw EEG signal obtained through electrodes placed in a specific pattern on the scalp. One of the electrodes needs to be selected as a Reference, relative to which electrical activity at all other electrodes shall be measured. It is usually said that electrodes at “inactive” regions of the scalp should be selected as the reference but this paper deals with analyzing what happens if any other electrode is selected as reference as there is no single best reference for all the purposes. The magnitude of electrical activity at various electrodes is indicated by color codes so that it is visually distinguishable. Findings: The difference in topography of EEG rhythms shall be observed and the extent up to which this is correct with different electrodes as reference would be seen. We observe that asymmetry arises if electrodes are at right or left rather than at Center line. We also observe that the electrodes which are at similar potential as that of the reference is indicated by Red color. Application/Improvements: The electrodes at very high potential than the reference are indicated by yellow whereas blue indicates medium activity w.r.t. the reference. These signals may be used for BCI application.