Motor Imagery Brain-computer Interface System Research Articles

Model and Data Dual-Driven Double-Point Observation Network for Ultra-Short MI EEG Classification.

Although deep networks have succeeded in various signal classification tasks, the time sequence samples used to train the deep models are usually required to reach a certain length. Especially, in brain computer interface (BCI) research, around 3.5s-long motor imagery (MI) Electroencephalography (EEG) samples are needed to obtain satisfactory classification performance. This time-span requirement of the training samples makes real-time MI BCI systems impossible to implement based on deep networks, which restricts the related researches within laboratory and makes practical application hard to accomplish. To address this issue, a double-point observation deep network (DoNet) is developed to classify ultra-short samples buried in noise. First, an analytical solution is developed theoretically to perform ultra-short signal classification based on double-point couples. Then, a signal-noise model is constructed to study the interference of noise on classification based on double-point couples. Based on which, an independent identical distribution condition is utilized to improve the classification accuracy in a data-driven manner. Combining the theoretical model and data-driven mechanism, DoNet can construct a steady data-distribution for the double-point couples of the samples with the same label. Therefore, the conditional probability of each double-point couple of a test sample can be obtained. With a voting strategy, the samples can be accurately classified by fusing these conditional probabilities. Meanwhile, the noise interference can be suppressed. DoNet has been evaluated on two public EEG datasets. Compared to most state-of-the-art methods, the 1s-long EEG signal classification accuracy has been improved by more than 3%.

A Decoding Method Using Riemannian Local Linear Feature Construction for a Lower-Limb Motor Imagery Brain–Computer Interface System

Recently, motor imagery brain–computer interfaces (BCIs) have been developed for use in motor function assistance and rehabilitation engineering. In particular, lower-limb motor imagery BCI systems are receiving increasing attention in the field of motor rehabilitation, because these systems could accurately and rapidly identify a patient’s lower-limb movement intention, which could improve the practicability of the motor rehabilitation. In this study, a novel lower-limb BCI system combining visual stimulation, auditory stimulation, functional electrical stimulation, and proprioceptive stimulation was designed to assist patients in lower-limb rehabilitation training. In addition, the Riemannian local linear feature construction (RLLFC) algorithm is proposed to improve the performance of decoding by using unsupervised basis learning and representation weight calculation in the motor imagery BCI system. Three in-house experiment were performed to demonstrate the effectiveness of the proposed system in comparison with other state-of-the-art methods. The experimental results indicate that the proposed system can learn low-dimensional features and correctly characterize the relationship between the testing trial and its k-nearest neighbors.

MSATNet: multi-scale adaptive transformer network for motor imagery classification.

Motor imagery brain-computer interface (MI-BCI) can parse user motor imagery to achieve wheelchair control or motion control for smart prostheses. However, problems of poor feature extraction and low cross-subject performance exist in the model for motor imagery classification tasks. To address these problems, we propose a multi-scale adaptive transformer network (MSATNet) for motor imagery classification. Therein, we design a multi-scale feature extraction (MSFE) module to extract multi-band highly-discriminative features. Through the adaptive temporal transformer (ATT) module, the temporal decoder and multi-head attention unit are used to adaptively extract temporal dependencies. Efficient transfer learning is achieved by fine-tuning target subject data through the subject adapter (SA) module. Within-subject and cross-subject experiments are performed to evaluate the classification performance of the model on the BCI Competition IV 2a and 2b datasets. The MSATNet outperforms benchmark models in classification performance, reaching 81.75 and 89.34% accuracies for the within-subject experiments and 81.33 and 86.23% accuracies for the cross-subject experiments. The experimental results demonstrate that the proposed method can help build a more accurate MI-BCI system.

Motor imagery brain-computer interface rehabilitation system enhances upper limb performance and improves brain activity in stroke patients: A clinical study.

This study compared the efficacy of Motor Imagery brain-computer interface (MI-BCI) combined with physiotherapy and physiotherapy alone in ischemic stroke before and after rehabilitation training. We wanted to explore whether the rehabilitation effect of MI-BCI is affected by the severity of the patient's condition and whether MI-BCI was effective for all patients. Forty hospitalized patients with ischemic stroke with motor deficits participated in this study. The patients were divided into MI and control groups. Functional assessments were performed before and after rehabilitation training. The Fugl-Meyer Assessment (FMA) was used as the primary outcome measure, and its shoulder and elbow scores and wrist scores served as secondary outcome measures. The motor assessment scale (MAS) was used to assess motor function recovery. We used non-contrast CT (NCCT) to investigate the influence of different types of middle cerebral artery high-density signs on the prognosis of ischemic stroke. Brain topographic maps can directly reflect the neural activity of the brain, so we used them to detect changes in brain function and brain topological power response after stroke. Compared the MI group and control group after rehabilitation training, better functional outcome was observed after MI-BCI rehabilitation, including a significantly higher probability of achieving a relevant increase in the Total FMA scores (MI = 16.70 ± 12.79, control = 5.34 ± 10.48), FMA shoulder and elbow scores (MI = 12.56 ± 6.37, control = 2.45 ± 7.91), FMA wrist scores (MI = 11.01 ± 3.48, control = 3.36 ± 5.79), the MAS scores (MI = 3.62 ± 2.48, control = 1.85 ± 2.89), the NCCT (MI = 21.94 ± 2.37, control = 17.86 ± 3.55). The findings demonstrate that MI-BCI rehabilitation training could more effectively improve motor function after upper limb motor dysfunction after stroke compared with routine rehabilitation training, which verifies the feasibility of active induction of neural rehabilitation. The severity of the patient's condition may affect the rehabilitation effect of the MI-BCI system.

An overview of methods of left and right foot motor imagery based on Tikhonov regularisation common spatial pattern.

The motor imagery brain-computer interface (MI-BCI) provides an interactive control channel for spinal cord injury patients. However, the limitations of feature extraction algorithms may lead to low accuracy and instability in decoding electroencephalogram (EEG) signals. In this study, we examined the classification performance of an MI-BCI system by focusing on the distinction of the left and right foot kinaesthetic motor imagery tasks in five subjects. Feature extraction was performed using the common space pattern (CSP) and the Tikhonov regularisation CSP (TRCSP) spatial filters. TRCSP overcomes the CSP problems of noise sensitivity and overfitting. Moreover, support vector machine (SVM) and linear discriminant analysis (LDA) were used for classification and recognition. We constructed four combined classification methods (TRCSP-SVM, TRCSP-LDA, CSP-SVM, and CSP-LDA) and evaluated them by comparing their accuracies, kappa coefficients, and receiver operating characteristic (ROC) curves. The results showed that the TRCSP-SVM method performed significantly better than others (average accuracy 97%, average kappa coefficient 0.91, and average area under ROC curve (AUC) 0.98). Using TRCSP instead of standard CSP improved accuracy by up to 10%. This study provides insights into the classification of EEG signals. The results of this study can aid lower limb MI-BCI systems in rehabilitation training.

Deep Convolution Generative Adversarial Network-Based Electroencephalogram Data Augmentation for Post-Stroke Rehabilitation with Motor Imagery.

The motor imagery brain-computer interface (MI-BCI) system is currently one of the most advanced rehabilitation technologies, and it can be used to restore the motor function of stroke patients. The deep learning algorithms in the MI-BCI system require lots of training samples, but the electroencephalogram (EEG) data of stroke patients is quite scarce. Therefore, the expansion of EEG data has become an important part of stroke clinical rehabilitation research. In this paper, a deep convolution generative adversarial network (DCGAN) model is proposed to generate artificial EEG data and further expand the scale of the stroke dataset. First, multichannel one-dimensional EEG data is converted into a two-dimensional EEG spectrogram using EEG2Image based on the modified S-transform. Then, DCGAN is used to artificially generate EEG data based on MI. Finally, the validity of the generated artificial EEG data is proved. This paper preliminarily indicates that generating artificial stroke data is a promising strategy, which contributes to the further development of stroke clinical rehabilitation.

Two-branch 3D convolutional neural network for motor imagery EEG decoding

Objective. The original motor imagery electroencephalography (MI-EEG) data contains not only temporal features but also a large number of spatial features related to the distribution of electrodes on the brain. However, in the process of MI-EEG decoding, most of the current convolutional neural network (CNN) based methods do not make the utmost of the spatial features related to electrode distribution. Approach. In this study, we adopt a concise 3D representation for the MI-EEG data to take full advantage of the spatial features and propose a two-branch 3D CNN (TB-3D CNN) for the 3D representation of MI-EEG data. First, the spatial and temporal features of the input 3D samples are extracted by the spatial and temporal feature learning branches, respectively, to avoid the mutual interference between the temporal and spatial features. Then, the central loss is introduced into the TB-3D CNN framework to further improve the MI-EEG decoding accuracy. And a 3D data augmentation method based on the cyclic translation of time dimension is proposed for the 3D representation method to alleviate the overfitting problem. Main results. Some experiments are conducted on the famous BCI competition IV 2a dataset to evaluate the effectiveness of the proposed MI-EEG decoding method. The experimental results comparison with some state-of-the-art methods demonstrates that the average accuracy of our method is 4.42% higher than that of the best of the comparative methods. Significance. The proposed MI-EEG decoding method has great promise to improve the performance of motor imagery brain-computer interface system.

EEG Motor-Imagery BCI System Based on Maximum Overlap Discrete Wavelet Transform (MODWT) and cubic SVM

Communication of the human brain with the surroundings became reality by using Brain- Computer Interface (BCI) based mechanism. Electroencephalography (EEG) being the non-invasive method has become popular for interaction with the brain. Traditionally, the devices were used for clinical applications to detect various brain diseases but with the advancement in technologies, companies like Emotiv, NeuoSky are coming up with low cost, easily portable EEG based consumer graded devices that can be used in various application domains like gaming, education etc as these devices are comfortable to wear also. This paper reviews the fields where the EEG has shown its impact and the way it has proved useful for individuals with severe motor disorder, rehabilitation and has become a means of communication to the real world. This paper investigates the use of Cubic SVM algorithm In the EEG classification. EEG feature extraction is Implemented by maximum overlap discrete wavelet transform (MODWT) to reduce the dimensionality of data. The Sliding Window Technique is used to calculate the mean within each window samples. The feature vectors are loaded into the support vector machine (SVM) and optimize tree.

EEG Motor-Imagery BCI System Based on Maximum Overlap Discrete Wavelet Transform (MODWT) and Machine learning algorithm

The ability of the human brain to communicate with its environment has become a reality through the use of a Brain-Computer Interface (BCI)-based mechanism. Electroencephalography (EEG) has gained popularity as a non-invasive way of brain connection. Traditionally, the devices were used in clinical settings to detect various brain diseases. However, as technology advances, companies such as Emotiv and NeuroSky are developing low-cost, easily portable EEG-based consumer-grade devices that can be used in various application domains such as gaming, education. This article discusses the parts in which the EEG has been applied and how it has proven beneficial for those with severe motor disorders, rehabilitation, and as a form of communicating with the outside world. This article examines the use of the SVM, k-NN, and decision tree algorithms to classify EEG signals. To minimize the complexity of the data, maximum overlap discrete wavelet transform (MODWT) is used to extract EEG features. The mean inside each window sample is calculated using the Sliding Window Technique. The vector machine (SVM), k-Nearest Neighbor, and optimize decision tree load the feature vectors.

Feature fusion for improving performance of motor imagery brain-computer interface system

A brain-computer interface (BCI) is a system that makes communication between an external device and the brain based on the brain’s neural activity. This communication is conducted by analyzing brain signals, so extracting and selecting those features of the brain signals that distinguish between humans’ different activities are essentially important. In this study, first, the brain signal is divided into frequency sub-bands using Constant-Q filters, which allows achieving better frequency resolution in lower frequencies and also the better temporal resolution in higher frequencies. Then, appropriate features in temporal, spatial, and spectral domains are extracted from the considered frequency sub-bands to improve the motor imagery classification. Three different ranking methods including Fisher’s method, ReliefF, and mRMR are used to select the features; due to their specific criteria, they can help the best selection for the motor imagery classification. Finally, the results obtained from the selection stage are fused using the Dempster-Shafer evidence method. The proposed technique is applied to the BCI 2008−2b competition dataset, which achieves a Kappa score of 0.718. The results show the capability and excellent performance of the proposed method in comparison with the state-of-the-art studies.

CNN-based classification of fNIRS signals in motor imagery BCI system

Objective. Development of a brain–computer interface (BCI) requires classification of brain neural activities to different states. Functional near-infrared spectroscopy (fNIRS) can measure the brain activities and has great potential for BCI. In recent years, a large number of classification algorithms have been proposed, in which deep learning methods, especially convolutional neural network (CNN) methods are successful. fNIRS signal has typical time series properties, we combined fNIRS data and kinds of CNN-based time series classification (TSC) methods to classify BCI task. Approach. In this study, participants were recruited for a left and right hand motor imagery experiment and the cerebral neural activities were recorded by fNIRS equipment (FOIRE-3000). TSC methods are used to distinguish the brain activities when imagining the left or right hand. We have tested the overall person, single person and overall person with single-channel classification results, and these methods achieved excellent classification results. We also compared the CNN-based TSC methods with traditional classification methods such as support vector machine. Main results. Experiments showed that the CNN-based methods have significant advantages in classification accuracy: the CNN-based methods have achieved remarkable results in the classification of left-handed and right-handed imagination tasks, reaching 98.6% accuracy on overall person, 100% accuracy on single person, and in the single-channel classification an accuracy of 80.1% has been achieved with the best-performing channel. Significance. These results suggest that using the CNN-based TSC methods can significantly improve the BCI performance and also lay the foundation for the miniaturization and portability of training rehabilitation equipment.

A novel motor imagery EEG decoding method based on feature separation

Objective. Motor imagery electroencephalography (EEG) decoding is a vital technology for the brain–computer interface (BCI) systems and has been widely studied in recent years. However, the original EEG signals usually contain a lot of class-independent information, and the existing motor imagery EEG decoding methods are easily interfered by this irrelevant information, which greatly limits the decoding accuracy of these methods. Approach. To overcome the interference of the class-independent information, a motor imagery EEG decoding method based on feature separation is proposed in this paper. Furthermore, a feature separation network based on adversarial learning (FSNAL) is designed for the feature separation of the original EEG samples. First, the class-related features and class-independent features are separated by the proposed FSNAL framework, and then motor imagery EEG decoding is performed only according to the class-related features to avoid the adverse effects of class-independent features. Main results. To validate the effectiveness of the proposed motor imagery EEG decoding method, we conduct some experiments on two public EEG datasets (the BCI competition IV 2a and 2b datasets). The experimental results comparison between our method and some state-of-the-art methods demonstrates that our motor imagery EEG decoding method outperforms all the compared methods on the two experimental datasets. Significance. Our motor imagery EEG decoding method can alleviate the interference of class-independent features, and it has great application potential for improving the performance of motor imagery BCI systems in the near future.

The Improvement of a Brain Computer Interface Based on EEG Signals

Purpose: Brain Computer Interface (BCI) has provided a novel way of communication that can significantly revolutionize life of people suffering from disabilities. Motor Imagery (MI) EEG BCI is one of the most promising solutions to address. The main phases of such systems include signal acquisition, pre-processing, feature extraction, classification and the intended interface. The challenging obstacles in such systems are to detect and extract efficient features that present reliability and robustness alongside promising classification accuracy. In this paper it is endeavored to present a robust method for a two-class MI BCI that results in high accuracy.
 Materials and Methods: For this purpose, the dataset 2b from BCI competition 2008, consisting of three channels (C3, C and Cz), was utilized. Firstly, the signals were bandpass filtered. Secondly, Common Spatial Pattern (CSP) was employed and then a number of features, including non-linear chaotic features were extracted from channels C3 and C4. After feature selection phase the number of features were reduced to 38 and 47. Finally, these features were fed into two classifiers, namely Support Vector Machine classifier (SVM) and Bagging to evaluate the performance of the system.
 Results: Classification accuracy and Cohen’s Kappa coefficient of the proposed method for two MI EEG channels are 96.40% and 0.92, respectively.
 Conclusion: These results indicate the high accuracy and stability of our method in comparison with similar studies. Therefore, it can be a promising approach in two-class MI BCI systems.

Advanced TSGL-EEGNet for Motor Imagery EEG-Based Brain-Computer Interfaces

Deep learning technology is rapidly spreading in recent years and has been extensive attempts in the field of Brain-Computer Interface (BCI). Though the accuracy of Motor Imagery (MI) BCI systems based on the deep learning have been greatly improved compared with some traditional algorithms, it is still a big problem to clearly interpret the deep learning models. To address the issues, this work first introduces a popular deep learning model EEGNet and compares it with the traditional algorithm Filter-Bank Common Spatial Pattern (FBCSP). After that, this work considers that the 1-D convolution of EEGNet can be explained by a special Discrete Wavelet Transform (DWT), and the depthwise convolution of EEGNet is similar to the Common Spatial Pattern (CSP) algorithm. Therefore, this work improves the EEGNet by using the algorithm Temporary Constrained Sparse Group Lasso (TCSGL) to enhance its performance. The proposed model TSGL-EEGNet is tested on the BCI Competition IV 2a and BCI Competition III IIIa datasets that both are 4-classes classification MI tasks. The testing results show that the proposed model has achieved 78.96% (0.7194) average classification accuracy (kappa) on the dataset BCI Competition IV 2a, which are greater than EEGNet, C2CM, MB3DCNN, SS-MEMDBF and FBCSP, especially on insensitive subjects. The proposed model has also achieved 85.30% (0.8040) average classification accuracy (kappa) on the dataset BCI Competition III IIIa, which are greater than the EEGNet, MFTFS et al. At last, this work uses average-validation and stacking to further enhance the effect of the model. The 4-classes classification average accuracy rates reach 81.34% and 88.89%, and the kappas reach 0.7511 and 0.8519 on dataset BCI Competition IV 2a and BCI Competition III IIIa, respectively. Additionally, this work also uses the Grad-CAM to visualize the frequency and spatial features that are learned by the neural network.

Task Transfer Learning for EEG Classification in Motor Imagery-Based BCI System.

The motor-imagery brain-computer interface system (MI-BCI) has a board prospect for development. However, long calibration time and lack of enough MI commands limit its use in practice. In order to enlarge the command set, we add the combinations of traditional MI commands as new commands into the command set. We also design an algorithm based on transfer learning so as to decrease the calibration time for collecting EEG signal and training model. We create feature extractor based on data from traditional commands and transfer patterns through the data from new commands. Through the comparison of the average accuracy between our algorithm and traditional algorithms and the visualization of spatial patterns in our algorithm, we find that the accuracy of our algorithm is much higher than traditional algorithms, especially as for the low-quality datasets. Besides, the visualization of spatial patterns is meaningful. The algorithm based on transfer learning takes the advantage of the information from source data. We enlarge the command set while shortening the calibration time, which is of significant importance to the MI-BCI application.

Identification of Motor and Mental Imagery EEG in Two and Multiclass Subject-Dependent Tasks Using Successive Decomposition Index.

The development of fast and robust brain–computer interface (BCI) systems requires non-complex and efficient computational tools. The modern procedures adopted for this purpose are complex which limits their use in practical applications. In this study, for the first time, and to the best of our knowledge, a successive decomposition index (SDI)-based feature extraction approach is utilized for the classification of motor and mental imagery electroencephalography (EEG) tasks. First of all, the public datasets IVa, IVb, and V from BCI competition III were denoised using multiscale principal analysis (MSPCA), and then a SDI feature was calculated corresponding to each trial of the data. Finally, six benchmark machine learning and neural network classifiers were used to evaluate the performance of the proposed method. All the experiments were performed for motor and mental imagery datasets in binary and multiclass applications using a 10-fold cross-validation method. Furthermore, computerized automatic detection of motor and mental imagery using SDI (CADMMI-SDI) is developed to describe the proposed approach practically. The experimental results suggest that the highest classification accuracy of 97.46% (Dataset IVa), 99.52% (Dataset IVb), and 99.33% (Dataset V) was obtained using feedforward neural network classifier. Moreover, a series of experiments, namely, statistical analysis, channels variation, classifier parameters variation, processed and unprocessed data, and computational complexity, were performed and it was concluded that SDI is robust for noise, and a non-complex and efficient biomarker for the development of fast and accurate motor and mental imagery BCI systems.

A self-paced BCI prototype system based on the incorporation of an intelligent environment-understanding approach for rehabilitation hospital environmental control

This paper presents a self-paced brain–computer interface (BCI) based on the incorporation of an intelligent environment-understanding approach into a motor imagery (MI) BCI system for rehabilitation hospital environmental control. The interface integrates four types of daily assistance tasks: medical calls, service calls, appliance control and catering services. The system introduces intelligent environment understanding technology to establish preliminary predictions concerning a user’s control intention by extracting potential operational objects in the current environment through an object detection neural network. According to the characteristics of the four types of control and services, we establish different response mechanisms and use an intelligent decision-making method to design and dynamically optimize the relevant control instruction set. The control feedback is communicated to the user via voice prompts; it avoids the use of visual channels throughout the interaction. The asynchronous and synchronous modes of the MI-BCI are designed to launch the control process and to select specific operations, respectively. In particular, the reliability of the MI-BCI is enhanced by the optimized identification algorithm. An online experiment demonstrated that the system can respond quickly and it generates an activation command in an average of 3.38s while effectively preventing false activations; the average accuracy of the BCI synchronization commands was 89.2%, which represents sufficiently effective control. The proposed system is efficient, applicable and can be used to both improve system information throughput and to reduce mental loads. The proposed system can be used to assist with the daily lives of patients with severe motor impairments.

A novel method for classification of BCI multi-class motor imagery task based on Dempster–Shafer theory

Brain-computer interface (BCI) is a promising technology to help disabled people to interact with the world only through their brain signals. These systems are designed based on recognizing the patient’s intention by processing his brain signal measurements. The essence of Motor-Imagery (MI) BCI systems is to train a model which classifies the brain signals into several major motions using several training sessions. Current methods mainly employ common spatial patterns (CSP) algorithm to extract meaningful features from the brain signals. Since the CSP algorithm primarily is designed for a two-class paradigm, often data fusion techniques are used to combine the results of CSP-based binary classifiers in multi-class problems. Noting the critical drawback of current data fusion methods due to losing information when combining the results of binary classifiers, here we propose a novel method based on Dempster–Shafer theory to fuse the results of constituent binary classifiers. We applied the proposed method to the benchmark BCI competition iv set 2a data set. The results show a significant improvement, which prove success of our method in modeling uncertainty. More specifically, the average kappa value is 75% which is higher than other competing methods. The proposed method is general and applicable to a wide range of applications.

Automated Selection of a Channel Subset Based on the Genetic Algorithm in a Motor Imagery Brain-Computer Interface System

Brain-computer interface (BCI) is a system for communication and control between the human brain and computers or other electronic devices. However, multichannel signal acquisition, which is time-consuming, laborious, and not conducive to subsequent real-time signal processing, can cause channel redundancy. When designing a BCI system, the selection of the optimal channels that match the expected pattern of potential cortical activity is useful for classifying brain activity during a mental task. The Stockwell transform and Bayesian linear discriminant analysis were applied to feature extraction and classification, respectively, and a genetic algorithm (GA) was used in the process of channel selection to extract the most relevant channel for classification. The superior performance of the algorithm is demonstrated by the test results on BCI Competition III dataset I. For a motor imagery paradigm, we show that the number of used channels can be reduced significantly, and the classification performance of the classifier can be improved. By comparing the performances of the algorithm with or without channel selection, the best channel combination was selected, and only 28 out of the 64 acquisition electrodes were used to realize classification sensitivity, specificity, precision, accuracy and Kappa coefficient values of 98%, 96%, 96.08%, 97% and 0.94, respectively, thereby outperforming existing algorithms. The proposed algorithm can reduce the number of channels and select the best channel combination to improve the classification performance, thereby overcoming the redundant channel problem. In addition, the signal processing framework that is adopted by this research can serve as a reference for related BCI application system research.

Signal processing techniques for motor imagery brain computer interface: A review

Abstract Motor Imagery Brain Computer Interface (MI-BCI) provides a non-muscular channel for communication to those who are suffering from neuronal disorders. The designing of an accurate and reliable MI-BCI system requires the extraction of informative and discriminative features. Common Spatial Pattern (CSP) has been potent and is widely used in BCI for extracting features in motor imagery tasks. The classifiers translate these features into device commands. Many classification algorithms have been devised, among those Support Vector Machine (SVM) and Linear Discriminate Analysis (LDA) have been widely used. In recent studies, the researchers are using deep neural networks for the classification of motor imagery tasks. This paper provides a comprehensive review of dominant feature extraction methods and classification algorithms in brain-computer interface for motor imagery tasks. Authors discuss existing challenges in the domain of motor imagery brain-computer interface and suggest possible research directions.