Imagery Based Brain Computer Interface Research Articles

Variance characteristic preserving common spatial pattern for motor imagery BCI.

The common spatial patterns (CSP) algorithm is the most popular technique for extracting electroencephalogram (EEG) features in motor imagery based brain-computer interface (BCI) systems. CSP algorithm embeds the dimensionality of multichannel EEG data to extract features of motor imagery tasks. Most previous studies focused on the optimization of the time domain and the spectrum domain of EEG signal to improve the effectiveness of CSP, whereas ignoring the constraint on the projected feature space. This study proposed a variance characteristic preserving CSP (VPCSP) that is modified by a regularization item based on graph theory. Specifically, we calculated the loss of abnormalities of the projected data while preserving the variance characteristic locally. Then the loss could be rewritten as a matrix with the introduction of the Laplace matrix, which turned it into a generalized eigenvalue problem equivalent to CSP. This study evaluated the proposed method on two public EEG datasets from the BCI competition. The modified method could extract robust and distinguishable features that provided higher classification performance. Experimental results showed that the proposed regularization improved the effectiveness of CSP significantly and achieved superior performance compared with reported modified CSP algorithms significantly. The classification accuracy of the proposed method achieved 87.88 %, 90.07 %, and 76.06 % on public dataset IV part I, III part IVa and the self-collected dataset, respectively. Comparative experiments are conducted on two public datasets and one self-collected dataset. Results showed that the proposed method outperformed the reported algorithm. The proposed method can extract robust features to increase the performance of BCI systems. And the proposal still has expandability. These results show that our proposal is a promising candidate for the performance improvement of MI-BCI.

Effects of motor imagery based brain-computer interface on upper limb function and attention in stroke patients with hemiplegia: a randomized controlled trial

BackgroundSeeking positive and comprehensive rehabilitation methods after stroke is an urgent problem to be solved, which is very important to improve the dysfunction of stroke. The aim of this study was to investigate the effects of motor imagery-based brain-computer interface training (MI-BCI) on upper limb function and attention in stroke patients with hemiplegia.MethodsSixty stroke patients with impairment of upper extremity function and decreased attention were randomly assigned to the control group (CR group) or the experimental group (BCI group) in a 1:1 ratio. Patients in the CR group received conventional rehabilitation. Patients in the BCI group received 20 min of MI-BCI training five times a week for 3 weeks (15 sessions) in addition to conventional rehabilitation. The primary outcome measures were the changes in Fugl-Meyer Motor Function Assessment of Upper Extremities (FMA-UE) and Attention Network Test (ANT) from baseline to 3 weeks.ResultsAbout 93% of the patients completed the allocated training. Compared with the CR group, among those in the BCI group, FMA-UE was increased by 8.0 points (95%CI, 5.0 to 10.0; P < 0.001). Alert network response time (32.4ms; 95%CI, 58.4 to 85.6; P < 0.001), orienting network response (5.6ms; 95%CI, 29.8 to 55.8; P = 0.010), and corrects number (8.0; 95%CI, 17.0 to 28.0; P < 0.001) also increased in the BCI group compared with the CR group. Additionally, the executive control network response time (− 105.9ms; 95%CI, − 68.3 to − 23.6; P = 0.002), the total average response time (− 244.8ms; 95%CI, − 155.8 to − 66.2; P = 0.002), and total time (− 122.0ms; 95%CI, − 80.0 to − 35.0; P = 0.001) were reduced in the BCI group compared with the CR group.ConclusionMI-BCI combined with conventional rehabilitation training could better enhance upper limb motor function and attention in stroke patients. This training method may be feasible and suitable for individuals with stroke.Trial registration: This study was registered in the Chinese Clinical Trial Registry with Portal Number ChiCTR2100050430(27/08/2021).

Multiscale Convolutional Transformer for EEG Classification of Mental Imagery in Different Modalities.

A new kind of sequence-to-sequence model called a transformer has been applied to electroencephalogram (EEG) systems. However, the majority of EEG-based transformer models have applied attention mechanisms to the temporal domain, while the connectivity between brain regions and the relationship between different frequencies have been neglected. In addition, many related studies on imagery-based brain-computer interface (BCI) have been limited to classifying EEG signals within one type of imagery. Therefore, it is important to develop a general model to learn various types of neural representations. In this study, we designed an experimental paradigm based on motor imagery, visual imagery, and speech imagery tasks to interpret the neural representations during mental imagery in different modalities. We conducted EEG source localization to investigate the brain networks. In addition, we propose the multiscale convolutional transformer for decoding mental imagery, which applies multi-head attention over the spatial, spectral, and temporal domains. The proposed network shows promising performance with 0.62, 0.70, and 0.72 mental imagery accuracy with the private EEG dataset, BCI competition IV 2a dataset, and Arizona State University dataset, respectively, as compared to the conventional deep learning models. Hence, we believe that it will contribute significantly to overcoming the limited number of classes and low classification performances in the BCI system.

Exploration of Pattern Recognition Methods for Motor Imagery EEG Signal with Convolutional Neural Network Approach

As an application of EEG, Motor Imagery based Brain-Computer Interface (MI BCI) plays a significant role in assisting patients with disability to communicate with their environment. MI BCI could now be realized through various methods such as machine learning. Many attempts using different machine learning approaches as MI BCI applications have been done with every one of them yielding various results. While some attempts managed to achieve agreeable results, some still failed. This failure may be caused by the separation of the feature extraction and classification steps, as this may lead to the loss of information which in turn causes lower classification accuracy. This problem can be solved by integrating feature extraction and classification by harnessing a classification algorithm that processed the input data as a whole until it produces the prediction, hence the use of convolutional neural network (CNN) approach which is known for its versatility in processing and classifying data all in one go. In this study, the CNN exploration involved a task to classify 5 different classes of fingers’ imaginary movement (thumb, index, middle, ring, and pinky) based on the processed raw signal provided. The CNN performance was observed for both non-augmented and augmented data with the data augmentation techniques used include sliding window, noise addition, and the combination of those two methods. From these experiments, the results show that the CNN model managed to achieve an averaged accuracy of 47%, meanwhile with the help of augmentation techniques of sliding window, noise addition, and the combined methods, the model achieved even higher averaged accuracy of 57,1%, 47,2%, and 57,5% respectively.

A Novel Classification Framework Using the Graph Representations of Electroencephalogram for Motor Imagery Based Brain-Computer Interface

The motor imagery (MI) based brain-computer interfaces (BCIs) have been proposed as a potential physical rehabilitation technology. However, the low classification accuracy achievable with MI tasks is still a challenge when building effective BCI systems. We propose a novel MI classification model based on measurement of functional connectivity between brain regions and graph theory. Specifically, motifs describing local network structures in the brain are extracted from functional connectivity graphs. A graph embedding model called Ego-CNNs is then used to build a classifier, which can convert the graph from a structural representation to a fixed-dimensional vector for detecting critical structure in the graph. We validate our proposed method on four datasets, and the results show that our proposed method produces high classification accuracies in two-class classification tasks (92.8% for dataset 1, 93.4% for dataset 2, 96.5% for dataset 3, and 80.2% for dataset 4) and multiclass classification tasks (90.33% for dataset 1). Our proposed method achieves a mean Kappa value of 0.88 across nine participants, which is superior to other methods we compared it to. These results indicate that there is a local structural difference in functional connectivity graphs extracted under different motor imagery tasks. Our proposed method has great potential for motor imagery classification in future studies.

A novel method for classification of multi-class motor imagery tasks based on feature fusion

Motor imagery based brain computer interface (MI-BCI) has the advantage of strong independence that can rely on the spontaneous brain activity of the user to operate external devices. However, MI-BCI still has the problem of poor control effect, which requires more effective feature extraction algorithms and classification methods to extract distinctly separable features from electroencephalogram (EEG) signals. This paper proposes a novel framework based on Bispectrum, Entropy and common spatial pattern (BECSP). Here we use three methods of bispectrum in higher order spectra, entropy and CSP to extract MI-EEG signal features, and then select the most contributing features through tree-based feature selection algorithm. By comparing the classification results of SVM, Random Forest, Naive Bayes, LDA, KNN, Xgboost and Adaboost, we finally decide to use the SVM algorithm based on RBF kernel function which obtained the best result among them for classification. The proposed method is applied to the BCI competition IV data set 2a and BCI competition III data set IVa. On data set 2a, the highest accuracy on the evaluation data set reaches 85%. The experiment on data set IVa can also achieve good results. Compared with other algorithms that use the same data set, the performance of our algorithm has also been improved.

An EEG channel selection method for motor imagery based brain–computer interface and neurofeedback using Granger causality

Motor imagery (MI) brain–computer interface (BCI) and neurofeedback (NF) with electroencephalogram (EEG) signals are commonly used for motor function improvement in healthy subjects and to restore neurological functions in stroke patients. Generally, in order to decrease noisy and redundant information in unrelated EEG channels, channel selection methods are used which provide feasible BCI and NF implementations with better performances. Our assumption is that there are causal interactions between the channels of EEG signal in MI tasks that are repeated in different trials of a BCI and NF experiment. Therefore, a novel method for EEG channel selection is proposed which is based on Granger causality (GC) analysis. Additionally, the machine-learning approach is used to cluster independent component analysis (ICA) components of the EEG signal into artifact and normal EEG clusters. After channel selection, using the common spatial pattern (CSP) and regularized CSP (RCSP), features are extracted and with the k-nearest neighbor (k-NN), support vector machine (SVM) and linear discriminant analysis (LDA) classifiers, MI tasks are classified into left and right hand MI. The goal of this study is to achieve a method resulting in lower EEG channels with higher classification performance in MI-based BCI and NF by causal constraint. The proposed method based on GC, with only eight selected channels, results in 93.03% accuracy, 92.93% sensitivity, and 93.12% specificity, with RCSP feature extractor and best classifier for each subject, after being applied on Physionet MI dataset, which is increased by 3.95%, 3.73%, and 4.13%, in comparison with correlation-based channel selection method.

The CSP-Based New Features Plus Non-Convex Log Sparse Feature Selection for Motor Imagery EEG Classification.

The common spatial pattern (CSP) is a very effective feature extraction method in motor imagery based brain computer interface (BCI), but its performance depends on the selection of the optimal frequency band. Although a lot of research works have been proposed to improve CSP, most of these works have the problems of large computation costs and long feature extraction time. To this end, three new feature extraction methods based on CSP and a new feature selection method based on non-convex log regularization are proposed in this paper. Firstly, EEG signals are spatially filtered by CSP, and then three new feature extraction methods are proposed. We called them CSP-wavelet, CSP-WPD and CSP-FB, respectively. For CSP-Wavelet and CSP-WPD, the discrete wavelet transform (DWT) or wavelet packet decomposition (WPD) is used to decompose the spatially filtered signals, and then the energy and standard deviation of the wavelet coefficients are extracted as features. For CSP-FB, the spatially filtered signals are filtered into multiple bands by a filter bank (FB), and then the logarithm of variances of each band are extracted as features. Secondly, a sparse optimization method regularized with a non-convex log function is proposed for the feature selection, which we called LOG, and an optimization algorithm for LOG is given. Finally, ensemble learning is used for secondary feature selection and classification model construction. Combing feature extraction and feature selection methods, a total of three new EEG decoding methods are obtained, namely CSP-Wavelet+LOG, CSP-WPD+LOG, and CSP-FB+LOG. Four public motor imagery datasets are used to verify the performance of the proposed methods. Compared to existing methods, the proposed methods achieved the highest average classification accuracy of 88.86, 83.40, 81.53, and 80.83 in datasets 1–4, respectively. The feature extraction time of CSP-FB is the shortest. The experimental results show that the proposed methods can effectively improve the classification accuracy and reduce the feature extraction time. With comprehensive consideration of classification accuracy and feature extraction time, CSP-FB+LOG has the best performance and can be used for the real-time BCI system.

Weighted Transfer Learning of Dynamic Time Warped Data for Motor Imagery based Brain Computer Interfaces.

A large amount of calibration data is typically needed to train an electroencephalogram (EEG)-based brain-computer interfaces (BCI) due to the non-stationary nature of EEG data. This paper proposes a novel weighted transfer learning algorithm using a dynamic time warping (DTW) based alignment method to alleviate this need by using data from other subjects. DTW-based alignment is first applied to reduce the temporal variations between a specific subject data and the transfer learning data from other subjects. Next, similarity is measured using Kullback Leibler divergence (KL) between the DTW aligned data and the specific subject data. The other subjects' data are then weighted based on their KL similarity to each trials of the specific subject data. This weighted data from other subjects are then used to train the BCI model of the specific subject. An experiment was performed on publicly available BCI Competition IV dataset 2a. The proposed algorithm yielded an average improvement of 9% compared to a subject-specific BCI model trained with 4 trials, and the results yielded an average improvement of 10% compared to naive transfer learning. Overall, the proposed DTW-aligned KL weighted transfer learning algorithm show promise to alleviate the need of large amount of calibration data by using only a short calibration data.

A Review of Using Common Spatial Pattern in Discrimination of Multiclass Motor Imagery–Based Brain-Computer Interface

Brain-computer interface (BCI) is now being one of the interesting and attractive topics for researchers in recent years. Common spatial pattern is the most common approach applied in motor imagery (MI)-based BCI because of its effectiveness in differentiating between 2 different MI classes. The main objective of this article is to review using common spatial pattern combining with different methods such as Hilbert transform, error correction output coding, wavelet transform, joint approximate diagonalization, and others to extract features that can be used to differentiate between multiclass MI-based BCI. Data set 2a from BCI competition IV is used as an example for all authors. In addition, results of using various classifiers are demonstrated.

A Comparative Study of Linear and Non-Linear Classifiers in Sensory Motor Imagery Based Brain Computer Interface

Over the period of 4–5 decades of the field electrophysiology on neural signal events in single cell recording, it can be concluded that there is production of far field potentials containing near synchronous field patterns might reach at the scalp electrodes. These neural signals then can be analyzed and converted into the control signals for computers and other electronic devices. Electroencephalography (EEG) is a method to acquire these neural signals from the scalp of human brain. EEG signals are simple, economical and have high temporal resolution properties. These properties make it advantageous to use widely in the medical as well as non-medical applications. The event related synchronization and desynchronization (ERS/ERD) pattern present in EEG during sensory motor imagery (SMI) process over the cortical area is an important feature to take BCI towards realistic approach. The accurate classification of these ERS/ERD pattern present in EEG signal is dependent on classification accuracy of different classifiers. So, the objective of the paper is to analyze the classification accuracy of linear and non-linear classifiers used for BCI system design. This paper also presents the comparative study of linear (Linear discriminant analysis and Support Vector Machine) and non-linear classifiers (Bayesian and Radial Basis Function-Support Vector Machine) using BCI competition IV dataset 2a. The result concluded that linear classifiers (LDA and SVM) have outperformed the non-linear classifiers on EEG data with the average performance across subjects 75.26±12.23, 72.42±11.12.

High Classification Accuracy of a Motor Imagery Based Brain-Computer Interface for Stroke Rehabilitation Training.

Motor imagery (MI) based brain-computer interfaces (BCI) extract commands in real-time and can be used to control a cursor, a robot or functional electrical stimulation (FES) devices. The control of FES devices is especially interesting for stroke rehabilitation, when a patient can use motor imagery to stimulate specific muscles in real-time. However, damage to motor areas resulting from stroke or other causes might impair control of a motor imagery BCI for rehabilitation. The current work presents a comparative evaluation of the MI-based BCI control accuracy between stroke patients and healthy subjects. Five patients who had a stroke that affected the motor system participated in the current study, and were trained across 10–24 sessions lasting about 1 h each with the recoveriX system. The participants' EEG data were classified while they imagined left or right hand movements, and real-time feedback was provided on a monitor. If the correct imagination was detected, the FES was also activated to move the left or right hand. The grand average mean accuracy was 87.4% for all patients and sessions. All patients were able to achieve at least one session with a maximum accuracy above 96%. Both the mean accuracy and the maximum accuracy were surprisingly high and above results seen with healthy controls in prior studies. Importantly, the study showed that stroke patients can control a MI BCI system with high accuracy relative to healthy persons. This may occur because these patients are highly motivated to participate in a study to improve their motor functions. Participants often reported early in the training of motor improvements and this caused additional motivation. However, it also reflects the efficacy of combining motor imagination, seeing continuous bar feedback, and real hand movement that also activates the tactile and proprioceptive systems. Results also suggested that motor function could improve even if classification accuracy did not, and suggest other new questions to explore in future work. Future studies will also be done with a first-person view 3D avatar to provide improved feedback and thereby increase each patients' sense of engagement.

Localized and Optimized Feature Extraction from a Sculpted Electroencephalographic Signal for Better Classification in Motor Imagery Based Brain Computer Interface

Localized and Optimized Feature Extraction from a Sculpted Electroencephalographic Signal for Better Classification in Motor Imagery Based Brain Computer Interface

Motor Imagery based Brain Computer Interface Paradigm for Upper Limb Stroke Rehabilitation.

Motor Imagery (MI) based Brain Computer Interface (BCI) systems have shown potential to serve as a tool for neurorehabilitation for post stroke patients to complement the standard therapy. The aim of this study was to develop an MI based BCI system that could potentially be used in neurorehabilitation of hand motor function in stroke patients. Two co-adaptive, three-class MI based BCI systems for realtime processing were developed and compared using the publicly available data from the BCI Competition III Dataset V as well as our own data. The first algorithm utilizes the Filterbank Common Spatial Pattern (FBCSP) for feature extraction, and the other utilizes the Separable Common Spatio-Spectral Pattern (SCSSP) - both combined with a Multi-layer Perceptron (MLP) for classification. The proposed system proved successful when using the competition data showing an average accuracy of 64.71 % for the SCSSP compared to 60.48% for the FBCSP. This proved superior to a related study using the same feature extraction methods, but with other classification methods. The proposed system, however did show results around chance level for the 3-class MI experimental data that we have collected in our laboratory. Further studies needs to be conducted to improve the performance as well as to realize such a system to put in use.

CSP-TSM: Optimizing the performance of Riemannian tangent space mapping using common spatial pattern for MI-BCI

BackgroundClassification of electroencephalography (EEG) signals for motor imagery based brain computer interface (MI-BCI) is an exigent task and common spatial pattern (CSP) has been extensively explored for this purpose. In this work, we focused on developing a new framework for classification of EEG signals for MI-BCI. MethodWe propose a single band CSP framework for MI-BCI that utilizes the concept of tangent space mapping (TSM) in the manifold of covariance matrices. The proposed method is named CSP-TSM. Spatial filtering is performed on the bandpass filtered MI EEG signal. Riemannian tangent space is utilized for extracting features from the spatial filtered signal. The TSM features are then fused with the CSP variance based features and feature selection is performed using Lasso. Linear discriminant analysis (LDA) is then applied to the selected features and finally classification is done using support vector machine (SVM) classifier. ResultsThe proposed framework gives improved performance for MI EEG signal classification in comparison with several competing methods. Experiments conducted shows that the proposed framework reduces the overall classification error rate for MI-BCI by 3.16%, 5.10% and 1.70% (for BCI Competition III dataset IVa, BCI Competition IV Dataset I and BCI Competition IV Dataset IIb, respectively) compared to the conventional CSP method under the same experimental settings. ConclusionThe proposed CSP-TSM method produces promising results when compared with several competing methods in this paper. In addition, the computational complexity is less compared to that of TSM method. Our proposed CSP-TSM framework can be potentially used for developing improved MI-BCI systems.

Hilbert-Huang Transform and Welch's Method for Motor imagery based Brain Computer Interface

The features extraction is the main step in a Brain-Computer Interface (BCI) design. Its goal is to create features easy to be interpreted in order to produce the most accurate control commands. For this end, these features must include all the original signal characteristics. The generated brain's signals' non-stationary and nonlinearity constitute a limitation to the improvement of the performances of systems based on traditional signal processing such as Fourier Transform. This work deals with the comparison of features extraction between Hilbert-Huang Transform (HHT) and Welch's method for Power Spectral Density estimation (PSD) then on the creation of an adaptive method combining the two. The parameters optimization of each method is firstly performed to reach the best classification accuracy rate. The study shows that the PSD estimation is sensitive to the parametric variation whereas the HHT method is mainly robust. The classification results show that an adaptive joint method can reach 90% of accuracy rate for a mental activity period of 1s.

Enhancing performance of a motor imagery based brain–computer interface by incorporating electrical stimulation-induced SSSEP

Objective. We proposed a novel simultaneous hybrid brain–computer interface (BCI) by incorporating electrical stimulation into a motor imagery (MI) based BCI system. The goal of this study was to enhance the overall performance of an MI-based BCI. In addition, the brain oscillatory pattern in the hybrid task was also investigated. Approach. 64-channel electroencephalographic (EEG) data were recorded during MI, selective attention (SA) and hybrid tasks in fourteen healthy subjects. In the hybrid task, subjects performed MI with electrical stimulation which was applied to bilateral median nerve on wrists simultaneously. Main results. The hybrid task clearly presented additional steady-state somatosensory evoked potential (SSSEP) induced by electrical stimulation with MI-induced event-related desynchronization (ERD). By combining ERD and SSSEP features, the performance in the hybrid task was significantly better than in both MI and SA tasks, achieving a ~14% improvement in total relative to the MI task alone and reaching ~89% in mean classification accuracy. On the contrary, there was no significant enhancement obtained in performance while separate ERD feature was utilized in the hybrid task. In terms of the hybrid task, the performance using combined feature was significantly better than using separate ERD or SSSEP feature. Significance. The results in this work validate the feasibility of our proposed approach to form a novel MI-SSSEP hybrid BCI outperforming a conventional MI-based BCI through combing MI with electrical stimulation.

A Fully Automated Trial Selection Method for Optimization of Motor Imagery Based Brain-Computer Interface.

Independent component analysis (ICA) as a promising spatial filtering method can separate motor-related independent components (MRICs) from the multichannel electroencephalogram (EEG) signals. However, the unpredictable burst interferences may significantly degrade the performance of ICA-based brain-computer interface (BCI) system. In this study, we proposed a new algorithm frame to address this issue by combining the single-trial-based ICA filter with zero-training classifier. We developed a two-round data selection method to identify automatically the badly corrupted EEG trials in the training set. The “high quality” training trials were utilized to optimize the ICA filter. In addition, we proposed an accuracy-matrix method to locate the artifact data segments within a single trial and investigated which types of artifacts can influence the performance of the ICA-based MIBCIs. Twenty-six EEG datasets of three-class motor imagery were used to validate the proposed methods, and the classification accuracies were compared with that obtained by frequently used common spatial pattern (CSP) spatial filtering algorithm. The experimental results demonstrated that the proposed optimizing strategy could effectively improve the stability, practicality and classification performance of ICA-based MIBCI. The study revealed that rational use of ICA method may be crucial in building a practical ICA-based MIBCI system.

Motor imagery based brain computer interface using transform domain features.

Brain Computer Interface (BCI) is a channel of communication between the human brain and an external device through brain electrical activity. In this paper, we extracted different features to boost the classification accuracy as well as the mutual information of BCI systems. The extracted features include the magnitude of the discrete Fourier transform and the wavelet coefficients for the EEG signals in addition to distance series values and invariant moments calculated for the reconstructed phase space of the EEG measurements. Different preprocessing, feature selection, and classification schemes were utilized to evaluate the performance of the proposed system for dataset III from BCI competition II. The maximum accuracy achieved was 90.7% while the maximum mutual information was 0.76 bit obtained using the distance series features.

Mutual information-based feature selection for low-cost BCIs based on motor imagery.

In the present study a feature selection algorithm based on mutual information (MI) was applied to electro-encephalographic (EEG) data acquired during three different motor imagery tasks from two dataset: Dataset I from BCI Competition IV including full scalp recordings from four subjects, and new data recorded from three subjects using the popular low-cost Emotiv EPOC EEG headset. The aim was to evaluate optimal channels and band-power (BP) features for motor imagery tasks discrimination, in order to assess the feasibility of a portable low-cost motor imagery based Brain-Computer Interface (BCI) system. The minimal sub set of features most relevant to task description and less redundant to each other was determined, and the corresponding classification accuracy was assessed offline employing linear support vector machine (SVM) in a 10-fold cross validation scheme. The analysis was performed: (a) on the original full Dataset I from BCI competition IV, (b) on a restricted channels set from Dataset I corresponding to available Emotiv EPOC electrodes locations, and (c) on data recorded with the EPOC system. Results from (a) showed that an offline classification accuracy above 80% can be reached using only 5 features. Limiting the analysis to EPOC channels caused a decrease of classification accuracy, although it still remained above chance level, both for data from (b) and (c). A top accuracy of 70% was achieved using 2 optimal features. These results encourage further research towards the development of portable low cost motor imagery-based BCI systems.