Motor Imagery Electroencephalogram Research Articles

Excellent fine-tuning: From specific-subject classification to cross-task classification for motor imagery

With the popularity of deep learning, motor imagery electroencephalogram (MI-EEG) recognition based on feature extractors and classifiers has performed well. However, the features extracted by most models are not discriminative enough and are limited to specific-subject classifi-cation. We proposed a novel model Metric-based Spatial Filtering Transformer (MSFT) that utilizes additive angular margin loss to enforce the deep model to improve inter-class separability while enhancing intra-class compactness. Besides, a data augmentation method called EEG pyramid was applied to the model. Our model not only outperforms many recent benchmarks in specific-subject classifi-cation, but also is used for cross-subject and even cross-task classification. We did some experiments using BCI competition IV 2a and 2b datasets to evaluate the average accuracy. The Specific-subject: 86.11 % for 2a, 88.39 % for 2b. The Cross-subject: 61.92 % for 2a. The Cross-task: training the feature extractor with 2a data and then fine-tuning the classifier with 2b can achieve an average accuracy of 83.38 %. Our method is more general than most benchmarks and can deal with different kinds of classification situations.

A Fused Multidimensional EEG Classification Method Based on an Extreme Tree Feature Selection.

When a brain-computer interface (BCI) is designed, high classification accuracy is difficult to obtain for motor imagery (MI) electroencephalogram (EEG) signals in view of their relatively low signal-to-noise ratio. In this paper, a fused multidimensional classification method based on extreme tree feature selection (FMCM-ETFS) is proposed for discerning motor imagery EEG tasks. First, the EEG signal was filtered by a Butterworth filter for preprocessing. Second, C3, C4, and CZ channels were selected to extract time-frequency domain and spatial domain features using autoregressive (AR), common spatial pattern (CSP), and discrete wavelet transform (DWT). The extracted features were fused for a further feature elimination. Then, the features were selected using three feature selection methods: recursive feature elimination (RFE), principal component analysis method (PCA), and extreme trees (ET). The selected feature vectors were classified using support vector machines (SVM). Finally, a total of twelve subjects' EEG data from Inner Mongolia University of Technology (IMUT data), the 2nd BCI competition in 2003, and the 4th BCI competition in 2008 were employed to show the effectiveness of this proposed FMCM-ETFS method. The results show that the classification accuracy using the multidimensional fused feature extraction (AR + CSP + DWT) is 3%–20% higher than those using the aforementioned three single feature extractions (AR, CSP, and DWT). Extreme trees (ET), which is a sort of tree-based model method, outperforms RFE and PCA by 1%–9% in term of classification accuracies, when these three methods were applied to the procedure of feature extraction, respectively.

Multiattention Adaptation Network for Motor Imagery Recognition

Brain–computer interface (BCI) based on motor imagery electroencephalogram (EEG) has been widely used in various applications. Despite the previous efforts, the remained major challenges are effective feature extraction and the time-consuming calibration procedure. To address these issues, a novel multiattention adaptation network integrating the multiple attention mechanism and transfer learning is proposed to classify the EEG signals. First, the multiattention layer is introduced to automatically capture the dominant brain regions relevant to mental tasks without incorporating any prior knowledge about the physiology. Then, a multiattention convolutional neural network is employed to extract deep representation from raw EEG signals. Especially, a domain discriminator is applied to deep representation to reduce the differences between sessions for target subjects. The extensive experiments are conducted on three public EEG datasets (Dataset IIa and IIb of BCI Competition IV, and High Gamma dataset), achieving the competitive performance with average classification accuracy of 81.48%, 82.54%, and 93.97%, respectively. All the results outperform the state-of-the-art algorithms demonstrate the effectiveness and robustness of the proposed method. Importantly, we confirm that it is easier and more appropriate to transfer the information from local brain regions than from the whole brain. This enhances the transfer ability of deep features and, hence, it improves the performance of BCI systems.

Unsupervised Motor Imagery Saliency Detection Based on Self-Attention Mechanism.

Detecting the salient parts of motor-imagery electroencephalogram (MI-EEG) signals can enhance the performance of the brain-computer interface (BCI) system and reduce the computational burden required for processing lengthy MI-EEG signals. In this paper, we propose an unsupervised method based on the self-attention mechanism to detect the salient intervals of MI-EEG signals automatically. Our suggested method can be used as a preprocessing step within any BCI algorithm to enhance its performance. The effectiveness of the suggested method is evaluated on the most widely used BCI algorithm, the common spatial pattern (CSP) algorithm, using dataset 2a from BCI competition IV. The results indicate that the proposed method can effectively prune MI-EEG signals and significantly enhance the performance of the CSP algorithm in terms of classification accuracy.

TCACNet: Temporal and channel attention convolutional network for motor imagery classification of EEG-based BCI

Brain–computer interface (BCI) is a promising intelligent healthcare technology to improve human living quality across the lifespan, which enables assistance of movement and communication, rehabilitation of exercise and nerves, monitoring sleep quality, fatigue and emotion. Most BCI systems are based on motor imagery electroencephalogram (MI-EEG) due to its advantages of sensory organs affection, operation at free will and etc. However, MI-EEG classification, a core problem in BCI systems, suffers from two critical challenges: the EEG signal’s temporal non-stationarity and the nonuniform information distribution over different electrode channels. To address these two challenges, this paper proposes TCACNet, a temporal and channel attention convolutional network for MI-EEG classification. TCACNet leverages a novel attention mechanism module and a well-designed network architecture to process the EEG signals. The former enables the TCACNet to pay more attention to signals of task-related time slices and electrode channels, supporting the latter to make accurate classification decisions. We compare the proposed TCACNet with other state-of-the-art deep learning baselines on two open source EEG datasets. Experimental results show that TCACNet achieves 11.4% and 7.9% classification accuracy improvement on two datasets respectively. Additionally, TCACNet achieves the same accuracy as other baselines with about 50% less training data. In terms of classification accuracy and data efficiency, the superiority of the TCACNet over advanced baselines demonstrates its practical value for BCI systems.

Motor imagery electroencephalogram classification based on sparse spatiotemporal decomposition and channel attention

Motor imagery electroencephalogram (EEG) signals are non-stationary time series with a low signal-to-noise ratio. Therefore, the single-channel EEG analysis method is difficult to effectively describe the interaction characteristics between multi-channel signals. This paper proposed a deep learning network model based on the multi-channel attention mechanism. First, we performed time-frequency sparse decomposition on the pre-processed data, which enhanced the difference of time-frequency characteristics of EEG signals. Then we used the attention module to map the data in time and space so that the model could make full use of the data characteristics of different channels of EEG signals. Finally, the improved time-convolution network (TCN) was used for feature fusion and classification. The BCI competition IV-2a data set was used to verify the proposed algorithm. The experimental results showed that the proposed algorithm could effectively improve the classification accuracy of motor imagination EEG signals, which achieved an average accuracy of 83.03% for 9 subjects. Compared with the existing methods, the classification accuracy of EEG signals was improved. With the enhanced difference features between different motor imagery EEG data, the proposed method is important for the study of improving classifier performance.

Empirical Mode Decomposition and a Bidirectional LSTM Architecture Used to Decode Individual Finger MI-EEG Signals

Brain-Computer Interface (BCI) paradigms based on Motor Imagery Electroencephalogram (MI-EEG) signals have been developed because the related signals can be generated voluntarily to control further applications. Researches using strong and stout limbs MI-EEG signals reported performing significant classification rates for BCI applied systems. However, MI-EEG signals produced by imagined movements of small limbs present a real classification challenge to be effectively used in BCI systems. It is due to a reduced signal level and increased noisy distorted effects. This study aims to decode individual right-hand fingers’ imagined movements for BCI applications, using MI-EEG signals from C3, Cz, P3, and Pz channels. For this purpose, the Empirical Mode Decomposition (EMD) preprocesses the non-stationary and non-linear EEG signals to finally use a Bidirectional Long Short-Term Memory (BiLSTM) to classify corresponding feature sequences. An average accuracy of 98.8 % was achieved for ring-finger movements decoding using k-fold cross-validation on a public dataset (Scientific-Data). The obtained results support that the proposed framework can be used for BCI control applications.

Low-quality training data detection method of EEG signals for motor imagery BCI system

BackgroundThe design and implementation of high-performance motor imagery-based brain computer interface (MI-BCI) requires high-quality training samples. However, fluctuation in subjects’ physiological and mental states as well as artifacts can produce the low-quality motor imagery electroencephalogram (EEG) signal, which will damage the performance of MI-BCI system. New methodIn order to select high-quality MI-EEG training data, this paper proposes a low-quality training data detection method combining independent component analysis (ICA) and weak classifier cluster. we also design and implement a new online BCI system based on motor imagery to verify the online processing performance of the proposed method. ResultIn order to verify the effectiveness of the proposed method, we conducted offline experiments on the public dataset called BCI Competition IV Data Set 2b. Furthermore, in order to verify the processing performance of the online system, we designed 60 groups of online experiments on 12 subjects. The online experimental results show that the twelve subjects can complete the system task efficiently (the best experiment is 135.6 s with 9 trials of subject S1). ConclusionThis paper demonstrated that the proposed low-quality training data detection method can effectively screen out low-quality training samples, so as to improve the performance of the MI-BCI system.

Bispectrum-based hybrid neural network for motor imagery classification

BackgroundThe performance of motor imagery electroencephalogram (MI-EEG) decoding systems is easily affected by noise. As a higher-order spectra (HOS), the bispectrum is capable of suppressing Gaussian noise and increasing the signal-to-noise ratio of signals. However, the sum of logarithmic amplitudes (SLA) and the first order spectral moment (FOSM) features extracted from the bispectrum only use the numerical values of the bispectrum, ignoring the related information between different frequency bins. New methodIn this study, we proposed a novel framework, termed a bispectrum-based hybrid neural network (BHNN), to make full use of bispectrum for improving the performance of the MI-based brain-computer interface (BCI). Specifically, the BHNN consisted of a convolutional neural network (CNN), gated recurrent units (GRU), and squeeze-and-excitation (SE) modules. The SE modules and CNNs are first used to learn the deep relation between frequency bins of the bispectrum estimated from different time window segmentations of the MI-EEG. Then, we used GRU to seek the overlooked sequential information of the bispectrum. ResultsTo validate the effectiveness of the proposed BHNN, three public BCI competition datasets were used in this study. The results demonstrated that the BHNN can achieve promising performance in decoding MI-EEG. Comparison with existing methodsThe statistical test results demonstrated that the proposed BHNN can significantly outperform other competing methods (p < =0.05). ConclusionThe proposed BHNN is a novel bispectrum-based neural network, which can enhance the decoding performance of MI-based BCIs.

A dynamic directed transfer function for brain functional network-based feature extraction

Directed transfer function (DTF) is good at characterizing the pairwise interactions from whole brain network and has been applied in discrimination of motor imagery (MI) tasks. Considering the fact that MI electroencephalogram signals are more non-stationary in frequency domain than in time domain, and the activated intensities of α band (8–13 Hz) and β band [13–30 Hz, with beta_{1}(13–21 Hz) and beta_{2}(21–30 Hz) included] have considerable differences for different subjects, a dynamic DTF (DDTF) with variable model order and frequency band is proposed to construct the brain functional networks (BFNs), whose information flows and outflows are further calculated as network features and evaluated by support vector machine. Extensive experiments are conducted based on a public BCI competition dataset and a real-world dataset, the highest recognition rate achieve 100% and 86%, respectively. The experimental results suggest that DDTF can reflect the dynamic evolution of BFN, the best subject-based DDTF appears in one of four frequency sub-bands (α, β, beta_{1} ,{ }beta_{2}) for discrimination of MI tasks and is much more related to the current and previous states. Besides, DDTF is superior compared to granger causality-based and traditional feature extraction methods, the t-test and Kappa values show its statistical significance and high consistency as well.

A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals

In the Motor Imagery (MI)-based Brain Computer Interface (BCI), users' intention is converted into a control signal through processing a specific pattern in brain signals reflecting motor characteristics. There are such restrictions as the limited size of the existing datasets and low signal to noise ratio in the classification of MI Electroencephalogram (EEG) signals. Machine learning (ML) methods, particularly Deep Learning (DL), have overcome these limitations relatively. In this study, three hybrid models were proposed to classify the EEG signal in the MI-based BCI. The proposed hybrid models consist of the convolutional neural networks (CNN) and the Long-Short Term Memory (LSTM). In the first model, the CNN with different number of convolutional-pooling blocks (from shallow to deep CNN) was examined; a two-block CNN model not affected by the vanishing gradient descent and yet able to extract desirable features employed; the second and third models contained pre-trained CNNs conducing to the exploration of more complex features. The transfer learning strategy and data augmentation methods were applied to overcome the limited size of the datasets by transferring learning from one model to another. This was achieved by employing two powerful pre-trained convolutional neural networks namely ResNet-50 and Inception-v3. The continuous wavelet transform (CWT) was used to generate images for the CNN. The performance of the proposed models was evaluated on the BCI Competition IV dataset 2a. The mean accuracy vlaues of 86%, 90%, and 92%, and mean Kappa values of 81%, 86%, and 88% were obtained for the hybrid neural network with the customized CNN, the hybrid neural network with ResNet-50 and the hybrid neural network with Inception-v3, respectively. Despite the promising performance of the three proposed models, the hybrid neural network with Inception-v3 outperformed the two other models. The best obtained result in the present study improved the previous best result in the literature by 7% in terms of classification accuracy. From the findings, it can be concluded that transfer learning based on a pre-trained CNN in combination with LSTM is a novel method in MI-based BCI. The study also has implications for the discrimination of motor imagery tasks in each EEG recording channel and in different brain regions which can reduce computational time in future works by only selecting the most effective channels.

Single-trial motor imagery electroencephalogram intention recognition by optimal discriminant hyperplane and interpretable discriminative rectangle mixture model.

The online version contains supplementary material available at 10.1007/s11571-021-09768-w.

Motor Imagery EEG Decoding Based on New Spatial-Frequency Feature and Hybrid Feature Selection Method

Feature extraction and selection are important parts of motor imagery electroencephalogram (EEG) decoding and have always been the focus and difficulty of brain-computer interface (BCI) system research. In order to improve the accuracy of EEG decoding and reduce model training time, new feature extraction and selection methods are proposed in this paper. First, a new spatial-frequency feature extraction method is proposed. The original EEG signal is preprocessed, and then the common spatial pattern (CSP) is used for spatial filtering and dimensionality reduction. Finally, the filter bank method is used to decompose the spatially filtered signals into multiple frequency subbands, and the logarithmic band power feature of each frequency subband is extracted. Second, to select the subject-specific spatial-frequency features, a hybrid feature selection method based on the Fisher score and support vector machine (SVM) is proposed. The Fisher score of each feature is calculated, then a series of threshold parameters are set to generate different feature subsets, and finally, SVM and cross-validation are used to select the optimal feature subset. The effectiveness of the proposed method is validated using two sets of publicly available BCI competition data and a set of self-collected data. The total average accuracy of the three data sets achieved by the proposed method is 82.39%, which is 2.99% higher than the CSP method. The experimental results show that the proposed method has a better classification effect than the existing methods, and at the same time, feature extraction and feature selection time also have greater advantages.

Wearable Supernumerary Robotic Limb System Using a Hybrid Control Approach Based on Motor Imagery and Object Detection

Motor disorder of upper limbs has seriously affected the daily life of the patients with hemiplegia after stroke. We developed a wearable supernumerary robotic limb (SRL) system using a hybrid control approach based on motor imagery (MI) and object detection for upper-limb motion assistance. SRL system included an SRL hardware subsystem and a hybrid control software subsystem. The system obtained the patient's motion intention through MI electroencephalogram (EEG) recognition method based on graph convolutional network (GCN) and gated recurrent unit network (GRU) to control the left and right movements of SRL, and the object detection technology was used together for a quick grasp of target objects to compensate for the disadvantages when using MI EEG alone like fewer control instructions and lower control efficiency. Offline training experiment was designed to obtain subjects' MI recognition models and evaluate the feasibility of the MI EEG recognition method; online control experiment was designed to verify the effectiveness of our wearable SRL system. The results showed that the proposed MI EEG recognition method (GCN+GRU) could effectively improve the MI classification accuracy (90.04% ± 2.36 %) compared with traditional methods; all subjects were able to complete the target object grasping tasks within 23 seconds by controlling the SRL, and the highest average grasping success rate achieved 90.67% in bag grasping task. The SRL system can effectively assist people with upper-limb motor disorder to perform upper-limb tasks in daily life by natural human-robot interaction, and improve their ability of self-help and enhance their confidence of life.

Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture

Motor Imagery Electroencephalogram (MI-EEG) signals are widely used in Brain-Computer Interfaces (BCI). MI-EEG signals of large limbs movements have been explored in recent researches because they deliver relevant classification rates for BCI systems. However, smaller and noisy signals corresponding to hand-finger imagined movements are less frequently used because they are difficult to classify. This study proposes a method for decoding finger imagined movements of the right hand. For this purpose, MI-EEG signals from C3, Cz, P3, and Pz sensors were carefully selected to be processed in the proposed framework. Therefore, a method based on Empirical Mode Decomposition (EMD) is used to tackle the problem of noisy signals. At the same time, the sequence classification is performed by a stacked Bidirectional Long Short-Term Memory (BiLSTM) network. The proposed method was evaluated using k-fold cross-validation on a public dataset, obtaining an accuracy of 82.26%.

Deep CNN model based on serial-parallel structure optimization for four-class motor imagery EEG classification

Motor imagery electroencephalogram (MI-EEG) is one of the most important brain-computer interface (BCI) signal. It is vital to analyze the MI-EEG for the manipulation of external BCI actuator. However, traditional methods usually undertake EEG feature extraction and classification separately, which may lose efficient feature information. It behaves beyond our satisfaction for multi-class MI activity evoked by space-close and cannot eliminate the influence of individual differences. To solve these problems, we propose a convolutional neural network (CNN) with an end-to-end serial-parallel (SP) structure followed by tranfer learning. In detail, we use the serial module to extract the rough features in time–frequency-space domain, and the parallel module for fine feature learning in different scales. Meanwhile, a freeze-and-retrain fune tuning transfer learning strategy is proposed to improve the cross-subject accuracy. When our model is compared with the other three typical networks, results show that the proposed model performs best with the average testing accuracy of 72.13% and the average loss of 0.47, among which one subject only takes 0.7 s to reach 89.17% as the highest one. Through transfer learning, we reduce the training parameters by 53%. The average cross-subject classification accuracy increases by approximate 15%, and the individual highest accuracy reaches 76.98%. In conclusion, the integrity and separability of SPCNN determine that we require no additional EEG signal feature analysis, which is conducive to the realization of an efficient online BCI. It can also get rid of the dependence on training time and subject data to rapidly advance BCI in the future.

Progress of classification algorithms for motor imagery electroencephalogram signals

Motor imagery (MI), motion intention of the specific body without actual movements, has attracted wide attention in fields as neuroscience. Classification algorithms for motor imagery electroencephalogram (MI-EEG) signals are able to distinguish different MI tasks based on the physiological information contained by the EEG signals, especially the features extracted from them. In recent years, there have been some new advances in classification algorithms for MI-EEG signals in terms of classifiers versus machine learning strategies. In terms of classifiers, traditional machine learning classifiers have been improved by some researchers, deep learning and Riemannian geometry classifiers have been widely applied as well. In terms of machine learning strategies, ensemble learning, adaptive learning, and transfer learning strategies have been utilized to improve classification accuracies or reach other targets. This paper reviewed the progress of classification algorithms for MI-EEG signals, summarized and evaluated the existing classifiers and machine learning strategies, to provide new ideas for developing classification algorithms with higher performance.

Combination of Variational Mode Decomposition for Feature Extraction and Deep Belief Network for Feature Classification in Motor Imagery Electroencephalogram Recognition

Combination of Variational Mode Decomposition for Feature Extraction and Deep Belief Network for Feature Classification in Motor Imagery Electroencephalogram Recognition

CluSem: Accurate clustering-based ensemble method to predict motor imagery tasks from multi-channel EEG data

BackgroundThe classification of motor imagery electroencephalogram (MI-EEG) is a pivotal task in the biosignal classification process in the brain-computer interface (BCI) applications. Currently, this bio-engineering-based technology is being employed by researchers in various fields to develop cutting-edge applications. The classification of real-time MI-EEG signals is the most challenging task in these applications. The prediction performance of the existing classification methods is still limited due to the high dimensionality and dynamic behaviors of the real-time EEG data. Proposed methodTo enhance the classification performance of real-time BCI applications, this paper presents a new clustering-based ensemble technique called CluSem to mitigate this problem. We also develop a new brain game called CluGame using this method to evaluate the classification performance of real-time motor imagery movements. In this game, real-time EEG signal classification and prediction tabulation through animated balls are controlled via threads. By playing this game, users can control the movements of the balls via the brain signals of motor imagery movements without using any traditional input devices. ResultsOur results demonstrate that CluSem is able to improve the classification accuracy between 5% and 15% compared to the existing methods on our collected as well as the publicly available EEG datasets. The source codes used to implement CluSem and CluGame are publicly available at https://github.com/MdOchiuddinMiah/MI-BCI_ML.

Classification of motor imagery electroencephalogram signals by using a divergence based convolutional neural network

Deep neural networks (DNNs) are observed to be successful in pattern classification. However, the high classification performances of DNNs are related to their large training sets. Unfortunately, in the literature, the datasets used to classify motor imagery (MI) electroencephalogram (EEG) signals contain a small number of samples. To achieve high performances with small-sized datasets, most of the studies have employed a transformation such as the common spatial patterns (CSP) before the classification process. However, the CSP is dependent on subjects and introduces computational load in real time applications. It is observed in the literature that data augmentation is not applied for increasing the classification performance of EEG signals. In this study, we have investigated the effect of the augmentation process on the classification performance of MI EEG signals instead of using a preceding transformation such as the CSP, and we have demonstrated that the augmentation process is able to compete with the CSP by generating high success rates for the classification of MI EEGs. In addition to the augmentation process, we have modified the DNN structure to increase the classification performance, to decrease the number of nodes in the structure, and to use less number of hyper parameters. A minimum distance network following the last layer of the convolutional neural network (CNN) was used as the classifier instead of a fully connected neural network (FCNN). By augmenting the EEG dataset and focusing solely on CNN’s training, the training algorithm of the proposed structure is strengthened without applying any transformation. We tested these improvements on brain–computer interface (BCI) competitions 2005 and 2008 databases with two and four classes, and the positive effects of the augmentation on the average accuracies are demonstrated.