Brain-computer Interface Methods Research Articles

EEG-based brain-computer interface methods with the aim of rehabilitating advanced stage ALS patients

Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease that leads to progressive muscle weakness and paralysis, ultimately resulting in the loss of ability to communicate and control the environment. EEG-based Brain-Computer Interface (BCI) methods have shown promise in providing communication and control with the aim of rehabilitating ALS patients. In particular, P300-based BCI has been widely studied and used for ALS rehabilitation. Other EEG-based BCI methods, such as Motor Imagery (MI) based BCI and Hybrid BCI, have also shown promise in ALS rehabilitation. Nonetheless, EEG-based BCI methods hold great potential for improvement. This review article introduces and reviews FFT, WPD, CSP, CSSP, CSP, and GC feature extraction methods. The Common Spatial Pattern (CSP) is an efficient and common technique for extracting data properties used in BCI systems. In addition, Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Neural Networks (NN), and Deep Learning (DL) classification methods were introduced and reviewed. SVM is the most appropriate classifier due to its insensitivity to the curse of dimensionality. Also, DL is used in the design of BCI systems and is a good choice for BCI systems based on motor imagery with big datasets. Despite the progress made in the field, there are still challenges to overcome, such as improving the accuracy and reliability of EEG signal detection and developing more intuitive and user-friendly interfaces By using BCI, disabled patients can communicate with their caregivers and control their environment using various devices, including wheelchairs, and robotic arms.

EEG rhythm separation and time-frequency analysis of fast multivariate empirical mode decomposition for motor imagery BCI.

Motor imagery electroencephalogram (EEG) is widely employed in brain-computer interface (BCI) systems. As a time-frequency analysis method for nonlinear and non-stationary signals, multivariate empirical mode decomposition (MEMD) and its noise-assisted version (NA-MEMD) has been widely used in the preprocessing step of BCI systems for separating EEG rhythms corresponding to specific brain activities. However, when applied to multichannel EEG signals, MEMD or NA-MEMD often demonstrate low robustness to noise and high computational complexity. To address these issues, we have explored the advantages of our recently proposed fast multivariate empirical mode decomposition (FMEMD) and its noise-assisted version (NA-FMEMD) for analyzing motor imagery data. We emphasize that FMEMD enables a more accurate estimation of EEG frequency information and exhibits a more noise-robust decomposition performance with improved computational efficiency. Comparative analysis with MEMD on simulation data and real-world EEG validates the above assertions. The joint average frequency measure is employed to automatically select intrinsic mode functions that correspond to specific frequency bands. Thus, FMEMD-based classification architecture is proposed. Using FMEMD as a preprocessing algorithm instead of MEMD can improve the classification accuracy by 2.3% on the BCI Competition IV dataset. On the Physiobank Motor/Mental Imagery dataset and BCI Competition IV Dataset 2a, FMEMD-based architecture also attained a comparable performance to complex algorithms. The results indicate that FMEMD proficiently extracts feature information from small benchmark datasets while mitigating dimensionality constraints resulting from computational complexity. Hence, FMEMD or NA-FMEMD can be a powerful time-frequency preprocessing method for BCI.

An improved cross-subject spatial filter transfer method for SSVEP-based BCI

Objective. Steady-state visual evoked potential (SSVEP) training feature recognition algorithms utilize user training data to reduce the interference of spontaneous electroencephalogram activities on SSVEP response for improved recognition accuracy. The data collection process can be tedious, increasing the mental fatigue of users and also seriously affecting the practicality of SSVEP-based brain-computer interface (BCI) systems. Approach. As an alternative, a cross-subject spatial filter transfer (CSSFT) method to transfer an existing user data model with good SSVEP response to new user test data has been proposed. The CSSFT method uses superposition averages of data for multiple blocks of data as transfer data. However, the amplitude and pattern of brain signals are often significantly different across trials. The goal of this study was to improve superposition averaging for the CSSFT method and propose an Ensemble scheme based on ensemble learning, and an Expansion scheme based on matrix expansion. Main results. The feature recognition performance was compared for CSSFT and the proposed improved CSSFT method using two public datasets. The results demonstrated that the improved CSSFT method can significantly improve the recognition accuracy and information transmission rate of existing methods. Significance. This strategy avoids a tedious data collection process, and promotes the potential practical application of BCI systems.

Control of a Robotic Arm With an Optimized Common Template-Based CCA Method for SSVEP-Based BCI.

Recently, the robotic arm control system based on a brain-computer interface (BCI) has been employed to help the disabilities to improve their interaction abilities without body movement. However, it's the main challenge to implement the desired task by a robotic arm in a three-dimensional (3D) space because of the instability of electroencephalogram (EEG) signals and the interference by the spontaneous EEG activities. Moreover, the free motion control of a manipulator in 3D space is a complicated operation that requires more output commands and higher accuracy for brain activity recognition. Based on the above, a steady-state visual evoked potential (SSVEP)-based synchronous BCI system with six stimulus targets was designed to realize the motion control function of the seven degrees of freedom (7-DOF) robotic arm. Meanwhile, a novel template-based method, which builds the optimized common templates (OCTs) from various subjects and learns spatial filters from the common templates and the multichannel EEG signal, was applied to enhance the SSVEP recognition accuracy, called OCT-based canonical correlation analysis (OCT-CCA). The comparison results of offline experimental based on a public benchmark dataset indicated that the proposed OCT-CCA method achieved significant improvement of detection accuracy in contrast to CCA and individual template-based CCA (IT-CCA), especially using a short data length. In the end, online experiments with five healthy subjects were implemented for achieving the manipulator real-time control system. The results showed that all five subjects can accomplish the tasks of controlling the manipulator to reach the designated position in the 3D space independently.

Uso do significado linguístico para suporte à comunicação da informação

Introduction:Ferdinand Saussure, linguist, semiologist, philosopher and one of the main founders of semiotics, affirms that "Meaning"(significance) is a representation of something created in the mind, an association that is useful for a "Signifier" that is a psychic impression of sound. Objective:In this context, the objective of this research is to verify the feasibility of generating communication based on images formed in the mind (Signified) and what it can represent cognitively related to the Signifier using a brain-computer interface Method:A computer brain interface has been developed and a user has been tested so that it uses neuro-muscular commands and pure mental commands that invoke Signified (records in the user's mind) that represent a goal of communicating. Result:The results allow to evaluate a relationship between signified and signifier of information drawing from thebrain. Psychic images of a communication intent were linked to sound images that are also mental entities, when brain activated, are converted in “speech” (physical sound) computationally. Conclusion:The results demonstrate the feasibility of communication in this modality, which could support the basic communication needs of people who do not communicate orally

User State Classification Based on Functional Brain Connectivity Using a Convolutional Neural Network

The brain–computer interface (BCI) is a promising technology where a user controls a robot or computer by thinking with no movement. There are several underlying principles to implement BCI, such as sensorimotor rhythms, P300, steady-state visually evoked potentials, and directional tuning. Generally, different principles are applied to BCI depending on the application, because strengths and weaknesses vary according to each BCI method. Therefore, BCI should be able to predict a user state to apply suitable principles to the system. This study measured electroencephalography signals in four states (resting, speech imagery, leg-motor imagery, and hand-motor imagery) from 10 healthy subjects. Mutual information from 64 channels was calculated as brain connectivity. We used a convolutional neural network to predict a user state, where brain connectivity was the network input. We applied five-fold cross-validation to evaluate the proposed method. Mean accuracy for user state classification was 88.25 ± 2.34%. This implies that the system can change the BCI principle using brain connectivity. Thus, a BCI user can control various applications according to their intentions.

A binary harmony search algorithm as channel selection method for motor imagery-based BCI

BackgroundChannel selection is a key topic in brain-computer interface (BCI). Task-irrelevant and redundant channels used in BCI may lead to low classification accuracy, high computational complexity, and inconvenience for application. By selecting optimal channels, the performance of BCI could enhance significantly. MethodIn this paper, a new binary harmony search (BHS) is proposed to select the optimal channel sets and optimize the system accuracy. The BHS is implemented on the training data sets to select the optimal channels and the test data sets are used to evaluate the classification performance on the selected channels. The sparse representation-based classification, linear discriminant analysis, and support vector machine are performed on the common spatial pattern (CSP) features for motor imagery (MI) classification. ResultsTwo public EEG datasets are employed to validate the proposed BHS method. The paired t-test is conducted on the test classification performance between the BHS and traditional CSP with all channels. The results reveal that the proposed BHS method significantly improved classification accuracy as compared to the conventional CSP method (p < 0.05). ConclusionThis study proposed the BHS method to select the optimal channels in MI -based BCI. On the one hand, the results confirm the validity of the BHS algorithm as a channel selection method for motor imagery data. On the other hand, the BHS method with costing shorter computation time relatively yields a better average test accuracy than the steady-state genetic algorithms. The proposed method could significantly improve the practicability and convenience of the BCI system.

The Improvement of a Brain Computer Interface Based on EEG Signals

Purpose: Brain Computer Interface (BCI) has provided a novel way of communication that can significantly revolutionize life of people suffering from disabilities. Motor Imagery (MI) EEG BCI is one of the most promising solutions to address. The main phases of such systems include signal acquisition, pre-processing, feature extraction, classification and the intended interface. The challenging obstacles in such systems are to detect and extract efficient features that present reliability and robustness alongside promising classification accuracy. In this paper it is endeavored to present a robust method for a two-class MI BCI that results in high accuracy.&#x0D; Materials and Methods: For this purpose, the dataset 2b from BCI competition 2008, consisting of three channels (C3, C and Cz), was utilized. Firstly, the signals were bandpass filtered. Secondly, Common Spatial Pattern (CSP) was employed and then a number of features, including non-linear chaotic features were extracted from channels C3 and C4. After feature selection phase the number of features were reduced to 38 and 47. Finally, these features were fed into two classifiers, namely Support Vector Machine classifier (SVM) and Bagging to evaluate the performance of the system.&#x0D; Results: Classification accuracy and Cohen’s Kappa coefficient of the proposed method for two MI EEG channels are 96.40% and 0.92, respectively.&#x0D; Conclusion: These results indicate the high accuracy and stability of our method in comparison with similar studies. Therefore, it can be a promising approach in two-class MI BCI systems.

A Novel RL-Assisted Deep Learning Framework for Task-Informative Signals Selection and Classification for Spontaneous BCIs

In this article, we formulate the problem of estimating and selecting task-relevant temporal signal segments from a single electroencephalogram (EEG) trial in the form of a Markov decision process and propose a novel reinforcement-learning mechanism that can be combined with the existing deep-learning-based brain–computer interface methods. To be specific, we devise an actor–critic network such that an agent can determine which timepoints need to be used (informative) or discarded (uninformative) in composing the intention-related features in a given trial, and thus enhancing the intention identification performance. To validate the effectiveness of our proposed method, we conduct experiments with a publicly available big motor imagery (MI) dataset and apply our novel mechanism to various recent deep-learning architectures designed for MI classification. Based on the exhaustive experiments, we observe that our proposed method helped achieve statistically significant improvements in performance.

A Computationally Efficient Method for Hybrid EEG-fNIRS BCI Based on the Pearson Correlation.

A hybrid brain computer interface (BCI) system considered here is a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG-fNIRS signals are simultaneously recorded to achieve high motor imagery task classification. This integration helps to achieve better system performance, but at the cost of an increase in system complexity and computational time. In hybrid BCI studies, channel selection is recognized as the key element that directly affects the system's performance. In this paper, we propose a novel channel selection approach using the Pearson product-moment correlation coefficient, where only highly correlated channels are selected from each hemisphere. Then, four different statistical features are extracted, and their different combinations are used for the classification through KNN and Tree classifiers. As far as we know, there is no report available that explored the Pearson product-moment correlation coefficient for hybrid EEG-fNIRS BCI channel selection. The results demonstrate that our hybrid system significantly reduces computational burden while achieving a classification accuracy with high reliability comparable to the existing literature.

Automatic Identification of Brain Independent Components in Electroencephalography Data Collected while Standing in a Virtually Immersive Environment - A Deep Learning-Based Approach.

Electroencephalography (EEG) is a commonly used method for monitoring brain activity. Automating an EEG signal processing pipeline is imperative to the exploration of real-time brain computer interface (BCI) applications. EEG analysis demands substantial training and time for removal of distinct unwanted independent components (ICs), generated via independent component analysis, corresponding to artifacts. The considerable subject-wise variations across these components motivates defining a procedural way to identify and eliminate these artifacts. We propose DeepIC-virtual, a convolutional neural network (CNN) deep learning classifier to automatically identify brain components in the ICs extracted from the subject's EEG data gathered while they are being immersed in a virtual reality (VR) environment. This work examined the feasibility of DL techniques to provide automated ICs classification on noisy and visually engaging upright stance EEG data. We collected the EEG data for six subjects while they were standing upright in a VR testing setup simulating pseudo-randomized variations in height and depth conditions and induced perturbations. An extensive 1432 IC representation images data set was generated and manually labelled via an expert as brain components or one of the six distinct removable artifacts. The supervised CNN architecture was utilized to categorize good brain ICs and bad artifactual ICs via generated images of topographical maps. Our model categorizing good versus bad IC topographical maps resulted in a binary classification accuracy and area under curve of 89.20% and 0.93 respectively. Despite significant imbalance, only 1 out of the 57 present brain ICs in the withheld testing set was miss-classified as an artifact. These results will hopefully encourage clinicians to integrate BCI methods and neurofeedback to control anxiety and provide a treatment of acrophobia, given the viability of automatic classification of artifactual ICs.

A Novel Deep Learning Scheme for Motor Imagery EEG Decoding Based on Spatial Representation Fusion

Motor imagery electroencephalography (MI-EEG), which is an important subfield of active brain-computer interface (BCI) systems, can be applied to help disabled people to consciously and directly control prosthesis or external devices, aiding them in certain daily activities. However, the low signal-to-noise ratio and spatial resolution make MI-EEG decoding a challenging task. Recently, some deep neural approaches have shown good improvements over state-of-the-art BCI methods. In this study, an end-to-end scheme that includes a multi-layer convolution neural network is constructed for an accurate spatial representation of multi-channel grouped MI-EEG signals, which is employed to extract the useful information present in a multi-channel MI signal. Then the invariant spatial representations are captured from across-subjects training for enhancing the generalization capability through a stacked sparse autoencoder framework, which is inspired by representative deep learning models. Furthermore, a quantitative experimental analysis is conducted on our private dataset and on a public BCI competition dataset. The results show the effectiveness and significance of the proposed methodology.

World's fastest brain-computer interface: Combining EEG2Code with deep learning.

We present a novel approach based on deep learning for decoding sensory information from non-invasively recorded Electroencephalograms (EEG). It can either be used in a passive Brain-Computer Interface (BCI) to predict properties of a visual stimulus the person is viewing, or it can be used to actively control a BCI application. Both scenarios were tested, whereby an average information transfer rate (ITR) of 701 bit/min was achieved for the passive BCI approach with the best subject achieving an online ITR of 1237 bit/min. Further, it allowed the discrimination of 500,000 different visual stimuli based on only 2 seconds of EEG data with an accuracy of up to 100%. When using the method for an asynchronous self-paced BCI for spelling, an average utility rate of 175 bit/min was achieved, which corresponds to an average of 35 error-free letters per minute. As the presented method extracts more than three times more information than the previously fastest approach, we suggest that EEG signals carry more information than generally assumed. Finally, we observed a ceiling effect such that information content in the EEG exceeds that required for BCI control, and therefore we discuss if BCI research has reached a point where the performance of non-invasive visual BCI control cannot be substantially improved anymore.

Classification of EEG-based single-trial motor imagery tasks using a B-CSP method for BCI

Classifying single-trial electroencephalogram (EEG) based motor imagery (MI) tasks is extensively used to control brain-computer interface (BCI) applications, as a communication bridge between humans and computers. However, the low signal-to-noise ratio and individual differences of EEG can affect the classification results negatively. In this paper, we propose an improved common spatial pattern (B-CSP) method to extract features for alleviating these adverse effects. First, for different subjects, the method of Bhattacharyya distance is used to select the optimal frequency band of each electrode including strong event-related desynchronization (ERD) and event-related synchronization (ERS) patterns; then the signals of the optimal frequency band are decomposed into spatial patterns, and the features that can describe the maximum differences of two classes of MI are extracted from the EEG data. The proposed method is applied to the public data set and experimental data set to extract features which are input into a back propagation neural network (BPNN) classifier to classify single-trial MI EEG. Another two conventional feature extraction methods, original common spatial pattern (CSP) and autoregressive (AR), are used for comparison. An improved classification performance for both data sets (public data set: 91.25%±1.77% for left hand vs. foot and 84.50%±5.42% for left hand vs. right hand; experimental data set: 90.43%±4.26% for left hand vs. foot) verifies the advantages of the B-CSP method over conventional methods. The results demonstrate that our proposed B-CSP method can classify EEG-based MI tasks effectively, and this study provides practical and theoretical approaches to BCI applications.

A two-step idle-state detection method for SSVEP BCI.

The role of detecting work/idle state in asynchronous Steady-state visual evoked potential (SSVEP) Brain-computer interface (BCI) or a self-paced SSVEP BCI has received increased attention in recent years. This study proposed a tree structure method which identifies the work/idle state based on the frequency recognition to detect work/idle state. Firstly, a frequency recognition estimated with task-related component analysis (TRCA). Then, the work/idle state is classified with step-wise linear discriminant analysis (SWLDA) using the data fusion of TRCA scores and power spectral density (PSD) as features. This method was evaluated by Electroencephalography (EEG) data from fourteen healthy participants with eight frequencies as work states and three idle state conditions. The averaged AUC of this method achieved 0.89 with data lengths of one second, which was significantly higher than that of the conventional power spectrum-based algorithm. The proposed method could identify the work/idle state fast and accurately, making the SSVEP BCI better suited for practical application.

Development of a Smart Helmet for Strategical BCI Applications.

Conducting electrophysiological measurements from human brain function provides a medium for sending commands and messages to the external world, as known as a brain–computer interface (BCI). In this study, we proposed a smart helmet which integrated the novel hygroscopic sponge electrodes and a combat helmet for BCI applications; with the smart helmet, soldiers can carry out extra tasks according to their intentions, i.e., through BCI techniques. There are several existing BCI methods which are distinct from each other; however, mutual issues exist regarding comfort and user acceptability when utilizing such BCI techniques in practical applications; one of the main challenges is the trade-off between using wet and dry electroencephalographic (EEG) electrodes. Recently, several dry EEG electrodes without the necessity of conductive gel have been developed for EEG data collection. Although the gel was claimed to be unnecessary, high contact impedance and low signal-to-noise ratio of dry EEG electrodes have turned out to be the main limitations. In this study, a smart helmet with novel hygroscopic sponge electrodes is developed and investigated for long-term usage of EEG data collection. The existing electrodes and EEG equipment regarding BCI applications were adopted to examine the proposed electrode. In the impedance test of a variety of electrodes, the sponge electrode showed performance averaging 118 kΩ, which was comparable with the best one among existing dry electrodes, which averaged 123 kΩ. The signals acquired from the sponge electrodes and the classic wet electrodes were analyzed with correlation analysis to study the effectiveness. The results indicated that the signals were similar to each other with an average correlation of 90.03% and 82.56% in two-second and ten-second temporal resolutions, respectively, and 97.18% in frequency responses. Furthermore, by applying the proposed differentiable power algorithm to the system, the average accuracy of 21 subjects can reach 91.11% in the steady-state visually evoked potential (SSVEP)-based BCI application regarding a simulated military mission. To sum up, the smart helmet is capable of assisting the soldiers to execute instructions with SSVEP-based BCI when their hands are not available and is a reliable piece of equipment for strategical applications.

A Brain-Computer Interface Based on Three-Dimensional Stereo Stimuli for Assisting Clinical Object Recognition Assessment in Patients With Disorders of Consciousness.

The coma recovery scale-revised (CRS-R) behavioral scale is commonly used for the clinical evaluation of patients with disorders of consciousness (DOC). However, since DOC patients generally cannot supply stable and efficient behavioral responses to external stimulation, evaluation results based on behavioral scales are not sufficiently accurate. In this paper, we proposed a novel brain-computer interface (BCI) based on 3D stereo audiovisual stimuli to supplement object recognition evaluation in the CRS-R. During the experiment, subjects needed to follow the instructions and to focus on the target object on the screen, whereas EEG data were recorded and analyzed in real time to determine the object of focus, and the detection result was output as feedback. Thirteen DOC patients participated in the object recognition assessments using the 3D audiovisual BCI and CRS-R. None of the patients showed object recognition function in the CRS-R assessment before the BCI experiment. However, six of these DOC patients achieved accuracies that were significantly higher than the chance level in the BCI-based assessment, indicating the successful detection of object recognition function in these six patients using our 3D audiovisual BCI system. These results suggest that the BCI method may provide a more sensitive object recognition evaluation compared with CRS-R and may be used to assist clinical CRS-R for DOC patients.

SimBCI - A framework for studying BCI methods by simulated EEG.

Brain-Computer Interface (BCI) methods are commonly studied using Electroencephalogram (EEG) data recorded from human experiments. For understanding and developing BCI signal processing techniques real data is costly to obtain and its composition is apriori unknown. The brain mechanisms generating the EEG are not directly observable and their states cannot be uniquely identified from the EEG. Subsequently, we do not have generative ground truth for real data. In this paper we propose a novel convenience framework called simBCI to alleviate testing and studying BCI signal processing methods in simulated, controlled conditions. The framework can be used to generate artificial BCI data and to test classification pipelines with such data. Models and parameters on both data generation and the signal processing side can be iterated over to examine the interplay of different combinations. The framework provides the first time open source implementations of several models and methods. We invite researchers to insert more advanced models. The proposed system does not intend to replace human experiments. Instead, it can be used to discover hypotheses, study algorithms, to educate about BCI, and to debug signal processing pipelines of other BCI systems. The proposed framework is modular, extensible and freely available as open source1. It currently requires Matlab.

The control of a virtual automatic car based on multiple patterns of motor imagery BCI.

Multiple degrees of freedom (DOF) commands are required for a brain-actuated virtual automatic car, which makes the brain-computer interface (BCI) control strategy a big challenge. In order to solve the challenging issue, a mixed model of BCI combining P300 potentials and motor imagery had been realized in our previous study. However, compared with single model BCI, more training procedures are needed for the mixed model and more mental workload for users to bear. In the present study, we propose a multiple patterns of motor imagery (MPMI) BCI method, which is based on the traditional two patterns of motor imagery. Our motor imagery BCI approach had been extended to multiple patterns: right-hand motor imagery, left-hand motor imagery, foot motor imagery, and both hands motor imagery resulting in turning right, turning left, acceleration, and deceleration for a virtual automatic car control. Ten healthy subjects participated in online experiments, the experimental results not only show the efficiency of our proposed MPMI-BCI strategy but also indicate that those users can control the virtual automatic car spontaneously and efficiently without any other visual attention. Furthermore, the metric of path length optimality ratio (1.23) is very encouraging and the time optimality ratio (1.28) is especially remarkable. Graphical Abstract The paradigm of multiple patterns of motor imagery detection and the relevant topographies of CSP weights for different MI patterns.

Classification of electroencephalogram signals using wavelet-CSP and projection extreme learning machine.

Brain-computer interface (BCI) systems establish a direct communication channel from the brain to an output device. As the basis of BCIs, recognizing motor imagery activities poses a considerable challenge to signal processing due to the complex and non-stationary characteristics. This paper introduces an optimal and intelligent method for motor imagery BCIs. Because of the robustness to noise, wavelet packet decomposition and common spatial pattern (CSP) methods were implemented to reduce the dimensions of preprocessed signals. And a novel and efficient classifier projection extreme learning machine (PELM) was employed to recognize the labels of electroencephalogram signals. Experiments have been performed on the BCI Competition Dataset to demonstrate the superiority of wavelet-CSP in BCI and the outperformance of the PELM-based method. Results show that the average recognition rate of PELM approaches approximately 70%, while the optimal rate of other methods is 72%, whose training time and classification time are relatively longer as 11.00 ms and 11.66 ms, respectively, compared with 4.75 ms and 4.87 ms obtained by using the proposed BCI system.