Real-world Brain-computer Interface Research Articles

BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems.

The Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs have added asynchronous features, including abstention and dynamic stopping, but it remains a open question of how to evaluate asynchronous BCI performance. In this work, we build on the BCI-Utility metric to create the first evaluation metric with special consideration of the asynchronous features of self-paced BCIs. This metric considers accuracy as all of the following three - probability of a correct selection when a selection was intended, probability of making a selection when a selection was intended, and probability of an abstention when an abstention was intended. Further, it considers the average time required for a selection when using dynamic stopping and the proportion of intended selections versus abstentions. We establish the validity of the derived metric via extensive simulations, and illustrate and discuss its practical usage on real-world BCI data. We describe the relative contribution of different inputs with plots of BCI-Utility curves under different parameter settings. Generally, the BCI-Utility metric increases as any of the accuracy values increase and decreases as the expected time for an intended selection increases. Furthermore, in many situations, we find shortening the expected time of an intended selection is the most effective way to improve the BCI-Utility, which necessitates the advancement of asynchronous BCI systems capable of accurate abstention and dynamic stopping.

Towards real-world BCI: CCSPNet, a compact subject-independent motor imagery framework

A conventional brain-computer interface (BCI) requires a complete data gathering, training, and calibration phase for each user before it can be used. In recent years, a number of subject-independent (SI) BCIs have been developed. Many of these methods yield a weaker performance compared to the subject-dependent (SD) approach, and some are computationally expensive. A potential real-world application would greatly benefit from a more accurate, compact, and computationally efficient subject-independent BCI. In this work, we propose a novel subject-independent BCI framework, named CCSPNet (Convolutional Common Spatial Pattern Network) that is trained on the motor imagery (MI) paradigm of a large-scale electroencephalography (EEG) signals database consisting of 400 trials for every 54 subjects who perform two-class hand-movement MI tasks. The proposed framework applies a wavelet kernel convolutional neural network (WKCNN) and a temporal convolutional neural network (TCNN) in order to represent and extract the spectral features of EEG signals. A common spatial pattern (CSP) algorithm is implemented for spatial feature extraction, and the number of CSP features is reduced by a dense neural network. Finally, the class label is determined by a linear discriminant analysis (LDA) classifier. The CCSPNet evaluation results show that it is possible to have a compact BCI that achieves both SD and SI state-of-the-art performance comparable to complex and computationally expensive models.

Asynchronous Video Target Detection Based on Single-Trial EEG Signals.

Event-related potentials (ERPs) are widely used in brain-computer interface (BCI) systems to detect sensitive targets. However, asynchronous BCI systems based on video-target-evoked ERPs can pose a challenge in real-world applications due to the absence of an explicit target onset time and the time jitter of the detection latency. To address this challenge, we developed an asynchronous detection framework for video target detection. In this framework, an ERP alignment method based on the principle of iterative minimum distance square error (MDSE) was proposed for constructing an ERP template and aligning signals on the same base to compensate for possible time jitter. Using this method, ERP response characteristics induced by video targets were estimated. Online video target detection results indicated that alignment methods reduced the false alarm more effectively than non-alignment methods. The false alarm of the proposed Aligned-MDSE method was one-third lower than that of existing alignment methods under the same right hit level using limited individual samples. Furthermore, cross-subject results indicated that untrained subjects could directly perform online detection tasks and achieve excellent performance by a general model trained from more than 10 subjects. The proposed asynchronous video target detection framework can thus have a significant impact on real-world BCI applications.

Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface

Brain-Computer Interface (BCI) is a system empowering humans to communicate with or control the outside world with exclusively brain intentions. Electroencephalography (EEG) based BCIs are promising solutions due to their convenient and portable instruments. Despite the extensive research of EEG in recent years, it is still challenging to interpret EEG signals effectively due to the massive noises in EEG signals (e.g., low signal-noise ratio and incomplete EEG signals), and difficulties in capturing the inconspicuous relationships between EEG signals and certain brain activities. Most existing works either only consider EEG as chain-like sequences neglecting complex dependencies between adjacent signals or requiring pre-processing such as transforming EEG waves into images. In this paper, we introduce both cascade and parallel convolutional recurrent neural network models for precisely identifying human intended movements and instructions effectively learning the compositional spatio-temporal representations of raw EEG streams. Extensive experiments on a large scale movement intention EEG dataset (108 subjects,3,145,160 EEG records) have demonstrated that both models achieve high accuracy near 98.3% and outperform a set of baseline methods and most recent deep learning based EEG recognition models, yielding a significant accuracy increase of 18% in the cross-subject validation scenario. The developed models are further evaluated with a real-world BCI and achieve a recognition accuracy of 93% over five instruction intentions. This suggests the proposed models are able to generalize over different kinds of intentions and BCI systems.

A Fast, Open EEG Classification Framework Based on Feature Compression and Channel Ranking

Superior feature extraction, channel selection and classification methods are essential for designing electroencephalography (EEG) classification frameworks. However, the performance of most frameworks is limited by their improper channel selection methods and too specifical design, leading to high computational complexity, non-convergent procedure and narrow expansibility. In this paper, to remedy these drawbacks, we propose a fast, open EEG classification framework centralized by EEG feature compression, low-dimensional representation, and convergent iterative channel ranking. First, to reduce the complexity, we use data clustering to compress the EEG features channel-wise, packing the high-dimensional EEG signal, and endowing them with numerical signatures. Second, to provide easy access to alternative superior methods, we structurally represent each EEG trial in a feature vector with its corresponding numerical signature. Thus, the recorded signals of many trials shrink to a low-dimensional structural matrix compatible with most pattern recognition methods. Third, a series of effective iterative feature selection approaches with theoretical convergence is introduced to rank the EEG channels and remove redundant ones, further accelerating the EEG classification process and ensuring its stability. Finally, a classical linear discriminant analysis (LDA) model is employed to classify a single EEG trial with selected channels. Experimental results on two real world brain-computer interface (BCI) competition datasets demonstrate the promising performance of the proposed framework over state-of-the-art methods.

A High-Rate BCI Speller Based on Eye-Closed EEG Signal

This paper aims to develop a Brain-computer interface (BCI) speller utilizing eyes-closed and double-blinking electroencephalogram (EEG) signals and based on asynchronous mechanism. The proposed system comprises a signal processing module and a graphical user interface (virtual keyboard-VK) with 26 English characters plus a special symbol. A detected “eyes-closed” (EC) event induces the “select” command, whereas a “double-blinking” (DB) event functions as the “undo” command. A three-class support vector machine classifier involving EEG signal analysis of three groups of events (“eyes-open” - idle state, EC, and DB) is proposed. The results show that the proposed BCI can achieve an overall accuracy of 93.8% for multi-class classification. The proposed speller is then employed in the online experiment of spelling the word “bcispeller” using a 1 s time window. Consequently, it achieves an average accuracy of 92.3% and an average spelling rate of 5 letters/min. Overall, this paper shows improvement with high accuracy and spelling rate demonstrating the feasibility and reliability of implementing a real-world BCI speller.

Electroencephalogram-Based Brain-Computer Interface and Lower-Limb Prosthesis Control: A Case Study.

The purpose of this study was to establish the feasibility of manipulating a prosthetic knee directly by using a brain-computer interface (BCI) system in a transfemoral amputee. Although the other forms of control could be more reliable and quick (e.g., electromyography control), the electroencephalography (EEG)-based BCI may provide amputees an alternative way to control a prosthesis directly from brain. A transfemoral amputee subject was trained to activate a knee-unlocking switch through motor imagery of the movement of his lower extremity. Surface scalp electrodes transmitted brain wave data to a software program that was keyed to activate the switch when the event-related desynchronization in EEG reached a certain threshold. After achieving more than 90% reliability for switch activation by EEG rhythm-feedback training, the subject then progressed to activating the knee-unlocking switch on a prosthesis that turned on a motor and unlocked a prosthetic knee. The project took place in the prosthetic department of a Veterans Administration medical center. The subject walked back and forth in the parallel bars and unlocked the knee for swing phase and for sitting down. The success of knee unlocking through this system was measured. Additionally, the subject filled out a questionnaire on his experiences. The success of unlocking the prosthetic knee mechanism ranged from 50 to 100% in eight test segments. The performance of the subject supports the feasibility for BCI control of a lower extremity prosthesis using surface scalp EEG electrodes. Investigating direct brain control in different types of patients is important to promote real-world BCI applications.

Sinusoidal Signal Assisted Multivariate Empirical Mode Decomposition for Brain-Computer Interfaces.

A brain-computer interface (BCI) is a communication approach that permits cerebral activity to control computers or external devices. Brain electrical activity recorded with electroencephalography (EEG) is most commonly used for BCI. Noise-assisted multivariate empirical mode decomposition (NA-MEMD) is a data-driven time-frequency analysis method that can be applied to nonlinear and nonstationary EEG signals for BCI data processing. However, because white Gaussian noise occupies a broad range of frequencies, some redundant components are introduced. To solve this leakage problem, in this study, we propose using a sinusoidal assisted signal that occupies the same frequency ranges as the original signals to improve MEMD performance. To verify the effectiveness of the proposed sinusoidal signal assisted MEMD (SA-MEMD) method, we compared the decomposition performances of MEMD, NA-MEMD, and the proposed SA-MEMD using synthetic signals and a real-world BCI dataset. The spectral decomposition results indicate that the proposed SA-MEMD can avoid the generation of redundant components and over decomposition, thus, substantially reduce the mode mixing and misalignment that occurs in MEMD and NA-MEMD. Moreover, using SA-MEMD as a signal preprocessing method instead of MEMD or NA-MEMD can significantly improve BCI classification accuracy and reduce calculation time, which indicates that SA-MEMD is a powerful spectral decomposition method for BCI.

Online and Offline Domain Adaptation for Reducing BCI Calibration Effort

Many real-world brain-computer interface (BCI) applications rely on single-trial classification of event-related potentials (ERPs) in EEG signals. However, because different subjects have different neural responses to even the same stimulus, it is very difficult to build a generic ERP classifier whose parameters fit all subjects. The classifier needs to be calibrated for each individual subject, using some labeled subject-specific data. This paper proposes both online and offline weighted adaptation regularization (wAR) algorithms to reduce this calibration effort, i.e., to minimize the amount of labeled subject-specific EEG data required in BCI calibration, and hence to increase the utility of the BCI system. We demonstrate using a visually-evoked potential oddball task and three different EEG headsets that both online and offline wAR algorithms significantly outperform several other algorithms. Moreover, through source domain selection, we can reduce their computational cost by about 50%, making them more suitable for real-time applications.

Extracting patterns of single-trial EEG using an adaptive learning algorithm.

The improvement of brain imaging technique brings about an opportunity for developing and investigating brain-computer interface (BCI) which is a way to interact with computer and environment. The measured brain activities usually constitute the signals of interest and noises. Applying the portable device and removing noise are the benefits to real-world BCI. In this study, one portable electroencephalogram (EEG) system non-invasively acquired brain dynamics through wireless transmission while six subjects participated in the rapid serial visual presentation (RSVP) paradigm. The event-related potential (ERP) was traditionally estimated by ensemble averaging (EA) to increase the signal-to-noise ratio. One adaptive filter of data-reusing radial basis function network (DR-RBFN) was also utilized as the estimator. The results showed that this portable EEG system stably acquired brain activities. Furthermore, the task-related potentials could be clearly explored from the limited samples of EEG data through DR-RBFN. According to the artifact-free data from the portable device, this study demonstrated the potential to move the BCI from laboratory research to real-life application in the near future.

Robust Common Spatial Filters with a Maxmin Approach

Electroencephalographic signals are known to be nonstationary and easily affected by artifacts; therefore, their analysis requires methods that can deal with noise. In this work, we present a way to robustify the popular common spatial patterns (CSP) algorithm under a maxmin approach. In contrast to standard CSP that maximizes the variance ratio between two conditions based on a single estimate of the class covariance matrices, we propose to robustly compute spatial filters by maximizing the minimum variance ratio within a prefixed set of covariance matrices called the tolerance set. We show that this kind of maxmin optimization makes CSP robust to outliers and reduces its tendency to overfit. We also present a data-driven approach to construct a tolerance set that captures the variability of the covariance matrices over time and shows its ability to reduce the nonstationarity of the extracted features and significantly improve classification accuracy. We test the spatial filters derived with this approach and compare them to standard CSP and a state-of-the-art method on a real-world brain-computer interface (BCI) data set in which we expect substantial fluctuations caused by environmental differences. Finally we investigate the advantages and limitations of the maxmin approach with simulations.

A study of the existing problems of estimating the information transfer rate in online brain–computer interfaces

Objective. Today, the brain–computer interface (BCI) community lacks a standard method to evaluate an online BCI's performance. Even the most commonly used metric, the information transfer rate (ITR), is often reported differently, even incorrectly, in many papers, which is not conducive to BCI research. This paper aims to point out many of the existing problems and give some suggestions and methods to overcome these problems. Approach. First, the preconditions inherent in ITR calculation based on Wolpaw's definition are summarized and several incorrect ITR calculations, which go against the preconditions, are indicated. Then, the issues affecting ITR estimation during the test of online BCI systems are discussed in detail. Finally, a task-oriented online BCI test platform was proposed, which may help BCI evaluations in real-world applications. Main results. The guidelines for ITR calculation in online BCIs testing are proposed. The platform executed in the Beijing BCI Competition 2010 shows that it can be used as a common way to compare the online performances (including the ITR) of existing BCI paradigms. Significance: The proposed guidelines and task-oriented test platform may reduce the uncertainty and artifacts of online BCI performance evaluation; they provide a relatively objective way to compare different BCI's performances in real-world BCI applications, which is a forward step toward developing standards for BCI performance evaluation.