Online Brain-computer Interface System Research Articles

Comprehensive evaluation methods for translating BCI into practical applications: usability, user satisfaction and usage of online BCI systems.

Although brain-computer interface (BCI) is considered a revolutionary advancement in human-computer interaction and has achieved significant progress, a considerable gap remains between the current technological capabilities and their practical applications. To promote the translation of BCI into practical applications, the gold standard for online evaluation for classification algorithms of BCI has been proposed in some studies. However, few studies have proposed a more comprehensive evaluation method for the entire online BCI system, and it has not yet received sufficient attention from the BCI research and development community. Therefore, the qualitative leap from analyzing and modeling for offline BCI data to the construction of online BCI systems and optimizing their performance is elaborated, and then user-centred is emphasized, and then the comprehensive evaluation methods for translating BCI into practical applications are detailed and reviewed in the article, including the evaluation of the usability (including effectiveness and efficiency of systems), the evaluation of the user satisfaction (including BCI-related aspects, etc.), and the evaluation of the usage (including the match between the system and user, etc.) of online BCI systems. Finally, the challenges faced in the evaluation of the usability and user satisfaction of online BCI systems, the efficacy of online BCI systems, and the integration of BCI and artificial intelligence (AI) and/or virtual reality (VR) and other technologies to enhance the intelligence and user experience of the system are discussed. It is expected that the evaluation methods for online BCI systems elaborated in this review will promote the translation of BCI into practical applications.

Online BCI systems: cross-subject motor imagery classification based on weighted time-domain feature extraction methods

Motor imagery electroencephalogram (MI-EEG) is becoming increasingly important. This paper solves the problem of online signal recognition for motor imagery across subjects by finding common features across multiple subjects to improve the generality of the classification model. We analysed the EEG data from left/right-hand motor imagery of eight subjects and proposed a weighted time-domain (WTD) feature extraction method based on a weighted channel screening method. The classification model constructed by combining this feature extraction method with the support vector machine (SVM) classification method was faster in classification and achieved good cross-subject classification accuracy (The average offline classification accuracy was 91.39%). In this paper, an online control system for asynchronous brain-controlled wheelchairs was built with good performance. The online average motor imagery classification accuracy was 81.67%, and the average response time was 1.36s. This method contributes to bringing the online Brain-computer interface (BCI) system out of the laboratory and into wider application.

A sub-region combination scheme for spatial coding in a high-frequency SSVEP-based BCI

Objective. In studying the spatial coding mechanism of visual evoked potentials, it is significant to construct a model that shows the relationship between steady-state visual evoked potential (SSVEP) responses to the local and global visual field stimulation. In order to investigate whether SSVEPs produced by sub-region stimulation can predict that produced by joint region stimulation, a sub-region combination scheme for spatial coding in a high-frequency SSVEP-based brain-computer interface (BCI) is developed innovatively. Approach. An annular visual field is divided equally into eight sub-regions. The 60 Hz visual stimuli in different sub-regions and joint regions are presented separately to participants. The SSVEP produced by the sub-region stimulation is superimposed to simulate the SSVEP produced by the joint region stimulation with different spatial combinations. A four-class spatially-coded BCI paradigm is used to evaluate the simulated classification performance, and the performance ranking of all simulated SSVEPs is obtained. Six representative stimulus patterns from two performance levels and three stimulus areas are applied to the online BCI system for each participant. Main results. The experimental result shows that the proposed scheme can implement a spatially-coded visual BCI system and realize satisfactory performance with imperceptible flicker. Offline analysis indicates that the classification accuracy and information transfer rate (ITR) are 89.69 ± 8.75% and 24.35 ± 7.09 bits min−1 with 3 s data length under the 3/8 stimulus area. The online BCI system reaches an average classification accuracy of 87.50 ± 9.13% with 3 s data length, resulting in an ITR of 22.48 ± 6.71 bits min−1 under the 3/8 stimulus area. Significance. This study proves the feasibility of using the sub-region’s response to predict the joint region’s response. It has the potential to extend to other frequency bands and lays a foundation for future research on more complex spatial coding methods.

Predictive neuromodulation of cingulo-frontal neural dynamics in major depressive disorder using a brain-computer interface system: A simulation study.

Deep brain stimulation (DBS) is a promising therapy for treatment-resistant major depressive disorder (MDD). MDD involves the dysfunction of a brain network that can exhibit complex nonlinear neural dynamics in multiple frequency bands. However, current open-loop and responsive DBS methods cannot track the complex multiband neural dynamics in MDD, leading to imprecise regulation of symptoms, variable treatment effects among patients, and high battery power consumption. Here, we develop a closed-loop brain-computer interface (BCI) system of predictive neuromodulation for treating MDD. We first use a biophysically plausible ventral anterior cingulate cortex (vACC)-dorsolateral prefrontal cortex (dlPFC) neural mass model of MDD to simulate nonlinear and multiband neural dynamics in response to DBS. We then use offline system identification to build a dynamic model that predicts the DBS effect on neural activity. We next use the offline identified model to design an online BCI system of predictive neuromodulation. The online BCI system consists of a dynamic brain state estimator and a model predictive controller. The brain state estimator estimates the MDD brain state from the history of neural activity and previously delivered DBS patterns. The predictive controller takes the estimated MDD brain state as the feedback signal and optimally adjusts DBS to regulate the MDD neural dynamics to therapeutic targets. We use the vACC-dlPFC neural mass model as a simulation testbed to test the BCI system and compare it with state-of-the-art open-loop and responsive DBS treatments of MDD. We demonstrate that our dynamic model accurately predicts nonlinear and multiband neural activity. Consequently, the predictive neuromodulation system accurately regulates the neural dynamics in MDD, resulting in significantly smaller control errors and lower DBS battery power consumption than open-loop and responsive DBS. Our results have implications for developing future precisely-tailored clinical closed-loop DBS treatments for MDD.

Non-Linear Adapted Spatio-Temporal Filter for Single-Trial Identification of Movement-Related Cortical Potential

The execution or imagination of a movement is reflected by a cortical potential that can be recorded by electroencephalography (EEG) as Movement-Related Cortical Potentials (MRCPs). The identification of MRCP from a single trial is a challenging possibility to get a natural control of a Brain–Computer Interface (BCI). We propose a novel method for MRCP detection based on optimal non-linear filters, processing different channels of EEG including delayed samples (getting a spatio-temporal filter). Different outputs can be obtained by changing the order of the temporal filter and of the non-linear processing of the input data. The classification performances of these filters are assessed by cross-validation on a training set, selecting the best ones (adapted to the user) and performing a majority voting from the best three to get an output using test data. The method is compared to another state-of-the-art filter recently introduced by our group when applied to EEG data recorded from 16 healthy subjects either executing or imagining 50 self-paced upper-limb palmar grasps. The new approach has a median accuracy on the overall dataset of 80%, which is significantly better than that of the previous filter (i.e., 63%). It is feasible for online BCI system design with asynchronous, self-paced applications.

Optimizing Stimulus Frequency Ranges for Building a High-Rate High Frequency SSVEP-BCI.

The brain-computer interfaces (BCIs) based on steady-state visual evoked potential (SSVEP) have been extensively explored due to their advantages in terms of high communication speed and smaller calibration time. The visual stimuli in the low- and medium-frequency ranges are adopted in most of the existing studies for eliciting SSVEPs. However, there is a need to further improve the comfort of these systems. The high-frequency visual stimuli have been used to build BCI systems and are generally considered to significantly improve the visual comfort, but their performance is relatively low. The distinguishability of 16-class SSVEPs encoded by the three frequency ranges, i.e., 31-34.75 Hz with an interval of 0.25 Hz, 31-38.5 Hz with an interval of 0.5 Hz, 31-46 Hz with an interval of 1 Hz, is explored in this study. We compare classification accuracy and information transfer rate (ITR) of the corresponding BCI system. According to the optimized frequency range, this study builds an online 16-target high frequency SSVEP-BCI and verifies the feasibility of the proposed system based on 21 healthy subjects. The BCI based on visual stimuli with the narrowest frequency range, i.e., 31-34.5 Hz, have the highest ITR. Therefore, the narrowest frequency range is adopted to build an online BCI system. An averaged ITR obtained from the online experiment is 153.79 ± 6.39 bits/min. These findings contribute to the development of more efficient and comfortable SSVEP-based BCIs.

Real-time regulation of room temperature based on individual thermal sensation using an online brain-computer interface.

Regulation of indoor temperature based on neurophysiological and psychological signals is one of the most promising technologies for intelligent buildings. In this study, we developed a system for closed-loop control of indoor temperature based on brain-computer interface (BCI) technology for the first time. Electroencephalogram (EEG) signals were collected from subjects for two room temperature categories (cool comfortable and hot uncomfortable) and used to build a thermal-sensation discrimination model (TSDM) with an ensemble learning method. Then, an online BCI system was developed based on the TSDM. In the online room temperature control experiment, when the TSDM detected that the subjects felt hot and uncomfortable, BCI would automatically turn on the air conditioner, and when the TSDM detected that the subjects felt cool and comfortable, BCI would automatically turn off the air conditioner. The results of online experiments in a hot environment showed that a BCI could significantly improve the thermal comfort of subjects (the subjective thermal comfort score decreased from 2.45 (hot uncomfortable) to 0.55 (cool comfortable), p < 0.001). A parallel experiment further showed that if the subjects wore thicker clothes during the experiment, the BCI would turn on the air conditioner for a longer time to ensure the thermal comfort of the subjects. This has further confirmed the effectiveness of TSDM model in evaluating thermal sensation under the dynamic change of room temperature and showed the model's good robustness. This study proposed a new paradigm of human-building interaction, which is expected to play a promising role in the development of human-centered intelligent buildings.

A novel online BCI system using speech imagery and ear-EEG for home appliances control

Background and ObjectiveThis paper investigates a novel way to interact with home appliances via a brain-computer interface (BCI), using electroencephalograph (EEG) signals acquired from around the user's ears with a custom-made wearable BCI headphone. MethodsThe users engage in speech imagery (SI), a type of mental task where they imagine speaking out a specific word without producing any sound, to control an interactive simulated home appliance. In this work, multiple models are employed to improve the performance of the system. Temporally-stacked multi-band covariance matrix (TSMBC) method is used to represent the neural activities during SI tasks with spatial, temporal, and spectral information included. To further increase the usability of our proposed system in daily life, a calibration session, where the pre-trained models are fine-tuned, is added to maintain performance over time with minimal training. Eleven participants were recruited to evaluate our method over three different sessions: a training session, a calibration session, and an online session where users were given the freedom to achieve a given goal on their own. ResultsIn the offline experiment, all participants were able to achieve a classification accuracy significantly higher than the chance level. In the online experiments, a few participants were able to use the proposed system to freely control the home appliance with high accuracy and relatively fast command delivery speed. The best participant achieved an average true positive rate and command delivery time of 0.85 and 3.79 s/command, respectively. ConclusionBased on the positive experimental results and user surveys, the novel ear-EEG-SI-based BCI paradigm is a promising approach for the wearable BCI system for daily life.

A user-friendly SSVEP-based BCI using imperceptible phase-coded flickers at 60Hz

A brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) was developed by four-class phase-coded stimuli. SSVEPs elicited by flickers at 60Hz, which is higher than the critical fusion frequency (CFF), were compared with those at 15Hz and 30Hz. SSVEP components in electroencephalogram (EEG) were detected using task related component analysis (TRCA) method. Offline analysis with 17 subjects indicated that the highest information transfer rate (ITR) was 29.80±4.65bpm with 0.5s data length for 60Hz and the classification accuracy was 70.07±4.15%. The online BCI system reached an averaged classification accuracy of 87.75±3.50% at 60Hz with 4s, resulting in an ITR of 16.73±1.63bpm. In particular, the maximum ITR for a subject was 80bpm with 0.5s at 60Hz. Although the BCI performance of 60Hz was lower than that of 15Hz and 30Hz, the results of the behavioral test indicated that, with no perception of flicker, the BCI system with 60Hz was more comfortable to use than 15Hz and 30Hz. Correlation analysis revealed that SSVEP with higher signal-to-noise ratio (SNR) corresponded to better classification performance and the improvement in comfortableness was accompanied by a decrease in performance. This study demonstrates the feasibility and potential of a user-friendly SSVEP-based BCI using imperceptible flickers.

Cortical Source Domain Based Motor Imagery and Motor Execution Framework for Enhanced Brain Computer Interface Applications

In the recent decades, there has been growing interest in novel applications of neuro-rehabilitation and motor power augmentation based on the experimental paradigm of Motor Imagery (MI) and Motor Execution (ME) respectively. Qualitative comparison between MI and ME reveals significant difference in response time, activation magnitude, and cortical source distribution. Based on these differences, different frameworks of MI and ME have been developed for enhanced Brain Computer Interface (BCI) applications by providing task specific information in time and cortical source space. Decoding of subject's intent using cortical source domain processing increases the accuracy in the MI and ME scenario by 5.18% and 3.68% respectively, compared to conventional sensor domain processing. The performance of the proposed methodology outperforms the stat-of-the-art techniques. The present study highlights the limitation of cross framework i.e., utilization of MI framework in ME experimental paradigm or vice versa. In particular, the performance degradation of MI and ME tasks was observed in the cross framework scenario. The experimental results are expected to provide an evidence for researchers when designing an imagery or execution based online (or offline) BCI system.

Design of an Online Brain-Computer Interface System Based on Field Programmable Gate Array

In the article, an online brain-computer interface (BCI) demo system based on Field Programmable Gate Array (FPGA) was designed. A colour visual stimulator based on a Video Graphics Array (VGA) monitor controlled by FPGA is designed to produce transient visual stimulations. An electroencephalography (EEG) acquisition module including active electrodes and driven-right-leg circuit is used to acquire the visual evoked potential (VEP) signal; Finite impulse response (FIR) filtering combined with an averaging method allows for extraction of the weak VEP signals. The template matching method is used in target identification. The FPGA is successfully used in the BCI system to complete multiple tasks such as the VGA controller, analog to digital (AD) converter controller, VEP signal processor and electronic device controller. Experiments show that the subjects’ transient VEP can successfully activate the electronic devices by the online BCI system which has relatively high identification accuracy. All these results show that the real time BCI demo system based on FPGA is feasible and effective.

Enhanced Performance of a Brain Switch by Simultaneous Use of EEG and NIRS Data for Asynchronous Brain-Computer Interface.

Previous studies have shown the superior performance of hybrid electroencephalography (EEG)/ near-infrared spectroscopy (NIRS) brain-computer interfaces (BCIs). However, it has been veiled whether the use of a hybrid EEG/NIRS modality can provide better performance for a brain switch that can detect the onset of the intention to turn on a BCI. In this study, we developed such a hybrid EEG/NIRS brain switch and compared its performance with single modality EEG- and NIRS-based brain switch respectively, in terms of true positive rate (TPR), false positive rate (FPR), onset detection time (ODT), and information transfer rate (ITR). In an offline analysis, the performance of a hybrid EEG/NIRS brain switch was significantly improved over that of EEG- and NIRS-based brain switches in general, and in particular a significantly lower FPR was observed for the hybrid EEG/NIRS brain switch. A pseudo-online analysis was additionally performed to confirm the feasibility of implementing an online BCI system with our hybrid EEG/NIRS brain switch. The overall trend of pseudo-online analysis results generally coincided with that of the offline analysis results. No significant difference in all performance measures was also found between offline and pseudo online analysis schemes when the amount of training data was same, with one exception for the ITRs of an EEG brain switch. These offline and pseudo-online results demonstrate that a hybrid EEG/NIRS brain switch can be used to provide a better onset detection performance than that of a single neuroimaging modality.

Adaptive Brain-Computer Interface with Attention Alterations in Patients with Amyotrophic Lateral Sclerosis.

The users' mental state such as attention variations can have an effect on the brain-computer interface (BCI) performance. In this project, we implemented an adaptive online BCI system with alterations in the users' attention. Twelve electroencephalography (EEG) signals were obtained from six patients with Amyotrophic Lateral Sclerosis (ALS). Participants were asked to execute 40 trials of ankle dorsiflexion concurrently with an auditory oddball task. EEG channels, classifiers and features with superior offline performance in the training phase of the classification of attention level were selected to use in the online mode for prediction the attention status. A feedback was provided to the users to reduce the amount of attention diversion created by the oddball task. The findings revealed that the users' attention can control an online BCI system and real-time neurofeedback can be applied to focus the attention of the user back onto the main task.

Feature subset and time segment selection for the classification of EEG data based motor imagery

The selection of feature subset and time segment is of great significance to the benefit of motor imagery classification. Hence, applying it in classification as well as itself alone has become an increasingly important research field in the brain-computer interface (BCI) systems. Most of the existing literatures only focus on binary-class classification situations in a fixed time segment. However, the modern BCI systems usually have to deal with more motor imagery classes. In this paper, we propose two Parzen window based methods to select the discriminative feature subset and subject-specific time segment. Further, we extend the proposed methods to multi-class issues. Finally, a soft Naive Bayesian classifier is designed to solve not only binary-class but also multi-class motor imagery problems. The proposed methods are validated on two well-known datasets, BCI competition IV dataset 2a and 2b. Experimental results reveal that the proposed methods achieve an improvement of 4.38 % for 2a and 2.54 % for 2b in comparison with the state-of-the-art methods, respectively. Meanwhile, both proposed methods achieve an improvement of 0.04 for 2a and 0.05 for 2b in kappa coefficient, respectively. Besides, the proposed multi-class methods would potentially contribute to the online BCI systems in practice.

Real-time neurofeedback is effective in reducing diversion of attention from a motor task in healthy individuals and patients with amyotrophic lateral sclerosis

Objective. The performance of brain-computer interface (BCI) systems is influenced by the user’s mental state, such as attention diversion. In this study, we propose a novel online BCI system able to adapt with variations in the users’ attention during real-time movement execution. Approach. Electroencephalography signals were recorded from healthy participants and patients with Amyotrophic Lateral Sclerosis while attention to the target task (a dorsiflexion movement) was drifted using an auditory oddball task. For each participant, the selected channels, classifiers and features from a training data set were used in the online phase to predict the attention status. Main results. For both healthy controls and patients, feedback to the user on attentional status reduced the amount of attention diversion. Significance. The findings presented here demonstrate successful monitoring of the users’ attention in a fully online BCI system, and further, that real-time neurofeedback on the users’ attention state can be implemented to focus the attention of the user back onto the main task.

Decoding spectro-temporal representation for motor imagery recognition using ECoG-based brain-computer interfaces.

One of the challenges in brain-computer interface systems is obtaining motor imagery recognition from brain activities. Brain-signal decoding robustness and system performance improvement during the motor imagery process are two of the essential issues in brain-computer interface research. In conventional approaches, ineffective decoding of features and high complexity of algorithms often lead to unsatisfactory performance. A novel method for the recognition of motor imagery tasks is developed based on employing a modified S-transforms for spectro-temporal representation to characterize the behavior of electrocorticogram activities. A classifier is trained by using a support vector machine, and an optimized wrapper approach is applied to guide selection to implement the representation selection obtained. A channel selection algorithm optimizes the wrapper approach by adding a cross-validation step, which effectively improves the classification performance. The modified S-transform can accurately capture event-related desynchronization/event-related synchronization phenomena and can effectively locate sensorimotor rhythm information. The optimized wrapper approach used in this scheme can effectively reduce the feature dimension and improve algorithm efficiency. The method is evaluated on a public electrocorticogram dataset with a recognition accuracy of 98% and an information transfer rate of 0.8586 bit/trial. To verify the effect of the channel selection, both electrocorticogram and electroencephalogram data are experimentally analyzed. Furthermore, the computational efficiency of this scheme demonstrates its potential for online brain-computer interface systems in future cognitive tasks.

Classification of Movement Preparation Between Attended and Distracted Self-Paced Motor Tasks.

Brain-computer interface (BCI) systems aim to control external devices by using brain signals. The performance of these systems is influenced by the user's mental state, such as attention. In this study, we classified two attention states to a target task (attended and distracted task level) while attention to the task is altered by one of three types of distractors. A total of 27 participants were allocated into three experimental groups and exposed to one type of distractor. An attended condition that was the same across the three groups comprised only the main task execution (self-paced dorsiflexion) while the distracted condition was concurrent execution of the main task and an oddball task (dual-task condition). Electroencephalography signals were recorded from 28 electrodes to classify the two attention states of attended or distracted task conditions by extracting temporal and spectral features. The results showed that the ensemble classification accuracy using the combination of temporal and spectral features (spectro-temporal features, 82.3 ± 2.7%) was greater than using temporal (69 ± 2.2%) and spectral (80.3 ± 2.6%) features separately. The classification accuracy was computed using a combination of different channel locations, and it was demonstrated that a combination of parietal and centrally located channels was superior for classification of two attention states during movement preparation (parietal channels: 84.6 ± 1.3%, central and parietal channels: 87.2 ± 1.5%). It is possible to monitor the users' attention to the task for different types of distractors. It has implications for online BCI systems where the requirement is for high accuracy of intention detection.

A research for single trial detection of error related negativity

Error related negativity (ERN) is generated in frontal and central cortical regions when individuals perceive errors. Because ERN has low signal-to-noise ratio and large individual difference, it is difficult for single trial ERN recognition. In current study, the optimized electroencephalograph (EEG) channels were selected based on the brain topography of ERN activity and ERN offline recognition rate, and the optimized EEG time segments were selected based on the ERN offline recognition rate, then the low frequency time domain and high frequency time-frequency domain features were analyzed based on wavelet transform, after which the ERN single detection algorithm was proposed based on the above procedures. Finally, we achieved average recognition rate of 72.0% ± 9.6% in 10 subjects by using the sample points feature in 0~3.9 Hz and the power and variance features in 3.9~15.6 Hz from the EEG segments of 200~600 ms on the selected 6 channels. Our work has the potential to help the error command real-time correction technique in the application of online brain-computer interface system.

Developing a Three- to Six-State EEG-Based Brain-Computer Interface for a Virtual Robotic Manipulator Control.

We develop an electroencephalography (EEG)-based noninvasive brain-computer interface (BCI) system having short training time (15 min) that can be applied for high-performance control of robotic prosthetic systems. A signal processing system for detecting user's mental intent from EEG data based on up to six-state BCI paradigm is developed and used. We examine the performance of the developed system on experimental data collected from 12 healthy participants and analyzed offline. Out of 12 participants 3 achieve an accuracy of six-state communication in 80%-90% range, while 2 participants do not achieve a satisfactory accuracy. We further implement an online BCI system for control of a virtual 3 degree-of-freedom (dof) prosthetic manipulator and test it with our three best participants. Two participants are able to successfully complete 100% of the test tasks, demonstrating on average the accuracy rate of 80% and requiring 5-10 s to execute a manipulator move. One participant failed to demonstrate a satisfactory performance in online trials. We show that our offline EEG BCI system can correctly identify different motor imageries in EEG data with high accuracy and our online BCI system can be used for control of a virtual 3 dof prosthetic manipulator. Our results prepare foundation for further development of higher performance EEG BCI-based robotic assistive systems and demonstrate that EEG-based BCI may be feasible for robotic control by paralyzed and immobilized individuals.

An Adaptive Calibration Framework for mVEP-Based Brain-Computer Interface.

Electroencephalogram signals and the states of subjects are nonstationary. To track changing states effectively, an adaptive calibration framework is proposed for the brain-computer interface (BCI) with the motion-onset visual evoked potential (mVEP) as the control signal. The core of this framework is to update the training set adaptively for classifier training. The updating procedure consists of two operations, that is, adding new samples to the training set and removing old samples from the training set. In the proposed framework, a support vector machine (SVM) and fuzzy C-mean clustering (fCM) are combined to select the reliable samples for the training set from the blocks close to the current blocks to be classified. Because of the complementary information provided by SVM and fCM, they can guarantee the reliability of information fed into classifier training. The removing procedure will aim to remove those old samples recorded a relatively long time before current new blocks. These two operations could yield a new training set, which could be used to calibrate the classifier to track the changing state of the subjects. Experimental results demonstrate that the adaptive calibration framework is effective and efficient and it could improve the performance of online BCI systems.