Traditional Brain-computer Interface Research Articles

A Survey in Implementation and Applications of Electroencephalograph (EEG)-Based Brain-Computer Interface

A Brain-Computer Interface (BCI) is an external system that controls activities and processes in the physical world based on brain signals. In Passive BCI, artificial signals are automatically generated by a computer program without any input from nerves in the body. This is useful for individuals with mobility issues. Traditional BCI has been dependent only on recording brain signals with Electroencephalograph (EEG) and has used a rule-based translation algorithm to generate control commands. These systems have developed very accurate translation systems. This paper is about the different methods for adapting the signals from the brain. It has been mentioned that various kinds of surveys in the past to serve the purpose of the present research. This paper shows a simple and easy analysis of each technique and its respective benefits and drawbacks, including signal acquisition, signal pre-processing, feature classification and classification. Finally, discussed is the application of EEG-based BCI.

Classification of motor imagery using a time-localised approach

Brain–computer interface (BCI) is getting increasing attention where classification of motor imagery (MI) using electroencephalography (EEG) signal plays a vital role. In traditional EEG-based BCI setup, after applying pre-processing like band-pass filtering and spatial filtering, features are extracted and are fed to the classifier. However, most of the traditional features are extracted from a single time window, which is usually the full-time frame of a cue-based MI signal. Such features are usually statistical characteristics like log-variance of the whole-time signal. Thus, the information, which is localised in time and crucial for subject-specific MI classification, is not best captured. In this work, a new time-localised approach is proposed where multiple time windows are used for feature extraction. We have developed a number of feature representations using those time windows. Our experimental results corroborate that the proposed approach can achieve higher accuracy of classification when compared to methods using conventional features using the same platform.

A Comparison of Mobile VR Display Running on an Ordinary Smartphone With Standard PC Display for P300-BCI Stimulus Presentation

A brain-computer interface (BCI) based on electroencephalography (EEG) is a promising technology for enhancing virtual reality (VR) applications-in particular, for gaming. We focus on the so-called P300-BCI, a stable and accurate BCI paradigm relying on the recognition of a positive event-related potential (ERP) occurring in the EEG about 300 ms post-stimulation. We implemented a basic version of such a BCI displayed on an ordinary and affordable smartphone-based head-mounted VR device: that is, a mobile and passive VR system (with no electronic components beyond the smartphone). The mobile phone performed the stimuli presentation, EEG synchronization (tagging) and feedback display. We compared the ERPs and the accuracy of the BCI on the VR device with a traditional BCI running on a personal computer (PC). We also evaluated the impact of subjective factors on the accuracy. The study was within-subjects, with 21 participants and one session in each modality. No significant difference in BCI accuracy was found between the PC and VR systems, although the P200 ERP was significantly wider and larger in the VR system as compared to the PC system.

A Dense Long Short‐Term Memory Model for Enhancing the Imagery‐Based Brain‐Computer Interface

Imagery‐based brain‐computer interfaces (BCIs) aim to decode different neural activities into control signals by identifying and classifying various natural commands from electroencephalogram (EEG) patterns and then control corresponding equipment. However, several traditional BCI recognition algorithms have the “one person, one model” issue, where the convergence of the recognition model’s training process is complicated. In this study, a new BCI model with a Dense long short‐term memory (Dense‐LSTM) algorithm is proposed, which combines the event‐related desynchronization (ERD) and the event‐related synchronization (ERS) of the imagery‐based BCI; model training and testing were conducted with its own data set. Furthermore, a new experimental platform was built to decode the neural activity of different subjects in a static state. Experimental evaluation of the proposed recognition algorithm presents an accuracy of 91.56%, which resolves the “one person one model” issue along with the difficulty of convergence in the training process.

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces.

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration.

Combining brain-computer interface and virtual reality for rehabilitation in neurological diseases: A narrative review.

The traditional rehabilitation for neurological diseases lacks the active participation of patients, its process is monotonous and tedious, and the effects need to be improved. Therefore, a new type of rehabilitation technology with more active participation combining brain-computer interface (BCI) with virtual reality (VR) has developed rapidly in recent years and has been used in rehabilitation in neurological diseases. This narrative review analyzed and characterized the development and application of the new training system (BCI-VR) in rehabilitation of neurological diseases from the perspective of the BCI paradigm, to provide a pathway for future research in this field. The review involved a search of the Web of Science-Science Citation Index/Social Sciences Citation Index and the China National Knowledge Infrastructure databases; 39 papers were selected. Advantages and challenges of BCI-VR-based neurological rehabilitation were analyzed in detail. Most BCI-VR studies included could be classified by 3 major BCI paradigms: motor imagery, P300, and steady-state visual-evoked potential. Integrating VR scenes into BCI systems could effectively promote the recovery process from nervous system injuries as compared with traditional methods. As compared with rehabilitation based on traditional BCI, rehabilitation based on BCI-VR can provide better feedback information for patients and promote the recovery of brain function. By solving the challenges and continual development, the BCI-VR system can be broadly applied to the clinical treatment of various neurological diseases.

Brain-Computer Interface-Based Humanoid Control: A Review.

A Brain-Computer Interface (BCI) acts as a communication mechanism using brain signals to control external devices. The generation of such signals is sometimes independent of the nervous system, such as in Passive BCI. This is majorly beneficial for those who have severe motor disabilities. Traditional BCI systems have been dependent only on brain signals recorded using Electroencephalography (EEG) and have used a rule-based translation algorithm to generate control commands. However, the recent use of multi-sensor data fusion and machine learning-based translation algorithms has improved the accuracy of such systems. This paper discusses various BCI applications such as tele-presence, grasping of objects, navigation, etc. that use multi-sensor fusion and machine learning to control a humanoid robot to perform a desired task. The paper also includes a review of the methods and system design used in the discussed applications.

Deep recurrent–convolutional neural network for classification of simultaneous EEG–fNIRS signals

Brain–computer interface (BCI) is a powerful system for communicating between the brain and outside world. Traditional BCI systems work based on electroencephalogram (EEG) signals only. Recently, researchers have used a combination of EEG signals with other signals to improve the performance of BCI systems. Among these signals, the combination of EEG with functional near-infrared spectroscopy (fNIRS) has achieved favourable results. In most studies, only EEGs or fNIRs have been considered as chain-like sequences, and do not consider complex correlations between adjacent signals, neither in time nor channel location. In this study, a deep neural network model has been introduced to identify the exact objectives of the human brain by introducing temporal and spatial features. The proposed model incorporates the spatial relationship between EEG and fNIRS signals. This could be implemented by transforming the sequences of these chain-like signals into hierarchical three-rank tensors. The tests show that the proposed model has a precision of 99.6%.

A Robust Screen-Free Brain-Computer Interface for Robotic Object Selection.

Brain signals represent a communication modality that can allow users of assistive robots to specify high-level goals, such as the object to fetch and deliver. In this paper, we consider a screen-free Brain-Computer Interface (BCI), where the robot highlights candidate objects in the environment using a laser pointer, and the user goal is decoded from the evoked responses in the electroencephalogram (EEG). Having the robot present stimuli in the environment allows for more direct commands than traditional BCIs that require the use of graphical user interfaces. Yet bypassing a screen entails less control over stimulus appearances. In realistic environments, this leads to heterogeneous brain responses for dissimilar objects—posing a challenge for reliable EEG classification. We model object instances as subclasses to train specialized classifiers in the Riemannian tangent space, each of which is regularized by incorporating data from other objects. In multiple experiments with a total of 19 healthy participants, we show that our approach not only increases classification performance but is also robust to both heterogeneous and homogeneous objects. While especially useful in the case of a screen-free BCI, our approach can naturally be applied to other experimental paradigms with potential subclass structure.

A novel hybrid paradigm based on steady state visually evoked potential & P300 to enhance information transfer rate

BackgroundSteady-State Visually Evoked Potential (SSVEP) has been one of the most common paradigms for Brain-Computer Interface (BCI) based applications with relatively good accuracy and Information Transfer Rate (ITR). However, limited decision options for a set number of paradigm frequencies, which is the main reason behind low ITR, is a huge hurdle the researchers have been facing in generalising SSVEP based BCI applications. New methodThis paper proposes a novel hybrid Brain-Computer Interface (hBCI) stimuli paradigm to improve ITR by increasing the number of decision options available for SSVEP BCI by introducing P300 as a Time Division Multiplexing (TDM) marker. One of the Hybrid BCI’s used distinct colours along with distinct flickering frequencies for targets, with an aim to improve the performance of the BCI based on the accuracy of classification, the elevation of ITR and reduction of FAR as compared to traditional BCIs. ResultsIt was established that the Novel SSVEP-P300 with distinct colours for target frequencies hybrid BCI had average parameters as following: classification accuracy of 92.30 %, ITR of 82.38 bits/min and FAR of 2.72 %. ComparisonA comparative study between the two novel paradigms, traditional SSVEP and P300 paradigms in the same environment was conducted. And a statistical inference was established sussing pairedt-tests. ConclusionThe results of the comparative study were conclusive that the hybrid BCI with distinct colours for each target frequency yielded best results and hence can be considered as a viable paradigm option for the development of an Assistive Device.

CNN With Large Data Achieves True Zero-Training in Online P300 Brain-Computer Interface

Each person has his or her own distinct event-related potential (ERP) signals. Thus, traditional brain-computer interface (BCI) systems require a calibration process in which the subject's data are extracted in order to train machine-learning classifiers. Despite past efforts to eliminate this process, often referred to as “zero-training,” BCI systems' best performance is achievable only with some level of calibration. This tedious process is one of the factors that have limited the use of BCI systems in the real world. Meanwhile, convolutional neural networks (CNN) have been proven to be useful in distinguishing neurophysiological features. In this study, we investigated whether an existing convolutional neural network (CNN) combined with large ERP samples (n = 99,000) can achieve zero-training in a P300 BCI speller system. As a result, the zero-trained CNN achieved comparable performance (89%, p 0.05) with calibration. Additionally, we illustrate a hybrid approach in order to further enhance performance, which adaptively updates a linear classifier using label information generated from a zero-trained CNN. With this technique, the hybrid approach achieved reasonable performance (92%), showing no statistical difference (p > 0.05) when compared to the traditional approach in the same offline data.

Regularized Group Sparse Discriminant Analysis for P300-Based Brain-Computer Interface.

Event-related potentials (ERPs) especially P300 are popular effective features for brain-computer interface (BCI) systems based on electroencephalography (EEG). Traditional ERP-based BCI systems may perform poorly for small training samples, i.e. the undersampling problem. In this study, the ERP classification problem was investigated, in particular, the ERP classification in the high-dimensional setting with the number of features larger than the number of samples was studied. A flexible group sparse discriminative analysis algorithm based on Moreau-Yosida regularization was proposed for alleviating the undersampling problem. An optimization problem with the group sparse criterion was presented, and the optimal solution was proposed by using the regularized optimal scoring method. During the alternating iteration procedure, the feature selection and classification were performed simultaneously. Two P300-based BCI datasets were used to evaluate our proposed new method and compare it with existing standard methods. The experimental results indicated that the features extracted via our proposed method are efficient and provide an overall better P300 classification accuracy compared with several state-of-the-art methods.

Effects of stimulus position on the classification of miniature asymmetric VEPs for brain-computer interfaces.

The speed of visual brain-computer interfaces (BCIs) has been greatly improved in recent years. However, traditional visual BCI paradigm requires users to directly gaze at the intensive flickering items, which would cause severe problems in practical applications, such as visual fatigue and excessive visual resources consumption. A promising solution is to use small visual stimuli outside the central visual area to encode instructions, which had been demonstrated to be effective in our previous study. This study aims to further investigate the effects of stimulus position on the classification of miniature asymmetric visual evoked potentials (aVEPs). Small peripheral visual stimuli were designed with different eccentricities (1° and 2°) and directions (0°, 45°, 90°, 135°, 180°, -135°, -90°, and -45°) to induce different kinds of miniature aVEPs. Five subjects participated in this experiment. Discriminative canonical pattern matching (DCPM) was used to classify all possible pairs of miniature aVEPs. Study results showed that visual stimuli with less eccentricity could induce more distinct miniature aVEPs. The highest single-trial accuracy achieved was about 83% for the binary classifications of miniature aVEPs pairs corresponding to (1°, -135°) Vs (1°, 0°), (1°, -45°) Vs (1°, -135°) and (1°, -45°) Vs (1°, 180°). This finding is very important for the design and development of the miniature aVEPs-based BCIs.

Multimodal Fuzzy Fusion for Enhancing the Motor-Imagery-Based Brain Computer Interface

Brain-computer interface technologies, such as steady-state visually evoked potential, P300, and motor imagery are methods of communication between the human brain and the external devices. Motor imagery-based brain-computer interfaces are popular because they avoid unnecessary external stimuli. Although feature extraction methods have been illustrated in several machine intelligent systems in motor imagery-based brain-computer interface studies, the performance remains unsatisfactory. There is increasing interest in the use of the fuzzy integrals, the Choquet and Sugeno integrals, that are appropriate for use in applications in which fusion of data must consider possible data interactions. To enhance the classification accuracy of brain-computer interfaces, we adopted fuzzy integrals, after employing the classification method of traditional brain-computer interfaces, to consider possible links between the data. Subsequently, we proposed a novel classification framework called the multimodal fuzzy fusion-based brain-computer interface system. Ten volunteers performed a motor imagery-based brain-computer interface experiment, and we acquired electroencephalography signals simultaneously. The multimodal fuzzy fusion-based brain-computer interface system enhanced performance compared with traditional brain-computer interface systems. Furthermore, when using the motor imagery-relevant electroencephalography frequency alpha and beta bands for the input features, the system achieved the highest accuracy, up to 78.81% and 78.45% with the Choquet and Sugeno integrals, respectively. Herein, we present a novel concept for enhancing brain-computer interface systems that adopts fuzzy integrals, especially in the fusion for classifying brain-computer interface commands.

Adaptive Spontaneous Brain–Computer Interfaces Based on Software Agents

Background: An adaptive Brain–Computer Interface (aBCI) is an extension of a traditional Brain–Computer Interface (BCI). In this work, trial rejection, median filter and software agent are included in a BCI to modify the parameters of its classifier and smoothen the signal feature during task execution. Methods: In this study, a database was used with five spontaneous mental tasks. A software agent was implemented to monitor a BCI for its adaptation. The software agent can learn from the environment and save this information. Results: The statistical significance and the effect size between a BCI and the aBCI proposed here were evaluated in this work. The Information Transfer Rate (ITR) in the aBCI was lower in comparison with BCI, however, the system has statistical significance and high effect size in the accuracy, sensitivity, specificity and kappa coefficient than the latter. Conclusions: Our aBCI improves the performance of a traditional BCI because the software agent can learn from its environment (brain signals) and adjust the BCI’s parameters. The signal quality was used as main factor to tune the feature extraction and parameters of the classifier.

Ternary Near-Infrared Spectroscopy Brain-Computer Interface With Increased Information Transfer Rate Using Prefrontal Hemodynamic Changes During Mental Arithmetic, Breath-Holding, and Idle State

The implementation of a multi-class brain-computer interface (BCI) is an efficient way to increase the information transfer rate (ITR) generally expressed in a unit of bits/trial. The improvement of ITR is of specific importance for near-infrared spectroscopy (NIRS)-BCI, because brain hemodynamic responses recorded by NIRS are much slower than the electrophysiological responses of the brain. In this paper, to implement a ternary NIRS-BCI with increased ITR, we used prefrontal cortex (PFC) hemodynamic changes induced by breath-holding (i.e., the voluntary suppression of breathing movements), which have never been used in the field of BCIs. Additionally, we used traditional BCI tasks such as mental arithmetic and idle state to implement ternary NIRS-BCI. As a result, an average offline ternary classification accuracy of 72.6 ± 10.7% could be achieved, which is the best performance of NIRS-BCI based on PFC hemodynamic changes reported to date. Because the number of available input commands was increased and the classification accuracy remained at an acceptable level, the ITR of the ternary BCI (0.51 ± 0.29 bits/trial) was 1.6 times higher than that of the traditional binary BCI (0.31 ± 0.21 bits/trial). Although the hemodynamic changes induced by the breath-holding were not caused by mental state changes, breath-holding proved to be a promising hybrid BCI task for implementing more efficient ternary NIRS-BCI.

Toward more intuitive brain-computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy.

In traditional brain-computer interface (BCI) studies, binary communication systems have generally been implemented using two mental tasks arbitrarily assigned to “yes” or “no” intentions (e.g., mental arithmetic calculation for “yes”). A recent pilot study performed with one paralyzed patient showed the possibility of a more intuitive paradigm for binary BCI communications, in which the patient’s internal yes/no intentions were directly decoded from functional near-infrared spectroscopy (fNIRS). We investigated whether such an “fNIRS-based direct intention decoding” paradigm can be reliably used for practical BCI communications. Eight healthy subjects participated in this study, and each participant was administered 70 disjunctive questions. Brain hemodynamic responses were recorded using a multichannel fNIRS device, while the participants were internally expressing “yes” or “no” intentions to each question. Different feature types, feature numbers, and time window sizes were tested to investigate optimal conditions for classifying the internal binary intentions. About 75% of the answers were correctly classified when the individual best feature set was employed (75.89% ± 1.39 and 74.08% ± 2.87 for oxygenated and deoxygenated hemoglobin responses, respectively), which was significantly higher than a random chance level (68.57% for p < 0.001). The kurtosis feature showed the highest mean classification accuracy among all feature types. The grand-averaged hemodynamic responses showed that wide brain regions are associated with the processing of binary implicit intentions. Our experimental results demonstrated that direct decoding of internal binary intention has the potential to be used for implementing more intuitive and user-friendly communication systems for patients with motor disabilities.

A P300 brain-computer interface based on a modification of the mismatch negativity paradigm.

The P300-based brain-computer interface (BCI) is an extension of the oddball paradigm, and can facilitate communication for people with severe neuromuscular disorders. It has been shown that, in addition to the P300, other event-related potential (ERP) components have been shown to contribute to successful operation of the P300 BCI. Incorporating these components into the classification algorithm can improve the classification accuracy and information transfer rate (ITR). In this paper, a single character presentation paradigm was compared to a presentation paradigm that is based on the visual mismatch negativity. The mismatch negativity paradigm showed significantly higher classification accuracy and ITRs than a single character presentation paradigm. In addition, the mismatch paradigm elicited larger N200 and N400 components than the single character paradigm. The components elicited by the presentation method were consistent with what would be expected from a mismatch paradigm and a typical P300 was also observed. The results show that increasing the signal-to-noise ratio by increasing the amplitude of ERP components can significantly improve BCI speed and accuracy. The mismatch presentation paradigm may be considered a viable option to the traditional P300 BCI paradigm.

A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control

Brain Computer Interfaces (BCIs) are developed to translate brain waves into machine instructions for external devices control. Recently, hybrid BCI systems are proposed for the multi-degree control of a real wheelchair to improve the systematical efficiency of traditional BCIs. However, it is difficult for existing hybrid BCIs to implement the multi-dimensional control in one command cycle. This paper proposes a novel hybrid BCI system that combines motor imagery (MI)-based bio-signals and steady-state visual evoked potentials (SSVEPs) to control the speed and direction of a real wheelchair synchronously. Furthermore, a hybrid modalities-based switch is firstly designed to turn on/off the control system of the wheelchair. Two experiments were performed to assess the proposed BCI system. One was implemented for training and the other one conducted a wheelchair control task in the real environment. All subjects completed these tasks successfully and no collisions occurred in the real wheelchair control experiment. The protocol of our BCI gave much more control commands than those of previous MI and SSVEP-based BCIs. Comparing with other BCI wheelchair systems, the superiority reflected by the index of path length optimality ratio validated the high efficiency of our control strategy. The results validated the efficiency of our hybrid BCI system to control the direction and speed of a real wheelchair as well as the reliability of hybrid signals-based switch control.

Toward a Semi-Self-Paced EEG Brain Computer Interface: Decoding Initiation State from Non-Initiation State in Dedicated Time Slots

Brain computer interfaces (BCIs) offer a broad class of neurologically impaired individuals an alternative means to interact with the environment. Many BCIs are “synchronous” systems, in which the system sets the timing of the interaction and tries to infer what control command the subject is issuing at each prompting. In contrast, in “asynchronous” BCIs subjects pace the interaction and the system must determine when the subject’s control command occurs. In this paper we propose a new idea for BCI which draws upon the strengths of both approaches. The subjects are externally paced and the BCI is able to determine when control commands are issued by decoding the subject’s intention for initiating control in dedicated time slots. A single task with randomly interleaved trials was designed to test whether it can be used as stimulus for inducing initiation and non-initiation states when the sensory and motor requirements for the two types of trials are very nearly identical. Further, the essential problem on the discrimination between initiation state and non-initiation state was studied. We tested the ability of EEG spectral power to distinguish between these two states. Among the four standard EEG frequency bands, beta band power recorded over parietal-occipital cortices provided the best performance, achieving an average accuracy of 86% for the correct classification of initiation and non-initiation states. Moreover, delta band power recorded over parietal and motor areas yielded a good performance and thus could also be used as an alternative feature to discriminate these two mental states. The results demonstrate the viability of our proposed idea for a BCI design based on conventional EEG features. Our proposal offers the potential to mitigate the signal detection challenges of fully asynchronous BCIs, while providing greater flexibility to the subject than traditional synchronous BCIs.