P300-based Brain-computer Interface Speller Research Articles

A Systematic Review of Computational Intelligence Techniques for Channel Selection in P300-Based Brain Computer Interface Speller

Electroencephalography (EEG) based P300 speller aid in restoring the communication and control capabilities in patients suffering from motor disabilities. However, the quality and quantity of the data collected from EEG recordings have a substantial influence on the P300 speller's performance. Hence, selecting the optimum number of recording electrodes, i.e., channels for each user, is a significant difficulty for the P300 speller. There are two fundamental objectives of the channel selection process: 1) To extract the most crucial information from the relevant channels, hence reducing the computing complexity of P300/Non-P300 signal processing operation, and 2) To lessen the potential overfitting that could result from using unwanted channels to boost performance. For obtaining the best channel subsets, different channel selection techniques, including manual, filtering, wrapper, and embedded approaches, have been applied by past researchers. This study presents a comprehensive review of recent developments, present status, challenges, and possible solution associated with channel selection strategies in P300 speller.

A novel multiclass-based framework for P300 detection in BCI matrix speller: Temporal EEG patterns of non-target trials vary based on their position to previous target stimuli

Brain–computer interface (BCI) provides a new communication pathway for severely disabled people and enables them to communicate with external world using only their brain activity. P300-based BCI speller helps patients spell words using their brain signals.Until now, binary-classification-based approaches have been used for P300 detection. This study demonstrates that binary-classification-based approaches may not be appropriate for P300 detection. We proved that temporal EEG patterns of non-target trials are different based on their position to previous target stimuli. Therefore, considering all non-target trials in only one group makes distinguishing target from non-target components difficult for machine learning algorithms and, consequently, deteriorate character recognition accuracy. This study introduced a novel approach for P300 detection in BCI spellers. In this study, we first divided non-target trials into several groups according to their temporal patterns in training stage. Then, we introduced a multiclass-based framework for P300 detection.Proposed approach is evaluated with three public datasets, BCI competition II, BCI competition III and the BNCI Horizon. In all three datasets, proposed multi-class approach outperformed common binary-classification-based approach in the same conditions. In addition, multiclass-based approach reached the highest accuracy (100%) in the 3th sequence for BCI competition II, for BCI competition III achieved an average accuracy of 74% and 98% in 5th and 15th sequences and, for the BNCI Horizon dataset achieved an average accuracy of 77.50% and 97.49% in 5th and 10th sequences, respectively. That means our proposed approach, regardless of its simplicity, achieved state-of-the-art character recognition performance in the existing methods. The results confirmed that binary-classification based methods is not appropriate for P300 detection in BCI spellers.

A new benchmark dataset for P300 ERP-based BCI applications

Because of its non-invasive nature, one of the most commonly used event-related potentials in brain-computer interface (BCI) system designs is the P300 electroencephalogram (EEG) signal. The fact that the P300 response can easily be stimulated and measured is particularly important for participants with severe motor disabilities. In order to train and test P300-based BCI speller systems in more realistic high-speed settings, there is a pressing need for a large and challenging benchmark dataset. Various datasets already exist in the literature but most of them are not publicly available, and they either have a limited number of participants or utilize relatively long stimulus duration (SD) and inter-stimulus intervals (ISI). They are also typically based on a 36 target (6×6) character matrix. The use of long ISI, in particular, not only reduces the speed and the information transfer rates (ITRs) but also oversimplifies the P300 detection. This leaves a limited challenge to state-of-the-art machine learning and signal processing algorithms. In fact, near-perfect P300 classification accuracies are reported with the existing datasets. Therefore, one certainly needs a large-scale dataset with challenging settings to fully exploit the recent advancements in algorithm design (machine learning and signal processing) and achieve high-performance speller results. To this end, in this article we introduce a new freely- and publicly-accessible P300 dataset obtained using 32-channel EEG, in the hope that it will lead to new research findings and eventually more efficient BCI designs. The introduced dataset comprises 18 participants performing a 40-target (5×8) cued-spelling task, with reduced SD (66.6 ms) and ISI (33.3 ms) for fast spelling. We have also processed, analyzed, and character-classified the introduced dataset and we presented the accuracy and ITR results as a benchmark. The introduced dataset and the codes of our experiments are publicly accessible at https://data.mendeley.com/datasets/vyczny2r4w.

ST-CapsNet: Linking Spatial and Temporal Attention with Capsule Network for P300 Detection Improvement.

A brain-computer interface (BCI), which provides an advanced direct human-machine interaction, has gained substantial research interest in the last decade for its great potential in various applications including rehabilitation and communication. Among them, the P300-based BCI speller is a typical application that is capable of identifying the expected stimulated characters. However, the applicability of the P300 speller is hampered for the low recognition rate partially attributed to the complex spatio-temporal characteristics of the EEG signals. Here, we developed a deep-learning analysis framework named ST-CapsNet to overcome the challenges regarding better P300 detection using a capsule network with both spatial and temporal attention modules. Specifically, we first employed spatial and temporal attention modules to obtain refined EEG signals by capturing event-related information. Then the obtained signals were fed into the capsule network for discriminative feature extraction and P300 detection. In order to quantitatively assess the performance of the proposed ST-CapsNet, two publicly-available datasets (i.e., Dataset IIb of BCI Competition 2003 and Dataset II of BCI Competition III) were applied. A new metric of averaged symbols under repetitions (ASUR) was adopted to evaluate the cumulative effect of symbol recognition under different repetitions. In comparison with several widely-used methods (i.e., LDA, ERP-CapsNet, CNN, MCNN, SWFP, and MsCNN-TL-ESVM), the proposed ST-CapsNet framework significantly outperformed the state-of-the-art methods in terms of ASUR. More interestingly, the absolute values of the spatial filters learned by ST-CapsNet are higher in the parietal lobe and occipital region, which is consistent with the generation mechanism of P300.

Feature selection method based on Menger curvature and LDA theory for a P300 brain–computer interface

Objective. Brain–computer interface (BCI) systems decode electroencephalogram (EEG) signals to establish a channel for direct interaction between the human brain and the external world without the need for muscle or nerve control. The P300 speller, one of the most widely used BCI applications, presents a selection of characters to the user and performs character recognition by identifying P300 event-related potentials from the EEG. Such P300-based BCI systems can reach good levels of accuracy but are difficult to use in day-to-day life due to redundancy and noisy signal. A room for improvement should be considered. We propose a novel hybrid feature selection method for the P300-based BCI system to address the problem of feature redundancy, which combines the Menger curvature and linear discriminant analysis. Approach. First, selected strategies are applied separately to a given dataset to estimate the gain for application to each feature. Then, each generated value set is ranked in descending order and judged by a predefined criterion to be suitable in classification models. The intersection of the two approaches is then evaluated to identify an optimal feature subset. Main results. The proposed method is evaluated using three public datasets, i.e. BCI Competition III dataset II, brain/neural computer interaction Horizon dataset, and Lausanne Federal Institute of Technology dataset. Experimental results indicate that compared with other typical feature selection and classification methods, our proposed method has better or comparable performance. Significance. Additionally, our proposed method can achieve the best classification accuracy after all epochs in three datasets. In summary, our proposed method provides a new way to enhance the performance of the P300-based BCI speller.

P300-Based Brain-Computer Interface Speller: Usability Evaluation of Three Speller Sizes by Severely Motor-Disabled Patients.

Brain-computer interface (BCI) spellers allow severe motor-disabled patients to communicate using their brain activity without muscular mobility. Different visual configurations of the widely studied P300-based BCI speller had been assessed with healthy and motor-disabled users. However, the speller size (in terms of cm) had only been assessed for healthy subjects. We think that the speller size might be limiting for some severely motor-disabled patients with restricted head and eye movements. The usability of three speller sizes was assessed for seven patients diagnosed with amyotrophic lateral sclerosis (ALS) and a participant diagnosed with Duchenne muscular dystrophy (DMD). This is the first usability evaluation of speller size with severely motor-disabled participants. Effectiveness (in the online results) and efficiency (in the workload test) of the medium speller was remarkably better. Satisfaction was significantly the highest with the medium size speller and the lowest with the small size. These results correlate with previously described findings in healthy subjects. In conclusion, the speller size should be considered when designing a speller paradigm, especially for motor-disabled individuals, since it might affect their performance and user experience while controlling a BCI speller.

Habituation of P300 in the Use of P300-based Brain-Computer Interface Spellers: Individuals With Amyotrophic Lateral Sclerosis Versus Age-Matched Controls

The P300-based brain-computer interface speller can provide motor independent communication to individuals with amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disorder that affects the motor system. P300 amplitude stability is critical for operation of the P300 speller. The P300 has good long-term stability, but to our knowledge, short-term habituation in the P300 speller has not been studied. In the current study, 15 participants: 8 ALS patients and 7 age-matched healthy volunteers (HVs), used 2 versions of P300 spellers, Face speller and Flash speller, each for 30 minutes. The ALS group performed as well as the HVs in both spellers and HVs did better with the Face speller than Flash speller while the ALS group performed equally well in both spellers. Neither intra-run P300 habituation nor inter-run P300 habituation was found. The P300 speller could be a reliable communication device for individuals with ALS.

EEG Waveform Analysis of P300 ERP with Applications to Brain Computer Interfaces.

The Electroencephalography (EEG) is not just a mere clinical tool anymore. It has become the de-facto mobile, portable, non-invasive brain imaging sensor to harness brain information in real time. It is now being used to translate or decode brain signals, to diagnose diseases or to implement Brain Computer Interface (BCI) devices. The automatic decoding is mainly implemented by using quantitative algorithms to detect the cloaked information buried in the signal. However, clinical EEG is based intensively on waveforms and the structure of signal plots. Hence, the purpose of this work is to establish a bridge to fill this gap by reviewing and describing the procedures that have been used to detect patterns in the electroencephalographic waveforms, benchmarking them on a controlled pseudo-real dataset of a P300-Based BCI Speller and verifying their performance on a public dataset of a BCI Competition.

Double ErrP Detection for Automatic Error Correction in an ERP-Based BCI Speller.

Brain-computer interface (BCI) is a useful device for people with severe motor disabilities. However, due to its low speed and low reliability, BCI still has a very limited application in daily real-world tasks. This paper proposes a P300-based BCI speller combined with a double error-related potential (ErrP) detection to automatically correct erroneous decisions. This novel approach introduces a second error detection to infer whether wrong automatic correction also elicits a second ErrP. Thus, two single-trial responses, instead of one, contribute to the final selection, improving the reliability of error detection. Moreover, to increase error detection, the evoked potential detected as target by the P300 classifier is combined with the evoked error potential at a feature-level. Discriminable error and positive potentials (response to correct feedback) were clearly identified. The proposed approach was tested on nine healthy participants and one tetraplegic participant. The online average accuracy for the first and second ErrPs were 88.4% and 84.8%, respectively. With automatic correction, we achieved an improvement around 5% achieving 89.9% in spelling accuracy for an effective 2.92 symbols/min. The proposed approach revealed that double ErrP detection can improve the reliability and speed of BCI systems.

Toward a Hybrid BCI: Self-Paced Operation of a P300-based Speller by Merging a Motor Imagery-Based “Brain Switch” into a P300 Spelling Approach

ABSTRACTThis study presents the self-paced operation of a brain–computer interface (BCI) speller, which can be voluntarily turned on/off by merging a motor imagery (MI)-based brain switch into a P300-based BCI speller. From an off state (idle state), the users can generate a “control signal” by consciously changing the cognitive state differential from the idle state to turn on a P300-based spelling system when he or she wants to spell words. With the system turned on, the user can spell words, and then, the spelling system can be voluntarily turned off and switched to the initial state using a command. In this paradigm, the participants tried to perform the two different cognitive tasks sequentially, rather than simultaneously, and multiple EEG components were processed sequentially. The practicability and effectiveness of the proposed approach were validated by eleven participants, and all of them achieved a satisfactory performance. For the P300 speller, they achieved an average PITR of 42.61 bits/min. The preliminary results indicated that the proposed hybrid BCI system with different mental strategies operating sequentially is feasible and has potential applications for practical self-paced control.

Effects of feedback latency on P300-based brain-computer interface.

Feedback has been shown to affect performance when using a Brain-Computer Interface (BCI) based on sensorimotor rhythms. In contrast, little is known about the influence of feedback on P300-based BCIs. There is still an open question whether feedback affects the regulation of P300 and consequently the operation of P300-based BCIs. In this paper, for the first time, the influence of feedback on the P300-based BCI speller task is systematically assessed. For this purpose, 24 healthy participants performed the classic P300-based BCI speller task, while only half of them received feedback. Importantly, the number of flashes per letter was reduced on a regular basis in order to increase the frequency of providing feedback. Experimental results showed that feedback could significantly improve the P300-based BCI speller performance, if it was provided in short time intervals (e.g. in sequences as short as 4 to 6 flashes per row/column). Moreover, our offline analysis showed that providing feedback remarkably enhanced the relevant ERP patterns and attenuated the irrelevant ERP patterns, such that the discrimination between target and non-target EEG trials increased.

Semi-supervised joint spatio-temporal feature selection for P300-based BCI speller.

In this paper, we address the important problem of feature selection for a P300-based brain computer interface (BCI) speller system in several aspects. Firstly, time segment selection and electroencephalogram channel selection are jointly performed for better discriminability of P300 and background signals. Secondly, in view of the situation that training data with labels are insufficient, we propose an iterative semi-supervised support vector machine for joint spatio-temporal feature selection as well as classification, in which both labeled training data and unlabeled test data are utilized. More importantly, the semi-supervised learning enables the adaptivity of the system. The performance of our algorithm has been evaluated through the analysis of a P300 dataset provided by BCI Competition 2005 and another dataset collected from an in-house P300 speller system. The results show that our algorithm for joint feature selection and classification achieves satisfactory performance, meanwhile it can significantly reduce the training effort of the system. Furthermore, this algorithm is implemented online and the corresponding results demonstrate that our algorithm can improve the adaptiveness of the P300-based BCI speller.

Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain–computer interface speller

The brain–computer interface P300 speller is aimed to help those patients unable to activate muscles to spell words by utilizing their brain activity. However, a problem associated with the use of this brain–computer interface paradigm is the generation mechanics of P300 related to responses to visual stimuli. Herein, we investigated the event-related potential (ERP) response for the P300-based brain–computer interface speller. A signal preprocessing method integrated coherent average, principal component analysis (PCA) and independent component analysis (ICA) to reduce the dimensions and noise in the raw data. The time–frequency analysis was based on wavelet and two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) were computed to indicate the evoked response (time-locked) and phase reset (phase-locked) activity, respectively. Results demonstrated that the proposed method was valid for the time-locked and phase-locked feature extraction and both the evoked response and phase reset contributed to the genesis of the P300 signal. These electrophysiological responses characteristics of ERPs would be used for BCI P300 speller design and its signal processing strategies.

Which Physiological Components are More Suitable for Visual ERP Based Brain–Computer Interface? A Preliminary MEG/EEG Study

We investigated which evoked response component occurring in the first 800 ms after stimulus presentation was most suitable to be used in a classical P300-based brain-computer interface speller protocol. Data was acquired from 275 Magnetoencephalographic sensors in two subjects and from 61 Electroencephalographic sensors in four. To better characterize the evoked physiological responses and minimize the effect of response overlap, a 1000 ms Inter Stimulus Interval was preferred to the short (<400 ms) trial length traditionally used in this class of BCIs. To investigate which scalp regions conveyed information suitable for BCI, a stepwise linear discriminant analysis classifier was used. The method iteratively analyzed each individual sensor and determined its performance indicators. These were then plotted on a 2-D topographic head map. Preliminary results for both EEG and MEG data suggest that components other than the P300 maximally represented in the occipital region, could be successfully used to improve classification accuracy and finally drive this class of BCIs.

Online detection of P300 and error potentials in a BCI speller.

Error potentials (ErrPs), that is, alterations of the EEG traces related to the subject perception of erroneous responses, have been suggested to be an elegant way to recognize misinterpreted commands in brain-computer interface (BCI) systems. We implemented a P300-based BCI speller that uses a genetic algorithm (GA) to detect P300s, and added an automatic error-correction system (ECS) based on the single-sweep detection of ErrPs. The developed system was tested on-line on three subjects and here we report preliminary results. In two out of three subjects, the GA provided a good performance in detecting P300 (90% and 60% accuracy with 5 repetitions), and it was possible to detect ErrP with an accuracy (roughly 60%) well above the chance level. In our knowledge, this is the first time that ErrP detection is performed on-line in a P300-based BCI. Preliminary results are encouraging, but further refinements are needed to improve performances.

A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system

In this paper, we first present a self-training semi-supervised support vector machine (SVM) algorithm and its corresponding model selection method, which are designed to train a classifier with small training data. Next, we prove the convergence of this algorithm. Two examples are presented to demonstrate the validity of our algorithm with model selection. Finally, we apply our algorithm to a data set collected from a P300-based brain computer interface (BCI) speller. This algorithm is shown to be able to significantly reduce training effort of the P300-based BCI speller.