P300 Speller Brain-computer Interface Research Articles

Signal alignment for cross-datasets in P300 brain-computer interfaces

Objective. Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment (SA) for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning. Approach. We proposed a linear SA that uses the P300’s latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm). Results. Although the standard approach without SA had an average precision (AP) score of 25.5%, the approach demonstrated a 35.8% AP score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets. Significance. We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.

A design and evaluation of wearable p300 brain-computer interface system based on Hololens2

Patients with amyotrophic lateral sclerosis ( ALS ) often have difficulty in expressing their intentions through language and behavior, which prevents them from communicating properly with the outside world and seriously affects their quality of life. The brain-computer interface (BCI) has received much attention as an aid for ALS patients to communicate with the outside world, but the heavy device causes inconvenience to patients in the application process. To improve the portability of the BCI system, this paper proposed a wearable P300-speller brain-computer interface system based on the augmented reality (MR-BCI). This system used Hololens2 augmented reality device to present the paradigm, an OpenBCI device to capture EEG signals, and Jetson Nano embedded computer to process the data. Meanwhile, to optimize the system's performance for character recognition, this paper proposed a convolutional neural network classification method with low computational complexity applied to the embedded system for real-time classification. The results showed that compared with the P300-speller brain-computer interface system based on the computer screen (CS-BCI), MR-BCI induced an increase in the amplitude of the P300 component, an increase in accuracy of 1.7% and 1.4% in offline and online experiments, respectively, and an increase in the information transfer rate of 0.7 bit/min. The MR-BCI proposed in this paper achieves a wearable BCI system based on guaranteed system performance. It has a positive effect on the realization of the clinical application of BCI.

An Evaluation of the P300 Brain-Computer Interface, EyeLink Board, and Eye-Tracking Camera as Augmentative and Alternative Communication Devices.

Augmentative and alternative communication (AAC) systems are important to support communication for individuals with complex communication needs. A recent addition to AAC system options is the brain-computer interface (BCI). This study aimed to compare the clinical application of the P300 speller BCI with two more common AAC systems, the EyeLink board, and an eye-tracking camera. Ten participants without communication impairment (18-35 years of age) used each of the three AAC systems to spell three-letter words in one session. Accuracy and speed of letter selection were measured, and questionnaires were administered to evaluate usability, cognitive workload, and user preferences. The results showed that the BCI was significantly less accurate, slower, and with lower usability and higher cognitive workload compared to the eye-tracking camera and EyeLink board. Participants rated the eye-tracking camera as the most favorable AAC system on all measures. The results demonstrated that while the P300 speller BCI was usable by most participants, it did not function as well as the eye-tracking camera and EyeLink board. The clinical use of the BCI is, therefore, currently difficult to justify for most individuals, particularly when considering the substantial cost and setup resourcing needed. https://doi.org/10.23641/asha.21291384.

An Interpretable Convolutional Neural Network for P300 Detection: Analysis of Time Frequency Features for Limited Data

In this study, a new deep learning-based methodology is developed for P300 detection in brain computer interface (BCI) systems based on time-frequency (TF) features of EEG signals coupled to deep learning. The TF distributions can transform EEG signals to the TF images by simultaneous representation of time and frequency properties of the signal. However, they do not display the energy distribution of signals at different scales identically and their advantages may be greatly incorporated by using them together. Here, four TF images of single-trail EEG signal are computed and the concatenation of the TF (cTF) images of each signal is developed to be used as training data for a simple and lightweight deep learning-based classifier. The applied TF distributions are spectrogram, Wigner–Ville distribution, Morlet-scalogram, and Bertrand distribution. Performance of method is evaluated over limited data acquired from the normal and amyotrophic lateral sclerosis (ALS) datasets and accuracy of 96.56%, and 96.84% are achieved respectively, which is superior to the other comparing algorithms. Moreover, results of cross-subject classification indicate the promising ability of the method in eliminating calibration in BCI systems. Furthermore, the heat maps of the P300 and non-P300 classes are produced to explain important regions of cTF image for classifier decision and investigate which TF may help better classification. Results revealed the efficiency of cTF images for accurate P300 detection in simple structure classifiers having the advantage of fewer data and less memory requirement. This method can be employed in P300 speller BCI systems to improve the character recognition performance in.

Mitigating the Impact of Psychophysical Effects During Adaptive Stimulus Selection in the P300 Speller Brain-Computer Interface.

Stimulus-driven brain-computer interfaces (BCIs), such as the P300 speller, rely on using sensory stimuli to elicit specific neural signal components called event-related potentials (ERPs) to control external devices. However, psychophysical factors, such as refractory effects and adjacency distractions, may negatively impact ERP elicitation and BCI performance. Although conventional BCI stimulus presentation paradigms usually design stimulus presentation schedules in a pseudo-random manner, recent studies have shown that controlling the stimulus selection process can enhance ERP elicitation. In prior work, we developed an algorithm to adaptively select BCI stimuli using an objective criterion that maximizes the amount of information about the user's intent that can be elicited with the presented stimuli given current data conditions. Here, we enhance this adaptive BCI stimulus selection algorithm to mitigate adjacency distractions and refractory effects by modeling temporal dependencies of ERP elicitation in the objective function and imposing spatial restrictions in the stimulus search space. Results from simulations using synthetic data and human data from a BCI study show that the enhanced adaptive stimulus selection algorithm can improve spelling speeds relative to conventional BCI stimulus presentation paradigms.Clinical relevance-Increased communication rates with our enhanced adaptive stimulus selection algorithm can potentially facilitate the translation of BCIs as viable communication alternatives for individuals with severe neuromuscular limitations.

3-D Interface for the P300 Speller BCI

A brain-computer interface (BCI) is a common device for communication between the human brain and a computer. This article investigates the efficiency of using a 3-D interface for BCI machines. For this purpose, the P300 speller, which is a BCI device that enables the user to spell characters on a screen using brain waves, is modified. The classical virtual keyboard of the P300 speller is replaced with 3-D stereoscopic images, which enhances the ergonomic features of the device. Moreover, the flashing paradigm on a 3-D interface can affect the performance of the device in three ways: accuracy, speed, and capacity. This article proposes two different flashing paradigms called the natural 3-D and parallel 2-D interfaces and studied their effects regarding the three mentioned measures. The former flashes the planes in 3-D space, and the latter comprises flashes of parallel keyboards at different 3-D depths. The theoretical analysis of these effects is presented. The results are validated by experimental data obtained from real subjects and were compared with the classical 2-D interface. Both presented keyboards increase the speed of the device, while parallel 2-D has a better total performance than natural 3-D.

The Effect That Auditory Distractions Have on a Visual P300 Speller While Utilizing Low-Cost Off-the-Shelf Equipment

This paper investigates the effect that selected auditory distractions have on the signal of a visual P300 Speller in terms of accuracy, amplitude, latency, user preference, signal morphology, and overall signal quality. In addition, it ensues the development of a hierarchical taxonomy aimed at categorizing distractions in the P300b domain and the effect thereof. This work is part of a larger electroencephalography based project and is based on the P300 speller brain–computer interface (oddball) paradigm and the xDAWN algorithm, with eight to ten healthy subjects, using a non-invasive brain–computer interface based on low-fidelity electroencephalographic (EEG) equipment. Our results suggest that the accuracy was best for the lab condition (LC) at 100%, followed by music at 90% (M90) at 98%, trailed by music at 30% (M30) and music at 60% (M60) equally at 96%, and shadowed by ambient noise (AN) at 92.5%, passive talking (PT) at 90%, and finally by active listening (AL) at 87.5%. The subjects’ preference prodigiously shows that the preferred condition was LC as originally expected, followed by M90, M60, AN, M30, AL, and PT. Statistical analysis between all independent variables shows that we accept our null hypothesis for both the amplitude and latency. This work includes data and comparisons from our previous papers. These additional results should give some insight into the practicability of the aforementioned P300 speller methodology and equipment to be used for real-world applications.

Comparison between a Passive and Active response task and their effect on the Amplitude and Latency of the P300 component for Visual Stimuli while using Low Fidelity Equipment.

In this paper, we investigate the effect, in terms of amplitude and latency, of the P300 component in a separate active and passive task response condition. This work is based on the P300 speller BCI (oddball) paradigm and the xDAWN algorithm, with five healthy subjects; while using a noninvasive Brain-Computer Interface (BCI) based on low fidelity electroencephalographic (EEG) equipment. Our results suggest that an active task yielded a larger P300 peak amplitude while there was no discriminable difference in the peak latency. The signal was also morphological consistent in both scenarios, even though they did not yield identical P300 components. This groundwork yields imperative data for future work where we plan to introduce several distractions, including communication with the user while performing the P300 speller paradigm.

Optimizing the Performance of the Visual P300-Speller Through Active Mental Tasks Based on Color Distinction and Modulation of Task Difficulty.

Objective: P300-speller is the most commonly used brain-computer interface (BCI) for providing a means of communication to patients with amyotrophic lateral sclerosis. However, the performance of the P300-speller BCI is still inadequate. We investigated whether the performance of P300-speller can be further improved by increasing the mental effort required of the user.Methods: We designed two active mental tasks for a P300-speller based on a differently colored smiling cartoon-face paradigm. The tasks were based on color distinction, and their difficulty was modulated. One of the active mental tasks (DC task) required participants to focus on and distinguish the color of a target, while the other task (CN + DC task) required participants to simultaneously count the number of times a target flashed and distinguish its color.Results: The amplitudes of the event-related potentials (ERPs) in both DC and CN + DC tasks were higher than that in the CN task. The significant difference in the amplitudes between the DC and CN tasks was observed around the parietal-central area from 440 to 800 ms (late positive component, LPC), and that between the CN + DC and CN tasks was observed around the left-frontal and right-frontal areas from 320 to 480 ms (P3a) and the parietal-central area from 480 to 800 ms (P3b and LPC). The latency of the P300 potential in the CN + DC task was significantly longer than that in the CN task at F3, Fz, F4, C4, Pz, and P4 (P < 0.05). Offline (P < 0.05 at superposing once, twice, and thrice) and online (P < 0.001) classification results showed that the average accuracies in the CN + DC task were significantly greater than that in the CN task. Similar results were found for online information transfer rates (ITRs; P < 0.001). In addition, we found that the average online accuracies in the DC task were greater than those in the CN task, although the difference was not statistically significant (P = 0.051).Significance: The active mental task based on task difficulty modulation can significantly improve the performance of the P300-speller, and that based on color distinction shows a trend of improved performance.

Weighted extreme learning machine for P300 detection with application to brain computer interface

Brain–computer interface (BCI) is a communication system, which brain signals can be analyzed and transformed into commands of external devices. The P300-speller BCI utilizes the P300 evoked potentials, which generated in oddball paradigms, to spell characters. In order to eliminate the imbalance existed between target and non-target stimuli data in electroencephalogram (EEG) signals, we apply weighted extreme learning machine (WELM), which consequently improves the P300 detection accuracy. For starter, the raw EEG signals of all channels are divided into several segments by a given time window. Afterwards, features of each segment are extracted by principal component analysis. Then, WELM executes a classification for above features. Last, outputs of WELM classifier are used to recognize the target character in the P300 speller. Experiments were carried out based on the third BCI Competition Data Set II and achieved an average accuracy of 97%, and it still reached 85% with the first eight high sensitivity channels for P300 responses.

Improving P300 spelling rate using language models and predictive spelling

The P300 speller brain-computer interface (BCI) provides a means of communication for those suffering from advanced neuromuscular diseases such as amyotrophic lateral sclerosis (ALS). Recent literature has incorporated language-based modeling, which uses previously chosen characters and the structure of natural language to modify the interface and classifier. Two complementary methods of incorporating language models have previously been independently studied: predictive spelling uses language models to generate suggestions of complete words to allow for the selection of multiple characters simultaneously, and language-model-based classifiers have used prior characters to create a prior probability distribution over the characters based on how likely they are to follow. In this study, we propose a combined method which extends a language-based classifier to generate prior probabilities for both individual characters and complete words. In order to gage the efficiency of this new model, results across 12 healthy subjects were measured. Incorporating predictive spelling increased typing speed using the P300 speller, with an average increase of 15.5% in typing rate across subjects, demonstrating that language models can be effectively utilized to create full word suggestions for predictive spelling. When combining predictive spelling with language-model classification, typing speed is significantly improved, resulting in better typing performance.

Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis

ObjectivesAmyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disease, restricts patients’ communication capacity a few years after onset. A proof-of-concept of brain–computer interface (BCI) has shown promise in ALS and “locked-in” patients, mostly in pre-clinical studies or with only a few patients, but performance was estimated not high enough to support adoption by people with physical limitation of speech. Here, we evaluated a visual BCI device in a clinical study to determine whether disabled people with multiple deficiencies related to ALS would be able to use BCI to communicate in a daily environment. MethodsAfter clinical evaluation of physical, cognitive and language capacities, 20 patients with ALS were included. The P300 speller BCI system consisted of electroencephalography acquisition connected to real-time processing software and separate keyboard-display control software. It was equipped with original features such as optimal stopping of flashes and word prediction. The study consisted of two 3-block sessions (copy spelling, free spelling and free use) with the system in several modes of operation to evaluate its usability in terms of effectiveness, efficiency and satisfaction. ResultsThe system was effective in that all participants successfully achieved all spelling tasks and was efficient in that 65% of participants selected more than 95% of the correct symbols. The mean number of correct symbols selected per minute ranged from 3.6 (without word prediction) to 5.04 (with word prediction). Participants expressed satisfaction: the mean score was 8.7 on a 10-point visual analog scale assessing comfort, ease of use and utility. Patients quickly learned how to operate the system, which did not require much learning effort. ConclusionWith its word prediction and optimal stopping of flashes, which improves information transfer rate, the BCI system may be competitive with alternative communication systems such as eye-trackers. Remaining requirements to improve the device for suitable ergonomic use are in progress.

Projected accuracy metric for the P300 Speller.

The P300 Speller brain-computer interface (BCI) is a virtual keyboard that allows users to type without requiring neuromuscular control. P300 Speller research commonly aims to improve the system accuracy, which is typically estimated by spelling a small number of characters and calculating the percent spelled correctly. In this paper we introduce a new method for estimating the long-term ("projected") accuracy, which utilizes all available flash data and a probabilistic model of the Speller system to produce an estimate with lower variance and lower granularity than the standard measure. We apply the new method to 110 previously-collected P300 Speller runs to confirm its consistency, and simulate spelling runs from real subject data to demonstrate lower variance on the accuracy estimate for any given amount of data.

Channel selection methods for the P300 Speller.

The P300 Speller brain-computer interface (BCI) allows a user to communicate without muscle activity by reading electrical signals on the scalp via electroencephalogram. Modern BCI systems use multiple electrodes ("channels") to collect data, which has been shown to improve speller accuracy; however, system cost and setup time can increase substantially with the number of channels in use, so it is in the user's interest to use a channel set of modest size. This constraint increases the importance of using an effective channel set, but current systems typically utilize the same channel montage for each user. We examine the effect of active channel selection for individuals on speller performance, using generalized standard feature-selection methods, and present a new channel selection method, termed jumpwise regression, that extends the Stepwise Linear Discriminant Analysis classifier. Simulating the selections of each method on real P300 Speller data, we obtain results demonstrating that active channel selection can improve speller accuracy for most users relative to a standard channel set, with particular benefit for users who experience low performance using the standard set. Of the methods tested, jumpwise regression offers accuracy gains similar to the best-performing feature-selection methods, and is robust enough for online use.

P300 speller BCI with a mobile EEG system: comparison to a traditional amplifier

Objective. In a previous study, we presented a low-cost, small and wireless EEG system enabling the recording of single-trial P300 amplitudes in a truly mobile, outdoor walking condition (Debener et al (2012 Psychophysiology 49 1449–53)). Small and wireless mobile EEG systems have substantial practical advantages as they allow for brain activity recordings in natural environments, but these systems may compromise the EEG signal quality. In this study, we aim to evaluate the EEG signal quality that can be obtained with the mobile system. Approach. We compared our mobile 14-channel EEG system with a state-of-the-art wired laboratory EEG system in a popular brain–computer interface (BCI) application. N = 13 individuals repeatedly performed a 6 × 6 matrix P300 spelling task. Between conditions, only the amplifier was changed, while electrode placement and electrode preparation, recording conditions, experimental stimulation and signal processing were identical. Main results. Analysis of training and testing accuracies and information transfer rate (ITR) revealed that the wireless mobile EEG amplifier performed as good as the wired laboratory EEG system. A very high correlation for testing ITR between both amplifiers was evident (r = 0.92). Moreover the P300 topographies and amplitudes were very similar for both devices, as reflected by high degrees of association (r > = 0.77). Significance. We conclude that efficient P300 spelling with a small, lightweight and quick to set up mobile EEG amplifier is possible. This technology facilitates the transfer of BCI applications from the laboratory to natural daily life environments, one of the key challenges in current BCI research.

An Online Semi-supervised Brain–Computer Interface

Practical brain-computer interface (BCI) systems should require only low training effort for the user, and the algorithms used to classify the intent of the user should be computationally efficient. However, due to inter- and intra-subject variations in EEG signal, intermittent training/calibration is often unavoidable. In this paper, we present an online semi-supervised P300 BCI speller system. After a short initial training (around or less than 1 min in our experiments), the system is switched to a mode where the user can input characters through selective attention. In this mode, a self-training least squares support vector machine (LS-SVM) classifier is gradually enhanced in back end with the unlabeled EEG data collected online after every character input. In this way, the classifier is gradually enhanced. Even though the user may experience some errors in input at the beginning due to the small initial training dataset, the accuracy approaches that of fully supervised method in a few minutes. The algorithm based on LS-SVM and its sequential update has low computational complexity; thus, it is suitable for online applications. The effectiveness of the algorithm has been validated through data analysis on BCI Competition III dataset II (P300 speller BCI data). The performance of the online system was evaluated through experimental results on eight healthy subjects, where all of them achieved the spelling accuracy of 85 % or above within an average online semi-supervised learning time of around 3 min.

Optimal linear spatial filters for event-related potentials based on a spatio-temporal model: Asymptotical performance analysis

In this paper, the estimation of spatio-temporal patterns in the context of event-related potentials or evoked potentials studies in neuroscience is addressed. The proposed framework (denoted xDAWN) has the advantage to require only the knowledge of the time of stimuli onsets which are determined by the experimental setup. A theoretical analysis of the xDAWN framework shows that it provides asymptotically optimal spatial filters under weak assumptions. The loss in signal to interference-plus-noise ratio due to finite sample effect is calculated in a closed form at the first order of perturbation and is then validated by simulations. This last result shows that the proposed method provides interesting performance and outperforms classical methods, such as independent component analysis, in a wide range of situations. Moreover, the xDAWN algorithm has the property to be robust with respect to the model parameter values. Finally, validations on real electro-encephalographic data confirm the good behavior of the proposed xDAWN framework in the context of a P300 speller brain–computer interface.

Towards the detection of error-related potentials and its integration in the context of a P300 speller brain–computer interface

A P300 Speller is a brain–computer interface (BCI) that enables subjects to spell text on a computer screen by detecting P300 Event-Related Potentials in their electroencephalograms (EEG). This BCI application is of particular interest to disabled patients who have lost all means of verbal and motor communication. Error-related Potentials (ErrPs) in the EEG are generated by the subject's perception of an error. We report on the possibility of using these ErrPs for improving the performance of a P300 Speller. Overall nine subjects were tested, allowing us to study their EEG responses to correct and incorrect performances of the BCI, compare our findings to previous studies, explore the possibility of detecting ErrPs and discuss the integration of ErrP classifiers into the P300 Speller system.

Signals from intraventricular depth electrodes can control a brain–computer interface

A brain–computer interface (BCI) is a device that enables severely disabled people to communicate and interact with their environments using their brain waves. Most research investigating BCI in humans have used scalp-recorded electroencephalography (EEG). We have recently demonstrated that signals from intracranial electrocorticography (ECoG) and stereotactic depth electrodes (SDE) in the hippocampus can be used to control a BCI P300 Speller paradigm. We report a case in which stereotactic depth electrodes positioned in the ventricle were able to obtain viable signals for a BCI. Our results demonstrate that event-related potentials from intraventricular electrodes can be used to reliably control the P300 Speller BCI paradigm.

Adaptive training session for a P300 speller brain–computer interface

With a brain–computer interface (BCI), it is nowadays possible to achieve a direct pathway between the brain and computers thanks to the analysis of some particular brain activities. The detection of even-related potentials, like the P300 in the oddball paradigm exploited in P300-speller, provides a way to create BCIs by assigning several detected ERP to a command. Due to the noise present in the electroencephalographic signal, the detection of an ERP and its different components requires efficient signal processing and machine learning techniques. As a consequence, a calibration session is needed for training the models, which can be a drawback if its duration is too long. Although the model depends on the subject, the goal is to provide a reliable model for the P300 detection over time. In this study, we propose a new method to evaluate the optimal number of symbols (i.e. the number of ERP that shall be detected given a determined target probability) that should be spelt during the calibration process. The goal is to provide a usable system with a minimum calibration duration and such that it can automatically switch between the training and online sessions. The method allows to adaptively adjust the number of training symbols to each subject. The evaluation has been tested on data recorded on 20 healthy subjects. This procedure lets drastically reduced the calibration session: height symbols during the training session reach an initialized system with an average accuracy of 80% after five epochs.