Brain-computer Interface Speller Research Articles

Habituation of P300 in the Use of P300-based Brain-Computer Interface Spellers: Individuals With Amyotrophic Lateral Sclerosis Versus Age-Matched Controls

The P300-based brain-computer interface speller can provide motor independent communication to individuals with amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disorder that affects the motor system. P300 amplitude stability is critical for operation of the P300 speller. The P300 has good long-term stability, but to our knowledge, short-term habituation in the P300 speller has not been studied. In the current study, 15 participants: 8 ALS patients and 7 age-matched healthy volunteers (HVs), used 2 versions of P300 spellers, Face speller and Flash speller, each for 30 minutes. The ALS group performed as well as the HVs in both spellers and HVs did better with the Face speller than Flash speller while the ALS group performed equally well in both spellers. Neither intra-run P300 habituation nor inter-run P300 habituation was found. The P300 speller could be a reliable communication device for individuals with ALS.

Brain-computer interface speller system design from electroencephalogram signals with channel selection algorithms

Background and objectiveBrain-computer interfaces (BCI) have started to be used with the development of computer technology in order to enable individuals who are in this situation to communicate with their environment or move. This study focused on the spelling system that transforms the brain activities obtained with EEG signals into writing. In BCI systems working with P300 obtained from 64 electrodes, data recording and processing cause high cost and high processing load. By reducing the number of electrodes used, the physical dimensions, costs, and processing loads of the systems can be reduced. The main problem at this stage is to determine which electrodes are more effective. Randomness-based optimization methods perform their experiments within the framework of a specific fitness function, resulting in near-best results rather than the best result. The electrodes chosen as a result of the study are expected to contribute positively to the classifier performance. At the same time, an unbalanced data set is balanced, and an increase in system performance is expected. MethodElectrode selection was performed in both the original dataset and ADASYN dataset using the Genetic Algorithm and Binary Particle Swarm Optimization methods. As a dataset, Wadsworth BCI Dataset (P300 Evoked Potentials) was used in the study. The channels chosen most frequently by optimization methods were determined and compared with the 64-channel classification results using LS-SVM and LDA. ResultAs a result of the optimization processes, the eight channels selected most frequently, the channels selected more than the average of all the selected channels and 64 channel results were compared. The highest accuracy was achieved with the LDA classifier for user A with 29 channels selected with BPSO with 97.250%. ConclusionsThe results obtained in the study showed that the number of channels decreased by optimization methods increases the classification performance. In addition, classifier training and test times have been greatly reduced. The application of the ADASYN method did not result in any significant difference.

Character encoding based on occurrence probability enhances the performance of SSVEP-based BCI spellers

ObjectiveSteady-state visual evoked potential (SSVEP) is a control signal which is widely used in brain-computer interface (BCI) systems. The SSVEP-based spellers with hierarchical structure have a limitation of low ITR. To improve the ITR in these spellers, we effectively applied the character encoding based on the character frequency rate. MethodsWe proposed the 1–2 level hierarchical structure that allows the user to spell the most used characters just in one stage, while other characters will be selected through two stages. We also considered the latency at the start of each trial, to enhance the SSVEP classification accuracy. To estimate the ITR more accurately, we used a novel ITR definition for the first time, which considers the symbol occurrence probability. ResultsThe proposed speller achieved the mean classification accuracy of 90.5%, the ITR of 48.3 bit/min, and the speed of 13.2 char/min. The latency varies for different subjects, and the mean value of 0.2 was determined across all individuals. ConclusionConsidering the character encoding enhances the performance of SSVEP-based BCI spellers. SignificanceThe proposed speller provides a reliable and easy-to-use assistive communication system for locked-in patients.

A novel hybrid BCI speller based on RSVP and SSVEP paradigm

Background and objectiveSteady-state visual evoked potential (SSVEP) and rapid serial visual presentation (RSVP) are useful methods in the brain-computer interface (BCI) systems. Hybrid BCI systems that combine these two approaches can enhance the proficiency of the P300 spellers. MethodsIn this study, a new hybrid RSVP/SSVEP BCI is proposed to increase the classification accuracy and information transfer rate (ITR) as compared with the other RSVP speller paradigms. In this paradigm, RSVP (eliciting a P300 response) and SSVEP stimulations are presented in such a way that the target group of characters is identified by RSVP stimuli, and the target character is recognized by SSVEP stimuli. ResultsThe proposed paradigm achieved accuracy of 93.06%, and ITR of 23.41 bit/min averaged across six subjects. ConclusionsThe new hybrid system demonstrates that by using SSVEP stimulation in Triple RSVP speller paradigm, we could enhance the performance of the system as compared with the traditional Triple RSVP paradigm. Our work is the first hybrid paradigm in RSVP spellers that could obtain the higher classification accuracy and information transfer rate in comparison with the previous RSVP spellers.

On Error-Related Potentials During Sensorimotor-Based Brain-Computer Interface: Explorations With a Pseudo-Online Brain-Controlled Speller.

Objective: Brain-computer interface (BCI) spelling is a promising communication solution for people in paralysis. Currently, BCIs suffer from imperfect decoding accuracy which calls for methods to handle spelling mistakes. Detecting error-related potentials (ErrPs) has been early identified as a potential remedy. Nevertheless, few works have studied the elicitation of ErrPs during engagement with other BCI tasks, especially when BCI feedback is provided continuously. Methods: Here, we test the possibility of correcting errors during pseudo-online Motor Imagery (MI) BCI spelling through ErrPs, and investigate whether BCI feedback hinders their generation. Ten subjects performed a series of MI spelling tasks with and without observing BCI feedback. Results: The average pseudo-online ErrP detection accuracy was found to be significantly above the chance level in both conditions and did not significantly differ between the two (74% with, and 78% without feedback). Conclusions: Our results support the possibility to detect ErrPs during MI-BCI spelling and suggest the absence of any BCI feedback-related interference.

CNN With Large Data Achieves True Zero-Training in Online P300 Brain-Computer Interface

Each person has his or her own distinct event-related potential (ERP) signals. Thus, traditional brain-computer interface (BCI) systems require a calibration process in which the subject's data are extracted in order to train machine-learning classifiers. Despite past efforts to eliminate this process, often referred to as “zero-training,” BCI systems' best performance is achievable only with some level of calibration. This tedious process is one of the factors that have limited the use of BCI systems in the real world. Meanwhile, convolutional neural networks (CNN) have been proven to be useful in distinguishing neurophysiological features. In this study, we investigated whether an existing convolutional neural network (CNN) combined with large ERP samples (n = 99,000) can achieve zero-training in a P300 BCI speller system. As a result, the zero-trained CNN achieved comparable performance (89%, p 0.05) with calibration. Additionally, we illustrate a hybrid approach in order to further enhance performance, which adaptively updates a linear classifier using label information generated from a zero-trained CNN. With this technique, the hybrid approach achieved reasonable performance (92%), showing no statistical difference (p > 0.05) when compared to the traditional approach in the same offline data.

Optimising non-invasive brain-computer interface systems for free communication between na\xefve human participants

Free communication is one of the cornerstones of modern civilisation. While manual keyboards currently allow us to interface with computers and manifest our thoughts, a next frontier is communication without manual input. Brain-computer interface (BCI) spellers often achieve this by decoding patterns of neural activity as users attend to flickering keyboard displays. To date, the highest performing spellers report typing rates of ~10.00 words/minute. While impressive, these rates are typically calculated for experienced users repetitively typing single phrases. It is therefore not clear whether naïve users are able to achieve such high rates with the added cognitive load of genuine free communication, which involves continuously generating and spelling novel words and phrases. In two experiments, we developed an open-source, high-performance, non-invasive BCI speller and examined its feasibility for free communication. The BCI speller required users to focus their visual attention on a flickering keyboard display, thereby producing unique cortical activity patterns for each key, which were decoded using filter-bank canonical correlation analysis. In Experiment 1, we tested whether seventeen naïve users could maintain rapid typing during prompted free word association. We found that information transfer rates were indeed slower during this free communication task than during typing of a cued character sequence. In Experiment 2, we further evaluated the speller’s efficacy for free communication by developing a messaging interface, allowing users to engage in free conversation. The results showed that free communication was possible, but that information transfer was reduced by voluntary textual corrections and turn-taking during conversation. We evaluated a number of factors affecting the suitability of BCI spellers for free communication, and make specific recommendations for improving classification accuracy and usability. Overall, we found that developing a BCI speller for free communication requires a focus on usability over reduced character selection time, and as such, future performance appraisals should be based on genuine free communication scenarios.

Peak Detection with Online Electroencephalography (EEG) Artifact Removal for Brain-Computer Interface (BCI) Purposes.

Brain–computer interfaces (BCIs) measure brain activity and translate it to control computer programs or external devices. However, the activity generated by the BCI makes measurements for objective fatigue evaluation very difficult, and the situation is further complicated due to different movement artefacts. The BCI performance could be increased if an online method existed to measure the fatigue objectively and accurately. While BCI-users are moving, a novel automatic online artefact removal technique is used to filter out these movement artefacts. The effects of this filter on BCI performance and mainly on peak frequency detection during BCI use were investigated in this paper. A successful peak alpha frequency measurement can lead to more accurately determining objective user fatigue. Fifteen subjects performed various imaginary and actual movements in separate tasks, while fourteen electroencephalography (EEG) electrodes were used. Afterwards, a steady-state visual evoked potential (SSVEP)-based BCI speller was used, and the users were instructed to perform various movements. An offline curve fitting method was used for alpha peak detection to assess the effect of the artefact filtering. Peak detection was improved by the filter, by finding 10.91% and 9.68% more alpha peaks during simple EEG recordings and BCI use, respectively. As expected, BCI performance deteriorated from movements, and also from artefact removal. Average information transfer rates (ITRs) were 20.27 bit/min, 16.96 bit/min, and 14.14 bit/min for the (1) movement-free, (2) the moving and unfiltered, and (3) the moving and filtered scenarios, respectively.

P300 ve DHGUP Tabanlı Hibrid BBA Heceleyicisi için Yeni bir Yaklaşım

P300 and steady state visual evoked potential (SSVEP) are type of electroencephalography (EEG) phenomena that widely used in brain computer interface (BCI) systems since both of them have high signal response and signal noise ratio. Classification accuracy rate of signal, and signal detection time affect overall performance of BCI systems. These both values are used for calculation information transfer rate (ITR) that is a key performance indicator for a BCI system. A P300 based BCI or a SSVEP based BCI have higher ITR values than other type of BCI systems. Thus, in this study our aim was to use together these both P300 and SSVEP phenomena in a BCI speller. We proposed a hybrid BCI speller based on P300 and SSVEP. Moreover, our proposed BCI speller interface allows to use only P300 stimuli, only SSVEP stimuli, or hybrid stimuli. In this BCI speller, there are numbers in 3 × 3 matrix form for elicitind P300 signal and also 9 white square flickering objects were placed near numbers for eliciting SSVEP. In this research, experiments were performed in two stage (training and online stages) with three sessions (only SSVEP stimuli session, only P300 stimuli session, and hybrid session). Five subjects participated experiments. We used support vector machine method for detection of P300 signal and SSVEP. According to experiment results, average classification accuracy values were 83.78%, 84.67%, and 90.89% with using only SSVEP stimuli, only P300 stimuli, and hybrid stimuli, respectively. Furhermore, average information transfer rate values were 6.81, 6.97, and, 8.19 bit/min with using only SSVEP stimuli, only P300 stimuli, and hybrid stimuli, respectively. Results showed that the proposed hybrid BCI speller based on P300 and SSVEP reached higher classification accuracy and ITR values than using only SSVEP stimuli or only P300 stimuli based BCI spellers.

Evaluation of emotional and neutral pictures as flashing stimuli using a P300 brain–computer interface speller

Objective. Previous works have reported that complex emotional and visual stimuli can increase the amplitude of the P300 brain potential. Thus, the aim of the present work is to assess these kinds of images in a P300 brain–computer interface (BCI) speller as flashing stimuli. Approach. Twenty-three volunteers controlled four spellers with different sets of flashing stimuli: flashing letters, neutral pictures (NP), emotional pleasant pictures (EPP) and emotional unpleasant pictures (EUP). Main results. The sets of pictures showed a higher performance than the letters in accuracy and information transfer rate. These results were supported by the analysis of the P300 signal, where the picture sets offered the greatest amplitudes. The NP and EPP sets were the best evaluated in the subjective questionnaire. Significance. In short, despite the fact that the effect of emotional stimuli could not be observed in the performance metrics, picture sets have offered a high performance and should be considered in future proposals for visual P300-based BCI applications.

Asynchronous Control of ERP-Based BCI Spellers Using Steady-State Visual Evoked Potentials Elicited by Peripheral Stimuli.

Brain-computer interface (BCI) spellers based on event related potentials (ERPs) are intrinsically synchronous systems. Therefore, selections are constantly made, even when users are not paying attention to the stimuli. This poses a major limitation in real-life applications, in which an asynchronous control is required. The aim of this study is to design, develop and test a novel method to discriminate whether the user is controlling the system (i.e., control state) or is engaged in other task (i.e., non-control state). To achieve such an asynchronous control, our method detects the steady-state visual evoked potentials (SSVEPs) elicited by peripheral stimuli of ERP-based spellers. A characterization experiment was conducted with 5 subjects to investigate general aspects of this phenomenon. Then, the proposed method was validated with 15 subjects in offline and online sessions. Results show that the proposed method provides a reliable asynchronous control, achieving an average accuracy of 95.5% for control state detection during the online sessions. Furthermore, our approach is independent of the ERP classification stage, and to the best of our knowledge, is the first procedure that does not need to extend the duration of the calibration sessions to acquire non-control observations.

Facilitating Calibration in High-Speed BCI Spellers via Leveraging Cross-Device Shared Latent Responses.

This paper proposes a novel device-to-device transfer-learning algorithm for reducing the calibration cost in a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) speller by leveraging electroencephalographic (EEG) data previously acquired by different EEG systems. The transferring is done by projecting the scalp-channel EEG signals onto a shared latent domain across devices. Three spatial filtering techniques, including channel averaging, canonical correlation analysis (CCA), and task-related component analysis (TRCA), were employed to extract the shared responses from different devices. The transferred data were integrated into a template-matching-based algorithm to detect SSVEPs. To evaluate its transferability, this paper conducted two sessions of simulated online BCI experiments with ten subjects using 40 visual stimuli modulated by joint frequency-phase coding method. In each session, two different EEG devices were used: first, the Quick-30 system (Cognionics, Inc.) with dry electrodes, and second, the ActiveTwo system (BioSemi, Inc.) with wet electrodes. The proposed method with CCA- and TRCA-based spatial filters achieved significantly higher classification accuracy compared with the calibration-free standard CCA-based method. This paper validated the feasibility and effectiveness of the proposed method in implementing calibration-free SSVEP-based BCIs. The proposed method has great potentials to enhance practicability and usability of real-world SSVEP-based BCI applications by leveraging user-specific data recorded in previous sessions even with different EEG systems and montages.

Histogram of Gradient Orientations of Signal Plots Applied to P300 Detection.

The analysis of Electroencephalographic (EEG) signals is of ulterior importance to aid in the diagnosis of mental disease and to increase our understanding of the brain. Traditionally, clinical EEG has been analyzed in terms of temporal waveforms, looking at rhythms in spontaneous activity, subjectively identifying troughs and peaks in Event-Related Potentials (ERP), or by studying graphoelements in pathological sleep stages. Additionally, the discipline of Brain Computer Interfaces (BCI) requires new methods to decode patterns from non-invasive EEG signals. This field is developing alternative communication pathways to transmit volitional information from the Central Nervous System. The technology could potentially enhance the quality of life of patients affected by neurodegenerative disorders and other mental illness. This work mimics what electroencephalographers have been doing clinically, visually inspecting, and categorizing phenomena within the EEG by the extraction of features from images of signal plots. These features are constructed based on the calculation of histograms of oriented gradients from pixels around the signal plot. It aims to provide a new objective framework to analyze, characterize and classify EEG signal waveforms. The feasibility of the method is outlined by detecting the P300, an ERP elicited by the oddball paradigm of rare events, and implementing an offline P300-based BCI Speller. The validity of the proposal is shown by offline processing a public dataset of Amyotrophic Lateral Sclerosis (ALS) patients and an own dataset of healthy subjects.

Development of a High-speed Mental Spelling System Combining Eye Tracking and SSVEP-based BCI with High Scalability.

Hybrid brain-computer interfaces (BCIs) have been proved to be more effective in mental control. In this study, a hybrid BCI speller system combining steady-state visual evoked potentials (SSVEPs) and eye tracking has been proposed. In this system, the eye tracker was used to detect eye gaze position for a 3×3 block selection, after that classification of the command was achieved through filter bank canonical correlation analysis (FBCCA) method. Results showed that the 40-classes hybrid speller system outperformed the SSVEP-only method, achieved a mean accuracy of 92.1% and a mean information transfer rate (ITR) of 180.8 bits/min during online experiments, and the scalability of the proposed system also has been tested with larger number of commands.

Comparison between a Passive and Active response task and their effect on the Amplitude and Latency of the P300 component for Visual Stimuli while using Low Fidelity Equipment.

In this paper, we investigate the effect, in terms of amplitude and latency, of the P300 component in a separate active and passive task response condition. This work is based on the P300 speller BCI (oddball) paradigm and the xDAWN algorithm, with five healthy subjects; while using a noninvasive Brain-Computer Interface (BCI) based on low fidelity electroencephalographic (EEG) equipment. Our results suggest that an active task yielded a larger P300 peak amplitude while there was no discriminable difference in the peak latency. The signal was also morphological consistent in both scenarios, even though they did not yield identical P300 components. This groundwork yields imperative data for future work where we plan to introduce several distractions, including communication with the user while performing the P300 speller paradigm.

Dynamic time window mechanism for time synchronous VEP-based BCIs-Performance evaluation with a dictionary-supported BCI speller employing SSVEP and c-VEP.

Brain-Computer Interfaces (BCIs) based on visual evoked potentials (VEPs) allow high communication speeds and accuracies. The fastest speeds can be achieved if targets are identified in a synchronous way (i.e., after a pre-set time period the system will produce a command output). The duration a target needs to be fixated on until the system classifies an output command affects the overall system performance. Hence, extracting a data window dedicated for the classification is of critical importance for VEP-based BCIs. Secondly, unintentional fixation on a target could easily lead to its selection. For the practical usability of BCI applications it is desirable to distinguish between intentional and unintentional fixations. This can be achieved by using threshold-based target identification methods. The study explores personalized dynamic classification time windows for threshold-based time synchronous VEP BCIs. The proposed techniques were tested employing the SSVEP and the c-VEP paradigm. Spelling performance was evaluated using an 8-target dictionary-supported BCI utilizing an n-gram word prediction model. The performance of twelve healthy participants was assessed with the information transfer rate (ITR) and accuracy. All participants completed sentence spelling tasks, reaching average accuracies of 94% and 96.3% for the c-VEP and the SSVEP paradigm, respectively. Average ITRs around 57 bpm were achieved for both paradigms.

Asynchronous non-invasive high-speed BCI speller with robust non-control state detection

Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Recent BCIs based on visual evoked potentials (VEPs) have shown to be suitable for high-speed communication. However, all recent high-speed BCIs are synchronous, which means that the system works with fixed time slots so that the user is not able to select a command at his own convenience, which poses a problem in real-world applications. In this paper, we present the first asynchronous high-speed BCI with robust distinction between intentional control (IC) and non-control (NC), with a nearly perfect NC state detection of only 0.075 erroneous classifications per minute. The resulting asynchronous speller achieved an average information transfer rate (ITR) of 122.7 bit/min using a 32 target matrix-keyboard. Since the method is based on random stimulation patterns it allows to use an arbitrary number of targets for any application purpose, which was shown by using an 55 target German QWERTZ-keyboard layout which allowed the participants to write an average of 16.1 (up to 30.7) correct case-sensitive letters per minute. As the presented system is the first asynchronous high-speed BCI speller with a robust non-control state detection, it is an important step for moving BCI applications out of the lab and into real-life.

On the Relative Contribution of Deep Convolutional Neural Networks for SSVEP-Based Bio-Signal Decoding in BCI Speller Applications.

Brain-computer interfaces (BCI) harnessing steady state visual evoked potentials (SSVEPs) manipulate the frequency and phase of visual stimuli to generate predictable oscillations in neural activity. For BCI spellers, oscillations are matched with alphanumeric characters allowing users to select target numbers and letters. Advances in BCI spellers can, in part, be accredited to subject-specific optimization, including; 1) custom electrode arrangements; 2) filter sub-band assessments; and 3) stimulus parameter tuning. Here, we apply deep convolutional neural networks (DCNNs) demonstrating cross-subject functionality for the classification of frequency and phase encoded SSVEP. Electroencephalogram (EEG) data are collected and classified using the same parameters across subjects. Subjects fixate forty randomly cued flickering characters ( 5 ×8 keyboard array) during concurrent wet-EEG acquisition. These data are provided by an open source SSVEP dataset. Our proposed DCNN, PodNet, achieves 86% and 77% offline accuracy of classification across-subjects for two data capture periods, respectively, 6-seconds (information transfer rate = 40 bpm) and 2-seconds (information transfer rate = 101 bpm). Subjects demonstrating sub-optimal (<70%) performance are classified to similar levels after a short subject-specific training period. PodNet outperforms filter-bank canonical correlation analysis for a low volume (3-channel) clinically feasible occipital electrode configuration. The networks defined in this study achieve functional performance for the largest number of SSVEP classes decoded via DCNN to date. Our results demonstrate PodNet achieves cross-subject, calibrationless classification and adaptability to sub-optimal subject data, and low-volume EEG electrode arrangements.

A Minimally Invasive Low-Power Platform for Real-Time Brain Computer Interaction Based on Canonical Correlation Analysis

A growing trend in human–computer interaction is to integrate computational capabilities into wearable devices, to enable sophisticated and natural interaction modalities. Acting directly by decoding neural activity is a very natural way of interaction and one of the fundamental paradigms of brain computer interfaces (BCIs) as well. In this paper, we present a wearable Internet of Things node designed for BCI spelling. The system is based on visual evoked potentials detection and runs the canonical correlation analysis on a low power microcontroller. Neural data is acquired by an array of electroencephalography active dry electrodes, suitable for a minimally intrusive interface. To evaluate our solution, we optimized the system on eight subjects and tested it on five different subjects for four and eight stimuli, reaching a peak transfer rate of 1.57 b/s, comparable with those achieved by state-of-the-art nonembedded systems. The power consumption of the device is less than 30 mW, resulting in 122 h of operation with a standard 1000-mAh battery.

A P300-Based Brain-Computer Interface for Improving Attention.

A Brain-computer Interface (BCI) can be used as a neurofeedback training tool to improve cognitive performance. BCIs aim to improve the effectiveness and efficiency of the conventional neurofeedback methods by focusing on the self-regulation of individualized neuromarkers rather than generic ones in a graphically appealing training environment. In this work, for the first time, we have modified a widely used P300-based speller BCI and used it as an engaging neurofeedack training game to enhance P300. According to the user's performance the game becomes more difficult in an adaptive manner, requiring the generation of a larger and stronger P300 (i.e., in terms of total energy) in response to target stimuli. Since the P300 is generated naturally without conscious effort in response to a target trial, unlike many rhythm-based neurofeedback tools, the ability to control the proposed P300-based neurofeedback training is obtained after a short calibration without undergoing tedious trial and error sessions. The performance of the proposed neurofeedback training was evaluated over a short time scale (approximately 30 min training) using 28 young adult participants who were randomly assigned to either the experimental group or the control group. In summary, our results show that the proposed P300-based BCI neurofeedback training yielded a significant enhancement in the ERP components of the target trials (i.e., 150–550 ms after the onset of stimuli which includes P300) as well as attenuation in the corresponding ERP components of the non-target trials. In addition, more centro-parietal alpha suppression was observed in the experimental group during the neurofeedback training as well as a post-training spatial attention task. Interestingly, a significant improvement in the response time of a spatial attention task performed immediately after the neurofeedback training was observed in the experimental group. This paper, as a proof-of-concept study, suggests that the proposed neurofeedback training tool is a promising tool for improving attention particularly for those who are at risk of attention deficiency.