c-VEP Brain-computer Interface Research Articles

Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs

One of the main problems that a brain-computer interface (BCI) face is that a training stage is required for acquiring training data to calibrate its classification model just before every use. Transfer learning is a promising method for addressing the problem. In this paper, we propose a Riemannian geometry-based transfer learning algorithm for code modulated visual evoked potential (c-VEP)-based BCIs, which can effectively reduce the calibration time without sacrificing the classification accuracy. The algorithm includes the main procedures of log-Euclidean data alignment (LEDA), super-trial construction, covariance matrix estimation, training accuracy-based subject selection (TSS) and minimum distance to mean classification. Among them, the LEDA reduces the difference in data distribution between subjects, whereas the TSS promotes the similarity between a target subject and the source subjects. The resulting performance of transfer learning is improved significantly. Sixteen subjects participated in a c-VEP BCI experiment and the recorded data were used in offline analysis. Leave-one subject-out (LOSO) cross-validation was used to evaluate the proposed algorithm on the data set. The results showed that the algorithm achieved much higher classification accuracy than the subject-specific (baseline) algorithm with the same number of training trials. Equivalently, the algorithm reduces the training time of the BCI at the same performance level and thus facilitates its application in real world.

From full calibration to zero training for a code-modulated visual evoked potentials for brain–computer interface

Objective. Typically, a brain–computer interface (BCI) is calibrated using user- and session-specific data because of the individual idiosyncrasies and the non-stationary signal properties of the electroencephalogram (EEG). Therefore, it is normal for BCIs to undergo a time-consuming passive training stage that prevents users from directly operating them. In this study, we systematically reduce the training data set in a stepwise fashion, to ultimately arrive at a calibration-free method for a code-modulated visually evoked potential (cVEP)-based BCI to fully eliminate the tedious training stage. Approach. In an extensive offline analysis, we compare our sophisticated encoding model with a traditional event-related potential (ERP) technique. We calibrate the encoding model in a standard way, with data limited to a single class while generalizing to all others and without any data. In addition, we investigate the feasibility of the zero-training cVEP BCI in an online setting. Main results. By adopting the encoding model, the training data can be reduced substantially, while maintaining both the classification performance as well as the explained variance of the ERP method. Moreover, with data from only one class or even no data at all, it still shows excellent performance. In addition, the zero-training cVEP BCI achieved high communication rates in an online spelling task, proving its feasibility for practical use. Significance. To date, this is the fastest zero-training cVEP BCI in the field, allowing high communication speeds without calibration while using only a few non-invasive water-based EEG electrodes. This allows us to skip the training stage altogether and spend all the valuable time on direct operation. This minimizes the session time and opens up new exciting directions for practical plug-and-play BCI. Fundamentally, these results validate that the adopted neural encoding model compresses data into event responses without the loss of explanatory power compared to using full ERPs as a template.

Ensemble or pool: A comprehensive study on transfer learning for c-VEP BCI during interpersonal interaction

BackgroundTo reduce calibration time of brain–computer interface (BCI) or even implement zero-training BCI, researchers have been studying how to effectively apply transfer learning in the field. In order to thoroughly investigate the performance of transfer learning in BCI and the key factors affecting transfer performance in the field, we carried out a comprehensive study. New methodIn general, transferring knowledge in BCI is implemented in two ways: ensemble or pool. In this work, we propose two different transfer approaches. One is to transfer the information of all channels as a whole from the source subjects to a target subject. The second approach is to transfer the information of corresponding channels between the subjects. A subject transfer framework is built by combining the two approaches with ensemble or pool. ResultsWe investigated the performances of eight implementations of this framework on a data set acquired by an interpersonal interaction (Chicken Game) experiment based on code-modulated visual evoked potential (c-VEP) BCI. The results show that transfer learning generally provides acceptable classification performance. Additionally, an in-depth analysis reveals that a target subject usually shares different brain signal distribution with different source subjects. In fact, this is a hypothesis usually implied by this kind of research. ConclusionsTransfer learning for c-VEP BCI can be qualified for reducing calibration time or starting the recognition of BCI without sufficient subjects’ own data. In addition, our finding suggests a solid validity of the hypothesis underlying transferring knowledge in BCI.

A Novel Dictionary-Driven Mental Spelling Application Based on Code-Modulated Visual Evoked Potentials

Brain–computer interfaces (BCIs) based on code-modulated visual evoked potentials (c-VEPs) typically utilize a synchronous approach to identify targets (i.e., after preset time periods the system produces command outputs). Hence, users have only a limited amount of time to fixate a desired target. This hinders the usage of more complex interfaces, as these require the BCI to distinguish between intentional and unintentional fixations. In this article, we investigate a dynamic sliding window mechanism as well as the implementation of software-based stimulus synchronization to enable the threshold-based target identification for the c-VEP paradigm. To further improve the usability of the system, an ensemble-based classification strategy was investigated. In addition, a software-based approach for stimulus on-set determination is proposed, which allows for an easier setup of the system, as it reduces additional hardware dependencies. The methods were tested with an eight-target spelling application utilizing an n-gram word prediction model. The performance of eighteen participants without disabilities was tested; all participants completed word- and sentence spelling tasks using the c-VEP BCI with a mean information transfer rate (ITR) of 75.7 and 57.8 bpm, respectively.

A multi-target brain-computer interface based on code modulated visual evoked potentials.

The number of selectable targets is one of the main factors that affect the performance of a brain-computer interface (BCI). Most existing code modulated visual evoked potential (c-VEP) based BCIs use a single pseudorandom binary sequence and its circularly shifting sequences to modulate different stimulus targets, making the number of selectable targets limited by the length of modulation codes. This paper proposes a novel paradigm for c-VEP BCIs, which divides the stimulus targets into four target groups and each group of targets are modulated by a unique pseudorandom binary code and its circularly shifting codes. Based on the paradigm, a four-group c-VEP BCI with a total of 64 stimulus targets was developed and eight subjects were recruited to participate in the BCI experiment. Based on the experimental data, the characteristics of the c-VEP BCI were explored by the analyses of auto- and cross-correlation, frequency spectrum, signal to noise ratio and correlation coefficient. On the basis, single-trial data with the length of one stimulus cycle were classified and the attended target was recognized. The averaged classification accuracy across subjects was 88.36% and the corresponding information transfer rate was as high as 184.6 bit/min. These results suggested that the c-VEP BCI paradigm is both feasible and effective, and provides a new solution for BCI study to substantially increase the number of available targets.

A Novel c-VEP BCI Paradigm for Increasing the Number of Stimulus Targets Based on Grouping Modulation With Different Codes.

In an existing brain-computer interface (BCI) based on code modulated visual evoked potentials (c-VEP), a method with which to increase the number of targets without increasing code length has not yet been established. In this paper, a novel c-VEP BCI paradigm, namely, grouping modulation with different codes that have good autocorrelation and crosscorrelation properties, is presented to increase the number of targets and information transfer rate (ITR). All stimulus targets are divided into several groups and each group of targets are modulated by a distinct pseudorandom binary code and its circularly shifting codes. Canonical correlation analysis is applied to each group for yielding a spatial filter and templates for all targets in a group are constructed based on spatially filtered signals. Template matching is applied to each group and the attended target is recognized by finding the maximal correlation coefficients of all groups. Based on the paradigm, a BCI with a total of 48 targets divided into three groups was implemented; 12 and 10 subjects participated in an off-line and a simulated online experiments, respectively. Data analysis of the offline experiment showed that the paradigm can massively increase the number of targets from 16 to 48 at the cost of slight compromise in accuracy (95.49% vs. 92.85%). Results of the simulated online experiment suggested that although the averaged accuracy across subjects of all three groups of targets was lower than that of a single group of targets (91.67% vs. 94.9%), the average ITR of the former was substantially higher than that of the later (181 bits/min vs. 135.6 bit/min) due to the large increase of the number of targets. The proposed paradigm significantly improves the performance of the c-VEP BCI, and thereby facilitates its practical applications such as high-speed spelling.

Grouping modulation with different codes for improving performance in cVEP‐based brain–computer interfaces

A novel paradigm for building code modulated visual evoked potential (cVEP) brain–computer interfaces (BCIs) is proposed. The paradigm is based on grouping modulation of stimuli with different codes in order to increase the number of stimuli and resulting information transfer rate (ITR). In a single group, an original code and its circularly shifting codes are used for modulating 16 stimuli. To achieve good separability of stimuli, the original codes for different groups should have good autocorrelation and cross-correlation properties. A pair of Golay complementary series and an almost perfect autocorrelation sequence are used as the original codes to implement a three-group cVEP BCI with 48 targets. Four subjects participated in a simulated online experiment and a high ITR of 129.58 bits/min was achieved due to the sharp increase of the number of targets.

Neural decoding of code modulated visual evoked potentials by spatio-temporal inverse filtering for brain computer interfaces.

This study addresses neural decoding of a code modulated visual evoked potentials (c-VEPs). c-VEP was recently developed, and applied to brain computer interfaces (BCIs). c-VEP BCI exhibits faster communication speed than existing VEP-based BCIs. In c-VEP BCI, the canonical correlation analysis (CCA) that maximizes the correlation between an averaged signal and single trial signals is often used for the spatial filter. However, CCA does not utilize information of given PN sequence, and hence, the filtered signal may not have properties of PN sequence. In this paper, we propose a decoding method to restore the given PN sequence from the observed VEP. We compare linear and nonlinear spatio-temporal inverse filtering methods. For the linear method, the least mean square error and lasso are used to obtain the filter coefficients. For the non-linear method, the artificial neural network is used. The proposed methods exhibited better decoding performance, and higher classification accuracies than conventional CCA spatial filtered c-VEP BCI.

VEP-based brain-computer interfaces modulated by Golay complementary series for improving performance.

The goal of a brain-computer interface (BCI) is to enable communication by pure brain activity without neural and muscle control. However, the practical use of BCIs is limited by low information transfer rate. Recently, code modulation visual evoked potential (c-VEP) based BCIs have exhibited great potential in establishing high-rate communication between the brain and the external world. This study aims at exploring a more effective modulation code than the commonly used pseudorandom M sequence for c-VEP based BCIs (c-VEP BCIs) in order to increase the detection accuracy of stimulus targets and the resulting information transfer rate. Golay complementary sequence pair is used for constructing the modulation code of c-VEP BCIs due to their superior autocorrelation property. The modulation code is created by concatenating a pair of Golay complementary sequences. Sixteen target stimuli are modulated by the Golay code and its time shift versions. Through offline analysis on data recorded from seven subjects and online test on five subjects, the Golay code modulated BCI yielded higher detection accuracy and information transfer rate than those achieved by M sequence. The Golay code modulated BCI demonstrates a high performance compared with the M sequence modulated systems, and it is applicable to persons with motor disabilities.

Stimulus Specificity of Brain-Computer Interfaces Based on Code Modulation Visual Evoked Potentials.

A brain-computer interface (BCI) based on code modulated visual evoked potentials (c-VEP) is among the fastest BCIs that have ever been reported, but it has not yet been given a thorough study. In this study, a pseudorandom binary M sequence and its time lag sequences are utilized for modulation of different stimuli and template matching is adopted as the method for target recognition. Five experiments were devised to investigate the effect of stimulus specificity on target recognition and we made an effort to find the optimal stimulus parameters for size, color and proximity of the stimuli, length of modulation sequence and its lag between two adjacent stimuli. By changing the values of these parameters and measuring classification accuracy of the c-VEP BCI, an optimal value of each parameter can be attained. Experimental results of ten subjects showed that stimulus size of visual angle 3.8°, white, spatial proximity of visual angle 4.8° center to center apart, modulation sequence of length 63 bits and the lag of 4 bits between adjacent stimuli yield individually superior performance. These findings provide a basis for determining stimulus presentation of a high-performance c-VEP based BCI system.

Spatial decoupling of targets and flashing stimuli for visual brain–computer interfaces

Objective. Recently, paradigms using code-modulated visual evoked potentials (c-VEPs) have proven to achieve among the highest information transfer rates for noninvasive brain–computer interfaces (BCIs). One issue with current c-VEP paradigms, and visual-evoked paradigms in general, is that they require direct foveal fixation of the flashing stimuli. These interfaces are often visually unpleasant and can be irritating and fatiguing to the user, thus adversely impacting practical performance. In this study, a novel c-VEP BCI paradigm is presented that attempts to perform spatial decoupling of the targets and flashing stimuli using two distinct concepts: spatial separation and boundary positioning. Approach. For the paradigm, the flashing stimuli form a ring that encompasses the intended non-flashing targets, which are spatially separated from the stimuli. The user fixates on the desired target, which is classified using the changes to the EEG induced by the flashing stimuli located in the non-foveal visual field. Additionally, a subset of targets is also positioned at or near the stimulus boundaries, which decouples targets from direct association with a single stimulus. This allows a greater number of target locations for a fixed number of flashing stimuli. Main results. Results from 11 subjects showed practical classification accuracies for the non-foveal condition, with comparable performance to the direct-foveal condition for longer observation lengths. Online results from 5 subjects confirmed the offline results with an average accuracy across subjects of 95.6% for a 4-target condition. The offline analysis also indicated that targets positioned at or near the boundaries of two stimuli could be classified with the same accuracy as traditional superimposed (non-boundary) targets. Significance. The implications of this research are that c-VEPs can be detected and accurately classified to achieve comparable BCI performance without requiring potentially irritating direct foveation of flashing stimuli. Furthermore, this study shows that it is possible to increase the number of targets beyond the number of stimuli without degrading performance. Given the superior information transfer rate of c-VEP paradigms, these results can lead to the development of more practical and ergonomic BCIs.

Online Adaptation of a c-VEP Brain-Computer Interface(BCI) Based on Error-Related Potentials and Unsupervised Learning

The goal of a Brain-Computer Interface (BCI) is to control a computer by pure brain activity. Recently, BCIs based on code-modulated visual evoked potentials (c-VEPs) have shown great potential to establish high-performance communication. In this paper we present a c-VEP BCI that uses online adaptation of the classifier to reduce calibration time and increase performance. We compare two different approaches for online adaptation of the system: an unsupervised method and a method that uses the detection of error-related potentials. Both approaches were tested in an online study, in which an average accuracy of 96% was achieved with adaptation based on error-related potentials. This accuracy corresponds to an average information transfer rate of 144 bit/min, which is the highest bitrate reported so far for a non-invasive BCI. In a free-spelling mode, the subjects were able to write with an average of 21.3 error-free letters per minute, which shows the feasibility of the BCI system in a normal-use scenario. In addition we show that a calibration of the BCI system solely based on the detection of error-related potentials is possible, without knowing the true class labels.

VEP-based brain-computer interfaces: time, frequency, and code modulations [Research Frontier

A brain computer interface (BCI) translates human intentions into control signals to establish a direct communication channel between the human brain and external devices. Because a BCI does not depend on the brain's normal output pathways of peripheral nerves and muscles, it can provide a new communication channel to people with severe motor disabilities. Electroencephalograms (EEGs) recorded from the surface of the scalp are widely used in current BCIs for their non-invasive nature and easy applications. Among EEG based BCIs, systems based on visual evoked potentials (VEPs) have received widespread attention in recent decades. We described the three stimulus modulation approaches used in current VEP based BCIs: time modulation (t-VEP), frequency modulation (f-VEP), and pseudorandom code modulation (c-VEP). We then carried out a detailed comparison of system performance between an f-VEP BCI and a c-VEP BCI. The results show that an f-VEP BCI has the advantage of little or no training and simple system configuration, while the c-VEP based BCI has a higher communication speed. The stimulus modulation design is the crux of VEP based BCI systems.