High-speed Brain-computer Interface Research Articles

A high-speed hybrid brain-computer interface with more than 200 targets

Objective. Brain-computer interfaces (BCIs) have recently made significant strides in expanding their instruction set, which has attracted wide attention from researchers. The number of targets and commands is a key indicator of how well BCIs can decode the brain’s intentions. No studies have reported a BCI system with over 200 targets. Approach. This study developed the first high-speed BCI system with up to 216 targets that were encoded by a combination of electroencephalography features, including P300, motion visual evoked potential (mVEP), and steady-state visual evoked potential (SSVEP). Specifically, the hybrid BCI paradigm used the time-frequency division multiple access strategy to elaborately tag targets with P300 and mVEP of different time windows, along with SSVEP of different frequencies. The hybrid features were then decoded by task-discriminant component analysis and linear discriminant analysis. Ten subjects participated in the offline and online cued-guided spelling experiments. Other ten subjects took part in online free-spelling experiments. Main results. The offline results showed that the mVEP and P300 components were prominent in the central, parietal, and occipital regions, while the most distinct SSVEP feature was in the occipital region. The online cued-guided spelling and free-spelling results showed that the proposed BCI system achieved an average accuracy of 85.37% ± 7.49% and 86.00% ± 5.98% for the 216-target classification, resulting in an average information transfer rate (ITR) of 302.83 ± 39.20 bits min−1 and 204.47 ± 37.56 bits min−1, respectively. Notably, the peak ITR could reach up to 367.83 bits min−1. Significance. This study developed the first high-speed BCI system with more than 200 targets, which holds promise for extending BCI’s application scenarios.

Application of virtual simulation situational model in Russian spatial preposition teaching.

The purpose is to improve the teaching quality of Russian spatial prepositions in colleges. This work takes teaching Russian spatial prepositions as an example to study the key technologies in 3D Virtual Simulation (VS) teaching. 3D VS situational teaching is a high-end visual teaching technology. VS situation construction focuses on Human-Computer Interaction (HCI) to explore and present a realistic language teaching scene. Here, the Steady State Visual Evoked Potential (SSVEP) is used to control Brain-Computer Interface (BCI). An SSVEP-BCI system is constructed through the Hybrid Frequency-Phase Modulation (HFPM). The acquisition system can obtain the current SSVEP from the user's brain to know which module the user is watching to complete instructions encoded by the module. Experiments show that the recognition accuracy of the proposed SSVEP-BCI system based on HFPM increases with data length. When the data length is 0.6-s, the Information Transfer Rate (ITR) reaches the highest: 242.21 ± 46.88 bits/min. Therefore, a high-speed BCI character input system based on SSVEP is designed using HFPM. The main contribution of this work is to build a SSVEP-BCI system based on joint frequency phase modulation. It is better than the currently-known brain computer interface character input system, and is of great value to optimize the performance of the virtual simulation situation system for Russian spatial preposition teaching.

Improving the Classification of Real-World SSVEP Data in Brain-Computer Interface Speller Systems Using Deep Convolutional Neural Networks

Purpose: Brain-Computer Interface (BCI) Speller systems help people with mobility impairments improve their cognitive and physical abilities. Steady-State Visual Evoked Potential (SSVEP) signals have been used to build high-speed BCI speller systems. SSVEP signals are a subtype of Visual Evoked Potential (VEP), a form of co-frequency, and the harmonics response elicited by a specific frequency stimulus. Noise and artifacts are critical issues for target detection in SSVEP-based BCI systems.
 Materials and Methods: Thus, it is essential to provide target detection techniques that operate well in the presence of noises. One solution for overcoming the noise impact is to employ approaches that automatically extract the appropriate features for target detection from the training data. Deep Convolutional Neural Network (DCNN) was utilized in this study to automatically extract features from SSVEP data in noisy conditions. Moreover, the BETA database, which contains SSVEP data from 70 individuals collected outside of the electromagnetic shielding room, was used. In this regard, a suitable DCNN structure for target stimulus frequency identification was first designed. The network was pre-trained with part of the data from the BETA database. Finally, at the single-subject level, this pre-trained network was retrained and evaluated.
 Results: The results showed that after retraining, the accuracy and Information Transfer Rate (ITR) increased (p-value < 0.01) for all participants.
 Conclusion:.The enhancement in accuracy and ITR are 25.72% and 43.10 bpm, respectively.

A Zero-Padding Frequency Domain Convolutional Neural Network for SSVEP Classification

The brain-computer interface (BCI) of steady-state visual evoked potential (SSVEP) is one of the fundamental ways of human-computer communication. The main challenge is that there may be a nonlinear relationship between different SSVEP in other states. For improving the performance of SSVEP BCI, a novel CNN algorithm model is proposed in this study. Based on the discrete Fourier transform to calculate the signal's power spectral density (PSD), we perform zero-padding in the signal's time domain to improve its performance on the PSD and make it more refined. In this way, the frequency point interval in the PSD of the SSVEP is consistent with the minimum gap between the stimulation frequency. Combining the nonlinear transformation capabilities of CNN in deep learning, a zero-padding frequency domain convolutional neural network (ZPFDCNN) model is proposed. Extensive experiments based on the SSVEP dataset validate the effectiveness of our method. The study verifies that the proposed ZPFDCNN method can improve the effectiveness of the SSVEP-based high-speed BCI ITR. It has massive potential in the application of BCI.

Effects of inter-stimulus intervals on concurrent P300 and SSVEP features for hybrid brain-computer interfaces

BackgroundRecently, we have implemented a high-speed brain-computer interface (BCI) system with a large instruction set using the concurrent P300 and steady-state visual evoked potential (SSVEP) features (also known as hybrid features). However, it remains unclear how to select inter-stimulus interval (ISI) for the proposed BCI system to balance the encoding efficiency and decoding performance. New methodThis study developed a 6 * 9 hybrid P300-SSVEP BCI system and investigated a series of ISIs ranged from −175–0 ms with a step of 25 ms. The influence of ISI on the hybrid features was analyzed from several aspects, including the amplitude of the induced features, classification accuracy, information transfer rate (ITR). Twelve naive subjects were recruited for the experiment. ResultsThe results showed the ISI factor had a significant impact on the hybrid features. Specifically, as the values of ISI decreased, the amplitudes of the induced features and accuracies decreased gradually, while the ITRs increased rapidly. It’s achieved the highest ITR of 158.50 bits/min when ISI equal to − 175 ms. Comparison with Existing MethodThe optimal ISI in this study achieved superior performance in comparison with the one we used in the previous study. ConclusionsThe ISI can exert an important influence on the P300-SSVEP BCI system and its optimal value is − 175 ms in this study, which is significant for developing the high-speed BCI system with larger instruction sets in the future.

Inter- and Intra-subject Template-Based Multivariate Synchronization Index Using an Adaptive Threshold for SSVEP-Based BCIs

The steady-state visually evoked potential (SSVEP) has been widely used in brain-computer interfaces (BCIs). Many studies have proved that the Multivariate synchronization index (MSI) is an efficient method for recognizing the frequency components in SSVEP-based BCIs. Despite its success, the recognition accuracy has not been satisfactory because the simplified pre-constructed sine-cosine waves lack abundant features from the real electroencephalogram (EEG) data. Recent advances in addressing this issue have achieved a significant improvement in recognition accuracy by using individual calibration data. In this study, a new extension based on inter- and intra-subject template signals is introduced to improve the performance of the standard MSI method. Through template transfer, inter-subject similarity and variability are employed to enhance the robustness of SSVEP recognition. Additionally, most existed methods for SSVEP recognition utilize a fixed time window (TW) to perform frequency domain analysis, which limits the information transfer rate (ITR) of BCIs. For addressing this problem, a novel adaptive threshold strategy is integrated into the extension of MSI, which uses a dynamic window to extract the temporal features of SSVEPs and recognizes the stimulus frequency based on a pre-set threshold. The pre-set threshold contributes to obtaining an appropriate and shorter signal length for frequency recognition and filtering ignored-invalid trials. The proposed method is evaluated on a 12-class SSVEP dataset recorded from 10 subjects, and the result shows that this achieves higher recognition accuracy and information transfer rate when compared with the CCA, MSI, Multi-set CCA, and Individual Template-based CCA. This paper demonstrates that the proposed method is a promising approach for developing high-speed BCIs.

Effects of stimulation frequency and stimulation waveform on steady-state visual evoked potentials using a computer monitor

Objective. A visual stimulator plays a vital part in brain–computer interfaces (BCIs) based on steady-state visual evoked potential (SSVEP). The properties of visual stimulation, such as frequency, color, and waveform, will influence SSVEP-based BCI performance to some extent. Recently, the computer monitor serves as a visual stimulator that is widespread in SSVEP-based BCIs because of its great flexibility in generating visual stimuli. However, stimulation properties based on a computer monitor have received very little attention. For a better comprehension of SSVEPs, this study explored the stimulation effects of waveforms and frequencies, when evoking SSVEPs through a computer monitor. Approach. This study utilized the approximation methods to realize sine- and square-wave temporal modulations at 18 stimulation frequencies ranging from 6 to 40 Hz on a conventional 120 Hz LCD screen. We collected electroencephalogram (EEG) datasets from 12 healthy subjects and compared the signal-to-noise ratios (SNRs), amplitudes, and topographic mapping of SSVEPs evoked by these two temporal modulation flickers (sine- and square-wave). In addition, a BCI experiment with two nine-target BCIs (i.e. low-frequency BCI and high-frequency BCI) was implemented to compare the two stimulation waveforms in terms of BCI performance. Main results. For both sine- and square-wave stimulation conditions, strong SSVEPs over the occipital area were observed for each stimulation frequency. SSVEP amplitudes at the stimulation frequency exhibited a global peak in the low-frequency band. The second harmonic SSVEP frequency-response functions showed the largest amplitude at 6 Hz and fell sharply for higher frequencies. In the BCI experiment, the classification performance of the square-wave stimuli was notably higher than that of the sine-wave stimuli when using shorter data lengths. Significance. These results suggested that the square-wave flicker was more efficient at implementing high-speed BCIs based on SSVEP when using a computer monitor as a visual stimulator.

Asynchronous non-invasive high-speed BCI speller with robust non-control state detection

Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Recent BCIs based on visual evoked potentials (VEPs) have shown to be suitable for high-speed communication. However, all recent high-speed BCIs are synchronous, which means that the system works with fixed time slots so that the user is not able to select a command at his own convenience, which poses a problem in real-world applications. In this paper, we present the first asynchronous high-speed BCI with robust distinction between intentional control (IC) and non-control (NC), with a nearly perfect NC state detection of only 0.075 erroneous classifications per minute. The resulting asynchronous speller achieved an average information transfer rate (ITR) of 122.7 bit/min using a 32 target matrix-keyboard. Since the method is based on random stimulation patterns it allows to use an arbitrary number of targets for any application purpose, which was shown by using an 55 target German QWERTZ-keyboard layout which allowed the participants to write an average of 16.1 (up to 30.7) correct case-sensitive letters per minute. As the presented system is the first asynchronous high-speed BCI speller with a robust non-control state detection, it is an important step for moving BCI applications out of the lab and into real-life.

Independent component analysis-based spatial filtering improves template-based SSVEP detection.

This study proposes a new algorithm to detect steady-state visual evoked potentials (SSVEPs) based on a template-matching approach combined with independent component analysis (ICA)-based spatial filtering. In recent studies, the effectiveness of the template-based SSVEP detection has been demonstrated in a high-speed brain-computer interface (BCI). Since SSVEPs can be considered as electroencephalogram (EEG) signals generated from underlying brain sources independent from other activities and artifacts, ICA has great potential to enhance the signal-to-noise ratio (SNR) of SSVEPs by separating them from artifacts. This study proposes to apply the ICA-based spatial filters to test data and individual templates obtained by averaging training trials, and then to use the correlation coefficients between the filtered data and templates as features for SSVEP classification. This study applied the proposed method to a 40-class SSVEP dataset to evaluate its classification accuracy against those obtained by conventional canonical correlation analysis (CCA)- and extended CCA-based methods. The study results showed that the ICA-based method outperformed the other methods in terms of the classification accuracy. Furthermore, its computational time was comparable to the CCA-based method, and was much shorter than that of the extended CCA-based method.

Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis.

This study proposes and evaluates a novel data-driven spatial filtering approach for enhancing steady-state visual evoked potentials (SSVEPs) detection toward a high-speed brain-computer interface (BCI) speller. Task-related component analysis (TRCA), which can enhance reproducibility of SSVEPs across multiple trials, was employed to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. An ensemble method was further developed to integrate TRCA filters corresponding to multiple stimulation frequencies. This study conducted a comparison of BCI performance between the proposed TRCA-based method and an extended canonical correlation analysis (CCA)-based method using a 40-class SSVEP dataset recorded from 12 subjects. An online BCI speller was further implemented using a cue-guided target selection task with 20 subjects and a free-spelling task with 10 of the subjects. The offline comparison results indicate that the proposed TRCA-based approach can significantly improve the classification accuracy compared with the extended CCA-based method. Furthermore, the online BCI speller achieved averaged information transfer rates (ITRs) of 325.33 ± 38.17 bits/min with the cue-guided task and 198.67 ± 50.48 bits/min with the free-spelling task. This study validated the efficiency of the proposed TRCA-based method in implementing a high-speed SSVEP-based BCI. The high-speed SSVEP-based BCIs using the TRCA method have great potential for various applications in communication and control.

A high-speed brain-computer interface (BCI) using dry EEG electrodes.

Recently, brain-computer interfaces (BCIs) based on visual evoked potentials (VEPs) have been shown to achieve remarkable communication speeds. As they use electroencephalography (EEG) as non-invasive method for recording neural signals, the application of gel-based EEG is time-consuming and cumbersome. In order to achieve a more user-friendly system, this work explores the usability of dry EEG electrodes with a VEP-based BCI. While the results show a high variability between subjects, they also show that communication speeds of more than 100 bit/min are possible using dry EEG electrodes. To reduce performance variability and deal with the lower signal-to-noise ratio of the dry EEG electrodes, an averaging method and a dynamic stopping method were introduced to the BCI system. Those changes were shown to improve performance significantly, leading to an average classification accuracy of 76% with an average communication speed of 46 bit/min, which is equivalent to a writing speed of 8.8 error-free letters per minute. Although the BCI system works substantially better with gel-based EEG, dry EEG electrodes are more user-friendly and still allow high-speed BCI communication.

A high-speed BCI based on code modulation VEP

Recently, electroencephalogram-based brain–computer interfaces (BCIs) have attracted much attention in the fields of neural engineering and rehabilitation due to their noninvasiveness. However, the low communication speed of current BCI systems greatly limits their practical application. In this paper, we present a high-speed BCI based on code modulation of visual evoked potentials (c-VEP). Thirty-two target stimuli were modulated by a time-shifted binary pseudorandom sequence. A multichannel identification method based on canonical correlation analysis (CCA) was used for target identification. The online system achieved an average information transfer rate (ITR) of 108 ± 12 bits min−1 on five subjects with a maximum ITR of 123 bits min−1 for a single subject.