SSVEP-based Brain-computer Interface Research Articles

SSVEP-Based Brain-Computer Interface for Part-Picking Robotic Co-Worker

AbstractOne of the expectations for the next generation of industrial robots is to work collaboratively with humans as robotic co-workers. Robotic co-workers must be able to communicate with human collaborators intelligently and seamlessly. However, industrial robots in prevalence are not good at understanding human intentions and decisions. We demonstrate a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) which can directly deliver human cognition to robots through a headset. The BCI is applied to a part-picking robot. The BCI sends decisions to the robot while operators visually inspecting the quality of parts. The BCI is verified through a human subject study. In the study, a camera by the side of the conveyor takes photos of each industrial part and presents it to the operator automatically. When the operator looks at the photo, the electroencephalography (EEG) is collected through the BCI. The inspection decision is extracted through SSVEPs in EEG. When a defective part is identified by the operator, the signal is communicated to the robot, which locates the defective part by a second camera and removes it from the conveyor. The robot can grasp various part with our random grasp planning algorithm (2FRG). We have developed a CNN-CCA model for SSVEP extraction. The model is trained on a dataset collected in our offline experiment. Our approach outperforms the existing CCA, CCA-SVM, and PSD-SVM models. The CNN-CCA model is further validated in an online experiment and achieved 93% accuracy in identifying and removing defective parts.

Classification of SSVEP-based BCIs using Genetic Algorithm

Brain computer interface (BCI) systems have been regarded as a new way of communication for humans. In this research, common methods such as wavelet transform are applied in order to extract features. However, genetic algorithm (GA), as an evolutionary method, is used to select features. Finally, classification was done using the two approaches support vector machine (SVM) and Bayesian method. Five features were selected and the accuracy of Bayesian classification was measured to be 80% with dimension reduction. Ultimately, the classification accuracy reached 90.4% using SVM classifier. The results of the study indicate a better feature selection and the effective dimension reduction of these features, as well as a higher percentage of classification accuracy in comparison with other studies.

Group ensemble learning enhances the accuracy and convenience of SSVEP-based BCIs via exploiting inter-subject information

Over the past years, Steady-state visual evoked potential (SSVEP) is widely used in brain–computer interfaces (BCIs) due to its high efficiency, robustness and convenience. Although the ensemble task-related component analysis (eTRCA) is a state-of-the-art signal processing algorithm for SSVEP signals, the tedious training process restricts its extensive application. In this study, a novel hybrid framework is proposed to conquer this problem by reusing the validation data recorded previously. Additionally, a new group ensemble learning method is also proposed to combine a filter bank canonical correlation analysis (FBCCA) detector and several eTRCA detectors to achieve a high-performance SSVEP detector. The offline analysis, which used a four-class SSVEP dataset recorded from 9 subjects in four days, demonstrates that the proposed method outperforms the FBCCA method and almost achieves the performance of the eTRCA method. The proposed method provides a good trade-off between performance and training cost, which is more suitable for application in real life.

Durağan-Durum Görsel Uyarılmış Potansiyellere Dayalı Beyin-Bilgisayar Arayüzünün Zamansal ve Frekanssal Derinliğinden Frekans Tanıma

Brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) have been acceleratingly used in different application areas from entertainment to rehabilitation, like clinical neuroscience, cognitive, and use of engineering researches. Of various electroencephalography paradigms, SSVEP-based BCI systems enable apoplectic people to communicate with outside world easily, due to their simple system structure, short or no training time, high temporal resolution, high information transfer rate, and affordable by comparing to other methods. SSVEP-based BCIs use multiple visual stimuli flickering at different frequencies to generate distinct commands. In this paper, we compared the classifier performances of combinations of binary commands flickering at seven different frequencies to determine which frequency pair gives the highest performance using temporal and spectral methods. For SSVEP frequency recognition, in total 25 temporal change characteristics of the signals and 15 frequency-based feature vectors extracted from the SSVEP signal. These feature vectors were applied to the input of seven well-known machine learning algorithms (Decision Tree, Discriminant Analysis, Logistic Regression, Naive Bayes, Support Vector Machines, Nearest Neighbour, and Ensemble Learning). In conclusion, we achieved 100% accuracy in 7.5 - 10 frequency pairs among these 2,520 distinct runs and we found that the most successful classifier is the Ensemble Learning classifier. The combination of these methods leads to an appropriate detailed and comparative analysis that represents the robustness and effectiveness of classical approaches.

Brain-Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance.

The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain–computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.

Enhancing detection of steady-state visual evoked potentials using channel ensemble method

Objective. This study proposed and evaluated a channel ensemble approach to enhance detection of steady-state visual evoked potentials (SSVEPs). Approach. Collected multi-channel electroencephalogram signals were classified into multiple groups of new analysis signals based on correlation analysis, and each group of analysis signals contained signals from a different number of electrode channels. These groups of analysis signals were used as the input of a training-free feature extraction model, and the obtained feature coefficients were converted into feature probability values using the softmax function. The ensemble value of multiple sets of feature probability values was determined and used as the final discrimination coefficient. Main results. Compared with canonical correlation analysis, likelihood ratio test, and multivariate synchronization index analysis methods using a standard approach, the recognition accuracies of the methods using a channel ensemble approach were improved by 5.05%, 3.87%, and 3.42%, and the information transfer rates (ITRs) were improved by 6.00%, 4.61%, and 3.71%, respectively. The channel ensemble method also obtained better recognition results than the standard algorithm on the public dataset. This study validated the efficiency of the proposed method to enhance the detection of SSVEPs, demonstrating its potential use in practical brain–computer interface (BCI) systems. Significance. A SSVEP-based BCI system using a channel ensemble method could achieve high ITR, indicating great potential of this design for various applications with improved control and interaction.

An Open Dataset for Wearable SSVEP-Based Brain-Computer Interfaces.

Brain-computer interfaces (BCIs) provide humans a new communication channel by encoding and decoding brain activities. Steady-state visual evoked potential (SSVEP)-based BCI stands out among many BCI paradigms because of its non-invasiveness, little user training, and high information transfer rate (ITR). However, the use of conductive gel and bulky hardware in the traditional Electroencephalogram (EEG) method hinder the application of SSVEP-based BCIs. Besides, continuous visual stimulation in long time use will lead to visual fatigue and pose a new challenge to the practical application. This study provides an open dataset, which is collected based on a wearable SSVEP-based BCI system, and comprehensively compares the SSVEP data obtained by wet and dry electrodes. The dataset consists of 8-channel EEG data from 102 healthy subjects performing a 12-target SSVEP-based BCI task. For each subject, 10 consecutive blocks were recorded using wet and dry electrodes, respectively. The dataset can be used to investigate the performance of wet and dry electrodes in SSVEP-based BCIs. Besides, the dataset provides sufficient data for developing new target identification algorithms to improve the performance of wearable SSVEP-based BCIs.

Muscle Artifact Removal Toward Mobile SSVEP-Based BCI: A Comparative Study

Steady-state visual evoked potential (SSVEP) serves as one of the extensively utilized paradigms for brain–computer interface (BCI). SSVEP-based BCI has advantages of good classification accuracy, high information transfer rate (ITR), and little user training. However, there exist few studies investigating SSVEP-based BCI in the mobile situation, where electroencephalography (EEG) data are inevitably contaminated by muscle activities, and meanwhile, a limited number of channels are preferred. In this study, two major muscle artifact removal schemes are investigated in the dynamic scenario. One scheme adopts state-of-the-art blind source separation (BSS) methods without electromyogram (EMG) reference, while the other utilizes auxiliary EMG channels as a reference in the denoising process. To evaluate the performance of two schemes, we have tested them on real EEG data collected from ten subjects. The results demonstrate that, in the few-channel condition of SSVEP data, there are classification accuracy improvements of 9.93% for the canonical correlation analysis (CCA) indicator and 15.72% for the task-related component analysis (TRCA) indicator after processed by recursive least-squares (RLS) filtering with auxiliary EMG, and in the single-channel condition, there are classification accuracy improvements of 22.00% for the CCA indicator and 24.42% for the TRCA indicator after RLS filtering, averaged across all the subjects and data lengths. However, the BSS-based methods fail to improve the classification accuracies of SSVEP. Thus, it is of necessity to mount additional EMG channels for effective muscle activity suppression. This study provides an insightful suggestion for future real-life BCI applications.

Phase-Approaching Stimulation Sequence for SSVEP-Based BCI: A Practical Use in VR/AR HMD

Steady-state visual evoked potential (SSVEP) has been used to implement brain-computer interface (BCI) due to its advantages of high information transfer rate (ITR) and high accuracy. In recent years, owing to the developments of head-mounted device (HMD), the HMD has become a popular device to implement SSVEP-based BCI. However, an HMD with fixed frame rate only can flash at its subharmonic frequencies which limits the available number of stimulation frequencies for SSVEP-based BCI. In order to increase the number of available commands for SSVEP-based BCI, we proposed a phase-approaching (PA) method to generate visual stimulation sequences at user-specified frequency on an HMD. The flickering sequence generated by our PA method (PAS sequence) tries to approximate user-specified stimulation frequency by means of minimizing the difference of accumulated phases between our PAS sequence and the ideal wave of user-specified frequency. The generated sequence of PA method determines the brightness state for each frame to approach the accumulated phase of the ideal wave. The SSVEPs evoked from stimulators, driven by PAS sequences, were analyzed using canonical correlation analysis (CCA) to identify user's gazed target. In this study, a six-command SSVEP-based BCI was designed to operate a flying drone. The ITR and detection accuracy are 36.84 bits/min and 93.30%, respectively.

SSVEP-Based Brain-Computer Interface With a Limited Number of Frequencies Based on Dual-Frequency Biased Coding.

How to encode as many targets as possible with a limited-frequency resource is a difficult problem in the practical use of a steady-state visual evoked potential (SSVEP) based brain-computer interface (BCI) speller. To solve this problem, this study developed a novel method called dual-frequency biased coding (DFBC) to tag targets in a SSVEP-based 48-character virtual speller, in which each target is encoded with a permutation sequence consisting of two permuted flickering periods that flash at different frequencies. The proposed paradigm was validated by 11 participants in an offline experiment and 7 participants in an online experiment. Three occipital channels (O1, Oz, and O2) were used to obtain the SSVEP signals for identifying the targets. Based on the coding characteristics of the DFBC method, the proposed approach has the ability of self-correction and thus achieves an accuracy of 76.6% and 79.3% for offline and online experiments, respectively, which outperforms the traditional multiple frequencies sequential coding (MFSC) method. This study demonstrates that DFBC is an efficient method for coding a high number of SSVEP targets with a small number of available frequencies.

Signal processing algorithms for SSVEP-based brain computer interface: State-of-the-art and recent developments

Over past two decades, steady-state evoked potentials (SSVEP)-based brain computer interface (BCI) systems have been extensively developed. As we all know, signal processing algorithms play an important role in this BCI. However, there is no comprehensive review of the latest development of signal processing algorithms for SSVEP-based BCI. By analyzing the papers published in authoritative journals in nearly five years, signal processing algorithms of preprocessing, feature extraction and classification modules are discussed in detail. In addition, other aspects existed in this BCI are mentioned. The following key problems are solved. (1) In recent years, which signal processing algorithms are frequently used in each module? (2) Which signal processing algorithms attract more attention in recent years? (3) Which modules are the key to signal processing in BCI field? This information is very important for choosing the appropriate algorithms, and can also be considered as a reference for further research. Simultaneously, we hope that this work can provide relevant BCI researchers with valuable information about the latest trends of signal processing algorithms for SSVEP-based BCI systems.

Convolutional Correlation Analysis for Enhancing the Performance of SSVEP-Based Brain-Computer Interface.

Currently, most of the high-performance models for frequency recognition of steady-state visual evoked potentials (SSVEPs) are linear. However, SSVEPs collected from different channels can have non-linear relationship among each other. Linearly combining electroencephalogram (EEG) from multiple channels is not the most accurate solution in SSVEPs classification. To further improve the performance of SSVEP-based brain-computer interface (BCI), we propose a convolutional neural network-based non-linear model, i.e. convolutional correlation analysis (Conv-CA). Different from pure deep learning models, Conv-CA use convolutional neural networks (CNNs) at the top of a self-defined correlation layer. The CNNs function on how to transform multiple channel EEGs into a single EEG signal. The correlation layer calculates the correlation coefficients between the transformed single EEG signal and reference signals. The CNNs provide non-linear operations to combine EEGs in different channels and different time. And the correlation layer constrains the fitting space of the deep learning model. A comparison study between the proposed Conv-CA method and the task-related component analysis (TRCA) based methods is conducted. Both methods are validated on a 40-class SSVEP benchmark dataset recorded from 35 subjects. The study verifies that the Conv-CA method significantly outperforms the TRCA-based methods. Moreover, Conv-CA has good explainability since its inputs of the correlation layer can be analyzed for visualizing what the model learnt from the data. Conv-CA is a non-linear extension of spatial filters. Its CNN structures can be further explored and tuned for reaching a better performance. The structure of combining neural networks and unsupervised features has the potential to be applied to the classification of other signals.

A Hybrid BCI Based on SSVEP and EOG for Robotic Arm Control.

Brain-computer interface (BCI) for robotic arm control has been studied to improve the life quality of people with severe motor disabilities. There are still challenges for robotic arm control in accomplishing a complex task with a series of actions. An efficient switch and a timely cancel command are helpful in the application of robotic arm. Based on the above, we proposed an asynchronous hybrid BCI in this study. The basic control of a robotic arm with six degrees of freedom was a steady-state visual evoked potential (SSVEP) based BCI with fifteen target classes. We designed an EOG-based switch which used a triple blink to either activate or deactivate the flash of SSVEP-based BCI. Stopping flash in the idle state can help to reduce visual fatigue and false activation rate (FAR). Additionally, users were allowed to cancel the current command simply by a wink in the feedback phase to avoid executing the incorrect command. Fifteen subjects participated and completed the experiments. The cue-based experiment obtained an average accuracy of 92.09%, and the information transfer rates (ITR) resulted in 35.98 bits/min. The mean FAR of the switch was 0.01/min. Furthermore, all subjects succeeded in asynchronously operating the robotic arm to grasp, lift, and move a target object from the initial position to a specific location. The results indicated the feasibility of the combination of EOG and SSVEP signals and the flexibility of EOG signal in BCI to complete a complicated task of robotic arm control.

Design of SSVEP Enhancement-Based Brain Computer Interface

Brain computer interface (BCI) system is a communication bridge between the brain and the external device, and it has been rapidly developed in the recent years. Here, steady state visually evoked potential (SSVEP) is one of the most frequently used control methods for BCI due to its advantages of low training requirement and high stability. However, some people present unobvious SSVEP feature at the location of the primary visual cortex, and this will reduce the performance of SSVEP-based BCIs. In this study, a novel field programmable gate array (FPGA)-based brain computer interface with SSVEP enhancement is proposed to improve the above issue. In the proposed system, a SSVEP-enhancement active dry electrode is designed to acquire good quality of electroencephalography (EEG) without conductive gels, and further enhance the local EEG signal. The experimental results show the proposed system can effectively improve the signal-to-noise ratio of SSVEP and the information transfer rate. Moreover, compared with the current SSVEP-based BCIs in the previous studies, the proposed BCI system contains the advantages of local EEG enhancement, wearablility, wireless transmission, front-end BCI translation, and it contains the potential of applying in many BCI applications in daily life.

Inter- and Intra-subject Template-Based Multivariate Synchronization Index Using an Adaptive Threshold for SSVEP-Based BCIs

The steady-state visually evoked potential (SSVEP) has been widely used in brain-computer interfaces (BCIs). Many studies have proved that the Multivariate synchronization index (MSI) is an efficient method for recognizing the frequency components in SSVEP-based BCIs. Despite its success, the recognition accuracy has not been satisfactory because the simplified pre-constructed sine-cosine waves lack abundant features from the real electroencephalogram (EEG) data. Recent advances in addressing this issue have achieved a significant improvement in recognition accuracy by using individual calibration data. In this study, a new extension based on inter- and intra-subject template signals is introduced to improve the performance of the standard MSI method. Through template transfer, inter-subject similarity and variability are employed to enhance the robustness of SSVEP recognition. Additionally, most existed methods for SSVEP recognition utilize a fixed time window (TW) to perform frequency domain analysis, which limits the information transfer rate (ITR) of BCIs. For addressing this problem, a novel adaptive threshold strategy is integrated into the extension of MSI, which uses a dynamic window to extract the temporal features of SSVEPs and recognizes the stimulus frequency based on a pre-set threshold. The pre-set threshold contributes to obtaining an appropriate and shorter signal length for frequency recognition and filtering ignored-invalid trials. The proposed method is evaluated on a 12-class SSVEP dataset recorded from 10 subjects, and the result shows that this achieves higher recognition accuracy and information transfer rate when compared with the CCA, MSI, Multi-set CCA, and Individual Template-based CCA. This paper demonstrates that the proposed method is a promising approach for developing high-speed BCIs.

The Validity of Steady-State Visual Evoked Potentials as Attention Tags and Input Signals: A Critical Perspective of Frequency Allocation and Number of Stimuli.

Steady-state visual evoked potential (SSVEP) is a periodic response to a repetitive visual stimulus at a specific frequency. Currently, SSVEP is widely treated as an attention tag in cognitive activities and is used as an input signal for brain–computer interfaces (BCIs). However, whether SSVEP can be used as a reliable indicator has been a controversial issue. We focused on the independence of SSVEP from frequency allocation and number of stimuli. First, a cue–target paradigm was adopted to examine the interaction between SSVEPs evoked by two stimuli with different frequency allocations under different attention conditions. Second, we explored whether signal strength and the performance of SSVEP-based BCIs were affected by the number of stimuli. The results revealed that no significant interaction of SSVEP responses appeared between attended and unattended stimuli under various frequency allocations, regardless of their appearance in the fundamental or second-order harmonic. The amplitude of SSVEP suffered no significant gain or loss under different numbers of stimuli, but the performance of SSVEP-based BCIs varied along with duration of stimuli; that is, the recognition rate was not affected by the number of stimuli when the duration of stimuli was long enough, while the information transfer rate (ITR) presented the opposite trend. It can be concluded that SSVEP is a reliable tool for marking and monitoring multiple stimuli simultaneously in cognitive studies, but much caution should be taken when choosing a suitable duration and the number of stimuli, in order to achieve optimal utility of BCIs in the future.

Data Analytics in Steady-State Visual Evoked Potential-Based Brain–Computer Interface: A Review

Electroencephalograph (EEG) has been widely applied for brain-computer interface (BCI) which enables paralyzed people to directly communicate with and control external devices, due to its portability, high temporal resolution, ease of use and low cost. Of various EEG paradigms, steady-state visual evoked potential (SSVEP)-based BCI system which uses multiple visual stimuli (such as LEDs or boxes on a computer screen) flickering at different frequencies has been widely explored in the past decades due to its fast communication rate and high signal-to-noise ratio. In this article, we review the current research in SSVEP-based BCI, focusing on the data analytics that enables continuous, accurate detection of SSVEPs and thus high information transfer rate. The main technical challenges, including signal pre-processing, spectrum analysis, signal decomposition, spatial filtering in particular canonical correlation analysis and its variations, and classification techniques are described in this article. Research challenges and opportunities in spontaneous brain activities, mental fatigue, transfer learning as well as hybrid BCI are also discussed.

Five-Class SSVEP Response Detection using Common-Spatial Pattern (CSP)-SVM Approach

Brain-computer interface (BCI) technologies significantly facilitate the interaction between physically impaired people and their surroundings. In electroencephalography (EEG) based BCIs, a variety of physiological responses including P300, motor imagery, movement-related potential, steady-state visual evoked potential (SSVEP) and slow cortical potential have been utilized. Because of the superior signal-to-noise ratio (SNR) together with quicker information transfer rate (ITR), the intentness of SSVEP-based BCIs is progressing significantly. This paper represents the feature extraction and classification frameworks to detect five classes EEG-SSVEP responses. The common-spatial pattern (CSP) has been employed to extract the features from SSVEP responses and these features have been classified through the support vector machine (SVM). The proposed architecture has achieved the highest classification accuracy of 88.3%. The experimental result proves that the proposed architecture could be utilized for the detection of SSVEP responses to develop any BCI applications. Keywords: EEG, BCI, SSVEP, CSP, SVM, Machine Learning

Optimizing SSVEP-Based BCI System towards Practical High-Speed Spelling.

The brain–computer interface (BCI) spellers based on steady-state visual evoked potentials (SSVEPs) have recently been widely investigated for their high information transfer rates (ITRs). This paper aims to improve the practicability of the SSVEP-BCIs for high-speed spelling. The system acquired the electroencephalogram (EEG) data from a self-developed dedicated EEG device and the stimulation was arranged as a keyboard. The task-related component analysis (TRCA) spatial filter was modified (mTRCA) for target classification and showed significantly higher performance compared with the original TRCA in the offline analysis. In the online system, the dynamic stopping (DS) strategy based on Bayesian posterior probability was utilized to realize alterable stimulating time. In addition, the temporal filtering process and the programs were optimized to facilitate the online DS operation. Notably, the online ITR reached 330.4 ± 45.4 bits/min on average, which is significantly higher than that of fixed stopping (FS) strategy, and the peak value of 420.2 bits/min is the highest online spelling ITR with a SSVEP-BCI up to now. The proposed system with portable EEG acquisition, friendly interaction, and alterable time of command output provides more flexibility for SSVEP-based BCIs and is promising for practical high-speed spelling.

Enhancing performance of SSVEP-based BCI by unsupervised learning information from test trials.

Steady-State Visual Evoked Potentials (SSVEPs) have become one of the most used neural signals for brain- computer interfaces (BCIs) due to their stability and high signal- to-noise rate. However, the performance of SSVEP-based BCIs would degrade with a few training samples. This study was proposed to enhance the detection of SSVEP by combining the supervised learning information from training samples and the unsupervised learning information from the trial to be tested. A new method, i.e. cyclic shift trials (CST), was proposed to generate new calibration samples from the test data, which were furtherly used to create the templates and spatial filters of task- related component analysis (TRCA). The test-trial templates and spatial filters were combined with training-sample templates and spatial filters to recognize SSVEP. The proposed algorithm was tested on a benchmark dataset. As a result, it reached significantly higher classification accuracy than traditional TRCA when only two training samples were used. Speciflcally, the accuracy was improved by 9.5% for 0.7s data. Therefore, this study demonstrates CST is effective to improve the performance of SSVEP-BCI.