SSVEP-based BCIs Research Articles

A Least-Square Unified Framework for Spatial Filtering in SSVEP-Based BCIs.

The steady-state visual evoked potential (SSVEP) has become one of the most prominent BCI paradigms with high information transfer rate, and has been widely applied in rehabilitation and assistive applications. This paper proposes a least-square (LS) unified framework to summarize the correlation analysis (CA)-based SSVEP spatial filtering methods from a machine learning perspective. Within this framework, the commonalities and differences between various spatial filtering methods appear apparent, the interpretation of computational factors becomes intuitive, and spatial filters can be determined by solving a generalized optimization problem with non-linear and regularization items. Moreover, the proposed LS framework provides the foundation of utilizing the knowledge behind these spatial filtering methods in further classification/regression model designs. Through a comparative analysis of existing representative spatial filtering methods, recommendations are made for the superior and robust design strategies. These recommended strategies are further integrated to fill the research gaps and demonstrate the ability of the proposed LS framework to promote algorithmic improvements, resulting in five new spatial filtering methods. This study could offer significant insights in understanding the relationships between various design strategies in the spatial filtering methods from the machine learning perspective, and would also contribute to the development of the SSVEP recognition methods with high performance.

A Novel Fast ICA-FBCCA Algorithm and Convolutional Neural Network for Single-Flicker SSVEP-Based BCIs

A Novel Fast ICA-FBCCA Algorithm and Convolutional Neural Network for Single-Flicker SSVEP-Based BCIs

Multi-Stimulus Least-Squares Transformation With Online Adaptation Scheme to Reduce Calibration Effort for SSVEP-Based BCIs.

The proposed ms-LST-OA consists of two parts. Firstly, to improve the precision of the transformation matrices, we propose the multi-stimulus LST (ms-LST) using cross-stimulus learning scheme as the cross-subject data transformation method. The ms-LST uses the data from neighboring stimuli to construct a higher precision transformation matrix for each stimulus to reduce the differences between transformed data and real data. Secondly, to further optimize the constructed spatial filters and reference templates, we use an online adaptation scheme to learn more features of the EEG signals of the target subject through an iterative process trial-by-trial. ms-LST-OA performance was measured for three datasets (Benchmark Dataset, BETA Dataset, and UCSD Dataset). Using few calibration data, the ITR of ms-LST-OA achieved 210.01±10.10 bits/min, 172.31±7.26 bits/min, and 139.04±14.90 bits/min for all three datasets, respectively. Using ms-LST-OA can reduce calibration effort for SSVEP-based BCIs.

Unsupervised multi-source variational domain adaptation for inter-subject SSVEP-based BCIs

Steady-State Visual Evoked Potential-based Brain–Computer Interfaces (SSVEP-based BCIs) are widely used to decode electroencephalography (EEG) data, often employing subject-specific models. However, these methods demand a substantial volume of labeled data, which tends to induce visual fatigue in subjects. To address this challenge, an efficient approach is to transfer known subject knowledge through inter-subject methods using unlabeled data. However, the previous state-of-the-art unsupervised inter-subject model is limited by the feature extraction capability of spatial filter methods. To overcome this limitation, we propose a novel Multi-source Variational Domain Adaptation (MVDA) method, which incorporates a Filter-bank Dynamic Graph Convolutional Network (FBDGCN). The FBDGCN aims to extract distinctive graph features under different harmonics from offline labeled source and unlabeled target EEG data. Subsequently, the source and target graph features are jointly trained using MVDA for domain adaptation. With proposed variational inference mechanism, MVDA can constrain the distribution manifold and minimize the domain discrepancy. Since the performance of knowledge transfer heavily relies on the similarity between source and target domains, we also propose an unsupervised subject correlation analysis method to select the source subjects from the dataset. Our experimental results demonstrate that the MVDA method yields a higher information transfer rate (ITR) (189.87 ± 24.98 bits/min) and mean accuracy (92.11 ± 2.85%). The proposed model can be calibrated only by unlabeled target subject data to obtain a higher recognition accuracy, which is more applicable to real-world scenarios.

Overview of recognition methods for SSVEP-based BCIs in World Robot Contest 2022: MATLAB undergraduate group

The steady-state visual evoked potential (SSVEP)-based speller has emerged as a widely adopted paradigm in current brain–computer interface (BCI) systems due to its rapid processing and consistent performance across different individuals. Calibration-free SSVEP algorithms, as opposed to their calibration-based counterparts, offer clear and intuitive mathematical principles, making them accessible to novice developers. During the World Robot Contest (WRC) 2022, participants in the undergraduate category utilized various approaches to accomplish target detection in the calibration-free setting, successfully implementing the algorithms using MATLAB. The winning approach achieved an average information transfer rate of 198.94 bits/min in the final test, which is notably high given the calibration-free scenario. This paper presents an introduction to the underlying principles of the selected methods, accompanied by a comparison of their effectiveness through analysis of results from both the final test and offline experiments. Additionally, we propose that the youth competition of WRC could serve as an ideal starting point for beginners interested in studying and developing their own BCI systems.

A deep learning method for classification of steady-state visual evoked potentials in a brain-computer interface speller

ABSTRACT The main drawback of SSVEP-based BCIs is the lack of a classifier that categorizes SSVEPs with high accuracy and Information Transfer Rate (ITR). Addressing this, we proposed a deep convolutional neural network (CNN) for classifying a 40-class SSVEP. Time windows of length 2 and 3.5 seconds were used for training and testing the model by leave-one-subject-out cross-validation, using nine, three, and single-channel EEG. The proposed model reached 88.5% average accuracy for the nine-channel EEG with the mean and max ITR of 72 and 91.23 bpm, respectively. It outperformed the previous deep learning methods for SSVEP-based BCIs, in terms of accuracy and ITR. In the three-channel experiment the mean accuracy and ITR were 76.02% and 40.1 bpm. In single-channel implementation, O1 channel achieved 77.38 % average accuracy (highest) and the mean ITR was 57.51 bpm. The model showed promising performance to put this technology forward and make it more practical.

A L1 normalization enhanced dynamic window method for SSVEP-based BCIs

BackgroundFilter bank canonical correlation analysis (FBCCA) has been widely applied to detect the frequency components of steady-state visual evoked potential (SSVEP). FBCCA with dynamic window (FBCCA-DW) is recently proposed to improve its performance. FBCCA-DW adaptively chooses a proper window length based on the signal-to-noise ratio (SNR) of SSVEP signals. It takes the output of FBCCA to evaluate the SNR of SSVEP signals, by using the softmax function and cost function. In practice, SSVEP signals always contain task-unrelated electroencephalogram (EEG), which degrades the SSVEP task. When the power of task-unrelated EEG changes, there would be an offset in the output of FBCCA. However, due to the insensitivity of softmax function to the offset, the SNR in FBCCA-DW ignores the interference of the task-unrelated EEG. Therefore, FBCCA-DW will analyze SSVEP signals at an inappropriate window length. New methodTo solve the issue, we replace the softmax function with the L1 normalization, which could respond a reasonable SNR to the offset. Since the proposed method takes task-unrelated EEG into account, it could choose a more appropriate window length. ResultsWe comprehensively validate the proposed method on three publicly available SSVEP datasets. The results indicate that the proposed method could improve the performance significantly. Comparison with existing methodsThe proposed method outperforms FBCCA and FBCCA-DW in terms of information transfer rate (ITR). ConclusionsThe proposed method enhances the correlation between the window length and the credibility of the recognition result. It shows its potential for practical applications in complex environments.

Does Real-Time Feedback Improve User Performance in SSVEP-based Brain-Computer Interfaces?

Offline and online experiments are both widely used in SSVEP-based BCI research and development for different purposes. One of the major differences between offline and online experiments is the existence of real-time feedback to the user while they are using the interface. However, the role of feedback in SSVEP-based BCIs has not yet been well studied. This work focuses on understanding the effect of feedback in SSVEP-based BCIs and if there exists any relationship between offline and online BCI performance. An experiment was designed to compare directly the accuracies of the BCI with and without feedback for participants. Results showed that feedback can improve performance in a complex task, but no clear improvement was observed in a simple task.

Facilitating Applications of SSVEP-Based BCIs by Within-Subject Information Transfer

The steady-state visual evoked potential based brain–computer interface (SSVEP–BCI) can provide high-speed alternative and augmentative communication in real-world applications. For individuals using a long-term BCI, within-subject (i.e., cross-day and cross-electrode) transfer learning could improve the BCI performance and reduce the calibration burden. To validate the within-subject transfer learning scheme, this study designs a 40-target SSVEP–BCI. Sixteen subjects are recruited, each of whom has performed experiments on three different days and has undergone the experiments of the SSVEP–BCIs based on the dry and wet electrodes. Several transfer directions, including the cross-day directions in parallel with the cross-electrode directions, are analyzed, and it is found that the transfer learning-based approach can maintain stable performance by zero training. Compared with the fully calibrated approaches, the transfer learning-based approach can achieve significantly better or comparable performance in different transfer directions. This result verifies that the transfer learning-based scheme is well suited for implementing a high-speed zero-training SSVEP–BCI, especially the dry electrode-based SSVEP–BCI system. A validation experiment of the cross-day wet-to-dry transfer, involving nine subjects, has shown that the average accuracy is 85.97 ± 5.60% for the wet-to-dry transfer and 77.69 ± 6.42% for the fully calibrated method with dry electrodes. By leveraging the electroencephalography data acquired on different days by different electrodes via transfer learning, this study lays the foundation for facilitating the long-term usage of the SSVEP–BCI and advancing the frontier of the dry electrode-based SSVEP–BCI in real-world applications.

Influence of Stimuli Spatial Proximity on a SSVEP-Based BCI Performance

Objective: Steady-State Visual Evoked Potentials based Brain-Computer Interfaces (SSVEP-based BCIs) systems have been shown as promising technology due to their short response time and ease of use. SSVEP-based BCIs use brain responses to a flickering visual stimulus as an input command to an external application or device, and it can be influenced by stimulus properties, signal recording, and signal processing. We aim to investigate the system performance varying the stimuli spatial proximity (a stimulus property).Material and methods: We performed a comparative analysis of two visual interface designs (named cross and square) for an SSVEP-based BCI. The power spectrum density (PSD) was used as feature extraction and the Support Machine Vector (SVM) as classification method. We also analyzed the effects of five flickering frequencies (6.67, 8.57, 10, 12 e 15 Hz) between and within interfaces.Results: We found higher accuracy rates for the flickering frequencies of 10, 12, and 15 Hz. The stimulus of 10 Hz presented the highest SSVEP amplitude response for both interfaces. The system presented the best performance (highest classification accuracy and information transfer rate) using the cross interface (lower visual angle).Conclusion: Our findings suggest that the system has the highest performance in the spatial proximity range from 4° to 13° (visual angle). In addition, we conclude that as the stimulus spatial proximity increases, the interference from other stimuli reduces, and the SSVEP amplitude response decreases, which reduces system accuracy. The inter-stimulus distance is a visual interface parameter that must be chosen carefully to increase the efficiency of an SSVEP-based BCI.

Cross-Subject Assistance: Inter- and Intra-Subject Maximal Correlation for Enhancing the Performance of SSVEP-Based BCIs.

The current state-of-the-art methods significantly improve the detection performance of the steady-state visual evoked potentials (SSVEPs) by using the individual calibration data. However, the time-consuming calibration sessions limit the number of training trials and may give rise to visual fatigue, which weakens the effectiveness of the individual training data. For addressing this issue, this study proposes a novel inter- and intra-subject maximal correlation (IISMC) method to enhance the robustness of SSVEP recognition via employing the inter- and intra-subject similarity and variability. Through efficient transfer learning, similar experience under the same task is shared across subjects. IISMC extracts subject-specific information and similar task-related information from oneself and other subjects performing the same task by maximizing the inter- and intra-subject correlation. Multiple weak classifiers are built from several existing subjects and then integrated to construct the strong classifiers by the average weighting. Finally, a powerful fusion predictor is obtained for target recognition. The proposed framework is validated on a benchmark data set of 35 subjects, and the experimental results demonstrate that IISMC obtains better performance than the state of the art task-related component analysis (TRCA). The proposed method has great potential for developing high-speed BCIs.

Comparing user-dependent and user-independent training of CNN for SSVEP BCI

Objective. We presented a comparative study on the training methodologies of a convolutional neural network (CNN) for the detection of steady-state visually evoked potentials (SSVEP). Two training scenarios were also compared: user-independent (UI) training and user-dependent (UD) training. Approach. The CNN was trained in both UD and UI scenarios on two types of features for SSVEP classification: magnitude spectrum features (M-CNN) and complex spectrum features (C-CNN). The canonical correlation analysis (CCA), widely used in SSVEP processing, was used as the baseline. Additional comparisons were performed with task-related components analysis (TRCA) and filter-bank canonical correlation analysis (FBCCA). The performance of the proposed CNN pipelines, CCA, FBCCA and TRCA were evaluated with two datasets: a seven-class SSVEP dataset collected on 21 healthy participants and a twelve-class publicly available SSVEP dataset collected on ten healthy participants. Main results. The UD based training methods consistently outperformed the UI methods when all other conditions were the same, as one would expect. However, the proposed UI-C-CNN approach performed similarly to the UD-M-CNN across all cases investigated on both datasets. On Dataset 1, the average accuracies of the different methods for 1 s window length were: CCA: 69.1% ± 10.8%, TRCA: 13.4% ± 1.5%, FBCCA: 64.8% ± 15.6%, UI-M-CNN: 73.5% ± 16.1%, UI-C-CNN: 81.6% ± 12.3%, UD-M-CNN: 87.8% ± 7.6% and UD-C-CNN: 92.5% ± 5%. On Dataset 2, the average accuracies of the different methods for data length of 1 s were: UD-C-CNN: 92.33% ± 11.1%, UD-M-CNN: 82.77% ± 16.7%, UI-C-CNN: 81.6% ± 18%, UI-M-CNN: 70.5% ± 22%, FBCCA: 67.1% ± 21%, CCA: 62.7% ± 21.5%, TRCA: 40.4% ± 14%. Using t-SNE, visualizing the features extracted by the CNN pipelines further revealed that the C-CNN method likely learned both the amplitude and phase related information from the SSVEP data for classification, resulting in superior performance than the M-CNN methods. The results suggested that UI-C-CNN method proposed in this study offers a good balance between performance and cost of training data. Significance. The proposed C-CNN based method is a suitable candidate for SSVEP-based BCIs and provides an improved performance in both UD and UI training scenarios.

Questionable Classification Accuracy Reported in "Designing a Sum of Squared Correlations Framework for Enhancing SSVEP-Based BCIs".

This commentary presents a replication study to verify the effectiveness of a sum of squared correlations (SSCOR)-based steady-state visual evoked potentials (SSVEPs) decoding method proposed by Kumar et al.. We implemented the SSCOR-based method in accordance with their descriptions and estimated its classification accuracy using a benchmark SSVEP dataset with cross validation. Our results showed significantly lower classification accuracy compared with the ones reported in Kumar et al.'s study. We further investigated the sources of performance discrepancy by simulating data leakage between training and test datasets. The classification performance of the simulation was remarkably similar to those reported by Kumar et al.. We, therefore, question the validity of evaluation and conclusions drawn in Kumar et al.'s study.

Adaptive canonical correlation analysis for harmonic stimulation frequencies recognition in SSVEP-based BCIs

Steady-state visual evoked potential (SSVEP) is the brain's response to quickly repetitive visual stimulus with a certain frequency. To increase the information transfer rate (ITR) in SSVEP-based systems, due to the frequency resolution restriction, we are forced to broaden the frequency range, which causes harmonic frequencies to come into the stimulation frequency range. Conventional canonical correlation analysis (CCA) may be associated with error for SSVEP frequency recognition at stimulation frequencies with harmonic relations. The number of harmonics considered to construct reference signals are determined adaptively; for frequencies whose second harmonic exists in the frequency range, two harmonics are used, and for other frequencies, just one harmonic is used. After constructing reference signals and recognizing the frequency corresponding to the maximum value of correlation by CCA, the target frequency is determined after a postprocessing step. Results show that for the 8-s time window length, the average classification accuracy for the adaptive CCA was 84 %, while the corresponding values for the CCA with one harmonic $(N=1)$ and two harmonics $(N=2)$ were 78 % and 74 %, respectively. For 4-s length, this accuracy for the adaptive CCA was 86 %, while it was 78\% for both harmonic selection modes of the standard CCA, $N=1$ and $N=2$. In SSVEP applications with harmonic stimulation frequencies, the adaptive CCA has significantly improved the frequency recognition accuracy in comparison with the popularly standard CCA method. The proposed method can be useful for SSVEP-based BCI systems that use broad ranges of stimulation frequencies with harmonic relation.

A study of SSVEP from below-the-hairline areas in low-, medium-, and high-frequency ranges

Recent studies have analyzed steady-state visual evoked potentials (SSVEPs) measured on belowthe-hairline areas, such as behind-the-ears (temporal area) and face (frontal area) using different montages of channels and frequency bands. This study aims to investigate how both reference electrode and frequency ranges (low, medium, and high bands) affect the SSVEP measured on hairless areas of temporal and frontal. The EEG signals were acquired from 12 individuals, and the elicited SSVEP was evaluated in terms of amplitude and signal-to-noise ratio (SNR). The best electrode combinations for measuring SSVEP on hairless areas are obtained with Fpz-Tp9 and Fpz-Tp10 (up to 40 Hz); however, for stimuli frequencies higher than 40 Hz, the best result is obtained with the temporal area and with the reference electrode located on the ear. The SSVEP amplitude and the SNR depend on the combination of the electrode reference and the range of visual stimulus frequency. These findings can aid in the development of more practical and comfortable SSVEP-based BCIs.

Effects of Mental Load and Fatigue on Steady-State Evoked Potential Based Brain Computer Interface Tasks: A Comparison of Periodic Flickering and Motion-Reversal Based Visual Attention.

Steady-state visual evoked potentials (SSVEP) based paradigm is a conventional BCI method with the advantages of high information transfer rate, high tolerance to artifacts and the robust performance across users. But the occurrence of mental load and fatigue when users stare at flickering stimuli is a critical problem in implementation of SSVEP-based BCIs. Based on electroencephalography (EEG) power indices α, θ, θ + α, ratio index θ/α and response properties of amplitude and SNR, this study quantitatively evaluated the mental load and fatigue in both of conventional flickering and the novel motion-reversal visual attention tasks. Results over nine subjects revealed significant mental load alleviation in motion-reversal task rather than flickering task. The interaction between factors of “stimulation type” and “fatigue level” also illustrated the motion-reversal stimulation as a superior anti-fatigue solution for long-term BCI operation. Taken together, our work provided an objective method favorable for the design of more practically applicable steady-state evoked potential based BCIs.

Impact of electrode positions and harmonic frequency components in SSVEP-based BCIs

Steady state visual evoked potential SSVEP based brain computer interfaces BCIs have been recognised as having major potential to be used practically in everyday daily life as they typically outperform more traditional BCIs in terms of detection accuracy and information transfer rates. It is also essential to improve the ease of use such as reducing the number of necessary electrodes without decreasing speed or accuracy. This study investigated the extent to which electrode positions, derivation methods, and harmonics will affect the performance of a SSVEP-based BCI. A pilot study was carried out on a 4-class SSVEP-based BCI with six healthy subjects. Detection accuracy increased from 63.75 ± 5.62% mean ± SD with monopolar derivation to 89.42 ± 5.58% with two-channel bipolar derivation using only harmonic H1. The significantly improved detection accuracy with two electrodes implies that it is possible to achieve high classification accuracy with a very low number of electrodes.

Accounting for Phase Drifts in SSVEP-Based BCIs by Means of Biphasic Stimulation

This study proposes a novel biphasic stimulation technique to solve the issue of phase drifts in steady-state visual evoked potential (SSVEPs) in phase-tagged systems. Phase calibration was embedded in stimulus sequences using a biphasic flicker, which is driven by a sequence with alternating reference and phase-shift states. Nine subjects were recruited to participate in off-line and online tests. Signals were bandpass filtered and segmented by trigger signals into reference and phase-shift epochs. Frequency components of SSVEP in the reference and phase-shift epochs were extracted using the Fourier method with a 50% overlapped sliding window. The real and imaginary parts of the SSVEP frequency components were organized into complex vectors in each epoch. Hotelling's t-square test was used to determine the significances of nonzero mean vectors. The rejection of noisy data segments and the validation of gaze detections were made based on p values. The phase difference between the valid mean vectors of reference and phase-shift epochs was used to identify user's gazed targets in this system. Data showed an average information transfer rate of 44.55 and 38.21 bits/min in off-line and online tests, respectively.