Brain-computer Interface Speller Research Articles

Signal alignment for cross-datasets in P300 brain-computer interfaces

Objective. Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment (SA) for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning. Approach. We proposed a linear SA that uses the P300’s latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm). Results. Although the standard approach without SA had an average precision (AP) score of 25.5%, the approach demonstrated a 35.8% AP score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets. Significance. We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.

Resting state EEG assisted imagined vowel phonemes recognition by native and non-native speakers using brain connectivity measures.

Communication is challenging for disabled individuals, but with advancement of brain-computer interface (BCI) systems, alternative communication systems can be developed. Current BCI spellers, such as P300, SSVEP, and MI, have drawbacks like reliance on external stimuli or conversation irrelevant mental tasks. In contrast to these systems, Imagined speech based BCI systems rely on directly decoding the vowels/words user is thinking, making them more intuitive, user friendly and highly popular among Brain-Computer-Interface (BCI) researchers. However, more research needs to be conducted on how subject-specific characteristics such as mental state, age, handedness, nativeness and resting state activity affects the brain's output during imagined speech. In an overt speech, it is evident that native and non-native speakers' brains function differently. Therefore, this paper explores how nativeness to language affects EEG signals while imagining vowel phonemes, using brain-map analysis and scalogram and also investigates the inclusion of features extracted from resting state EEG with imagined state EEG. The Fourteen-channel EEG for Imagined Speech (FEIS) dataset was used to analyse the EEG signals recorded while imagining vowel phonemes for 16 subjects (nine native English and seven non-native Chinese). For the classification of vowel phonemes, different connectivity measures such as covariance, coherence, and Phase Synchronous Index-PSI were extracted and analysed using statistics based Multivariate Analysis of Variance (MANOVA) approach. Different fusion strategies (difference, concatenation, Common Spatial Pattern-CSP and Canonical Correlation Analysis-CCA) were carried out to incorporate resting state EEG connectivity measures with imagined state connectivity measures for enhancing the accuracy of imagined vowel phoneme recognition. Simulation results revealed that concatenating imagined state and rest state covariance and PSI features provided the maximum accuracy of 92.78% for native speakers and 94.07% for non-native speakers.

A Modified Hybrid Brain-Computer Interface Speller Based on Steady-State Visual Evoked Potentials and Electromyogram.

To enhance the information transfer rate (ITR) of a steady-state visual evoked potential (SSVEP)-based speller, more characters with flickering symbols should be used. Increasing the number of symbols might reduce the classification accuracy. A hybrid brain-computer interface (BCI) improves the overall performance of a BCI system by taking advantage of two or more control signals. In a simultaneous hybrid BCI, various modalities work with each other simultaneously, which enhances the ITR. In our proposed speller, simultaneous combination of electromyogram (EMG) and SSVEP was applied to increase the ITR. To achieve 36 characters, only nine stimulus symbols were used. Each symbol allowed the selection of four characters based on four states of muscle activity. The SSVEP detected which symbol the subject was focusing on and the EMG determined the target character out of the four characters dedicated to that symbol. The frequency rate for character encoding was applied in the EMG modality and latency was considered in the SSVEP modality. Online experiments were carried out on 10 healthy subjects. The average ITR of this hybrid system was 96.1 bit/min with an accuracy of 91.2%. The speller speed was 20.9 char/min. Different subjects had various latency values. We used an average latency of 0.2 s across all subjects. Evaluation of each modality showed that the SSVEP classification accuracy varied for different subjects, ranging from 80% to 100%, while the EMG classification accuracy was approximately 100% for all subjects. Our proposed hybrid BCI speller showed improved system speed compared with state-of-the-art systems based on SSVEP or SSVEP-EMG, and can provide a user-friendly, practical system for speller applications.

A Systematic Review of Computational Intelligence Techniques for Channel Selection in P300-Based Brain Computer Interface Speller

Electroencephalography (EEG) based P300 speller aid in restoring the communication and control capabilities in patients suffering from motor disabilities. However, the quality and quantity of the data collected from EEG recordings have a substantial influence on the P300 speller's performance. Hence, selecting the optimum number of recording electrodes, i.e., channels for each user, is a significant difficulty for the P300 speller. There are two fundamental objectives of the channel selection process: 1) To extract the most crucial information from the relevant channels, hence reducing the computing complexity of P300/Non-P300 signal processing operation, and 2) To lessen the potential overfitting that could result from using unwanted channels to boost performance. For obtaining the best channel subsets, different channel selection techniques, including manual, filtering, wrapper, and embedded approaches, have been applied by past researchers. This study presents a comprehensive review of recent developments, present status, challenges, and possible solution associated with channel selection strategies in P300 speller.

A method for synchronized use of EEG and eye tracking in fully immersive VR.

This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

Average Time Consumption Per Character - a Practical Performance Metric for Generic Synchronous BCI Spellers.

The information transfer rate (ITR) is widely accepted as a performance metric for generic brain-computer interface (BCI) spellers, while it is noticeable that the communication speed given by ITR is actually an upper bound which however can never be reached in real systems. A new performance metric is therefore needed. In this paper, a new metric named average time consumption per character (ATCPC) is proposed. It quantifies how long it takes on average to type one character using a typical synchronous BCI speller. To analytically derive ATCPC, the real typing process is modelled with a random walk on a graph. Misclassification and backspace are carefully characterized. A close-form formula of ATCPC is obtained through computing the hitting time of the random walk. The new metric is validated through simulated typing experiments and compared with ITR. Firstly, the formula and simulation show a good consistency. Secondly, ITR always tends to overestimate the communication speed, while ATCPC is more realistic. The proposed ATCPC metric is valid. ATCPC is a qualified substitute for ITR. ATCPC also reveals the great potential of keyboard optimization to further enhance the performance of BCI spellers, which was hardly investigated before.

Non-invasive EEG-based BCI spellers from the beginning to today: a mini-review.

The defeat of the central motor neuron leads to the motor disorders. Patients lose the ability to control voluntary muscles, for example, of the upper limbs, which introduces a fundamental dissonance in the possibility of daily use of a computer or smartphone. As a result, the patients lose the ability to communicate with other people. The article presents the most popular paradigms used in the brain-computer-interface speller system and designed for typing by people with severe forms of the movement disorders. Brain-computer interfaces (BCIs) have emerged as a promising technology for individuals with communication impairments. BCI-spellers are systems that enable users to spell words by selecting letters on a computer screen using their brain activity. There are three main types of BCI-spellers: P300, motor imagery (MI), and steady-state visual evoked potential (SSVEP). However, each type has its own limitations, which has led to the development of hybrid BCI-spellers that combine the strengths of multiple types. Hybrid BCI-spellers can improve accuracy and reduce the training period required for users to become proficient. Overall, hybrid BCI-spellers have the potential to improve communication for individuals with impairments by combining the strengths of multiple types of BCI-spellers. In conclusion, BCI-spellers are a promising technology for individuals with communication impairments. P300, MI, and SSVEP are the three main types of BCI-spellers, each with their own advantages and limitations. Further research is needed to improve the accuracy and usability of BCI-spellers and to explore their potential applications in other areas such as gaming and virtual reality.

A novel multiclass-based framework for P300 detection in BCI matrix speller: Temporal EEG patterns of non-target trials vary based on their position to previous target stimuli

Brain–computer interface (BCI) provides a new communication pathway for severely disabled people and enables them to communicate with external world using only their brain activity. P300-based BCI speller helps patients spell words using their brain signals.Until now, binary-classification-based approaches have been used for P300 detection. This study demonstrates that binary-classification-based approaches may not be appropriate for P300 detection. We proved that temporal EEG patterns of non-target trials are different based on their position to previous target stimuli. Therefore, considering all non-target trials in only one group makes distinguishing target from non-target components difficult for machine learning algorithms and, consequently, deteriorate character recognition accuracy. This study introduced a novel approach for P300 detection in BCI spellers. In this study, we first divided non-target trials into several groups according to their temporal patterns in training stage. Then, we introduced a multiclass-based framework for P300 detection.Proposed approach is evaluated with three public datasets, BCI competition II, BCI competition III and the BNCI Horizon. In all three datasets, proposed multi-class approach outperformed common binary-classification-based approach in the same conditions. In addition, multiclass-based approach reached the highest accuracy (100%) in the 3th sequence for BCI competition II, for BCI competition III achieved an average accuracy of 74% and 98% in 5th and 15th sequences and, for the BNCI Horizon dataset achieved an average accuracy of 77.50% and 97.49% in 5th and 10th sequences, respectively. That means our proposed approach, regardless of its simplicity, achieved state-of-the-art character recognition performance in the existing methods. The results confirmed that binary-classification based methods is not appropriate for P300 detection in BCI spellers.

A spatial-temporal linear feature learning algorithm for P300-based brain-computer interfaces

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1–5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

Brain-Computer Interface Technologies for Neurological Diseases

A Brain-Computer Interface (BCI) is a device that monitors central nervous system (CNS) activity and transforms it into artificial output that can be used to replace, improve, supplement, or restore the natural CNS output. Brain-Computer Interface (BCI) technologies for neurological diseases are vital for the well-being of related patients. This paper presents 4 forms of BCI technologies used for neurological diseases: EEG-based BCI for the treatment of Attention Deficit Hyperactivity Disorder (ADHD), Neural Cursor and BCI Spellers for ALS patients, Seizure Prediction BCI for those with epilepsy, and Motor Imagery BCI (MI-BCI) for the treatment of upper-limb strokes. The development and applications of the aforementioned BCI technologies are covered in detail. This literature review's objective is to examine the key characteristics of BCI technologies used to make the life quality of neurological disease sufferers better. The author aims to assist concerned researchers and individuals with the gathered information and help future researchers to learn better from previous studies.

A hybrid P300-SSVEP brain-computer interface speller with a frequency enhanced row and column paradigm.

This study proposes a new hybrid brain-computer interface (BCI) system to improve spelling accuracy and speed by stimulating P300 and steady-state visually evoked potential (SSVEP) in electroencephalography (EEG) signals. A frequency enhanced row and column (FERC) paradigm is proposed to incorporate the frequency coding into the row and column (RC) paradigm so that the P300 and SSVEP signals can be evoked simultaneously. A flicker (white-black) with a specific frequency from 6.0 to 11.5 Hz with an interval of 0.5 Hz is assigned to one row or column of a 6 × 6 layout, and the row/column flashes are carried out in a pseudorandom sequence. A wavelet and support vector machine (SVM) combination is adopted for P300 detection, an ensemble task-related component analysis (TRCA) method is used for SSVEP detection, and the two detection possibilities are fused using a weight control approach. The implemented BCI speller achieved an accuracy of 94.29% and an information transfer rate (ITR) of 28.64 bit/min averaged across 10 subjects during the online tests. An accuracy of 96.86% is obtained during the offline calibration tests, higher than that of only using P300 (75.29%) or SSVEP (89.13%). The SVM in P300 outperformed the previous linear discrimination classifier and its variants (61.90-72.22%), and the ensemble TRCA in SSVEP outperformed the canonical correlation analysis method (73.33%). The proposed hybrid FERC stimulus paradigm can improve the performance of the speller compared with the classical single stimulus paradigm. The implemented speller can achieve comparable accuracy and ITR to its state-of-the-art counterparts with advanced detection algorithms.

A new grid stimulus with subtle flicker perception for user-friendly SSVEP-based BCIs

Objective. The traditional uniform flickering stimulation pattern shows strong steady-state visual evoked potential (SSVEP) responses and poor user experience with intense flicker perception. To achieve a balance between performance and comfort in SSVEP-based brain–computer interface (BCI) systems, this study proposed a new grid stimulation pattern with reduced stimulation area and low spatial contrast. Approach. A spatial contrast scanning experiment was conducted first to clarify the relationship between the SSVEP characteristics and the signs and values of spatial contrast. Four stimulation patterns were involved in the experiment: the ON and OFF grid stimulation patterns that separately activated the positive or negative contrast information processing pathways, the ON–OFF grid stimulation pattern that simultaneously activated both pathways, and the uniform flickering stimulation pattern that served as a control group. The contrast-intensity and contrast-user experience curves were obtained for each stimulation pattern. Accordingly, the optimized stimulation schemes with low spatial contrast (the ON-50% grid stimulus, the OFF-50% grid stimulus, and the Flicker-30% stimulus) were applied in a 12-target and a 40-target BCI speller and compared with the traditional uniform flickering stimulus (the Flicker-500% stimulus) in the evaluation of BCI performance and subjective experience. Main results. The OFF-50% grid stimulus showed comparable online performance (12-target, 2 s: 69.87 ± 0.74 vs. 69.76 ± 0.58 bits min−1, 40-target, 4 s: 57.02 ± 2.53 vs. 60.79 ± 1.08 bits min−1) and improved user experience (better comfortable level, weaker flicker perception and higher preference level) compared to the traditional Flicker-500% stimulus in both multi-targets BCI spellers. Significance. Selective activation of the negative contrast information processing pathway using the new OFF-50% grid stimulus evoked robust SSVEP responses. On this basis, high-performance and user-friendly SSVEP-based BCIs have been developed and implemented, which has important theoretical significance and application value in promoting the development of the visual BCI technology.

A new benchmark dataset for P300 ERP-based BCI applications

Because of its non-invasive nature, one of the most commonly used event-related potentials in brain-computer interface (BCI) system designs is the P300 electroencephalogram (EEG) signal. The fact that the P300 response can easily be stimulated and measured is particularly important for participants with severe motor disabilities. In order to train and test P300-based BCI speller systems in more realistic high-speed settings, there is a pressing need for a large and challenging benchmark dataset. Various datasets already exist in the literature but most of them are not publicly available, and they either have a limited number of participants or utilize relatively long stimulus duration (SD) and inter-stimulus intervals (ISI). They are also typically based on a 36 target (6×6) character matrix. The use of long ISI, in particular, not only reduces the speed and the information transfer rates (ITRs) but also oversimplifies the P300 detection. This leaves a limited challenge to state-of-the-art machine learning and signal processing algorithms. In fact, near-perfect P300 classification accuracies are reported with the existing datasets. Therefore, one certainly needs a large-scale dataset with challenging settings to fully exploit the recent advancements in algorithm design (machine learning and signal processing) and achieve high-performance speller results. To this end, in this article we introduce a new freely- and publicly-accessible P300 dataset obtained using 32-channel EEG, in the hope that it will lead to new research findings and eventually more efficient BCI designs. The introduced dataset comprises 18 participants performing a 40-target (5×8) cued-spelling task, with reduced SD (66.6 ms) and ISI (33.3 ms) for fast spelling. We have also processed, analyzed, and character-classified the introduced dataset and we presented the accuracy and ITR results as a benchmark. The introduced dataset and the codes of our experiments are publicly accessible at https://data.mendeley.com/datasets/vyczny2r4w.

A deep learning method for classification of steady-state visual evoked potentials in a brain-computer interface speller

ABSTRACT The main drawback of SSVEP-based BCIs is the lack of a classifier that categorizes SSVEPs with high accuracy and Information Transfer Rate (ITR). Addressing this, we proposed a deep convolutional neural network (CNN) for classifying a 40-class SSVEP. Time windows of length 2 and 3.5 seconds were used for training and testing the model by leave-one-subject-out cross-validation, using nine, three, and single-channel EEG. The proposed model reached 88.5% average accuracy for the nine-channel EEG with the mean and max ITR of 72 and 91.23 bpm, respectively. It outperformed the previous deep learning methods for SSVEP-based BCIs, in terms of accuracy and ITR. In the three-channel experiment the mean accuracy and ITR were 76.02% and 40.1 bpm. In single-channel implementation, O1 channel achieved 77.38 % average accuracy (highest) and the mean ITR was 57.51 bpm. The model showed promising performance to put this technology forward and make it more practical.

A flexible speller based on time-space frequency conversion SSVEP stimulation paradigm under dry electrode.

Speller is the best way to express the performance of the brain-computer interface (BCI) paradigm. Due to its advantages of short analysis time and high accuracy, the SSVEP paradigm has been widely used in the BCI speller system based on the wet electrode. It is widely known that the wet electrode operation is cumbersome and that the subjects have a poor experience. In addition, in the asynchronous SSVEP system based on threshold analysis, the system flickers continuously from the beginning to the end of the experiment, which leads to visual fatigue. The dry electrode has a simple operation and provides a comfortable experience for subjects. The EOG signal can avoid the stimulation of SSVEP for a long time, thus reducing fatigue. This study first designed the brain-controlled switch based on continuous blinking EOG signal and SSVEP signal to improve the flexibility of the BCI speller. Second, in order to increase the number of speller instructions, we designed the time-space frequency conversion (TSFC) SSVEP stimulus paradigm by constantly changing the time and space frequency of SSVEP sub-stimulus blocks, and designed a speller in a dry electrode environment. Seven subjects participated and completed the experiments. The results showed that the accuracy of the brain-controlled switch designed in this study was up to 94.64%, and all the subjects could use the speller flexibly. The designed 60-character speller based on the TSFC-SSVEP stimulus paradigm has an accuracy rate of 90.18% and an information transmission rate (ITR) of 117.05 bits/min. All subjects can output the specified characters in a short time. This study designed and implemented a multi-instruction SSVEP speller based on dry electrode. Through the combination of EOG and SSVEP signals, the speller can be flexibly controlled. The frequency of SSVEP stimulation sub-block is recoded in time and space by TSFC-SSVEP stimulation paradigm, which greatly improves the number of output instructions of BCI system in dry electrode environment. This work only uses FBCCA algorithm to test the stimulus paradigm, which requires a long stimulus time. In the future, we will use trained algorithms to study stimulus paradigm to improve its overall performance.

Transfer learning of an ensemble of DNNs for SSVEP BCI spellers without user-specific training

Objective. Steady-state visually evoked potentials (SSVEPs), measured with electroencephalogram (EEG), yield decent information transfer rates (ITRs) in brain-computer interface (BCI) spellers. However, the current high performing SSVEP BCI spellers in the literature require an initial lengthy and tiring user-specific training for each new user for system adaptation, including data collection with EEG experiments, algorithm training and calibration (all are before the actual use of the system). This impedes the widespread use of BCIs. To ensure practicality, we propose a novel target identification method based on an ensemble of deep neural networks (DNNs), which does not require any sort of user-specific training. Approach. We exploit already-existing literature datasets from participants of previously conducted EEG experiments to train a global target identifier DNN first, which is then fine-tuned to each participant. We transfer this ensemble of fine-tuned DNNs to the new user instance, determine the k most representative DNNs according to the participants’ statistical similarities to the new user, and predict the target character through a weighted combination of the ensemble predictions. Main results. The proposed method significantly outperforms all the state-of-the-art alternatives for all stimulation durations in [0.2–1.0] s on two large-scale benchmark and BETA datasets, and achieves impressive 155.51 bits/min and 114.64 bits/min ITRs. Code is available for reproducibility: https://github.com/osmanberke/Ensemble-of-DNNs. Significance. Our Ensemble-DNN method has the potential to promote the practical widespread deployment of BCI spellers in daily lives as we provide the highest performance while enabling the immediate system use without any user-specific training.

ST-CapsNet: Linking Spatial and Temporal Attention with Capsule Network for P300 Detection Improvement.

A brain-computer interface (BCI), which provides an advanced direct human-machine interaction, has gained substantial research interest in the last decade for its great potential in various applications including rehabilitation and communication. Among them, the P300-based BCI speller is a typical application that is capable of identifying the expected stimulated characters. However, the applicability of the P300 speller is hampered for the low recognition rate partially attributed to the complex spatio-temporal characteristics of the EEG signals. Here, we developed a deep-learning analysis framework named ST-CapsNet to overcome the challenges regarding better P300 detection using a capsule network with both spatial and temporal attention modules. Specifically, we first employed spatial and temporal attention modules to obtain refined EEG signals by capturing event-related information. Then the obtained signals were fed into the capsule network for discriminative feature extraction and P300 detection. In order to quantitatively assess the performance of the proposed ST-CapsNet, two publicly-available datasets (i.e., Dataset IIb of BCI Competition 2003 and Dataset II of BCI Competition III) were applied. A new metric of averaged symbols under repetitions (ASUR) was adopted to evaluate the cumulative effect of symbol recognition under different repetitions. In comparison with several widely-used methods (i.e., LDA, ERP-CapsNet, CNN, MCNN, SWFP, and MsCNN-TL-ESVM), the proposed ST-CapsNet framework significantly outperformed the state-of-the-art methods in terms of ASUR. More interestingly, the absolute values of the spatial filters learned by ST-CapsNet are higher in the parietal lobe and occipital region, which is consistent with the generation mechanism of P300.

Small Data Least-Squares Transformation (sd-LST) for Fast Calibration of SSVEP-Based BCIs.

Steady-state visual evoked potential (SSVEP) is one of the most popular brain-computer interface (BCI) paradigms, with high information transmission rate and signal-to-noise ratio. Many calibration-free and calibration-based approaches have been proposed to improve the performance of SSVEP-based BCIs. This paper considers a quick calibration scenario, where there are plenty of data from multiple source subjects, but only a small number of calibration trials from a subset of stimulus frequencies for the new subject. We propose small data least-squares transformation (sd-LST) to solve this problem. Experiments on three publicly available SSVEP datasets demonstrated that sd-LST outperformed several classical or state-of-the-art approaches, with only about 10 calibration trials for 40-target SSVEP-based BCI spellers.

Eliminating or Shortening the Calibration for a P300 Brain-Computer Interface Based on a Convolutional Neural Network and Big Electroencephalography Data: An Online Study.

A brain-computer interface (BCI) measures and analyzes brain activity and converts it into computer commands to control external devices. Traditional BCIs usually require full calibration, which is time-consuming and makes BCI systems inconvenient to use. In this study, we propose an online P300 BCI spelling system with zero or shortened calibration based on a convolutional neural network (CNN) and big electroencephalography (EEG) data. Specifically, three methods are proposed to train CNNs for the online detection of P300 potentials: (i) training a subject-independent CNN with data collected from 150 subjects; (ii) adapting the CNN online via a semisupervised learning/self-training method based on unlabeled data collected during the user's online operation; and (iii) fine-tuning the CNN with a transfer learning method based on a small quantity of labeled data collected before the user's online operation. Note that the calibration process is eliminated in the first two methods and dramatically shortened in the third method. Based on these methods, an online P300 spelling system is developed. Twenty subjects participated in our online experiments. Average accuracies of 89.38%, 94.00% and 93.50% were obtained by the subject-independent CNN, the self-training-based CNN and the transfer learning-based CNN, respectively. These results demonstrate the effectiveness of our methods, and thus, the convenience of the online P300-based BCI system is substantially improved.

Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation.

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.