P300-based Brain-computer Interface Research Articles

EEG-based brain-computer interface methods with the aim of rehabilitating advanced stage ALS patients

Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease that leads to progressive muscle weakness and paralysis, ultimately resulting in the loss of ability to communicate and control the environment. EEG-based Brain-Computer Interface (BCI) methods have shown promise in providing communication and control with the aim of rehabilitating ALS patients. In particular, P300-based BCI has been widely studied and used for ALS rehabilitation. Other EEG-based BCI methods, such as Motor Imagery (MI) based BCI and Hybrid BCI, have also shown promise in ALS rehabilitation. Nonetheless, EEG-based BCI methods hold great potential for improvement. This review article introduces and reviews FFT, WPD, CSP, CSSP, CSP, and GC feature extraction methods. The Common Spatial Pattern (CSP) is an efficient and common technique for extracting data properties used in BCI systems. In addition, Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Neural Networks (NN), and Deep Learning (DL) classification methods were introduced and reviewed. SVM is the most appropriate classifier due to its insensitivity to the curse of dimensionality. Also, DL is used in the design of BCI systems and is a good choice for BCI systems based on motor imagery with big datasets. Despite the progress made in the field, there are still challenges to overcome, such as improving the accuracy and reliability of EEG signal detection and developing more intuitive and user-friendly interfaces By using BCI, disabled patients can communicate with their caregivers and control their environment using various devices, including wheelchairs, and robotic arms.

Comparing Several P300-Based Visuo-Auditory Brain-Computer Interfaces for a Completely Locked-in ALS Patient: A Longitudinal Case Study

Comparing Several P300-Based Visuo-Auditory Brain-Computer Interfaces for a Completely Locked-in ALS Patient: A Longitudinal Case Study

A Systematic Review of Computational Intelligence Techniques for Channel Selection in P300-Based Brain Computer Interface Speller

Electroencephalography (EEG) based P300 speller aid in restoring the communication and control capabilities in patients suffering from motor disabilities. However, the quality and quantity of the data collected from EEG recordings have a substantial influence on the P300 speller's performance. Hence, selecting the optimum number of recording electrodes, i.e., channels for each user, is a significant difficulty for the P300 speller. There are two fundamental objectives of the channel selection process: 1) To extract the most crucial information from the relevant channels, hence reducing the computing complexity of P300/Non-P300 signal processing operation, and 2) To lessen the potential overfitting that could result from using unwanted channels to boost performance. For obtaining the best channel subsets, different channel selection techniques, including manual, filtering, wrapper, and embedded approaches, have been applied by past researchers. This study presents a comprehensive review of recent developments, present status, challenges, and possible solution associated with channel selection strategies in P300 speller.

Improving the performance of P300-based BCIs by mitigating the effects of stimuli-related evoked potentials through regularized spatial filtering

Objective. the P300-based brain–computer interface (BCI) establishes a communication channel between the mind and a computer by translating brain signals into commands. These systems typically employ a visual oddball paradigm, where different objects (linked to specific commands) are randomly and frequently intensified. Upon observing the target object, users experience an elicitation of a P300 event-related potential in their electroencephalography (EEG). However, detecting the P300 signal can be challenging due to its very low signal-to-noise ratio (SNR), often compromised by the sequence of visual evoked potentials (VEPs) generated in the occipital regions of the brain in response to periodic visual stimuli. While various approaches have been explored to enhance the SNR of P300 signals, the impact of VEPs has been largely overlooked. The main objective of this study is to investigate how VEPs impact P300-based BCIs. Subsequently, the study aims to propose a method for EEG spatial filtering to alleviate the effect of VEPs and enhance the overall performance of these BCIs. Approach. our approach entails analyzing recorded EEG signals from visual P300-based BCIs through temporal, spectral, and spatial analysis techniques to identify the impact of VEPs. Subsequently, we introduce a regularized version of the xDAWN algorithm, a well-established spatial filter known for enhancing single-trial P300s. This aims to simultaneously enhance P300 signals and suppress VEPs, contributing to an improved overall signal quality. Main results. analyzing EEG signals shows that VEPs can significantly contaminate P300 signals, resulting in a decrease in the overall performance of P300-based BCIs. However, our proposed method for simultaneous enhancement of P300 and suppression of VEPs demonstrates improved performance in P300-based BCIs. This improvement is verified through several experiments conducted with real P300 data. Significance. this study focuses on the effects of VEPs on the performance of P300-based BCIs, a problem that has not been adequately addressed in previous studies. It opens up a new path for investigating these BCIs. Moreover, the proposed spatial filtering technique has the potential to further enhance the performance of these systems.

Functional electrical stimulation therapy controlled by a P300-based brain-computer interface, as a therapeutic alternative for upper limb motor function recovery in chronic post-stroke patients. A non-randomized pilot study.

Up to 80% of post-stroke patients present upper-limb motor impairment (ULMI), causing functional limitations in daily activities and loss of independence. UMLI is seldom fully recovered after stroke when using conventional therapeutic approaches. Functional Electrical Stimulation Therapy (FEST) controlled by Brain-Computer Interface (BCI) is an alternative that may induce neuroplastic changes, even in chronic post-stroke patients. The purpose of this work was to evaluate the effects of a P300-based BCI-controlled FEST intervention, for ULMI recovery of chronic post-stroke patients. A non-randomized pilot study was conducted, including 14 patients divided into 2 groups: BCI-FEST, and Conventional Therapy. Assessments of Upper limb functionality with Action Research Arm Test (ARAT), performance impairment with Fugl-Meyer assessment (FMA), Functional Independence Measure (FIM) and spasticity through Modified Ashworth Scale (MAS) were performed at baseline and after carrying out 20 therapy sessions, and the obtained scores compared using Chi square and Mann-Whitney U statistical tests (𝛼 = 0.05). After training, we found statistically significant differences between groups for FMA (p = 0.012), ARAT (p < 0.001), and FIM (p = 0.025) scales. It has been shown that FEST controlled by a P300-based BCI, may be more effective than conventional therapy to improve ULMI after stroke, regardless of chronicity. The results of the proposed BCI-FEST intervention are promising, even for the most chronic post-stroke patients often relegated from novel interventions, whose expected recovery with conventional therapy is very low. It is necessary to carry out a randomized controlled trial in the future with a larger sample of patients.

Usability of three software platforms for modifying graphical layout in visual P300-based brain-computer interface

Individuals afflicted by neurodegenerative conditions such as amyotrophic lateral sclerosis may eventually reach a point where they lose the ability to communicate with the outside world through conventional muscular pathways. In these cases, brain-computer interfaces (BCIs) can be a suitable alternative, as they directly transform brain activity into external commands. A P300-based speller is a BCI for communication purposes. The most popular software platforms used to develop spellers are BCI2000 and Open-ViBE. However, these platforms can be relatively complex to set up without advanced technical knowledge. For this reason, the UMA-BCI Speller platform was recently developed, intended to facilitate the control of the system. Hence, the objective of this research was to assess and compare the user-friendliness of the three platforms put forth. A group of fifteen participants were tasked with configuring a designated speller layout using the three platforms. The findings acquired demonstrated that the UMA-BCI Speller platform exhibited the utmost level of usability, as it required the shortest time to complete the tasks and received the best feedback in the questionnaires. Overall, there was no difference between the BCI2000 and Open-ViBE platforms. In short, the UMA-BCI Speller offered the best usability and showed itself to be an easy application to use that provides many options to configure a speller graphical layout.

Adaptive P300-Based Brain-Computer Interface for Attention Training: Protocol for a Randomized Controlled Trial.

The number of people with cognitive deficits and diseases, such as stroke, dementia, or attention-deficit/hyperactivity disorder, is rising due to an aging, or in the case of attention-deficit/hyperactivity disorder, a growing population. Neurofeedback training using brain-computer interfaces is emerging as a means of easy-to-use and noninvasive cognitive training and rehabilitation. A novel application of neurofeedback training using a P300-based brain-computer interface has previously shown potential to improve attention in healthy adults. This study aims to accelerate attention training using iterative learning control to optimize the task difficulty in an adaptive P300 speller task. Furthermore, we hope to replicate the results of a previous study using a P300 speller for attention training, as a benchmark comparison. In addition, the effectiveness of personalizing the task difficulty during training will be compared to a nonpersonalized task difficulty adaptation. In this single-blind, parallel, 3-arm randomized controlled trial, 45 healthy adults will be recruited and randomly assigned to the experimental group or 1 of 2 control groups. This study involves a single training session, where participants receive neurofeedback training through a P300 speller task. During this training, the task's difficulty is progressively increased, which makes it more difficult for the participants to maintain their performance. This encourages the participants to improve their focus. Task difficulty is either adapted based on the participants' performance (in the experimental group and control group 1) or chosen randomly (in control group 2). Changes in brain patterns before and after training will be analyzed to study the effectiveness of the different approaches. Participants will complete a random dot motion task before and after the training so that any transfer effects of the training to other cognitive tasks can be evaluated. Questionnaires will be used to estimate the participants' fatigue and compare the perceived workload of the training between groups. This study has been approved by the Maynooth University Ethics Committee (BSRESC-2022-2474456) and is registered on ClinicalTrials.gov (NCT05576649). Participant recruitment and data collection began in October 2022, and we expect to publish the results in 2023. This study aims to accelerate attention training using iterative learning control in an adaptive P300 speller task, making it a more attractive training option for individuals with cognitive deficits due to its ease of use and speed. The successful replication of the results from the previous study, which used a P300 speller for attention training, would provide further evidence to support the effectiveness of this training tool. ClinicalTrials.gov NCT05576649; https://clinicaltrials.gov/ct2/show/NCT05576649. DERR1-10.2196/46135.

Enhancing P300-Based Brain-Computer Interfaces with Hybrid Transfer Learning: A Data Alignment and Fine-Tuning Approach

A new approach is introduced to address the subject dependency problem in P300-based brain-computer interfaces (BCI) by using transfer learning. The occurrence of P300, an event-related potential, is primarily associated with changes in natural neuron activity and elicited in response to infrequent stimuli, which can be monitored non-invasively through an electroencephalogram. However, implementing P300-based BCI in real-time requires many training samples and time-consuming calibration, making it challenging to use in practical applications. To tackle these challenges, the proposed approach harnesses the high-level feature extraction capability of a deep neural network, achieved through fine-tuning. To ensure similar distributions of feature extraction data, the approach of aligning data in Euclidean space is employed, which is then applied to a discriminatively restricted Boltzmann machine with a single layer for P300 detection. The performance of the proposed method on the BCI Competition III dataset II and the BCI competition II dataset II, the state-of-the-art dataset, was evaluated and compared with previous studies. The results showed that robust performance could be achieved using a small number of training samples, demonstrating the effectiveness of the transfer learning approach in P300-based BCI applications.

A novel multiclass-based framework for P300 detection in BCI matrix speller: Temporal EEG patterns of non-target trials vary based on their position to previous target stimuli

Brain–computer interface (BCI) provides a new communication pathway for severely disabled people and enables them to communicate with external world using only their brain activity. P300-based BCI speller helps patients spell words using their brain signals.Until now, binary-classification-based approaches have been used for P300 detection. This study demonstrates that binary-classification-based approaches may not be appropriate for P300 detection. We proved that temporal EEG patterns of non-target trials are different based on their position to previous target stimuli. Therefore, considering all non-target trials in only one group makes distinguishing target from non-target components difficult for machine learning algorithms and, consequently, deteriorate character recognition accuracy. This study introduced a novel approach for P300 detection in BCI spellers. In this study, we first divided non-target trials into several groups according to their temporal patterns in training stage. Then, we introduced a multiclass-based framework for P300 detection.Proposed approach is evaluated with three public datasets, BCI competition II, BCI competition III and the BNCI Horizon. In all three datasets, proposed multi-class approach outperformed common binary-classification-based approach in the same conditions. In addition, multiclass-based approach reached the highest accuracy (100%) in the 3th sequence for BCI competition II, for BCI competition III achieved an average accuracy of 74% and 98% in 5th and 15th sequences and, for the BNCI Horizon dataset achieved an average accuracy of 77.50% and 97.49% in 5th and 10th sequences, respectively. That means our proposed approach, regardless of its simplicity, achieved state-of-the-art character recognition performance in the existing methods. The results confirmed that binary-classification based methods is not appropriate for P300 detection in BCI spellers.

Beyond passive observation: feedback anticipation and observation activate the mirror system in virtual finger movement control via P300-BCI.

Action observation (AO) is widely used as a post-stroke therapy to activate sensorimotor circuits through the mirror neuron system. However, passive observation is often considered to be less effective and less interactive than goal-directed movement observation, leading to the suggestion that observation of goal-directed actions may have stronger therapeutic potential, as goal-directed AO has been shown to activate mechanisms for monitoring action errors. Some studies have also suggested the use of AO as a form of Brain-computer interface (BCI) feedback. In this study, we investigated the potential for observation of virtual hand movements within a P300-based BCI as a feedback system to activate the mirror neuron system. We also explored the role of feedback anticipation and estimation mechanisms during movement observation. Twenty healthy subjects participated in the study. We analyzed event-related desynchronization and synchronization (ERD/S) of sensorimotor EEG rhythms and Error-related potentials (ErrPs) during observation of virtual hand finger flexion presented as feedback in the P300-BCI loop and compared the dynamics of ERD/S and ErrPs during observation of correct feedback and errors. We also analyzed these EEG markers during passive AO under two conditions: when subjects anticipated the action demonstration and when the action was unexpected. A pre-action mu-ERD was found both before passive AO and during action anticipation within the BCI loop. Furthermore, a significant increase in beta-ERS was found during AO within incorrect BCI feedback trials. We suggest that the BCI feedback may exaggerate the passive-AO effect, as it engages feedback anticipation and estimation mechanisms as well as movement error monitoring simultaneously. The results of this study provide insights into the potential of P300-BCI with AO-feedback as a tool for neurorehabilitation.

A new benchmark dataset for P300 ERP-based BCI applications

Because of its non-invasive nature, one of the most commonly used event-related potentials in brain-computer interface (BCI) system designs is the P300 electroencephalogram (EEG) signal. The fact that the P300 response can easily be stimulated and measured is particularly important for participants with severe motor disabilities. In order to train and test P300-based BCI speller systems in more realistic high-speed settings, there is a pressing need for a large and challenging benchmark dataset. Various datasets already exist in the literature but most of them are not publicly available, and they either have a limited number of participants or utilize relatively long stimulus duration (SD) and inter-stimulus intervals (ISI). They are also typically based on a 36 target (6×6) character matrix. The use of long ISI, in particular, not only reduces the speed and the information transfer rates (ITRs) but also oversimplifies the P300 detection. This leaves a limited challenge to state-of-the-art machine learning and signal processing algorithms. In fact, near-perfect P300 classification accuracies are reported with the existing datasets. Therefore, one certainly needs a large-scale dataset with challenging settings to fully exploit the recent advancements in algorithm design (machine learning and signal processing) and achieve high-performance speller results. To this end, in this article we introduce a new freely- and publicly-accessible P300 dataset obtained using 32-channel EEG, in the hope that it will lead to new research findings and eventually more efficient BCI designs. The introduced dataset comprises 18 participants performing a 40-target (5×8) cued-spelling task, with reduced SD (66.6 ms) and ISI (33.3 ms) for fast spelling. We have also processed, analyzed, and character-classified the introduced dataset and we presented the accuracy and ITR results as a benchmark. The introduced dataset and the codes of our experiments are publicly accessible at https://data.mendeley.com/datasets/vyczny2r4w.

Electromagnetic Brain–Computer–Metasurface Holography

As an important bridge between the human brain and external devices, the brain–computer interface (BCI) has received much attention for decades. To further explore the connection between the human brain signals and electromagnetic (EM) information, here we propose a brain–computer–metasurface holography (BCMH) system to enable the brain control of EM imaging. By using a P300-based BCI and a 1-bit programmable metasurface, BCMH can precisely reproduce specific EM holographic images according to the operator’s intentions. Twenty distinct images are designed and tested with BCMH, showing good consistency with the theoretical expectation. The proposed BCMH, integrating the EM manipulation capability with intelligence of the human brain, may provide a new paradigm of interactions among the human, intelligent metasurfaces, and machines, and hence promote the visual-reality (VR) technology.

Multiple Scale Convolutional Few-Shot Learning Networks for Online P300-Based Brain–Computer Interface and Its Application to Patients With Disorder of Consciousness

P300 brain-computer interfaces (BCIs) have significant potential for detecting and assessing residual consciousness in patients with disorders of consciousness (DoC) but are limited by insufficient data collected from them. In this study, a multiple scale convolutional few-shot learning network (MSCNN-FSL) was proposed to detect and recognize small sample P300 signals. A multiple scale convolutional neural network (MSCNN) was developed to learn different scale features from different scales of receptive fields to obtain more information from electroencephalograms (EEG). Then, a prototypical network with cosine distance was introduced as a classifier to classify and small sample P300 signals. The MSCNN-FSL was evaluated in two independent online BCI experiments. In the first P300 speller experiment, the presented network achieves good character recognition performance with average accuracies of 98.02%±1.70%. In the second experiment, eight healthy controls achieved photo recognition performance with average accuracies of 98.75%±1.49% and three of the twelve DoC patients achieved more than 64% online accuracies with significance. Our results indicated that the proposed MSCNN-FSL could correctly assess the three patients who may have residual consciousness but were misdiagnosed by the Coma Recovery Scale-Revised (CRS-R). Clinical evaluation after two months did prove our BCI assessment results for the three patients.

Eliminating or Shortening the Calibration for a P300 Brain-Computer Interface Based on a Convolutional Neural Network and Big Electroencephalography Data: An Online Study.

A brain-computer interface (BCI) measures and analyzes brain activity and converts it into computer commands to control external devices. Traditional BCIs usually require full calibration, which is time-consuming and makes BCI systems inconvenient to use. In this study, we propose an online P300 BCI spelling system with zero or shortened calibration based on a convolutional neural network (CNN) and big electroencephalography (EEG) data. Specifically, three methods are proposed to train CNNs for the online detection of P300 potentials: (i) training a subject-independent CNN with data collected from 150 subjects; (ii) adapting the CNN online via a semisupervised learning/self-training method based on unlabeled data collected during the user's online operation; and (iii) fine-tuning the CNN with a transfer learning method based on a small quantity of labeled data collected before the user's online operation. Note that the calibration process is eliminated in the first two methods and dramatically shortened in the third method. Based on these methods, an online P300 spelling system is developed. Twenty subjects participated in our online experiments. Average accuracies of 89.38%, 94.00% and 93.50% were obtained by the subject-independent CNN, the self-training-based CNN and the transfer learning-based CNN, respectively. These results demonstrate the effectiveness of our methods, and thus, the convenience of the online P300-based BCI system is substantially improved.

ST-CapsNet: Linking Spatial and Temporal Attention with Capsule Network for P300 Detection Improvement.

A brain-computer interface (BCI), which provides an advanced direct human-machine interaction, has gained substantial research interest in the last decade for its great potential in various applications including rehabilitation and communication. Among them, the P300-based BCI speller is a typical application that is capable of identifying the expected stimulated characters. However, the applicability of the P300 speller is hampered for the low recognition rate partially attributed to the complex spatio-temporal characteristics of the EEG signals. Here, we developed a deep-learning analysis framework named ST-CapsNet to overcome the challenges regarding better P300 detection using a capsule network with both spatial and temporal attention modules. Specifically, we first employed spatial and temporal attention modules to obtain refined EEG signals by capturing event-related information. Then the obtained signals were fed into the capsule network for discriminative feature extraction and P300 detection. In order to quantitatively assess the performance of the proposed ST-CapsNet, two publicly-available datasets (i.e., Dataset IIb of BCI Competition 2003 and Dataset II of BCI Competition III) were applied. A new metric of averaged symbols under repetitions (ASUR) was adopted to evaluate the cumulative effect of symbol recognition under different repetitions. In comparison with several widely-used methods (i.e., LDA, ERP-CapsNet, CNN, MCNN, SWFP, and MsCNN-TL-ESVM), the proposed ST-CapsNet framework significantly outperformed the state-of-the-art methods in terms of ASUR. More interestingly, the absolute values of the spatial filters learned by ST-CapsNet are higher in the parietal lobe and occipital region, which is consistent with the generation mechanism of P300.

Bayesian Uncertainty Modeling for P300-Based Brain-Computer Interface.

P300 potential is important to cognitive neuroscience research, and has also been widely applied in brain-computer interfaces (BCIs). To detect P300, many neural network models, including convolutional neural networks (CNNs), have achieved outstanding results. However, EEG signals are usually high-dimensional. Moreover, since collecting EEG signals is time-consuming and expensive, EEG datasets are typically small. Therefore, data-sparse regions usually exist within EEG dataset. However, most existing models compute predictions based on point-estimate. They cannot evaluate prediction uncertainty and tend to make overconfident decisions on samples located in data-sparse regions. Hence, their predictions are unreliable. To solve this problem, we propose a Bayesian convolutional neural network (BCNN) for P300 detection. The network places probability distributions over weights to capture model uncertainty. In prediction phase, a set of neural networks can be obtained by Monte Carlo sampling. Integrating the predictions of these networks implies ensembling. Therefore, the reliability of prediction can be improved. Experimental results demonstrate that BCNN can achieve better P300 detection performance than point-estimate networks. In addition, placing a prior distribution over the weight acts as a regularization technique. Experimental results show that it improves the robustness of BCNN to overfitting on small dataset. More importantly, with BCNN, both weight uncertainty and prediction uncertainty can be obtained. The weight uncertainty is then used to optimize the network through pruning, and the prediction uncertainty is applied to reject unreliable decisions so as to reduce detection error. Therefore, uncertainty modeling provides important information to further improve BCI systems.

An optimized deep learning approach based on autoencoder network for P300 detection in brain computer interface systems

Background. Brain computer interface (BCI) systems by extracting knowledge from brain signals provide a connection channel to the outside world for disabled people, without physiological interfaces. Event-related potentials (ERPs) are a specific type of electroencephalography signals and P300 is one of the most important ERP components. The critical part of P300-based BCI systems is classification step. In this research, an approach is proposed for P300 classification based on novel machine learning methods using convolutional neural networks (CNN) and autoencoder networks. Methods. In the pre-processing step, channel selection, data augmentation (by ADASYN method), filtering and base-line drift were done. Then, in the classification step, four different CNN classifiers including CNN1D, CNN2D, CNN1D_Autoencoder, and CNN2D-Autoencoder were used for P300 classification. Results. After implementation and tuning the networks, 92% as a best accuracy was achieved by CNN2D_Autoencoder. This result was achieved with a considerable tradeoff between complexity and stability. Conclusion. The acquired results emphasize the ability of the deep learning methods in P300 classification and approve the advantage of using them in BCI systems. Furthermore, autoencoder versions of CNN networks are more stable and have a faster convergence. Meanwhile, ADASYN is a suitable method for augmentation of P300 data and even ERPs by sustaining the premier feature space without copying data. Practical Implications. Our results can increase the accuracy of P300 detection and simultaneously reduce the volume of data using the proposed model. Consequently, they can improve character recognition in P300-speller systems generally used by amyotrophic lateral sclerosis (ALS) patients.

Subject-Independent Classification of P300 Event-Related Potentials Using a Small Number of Training Subjects

The intersubject variability present in electroencephalographic (EEG) signals can affect the performance of the brain–computer interface (BCI) systems. Despite the significant progress in the field, the variability in neural data remains one of the most critical challenges in constructing accurate predictive models of human intention. As a result, the majority of the previous studies have focused either on devising subject-specific signal processing and machine learning algorithms, used some data from a target user to update and calibrate a pretrained classifier, or have used data collected from a relatively large number of training subjects to construct generic classifiers for new subjects. In this work, we investigate the feasibility of using a relatively small number of training subjects to achieve subject-independent classification of event-related potentials (ERPs) in P300-based BCIs. To this end, we employ convolutional neural networks (CNNs) and propose a leave-one-subject-out cross-validation (LOSO-CV) for model selection; that is to say, for tuning CNN hyperparameters including number of layers, filters, kernel size, and epoch. The utility of the proposed model selection is warranted because LOSO-CV simulates the effect of subject-independent classification within the training data. The entire process of training (including model selection) is validated by applying another LOSO-CV external to the training process. Our empirical results obtained on four publicly available datasets confirm the capability of LOSO-CV model selection with CNN to capture intrinsic ERP features from a small group of subjects to classify observations collected from unseen subjects.

Leveraging Deep Learning Techniques to Improve P300-Based Brain Computer Interfaces.

Brain-Computer Interface (BCI) has become an established technology to interconnect a human brain and an external device. One of the most popular protocols for BCI is based on the extraction of the so-called P300 wave from electroencephalography (EEG) recordings. P300 wave is an event-related potential with a latency of 300 ms after the onset of a rare stimulus. In this paper, we used deep learning architectures, namely convolutional neural networks (CNNs), to improve P300-based BCIs. We propose a novel BCI classifier, called P3CNET, that improved P300 classification accuracy performances of the best state-of-the-art classifier. In addition, we explored pre-processing and training choices that improved the usability of BCI systems. For the pre-processing of EEG data, we explored the optimal signal interval that would improve classification accuracies. Then, we explored the minimum number of calibration sessions to balance higher accuracy and shorter calibration time. To improve the explainability of deep learning architectures, we analyzed the saliency maps of the input EEG signal leading to a correct P300 classification, and we observed that the elimination of less informative electrode channels from the data did not result in better accuracy. All the methodologies and explorations were performed and validated on two different CNN classifiers, demonstrating the generalizability of the obtained results. Finally, we showed the advantages given by transfer learning when using the proposed novel architecture on other P300 datasets. The presented architectures and practical suggestions can be used by BCI practitioners to improve its effectiveness.

Effects of Low Mental Energy from Long Periods of Work on Brain-Computer Interfaces.

Brain-computer interfaces (BCIs) provide novel hands-free interaction strategies. However, the performance of BCIs is affected by the user’s mental energy to some extent. In this study, we aimed to analyze the combined effects of decreased mental energy and lack of sleep on BCI performance and how to reduce these effects. We defined the low-mental-energy (LME) condition as a combined condition of decreased mental energy and lack of sleep. We used a long period of work (>=18 h) to induce the LME condition, and then P300- and SSVEP-based BCI tasks were conducted in LME or normal conditions. Ten subjects were recruited in this study. Each subject participated in the LME- and normal-condition experiments within one week. For the P300-based BCI, we used two decoding algorithms: stepwise linear discriminant (SWLDA) and least square regression (LSR). For the SSVEP-based BCI, we used two decoding algorithms: canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA). Accuracy and information transfer rate (ITR) were used as performance metrics. The experimental results showed that for the P300-based BCI, the average accuracy was reduced by approximately 35% (with a SWLDA classifier) and approximately 40% (with a LSR classifier); the average ITR was reduced by approximately 6 bits/min (with a SWLDA classifier) and approximately 7 bits/min (with an LSR classifier). For the SSVEP-based BCI, the average accuracy was reduced by approximately 40% (with a CCA classifier) and approximately 40% (with a FBCCA classifier); the average ITR was reduced by approximately 20 bits/min (with a CCA classifier) and approximately 19 bits/min (with a FBCCA classifier). Additionally, the amplitude and signal-to-noise ratio of the evoked electroencephalogram signals were lower in the LME condition, while the degree of fatigue and the task load of each subject were higher. Further experiments suggested that increasing stimulus size, flash duration, and flash number could improve BCI performance in LME conditions to some extent. Our experiments showed that the LME condition reduced BCI performance, the effects of LME on BCI did not rely on specific BCI types and specific decoding algorithms, and optimizing BCI parameters (e.g., stimulus size) can reduce these effects.