Best practice for single-trial detection of event-related potentials: Application to brain-computer interfaces

The event-related potential (ERP) is the brain response measured in electroencephalography (EEG), which reflects the process of human cognitive activity. ERP has been introduced into brain computer interfaces (BCIs) to communicate the computer with the subject's intention. Due to the low signal-to-noise ratio of EEG, most ERP studies are based on grand-averaging over many trials. Recently single-trial ERP detection attracts more attention, which enables real time processing tasks as rapid face identification. All the targets needed to be retrieved may appear only once, and there is no knowledge of target label for averaging. More interestingly, how the features contribute temporally and spatially to single-trial ERP detection has not been fully investigated. In this paper, we propose to implement a local-learning-based (LLB) feature extraction method to investigate the importance of spatial-temporal components of ERP in a task of rapid face identification using single-trial detection. Comparing to previous methods, LLB method preserves the nonlinear structure of EEG signal distribution, and analyze the importance of original spatial-temporal components via optimization in feature space. As a data-driven methods, the weighting of the spatial-temporal component does not depend on the ERP detection method. The importance weights are optimized by making the targets more different from non-targets in feature space, and regularization penalty is introduced in optimization for sparse weights. This spatial-temporal feature extraction method is evaluated on the EEG data of 15 participants in performing a face identification task using rapid serial visual presentation paradigm. Comparing with other methods, the proposed spatial-temporal analysis method uses sparser (only 10% of the total) features, and could achieve comparable performance (98%) of single-trial ERP detection as the whole features across different detection methods. The interesting finding is that the N250 is the earliest temporal component that contributes to single-trial ERP detection in face identification. And the importance of N250 components is more laterally distributed toward the left hemisphere. We show that using only the left N250 component over-performs the right N250 in the face identification task using single-trial ERP detection. The finding is also important in building a fast and efficient (fewer electrodes) BCI system for rapid face identification.

Toward shift invariant detection of event-related potentials in non-invasive brain-computer interface

Non-invasive Brain-Computer Interfaces (BCIs) using electroencephalography (EEG) recordings are the most common type of BCI. The detection of Event-Related Potentials (ERP) corresponding to the presentation of visual stimuli is one of the main paradigms in BCI, such as for the detection of the P300 ERP component that is used in the P300 speller. The typing speed and the information transfer rate in a BCI speller are directly related to the single-trial detection performance. It corresponds to the binary classification of brain evoked responses corresponding to the presentation of stimuli representing targets vs. non-targets. Many techniques have been proposed in the literature, ranging from shallow approaches using linear discriminant analysis to hierarchical and deep learning methods. For BCIs that require a calibration session, reducing its duration is critical for the implementation of BCIs in clinical settings. For this reason, data augmentation approaches allowing to increase the size of the training database can improve performance while keeping the same number of trials for the calibration session. In this paper, we propose to generate artificial trials based on the properties of the distribution of the signals after spatial filtering using the coloring transformation. The approach is compared with other approaches on the single-trial detection of ERPs from a public database of 8 subjects with amyotrophic lateral sclerosis. The results support the conclusion that artificial trials based on the coloring transformation can be used for training a classifier. However, they do not provide a substantial improvement then added as a data augmentation technique, compared to data augmentation using examples shifted temporally.KeywordsBrain-Computer InterfaceEvent-Related PotentialsArtificial examples

Event-related potentials (ERPs) are often used in brain-computer interfaces (BCIs) for communication or system control for enhancing or regaining control for motor-disabled persons. Especially results from single-trial EEG classification approaches for BCIs support correlations between single-trial ERP detection performance and ERP expression. Hence, BCIs can be considered as a paradigm shift contributing to new methods with strong influence on both neuroscience and clinical applications. Here, we investigate the relevance of the choice of training data and classifier transfer for the interpretability of results from single-trial ERP detection. In our experiments, subjects performed a visual-motor oddball task with motor-task relevant infrequent (targets), motor-task irrelevant infrequent (deviants), and motor-task irrelevant frequent (standards) stimuli. Under dual-task condition, a secondary senso-motor task was performed, compared to the simple-task condition. For evaluation, average ERP analysis and single-trial detection analysis with different numbers of electrodes were performed. Further, classifier transfer was investigated between simple and dual task. Parietal positive ERPs evoked by target stimuli (but not by deviants) were expressed stronger under dual-task condition, which is discussed as an increase of task emphasis and brain processes involved in task coordination and change of task set. Highest classification performance was found for targets irrespective whether all 62, 6 or 2 parietal electrodes were used. Further, higher detection performance of targets compared to standards was achieved under dual-task compared to simple-task condition in case of training on data from 2 parietal electrodes corresponding to results of ERP average analysis. Classifier transfer between tasks improves classification performance in case that training took place on more varying examples (from dual task). In summary, we showed that P300 and overlaying parietal positive ERPs can successfully be detected while subjects are performing additional ongoing motor activity. This supports single-trial detection of ERPs evoked by target events to, e.g., infer a patient's attentional state during therapeutic intervention.

Accurate detection of single-trial event-related potentials (ERPs) in the electroencephalogram (EEG) is a difficult problem that requires efficient signal processing and machine learning techniques. Supervised spatial filtering methods that enhance the discriminative information in EEG data are commonly used to improve single-trial ERP detection. We propose a convolutional neural network (CNN) with a layer dedicated to spatial filtering for the detection of ERPs and with training based on the maximization of the area under the receiver operating characteristic curve (AUC). The CNN is compared with three common classifiers: 1) Bayesian linear discriminant analysis; 2) multilayer perceptron (MLP); and 3) support vector machines. Prior to classification, the data were spatially filtered with xDAWN (for the maximization of the signal-to-signal-plus-noise ratio), common spatial pattern, or not spatially filtered. The 12 analytical techniques were tested on EEG data recorded in three rapid serial visual presentation experiments that required the observer to discriminate rare target stimuli from frequent nontarget stimuli. Classification performance discriminating targets from nontargets depended on both the spatial filtering method and the classifier. In addition, the nonlinear classifier MLP outperformed the linear methods. Finally, training based AUC maximization provided better performance than training based on the minimization of the mean square error. The results support the conclusion that the choice of the systems architecture is critical and both spatial filtering and classification must be considered together.

In non-invasive brain-computer interface (BCI), the analysis of event-related potentials (ERP) has typically focused on averaged trials, a current trend is to analyze single-trial evoked response individually with new approaches in pattern recognition and signal processing. Such single trial detection requires a robust response that can be detected in a variety task conditions. Here, we investigated the influence of target probability, a key factor known to influence the amplitude of the evoked response, on single trial target classification in a difficult rapid serial visual presentation (RSVP) task. Our classification approach for detecting target vs. non target responses, considers spatial filters obtained through the maximization of the signal to signal-plus-noise ratio, and then uses the resulting information as inputs to a Bayesian discriminant analysis. The method is evaluated across eight healthy subjects, on four probability conditions (P=0.05, 0.10, 0.25, 0.50). We show that the target probability has a statistically significant effect on both the behavioral performance and the target detection. The best mean area under the ROC curve is achieved with P=0.10, AUC=0.82. These results suggest that optimal performance of ERP detection in RSVP tasks is critically dependent on target probability.

The detection of event-related potentials (ERPs) in brain-computer interface (BCI) depends on the ability of the subject to pay attention to specific stimuli presented during the BCI task. For healthy users, a BCI shall be used as a complement to other existing devices, which involve the response to other tasks. Those tasks may impair selective attention, particularly if the stimuli have the same modality e.g. visual. It is therefore critical to analyze how single-trial detection of brain evoked response is impaired by the addition of tasks concerning the same modality. We tested 10 healthy participants using an application that has two visual target detection tasks. The first one corresponds to a rapid serial visual presentation paradigm where target detection is achieved by brain-evoked single-trial detection in the recorded electroencephalogram (EEG) signal. The second task is the detection of a visual event on a tactical map by a behavioral response. These tasks were tested individually (single task) and in parallel (dual-task). Whereas the performance of single-trial detection was not impaired between single and dual-task conditions, the behavioral performance decreased during the dual-task condition. These results quantify the performance drop that can occur in a dual-task system using both brain-evoked responses and behavioral responses.

To propose a reliable and robust Brain-Computer Interface (BCI), efficient machine learning and signal processing methods have to be used. However, it is often necessary to have a sufficient number of labeled brain responses to create a model. A large database that would represent all of the possible variabilities of the signal is not always possible to obtain, because calibration sessions have to be short. In the case of BCIs based on the detection of event-related potentials (ERPs), we propose to tackle this problem by including additional deformed patterns in the training database to increase the number of labeled brain responses. The creation of the additional deformed patterns is based on two approaches: (i) smooth deformation fields, and (ii) right and left shifted signals. The evaluation is performed with data from 10 healthy subjects participating in a P300 speller experiment. The results show that small shifts of the signal allow a better estimation of both spatial filters, and a linear classifier. The best performance, AUC=0.828 ± 0.061, is obtained by combining the smooth deformation fields and the shifts, after spatial filtering, compared to AUC=0.543 ± 0.025, without additional deformed patterns. The results support the conclusion that adding signals with small deformations can significantly improve the performance of single-trial detection when the amount of training data is limited.

Single-trial detection of event-related potentials (ERPs) with electroencephalography (EEG) signals during Rapid Serial Visual Presentation (RSVP) tasks is a difficult problem. It is also a difficult and tedious task for participants who must keep their attention throughout the total duration of the task. Long EEG experimental sessions can be boring and impact the quality of the recorded signals and the user experience as participant. It is necessary to provide tools that allow participants to better focus on the desired stimuli. Several approaches can be performed to enhance single-trial detection, including the development of machine learning techniques. In this paper, we propose to enhance the experimental conditions by adding the Lilac Chaser visual illusion. We assessed the effect of the Lilac Chaser visual illusion during an RSVP task with targets and non-targets with 10 participants with images of human faces. While the Lilac Chaser brings additional visual stimuli that can be considered as distractors during the task, the performance of single-trial detection using the area under the ROC curve as a measure of performance does not change (about 0.89). The results suggest that the Lilac Chaser can be added as a means for users to be self-aware about their attention to the task, without decreasing the performance of ERP single-trial detection.

Searching for target images in large volume imagery is a challenging problem and the rapid serial visual presentation (RSVP) triage is potentially a promising solution to the problem. RSVP triage is essentially a cortically-coupled computer vision technique that relies on single-trial detection of event-related potentials (ERP). In RSVP triage, images are shown to a subject in a rapid serial sequence. When a target image is seen by the subject, unique ERP characterized by P300 are elicited. Thus, in RSVP triage, accurate detection of such distinct ERP allows for fast searching of target images in large volume imagery. The accuracy of the distinct ERP detection in RSVP triage depends on the feature extraction method, for which the common spatial pattern analysis (CSP) was used with limited success. This paper presents a novel feature extraction method, termed common spatio-temporal pattern (CSTP), which is critical for robust single-trial detection of ERP. Unlike the conventional CSP, whereby only spatial patterns of ERP are considered, the present proposed method exploits spatial and temporal patterns of ERP separately, providing complementary spatial and temporal features for high accurate single-trial ERP detection. Numerical study using data collected from 20 subjects in RSVP triage experiments demonstrates that the proposed method offers significant performance improvement over the conventional CSP method (corrected p-value < 0.05, Pearson r = 0.64) and other competing methods in the literature. This paper further shows that the main idea of CSTP can be easily applied to other methods.

Brain Computer Interfaces (BCIs) use brain signals to communicate with the external world. The main challenges to address are speed, accuracy and adaptability. Here, a novel algorithm for P300 based BCI spelling system is presented, specifically suited for single-trial detection of Event-Related Potentials (ERPs) by combining spatial filtering and new feature extraction methods. The adaptive spatial filtering technique, axDAWN, removes the need for calibration of the system thereby improving the overall speed of the system. Besides, axDAWN enhances the P300 response to target stimuli. The wavelet decomposition and entropy of the recorded ERPs are shown to be correlated with the presence of the P300 responses. The proposed scheme is validated thoroughly in a P300 speller and provides a solution to achieve high accuracy results for single-trial detection of ERPs, being the system user independent.

Current brain-computer interfaces (BCIs) are usually based on various, often supervised, signal processing methods. The disadvantage of supervised methods is the requirement to calibrate them with recently acquired subject-specific training data. Here, we present a novel algorithm for dimensionality reduction (spatial filter), that is ideally suited for single-trial detection of event-related potentials (ERPs) and can be adapted online to a new subject to minimize or avoid calibration time. The algorithm is based on the well-known xDAWN filter, but uses generalized eigendecomposition to allow an incremental training by recursive least squares (RLS) updates of the filter coefficients. We analyze the effectiveness of the spatial filter in different transfer scenarios and combinations with adaptive classifiers. The results show that it can compensate changes due to switching between different users, and therefore allows to reuse training data that has been previously recorded from other subjects. The presented approach allows to reduce or completely avoid a calibration phase and to instantly use the BCI system with only a minor decrease of performance. The novel filter can adapt a precomputed spatial filter to a new subject and make a BCI system user independent.

A framework for rapid visual image search using single-trial brain evoked responses

Brain–computer interfacing (BCI) is a promising technique for regaining communication and control in severely paralyzed people. Many BCI implementations are based on the recognition of task-specific event-related potentials (ERP) such as P300 responses. However, because of the high signal-to-noise ratio in noninvasive brain recordings, reliable detection of single trial ERPs is challenging. Furthermore, the relevant signal is often heterogeneously distributed over several channels. In this paper, we introduce a new approach for recognizing a sequence of attended events from multi-channel brain recordings. The framework utilizes spatial filtering to reduce both noise and signal space considerably. We introduce different models that can be used to construct the spatial filter and evaluate the approach using magnetoencephalography (MEG) data involving P300 responses, recorded during a BCI experiment. Compared to the accuracy achieved in the BCI experiment performed without spatial filtering, the recognition rate increased significantly to up to 95.3% on average (SD: 5.3%). In combination with the data-driven spatial filter construction we introduce here, our framework represents a powerful method to reliably recognize a sequence of brain potentials from high-density electrophysiological data, which could greatly improve the control of BCIs.

Many brain-computer interface (BCI) classification techniques rely on a large number of labeled brain responses to create efficient classifiers. A large database representing all of the possible variability in the signal is impossible to obtain in a short period of time, and prolonged calibration times prevent efficient BCI use. We propose to improve BCIs based on the detection of event-related potentials (ERPs) in two ways. First, we increase the size of the training database by considering additional deformed trials. The creation of the additional deformed trials is based on the addition of Gaussian noise, and on the variability of the ERP latencies. Second, we exploit the variability of the ERP latencies by combining decisions across multiple deformed trials. These new methods are evaluated on data from 16 healthy subjects participating in a rapid serial visual presentation task. The results show a significant increase in the performance of single-trial detection with the addition of artificial trials, and the combination of decisions obtained from altered trials. When the number of trials to train a classifier is low, the proposed approach allows us improve performance from an AUC of 0.533±0.080 to 0.905±0.053. This improvement represents approximately an 80% reduction in classification error. These results demonstrate that artificially increasing the training dataset leads to improved single-trial detection. Calibration sessions can be shortened for BCIs based on ERP detection.