Classification Of Event-related Potentials Research Articles

A framework for Interpretable deep learning in cross-subject detection of event-related potentials

Event-related potential-based Brain-Computer Interfaces are becoming widely popular due to their ability to send commands quickly with high accuracy. However, the stationary characteristics of electroencephalographic signals, coupled with their low signal-to-noise ratio, lead to variations in amplitude, time period, and latency in the patterns of event-related potentials across different trials, sessions, days and subjects. Conventional feature extraction and machine learning algorithms are not designed to handle these differences, requiring the development of methods that can address these variations. Here, we propose a novel lightweight deep neural network for event-related potential classification, consisting of three modules. In this model, we have a spatio-temporal module that learns local features simultaneously across channels and time points. Following this, there's a component extractor module comprising depthwise convolutions, inspired by mixed depthwise convolutions, to capture the event-related potential characteristics with different temporal durations. Lastly, an advanced temporal layer addresses event-related potential shape and scale variations using deformable convolutions. We conducted experiments on event-related potential detection in a subject-independent scenario using one error-related negativity potential dataset and three perturbation-evoked potential datasets. Comparisons were made with established methods including two conventional machine learning algorithms and three well-known deep learning architectures, demonstrating that our model outperformed them in terms of classification accuracy and parameter efficiency. In our analysis, we aimed to understand the model's performance using gradient-weighted class activation mapping and t-distributed stochastic neighbor embedding. These methods facilitated the visualization and interpretation of our model's effectiveness, providing insights into its relationship with the neuroscientific characteristics of event-related potentials.

Event-related potentials emotion features of sleep-deprived and non-sleep-deprived drivers: a preliminary study

This paper explains the study of Event-Related Potentials (ERP) emotion features related to sleep-deprived and non-sleep-deprived drivers. The study's research protocol consists of four stages - initial brainwave recording, four one-minute emotional stimulation experiments for happy, fear, sad, and calm, three sets of driving tests, and baseline brainwave recording. The data collected is pre-processed and then classified using Binary Linear Regression (BLR). The BLR classification of emotion-based ERP for sleep-deprived and non-sleep-deprived signals shows 86.1% accuracy. Though this result is still at the preliminary stage, it concluded that emotion-based ERPs can be used as promising features of brain signals to detect and classify the sleep-deprived and non-sleep-deprived.

MOCNN: A Multiscale Deep Convolutional Neural Network for ERP-Based Brain-Computer Interfaces.

Event-related potentials (ERPs) reflect neurophysiological changes of the brain in response to external events and their associated underlying complex spatiotemporal feature information is governed by ongoing oscillatory activity within the brain. Deep learning methods have been increasingly adopted for ERP-based brain-computer interfaces (BCIs) due to their excellent feature representation abilities, which allow for deep analysis of oscillatory activity within the brain. Features with higher spatiotemporal frequencies usually represent detailed and localized information, while features with lower spatiotemporal frequencies usually represent global structures. Mining EEG features from multiple spatiotemporal frequencies is conducive to obtaining more discriminative information. A multiscale feature fusion octave convolution neural network (MOCNN) is proposed in this article. MOCNN divides the ERP signals into high-, medium- and low-frequency components corresponding to different resolutions and processes them in different branches. By adding mid- and low-frequency components, the feature information used by MOCNN can be enriched, and the required amount of calculations can be reduced. After successive feature mapping using temporal and spatial convolutions, MOCNN realizes interactive learning among different components through the exchange of feature information among branches. Classification is accomplished by feeding the fused deep spatiotemporal features from various components into a fully connected layer. The results, obtained on two public datasets and a self-collected ERP dataset, show that MOCNN can achieve state-of-the-art ERP classification performance. In this study, the generalized concept of octave convolution is introduced into the field of ERP-BCI research, which allows effective spatiotemporal features to be extracted from multiscale networks through branch width optimization and information interaction at various scales.

Analysis and Classification of Event-Related Potentials During Image Observation.

In the field of cognitive neuroscience, researchers have conducted extensive studies on object categorization using Event-Related Potential (ERP) analysis, specifically by analyzing electroencephalographic (EEG) response signals triggered by visual stimuli. The most common approach for visual ERP analysis is to use a low presentation rate of images and an active task where participants actively discriminate between target and non-target images. However, researchers are also interested in understanding how the human brain processes visual information in real-world scenarios. To simulate real-life object recognition, this study proposes an analysis pipeline of visual ERPs evoked by images presented in a Rapid Serial Visual Presentation (RSVP) paradigm. Such an approach allows for the investigation of recurrent patterns of visual ERP signals across specific categories and subjects. The pipeline includes segmentation of the EEGs in epochs, and the use of the resulting features as inputs for Support Vector Machine (SVM) classification. Results demonstrate common ERP patterns across the selected categories and the ability to obtain discriminant information from single visual stimuli presented in the RSVP paradigm.

Multidomain Convolution Neural Network Models for Improved Event-Related Potential Classification.

Two convolution neural network (CNN) models are introduced to accurately classify event-related potentials (ERPs) by fusing frequency, time, and spatial domain information acquired from the continuous wavelet transform (CWT) of the ERPs recorded from multiple spatially distributed channels. The multidomain models fuse the multichannel Z-scalograms and the V-scalograms, which are generated from the standard CWT scalogram by zeroing-out and by discarding the inaccurate artifact coefficients that are outside the cone of influence (COI), respectively. In the first multidomain model, the input to the CNN is generated by fusing the Z-scalograms of the multichannel ERPs into a frequency-time-spatial cuboid. The input to the CNN in the second multidomain model is formed by fusing the frequency-time vectors of the V-scalograms of the multichannel ERPs into a frequency-time-spatial matrix. Experiments are designed to demonstrate (a) customized classification of ERPs, where the multidomain models are trained and tested with the ERPs of individual subjects for brain-computer interface (BCI)-type applications, and (b) group-based ERP classification, where the models are trained on the ERPs from a group of subjects and tested on single subjects not included in the training set for applications such as brain disorder classification. Results show that both multidomain models yield high classification accuracies for single trials and small-average ERPs with a small subset of top-ranked channels, and the multidomain fusion models consistently outperform the best unichannel classifiers.

Subject-Independent Classification of P300 Event-Related Potentials Using a Small Number of Training Subjects

The intersubject variability present in electroencephalographic (EEG) signals can affect the performance of the brain–computer interface (BCI) systems. Despite the significant progress in the field, the variability in neural data remains one of the most critical challenges in constructing accurate predictive models of human intention. As a result, the majority of the previous studies have focused either on devising subject-specific signal processing and machine learning algorithms, used some data from a target user to update and calibrate a pretrained classifier, or have used data collected from a relatively large number of training subjects to construct generic classifiers for new subjects. In this work, we investigate the feasibility of using a relatively small number of training subjects to achieve subject-independent classification of event-related potentials (ERPs) in P300-based BCIs. To this end, we employ convolutional neural networks (CNNs) and propose a leave-one-subject-out cross-validation (LOSO-CV) for model selection; that is to say, for tuning CNN hyperparameters including number of layers, filters, kernel size, and epoch. The utility of the proposed model selection is warranted because LOSO-CV simulates the effect of subject-independent classification within the training data. The entire process of training (including model selection) is validated by applying another LOSO-CV external to the training process. Our empirical results obtained on four publicly available datasets confirm the capability of LOSO-CV model selection with CNN to capture intrinsic ERP features from a small group of subjects to classify observations collected from unseen subjects.

Review of training-free event-related potential classification approaches in the World Robot Contest 2021

Recently, rapid serial visual presentation (RSVP), as a new event- related potential (ERP) paradigm, has become one of the most popular forms in electroencephalogram signal processing technologies. Several improvement approaches have been proposed to improve the performance of RSVP analysis. In brain–computer interface systems based on RSVP, the family of approaches that do not depend on training specific parameters is essential. The participating teams proposed several effective training-free frameworks of algorithms in the ERP competition of the BCI Controlled Robot Contest in World Robot Contest 2021. This paper discusses the effectiveness of various approaches in improving the performance of the system without requiring training and suggests how to apply these approaches in a practical system. First, appropriate preprocessing techniques will greatly improve the results. Then, the non-deep learning algorithm may be more stable than the deep learning approach. Furthermore, ensemble learning can make the model more stable and robust.

Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming

The usability of EEG-based visual brain–computer interfaces (BCIs) based on event-related potentials (ERPs) benefits from reducing the calibration time before BCI operation. Linear decoding models, such as the spatiotemporal beamformer model, yield state-of-the-art accuracy. Although the training time of this model is generally low, it can require a substantial amount of training data to reach functional performance. Hence, BCI calibration sessions should be sufficiently long to provide enough training data. This work introduces two regularized estimators for the beamformer weights. The first estimator uses cross-validated L2-regularization. The second estimator exploits prior information about the structure of the EEG by assuming Kronecker–Toeplitz-structured covariance. The performances of these estimators are validated and compared with the original spatiotemporal beamformer and a Riemannian-geometry-based decoder using a BCI dataset with P300-paradigm recordings for 21 subjects. Our results show that the introduced estimators are well-conditioned in the presence of limited training data and improve ERP classification accuracy for unseen data. Additionally, we show that structured regularization results in lower training times and memory usage, and a more interpretable classification model.

Hierarchical Classification of Event-Related Potentials for the Recognition of Gender Differences in the Attention Task.

Research on the functioning of human cognition has been a crucial problem studied for years. Electroencephalography (EEG) classification methods may serve as a precious tool for understanding the temporal dynamics of human brain activity, and the purpose of such an approach is to increase the statistical power of the differences between conditions that are too weak to be detected using standard EEG methods. Following that line of research, in this paper, we focus on recognizing gender differences in the functioning of the human brain in the attention task. For that purpose, we gathered, analyzed, and finally classified event-related potentials (ERPs). We propose a hierarchical approach, in which the electrophysiological signal preprocessing is combined with the classification method, enriched with a segmentation step, which creates a full line of electrophysiological signal classification during an attention task. This approach allowed us to detect differences between men and women in the P3 waveform, an ERP component related to attention, which were not observed using standard ERP analysis. The results provide evidence for the high effectiveness of the proposed method, which outperformed a traditional statistical analysis approach. This is a step towards understanding neuronal differences between men’s and women’s brains during cognition, aiming to reduce the misdiagnosis and adverse side effects in underrepresented women groups in health and biomedical research.

Use of Empirical Mode Decomposition in ERP Analysis to Classify Familial Risk and Diagnostic Outcomes for Autism Spectrum Disorder.

Event-related potentials (ERPs) activated by faces and gaze processing are found in individuals with autism spectrum disorder (ASD) in the early stages of their development and may serve as a putative biomarker to supplement behavioral diagnosis. We present a novel approach to the classification of visual ERPs collected from 6-month-old infants using intrinsic mode functions (IMFs) derived from empirical mode decomposition (EMD). Selected features were used as inputs to two machine learning methods (support vector machines and k-nearest neighbors (k-NN)) using nested cross validation. Different runs were executed for the modelling and classification of the participants in the control and high-risk (HR) groups and the classification of diagnosis outcome within the high-risk group: HR-ASD and HR-noASD. The highest accuracy in the classification of familial risk was 88.44%, achieved using a support vector machine (SVM). A maximum accuracy of 74.00% for classifying infants at risk who go on to develop ASD vs. those who do not was achieved through k-NN. IMF-based extracted features were highly effective in classifying infants by risk status, but less effective by diagnostic outcome. Advanced signal analysis of ERPs integrated with machine learning may be considered a first step toward the development of an early biomarker for ASD.

Improving Covariance Matrices Derived from Tiny Training Datasets for the Classification of Event-Related Potentials with Linear Discriminant Analysis

Electroencephalogram data used in the domain of brain–computer interfaces typically has subpar signal-to-noise ratio and data acquisition is expensive. An effective and commonly used classifier to discriminate event-related potentials is the linear discriminant analysis which, however, requires an estimate of the feature distribution. While this information is provided by the feature covariance matrix its large number of free parameters calls for regularization approaches like Ledoit–Wolf shrinkage. Assuming that the noise of event-related potential recordings is not time-locked, we propose to decouple the time component from the covariance matrix of event-related potential data in order to further improve the estimates of the covariance matrix for linear discriminant analysis. We compare three regularized variants thereof and a feature representation based on Riemannian geometry against our proposed novel linear discriminant analysis with time-decoupled covariance estimates. Extensive evaluations on 14 electroencephalogram datasets reveal, that the novel approach increases the classification performance by up to four percentage points for small training datasets, and gracefully converges to the performance of standard shrinkage-regularized LDA for large training datasets. Given these results, practitioners in this field should consider using our proposed time-decoupled covariance estimation when they apply linear discriminant analysis to classify event-related potentials, especially when few training data points are available.

A Low-Cost Computational Method for Characterizing Event-Related Potentials for BCI Applications and Beyond

Event-related potentials (ERPs) are important neurophysiological markers widely used in scientific, medical and engineering contexts. Proper ERP detection contributes to widening the scope of use and, in general, improving functionality. The morphology and latency of ERPs are variable among subject sessions, which complicates their detection. Although variability is an intrinsic feature of neuronal activity, it can be addressed with novel views on ERP detection techniques. In this paper, we propose an agile method for characterizing and thus detecting variable ERPs, which keeps track of their temporal and spatial information through the continuous measurement of the area under the curve in ERP components. We illustrate the usefulness of the proposed ERP characterization for electrode selection in brain-computer interfaces (BCIs) and compare the results with other standard methods. We assess ERP classification for BCI use with Bayesian linear discriminant analysis (BLDA) and cross-validation. We also evaluate performance with both the information transfer rate and BCI utility. The results of our validation tests show that this characterization helps to take advantage of the information on the evolution of positive and negative ERP components and, therefore, to efficiently select electrodes for optimized ERP detection. The proposed method improves the classification accuracy and bitrate of all sets of electrodes analyzed. Furthermore, the method is robust between different day sessions. Our work contributes to the efficient detection of ERPs, manages inter- and intrasubject variability, decreases the computational cost of classic detection methods and contributes to promoting low-cost personalized brain-computer interfaces.

Asynchronous Control of ERP-Based BCI Spellers Using Steady-State Visual Evoked Potentials Elicited by Peripheral Stimuli.

Brain-computer interface (BCI) spellers based on event related potentials (ERPs) are intrinsically synchronous systems. Therefore, selections are constantly made, even when users are not paying attention to the stimuli. This poses a major limitation in real-life applications, in which an asynchronous control is required. The aim of this study is to design, develop and test a novel method to discriminate whether the user is controlling the system (i.e., control state) or is engaged in other task (i.e., non-control state). To achieve such an asynchronous control, our method detects the steady-state visual evoked potentials (SSVEPs) elicited by peripheral stimuli of ERP-based spellers. A characterization experiment was conducted with 5 subjects to investigate general aspects of this phenomenon. Then, the proposed method was validated with 15 subjects in offline and online sessions. Results show that the proposed method provides a reliable asynchronous control, achieving an average accuracy of 95.5% for control state detection during the online sessions. Furthermore, our approach is independent of the ERP classification stage, and to the best of our knowledge, is the first procedure that does not need to extend the duration of the calibration sessions to acquire non-control observations.

Regularized Group Sparse Discriminant Analysis for P300-Based Brain-Computer Interface.

Event-related potentials (ERPs) especially P300 are popular effective features for brain-computer interface (BCI) systems based on electroencephalography (EEG). Traditional ERP-based BCI systems may perform poorly for small training samples, i.e. the undersampling problem. In this study, the ERP classification problem was investigated, in particular, the ERP classification in the high-dimensional setting with the number of features larger than the number of samples was studied. A flexible group sparse discriminative analysis algorithm based on Moreau-Yosida regularization was proposed for alleviating the undersampling problem. An optimization problem with the group sparse criterion was presented, and the optimal solution was proposed by using the regularized optimal scoring method. During the alternating iteration procedure, the feature selection and classification were performed simultaneously. Two P300-based BCI datasets were used to evaluate our proposed new method and compare it with existing standard methods. The experimental results indicated that the features extracted via our proposed method are efficient and provide an overall better P300 classification accuracy compared with several state-of-the-art methods.

3D Convolutional Neural Networks for Event-Related Potential detection.

Deep learning techniques have recently been successful in the classification of brain evoked responses for multiple applications, including brain-machine interface. Single-trial detection in the electroencephalogram (EEG) of brain evoked responses, like event-related potentials (ERPs), requires multiple processing stages, in the spatial and temporal domains, to extract high level features. Convolutional neural networks, as a type of deep learning method, have been used for EEG signal detection as the underlying structure of the EEG signal can be included in such system, facilitating the learning step. The EEG signal is typically decomposed into 2 main dimensions: space and time. However, the spatial dimension can be decomposed into 2 dimensions that better represent the relationships between the sensors that are involved in the classification. We propose to analyze the performance of 2D and 3D convolutional neural networks for the classification of ERPs with a dataset based on 64 EEG channels. We propose and compare 6 conv net architectures: 4 using 3D convolutions, that vary in relation to the number of layers and feature maps, and 2 using 2D convolutions. The results support the conclusion that 3D convolutions provide better performance than 2D convolutions for the binary classification of ERPs.

Cardiac afferent activity modulates early neural signature of error detection during skilled performance

Behavioral adaptations during performance rely on predicting and evaluating the consequences of our actions through action monitoring. Previous studies revealed that proprioceptive and exteroceptive signals contribute to error-monitoring processes, which are implemented in the posterior medial frontal cortex. Interestingly, errors also trigger changes in autonomic nervous system activity such as pupil dilation or heartbeat deceleration. Yet, the contribution of implicit interoceptive signals of bodily states to error-monitoring during ongoing performance has been overlooked. This study investigated whether cardiovascular interoceptive signals influence the neural correlates of error processing during performance, with an emphasis on the early stages of error processing. We recorded musicians’ electroencephalography and electrocardiogram signals during the performance of highly-trained music pieces. Previous event-related potential (ERP) studies revealed that pitch errors during skilled musical performance are preceded by an error detection signal, the pre-error-negativity (preERN), and followed by a later error positivity (PE). In this study, by combining ERP, source localization and multivariate pattern classification analysis, we found that the error-minus-correct ERP waveform had an enhanced amplitude within 40–100 ms following errors in the systolic period of the cardiac cycle. This component could be decoded from single-trials, was dissociated from the preERN and PE, and stemmed from the inferior parietal cortex, which is a region implicated in cardiac autonomic regulation. In addition, the phase of the cardiac cycle influenced behavioral alterations resulting from errors, with a smaller post-error slowing and less perturbed velocity in keystrokes following pitch errors in the systole relative to the diastole phase of the cardiac cycle. Lastly, changes in the heart rate anticipated the upcoming occurrence of errors. This study provides the first evidence of preconscious visceral information modulating neural and behavioral responses related to early error monitoring during skilled performance.

Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approach.

This paper targets a major challenge in developing practical electroencephalogram (EEG)-based brain-computer interfaces (BCIs): how to cope with individual differences so that better learning performance can be obtained for a new subject, with minimum or even no subject-specific data? We propose a novel approach to align EEG trials from different subjects in the Euclidean space to make them more similar, and hence improve the learning performance for a new subject. Our approach has three desirable properties: first, it aligns the EEG trials directly in the Euclidean space, and any signal processing, feature extraction, and machine learning algorithms can then be applied to the aligned trials; second, its computational cost is very low; and third, it is unsupervised and does not need any label information from the new subject. Both offline and simulated online experiments on motor imagery classification and event-related potential classification verified that our proposed approach outperformed a state-of-the-art Riemannian space data alignment approach, and several approaches without data alignment. The proposed Euclidean space EEG data alignment approach can greatly facilitate transfer learning in BCIs. Our proposed approach is effective, efficient, and easy to implement. It could be an essential pre-processing step for EEG-based BCIs.

Multivariate weighted recurrence network analysis of EEG signals from ERP-based smart home system.

Smart home has been widely used to improve the living quality of people. Recently, the brain-computer interface (BCI) contributes greatly to the smart home system. We design a BCI-based smart home system, in which the event-related potentials (ERP) are induced by the image interface based on the oddball paradigm. Then, we investigate the influence of mental fatigue on the ERP classification by the Fisher linear discriminant analysis. The results indicate that the classification accuracy of ERP decreases as the brain evolves from the normal stage to the mental fatigue stage. In order to probe into the difference of the brain, cognitive process between mental fatigue and normal states, we construct multivariate weighted recurrence networks and analyze the variation of the weighted clustering coefficient and weighted global efficiency corresponding to these two brain states. The findings suggest that these two network metrics allow distinguishing normal and mental fatigue states and yield novel insights into the brain fatigue behavior resulting from a long use of the ERP-based smart home system. These properties render the multivariate recurrence network, particularly useful for analyzing electroencephalographic recordings from the ERP-based smart home system.

Data-Adaptive Spatiotemporal ERP Cleaning for Single-Trial BCI Implementation.

This paper presents a data-adaptive approach to enhance the discriminative information of event-related potential (ERP) for the implementation of a brain-computer interface (BCI). The use of single-trial ERP in a real-time BCI application is challenging, due to its inherent noise contamination. Usually, multiple-trial ERPs are averaged to derive discriminative features of different classes by reducing their noise effects. Time-domain filtering is implemented here using an array wavelet transform. Sometimes, several channels can carry the signals, which are irrelevant to actual EPR information against the respective stimuli. A spatial filtering method based on clustering is introduced, to suppress such channels if any. Hence, the single-trial ERP is filtered in both the spatial and temporal domains to improve its discriminative features. The spatial-temporal discriminate analysis is employed to derive the features leading to the performance of target and non-target classification by using linear discriminant analysis. The proposed method is validated using a data set recorded from our experiments. The experimental results show that the performance of the proposed method is superior to that of the recently developed algorithms for single-trial ERP classification.

Online and Offline Domain Adaptation for Reducing BCI Calibration Effort

Many real-world brain-computer interface (BCI) applications rely on single-trial classification of event-related potentials (ERPs) in EEG signals. However, because different subjects have different neural responses to even the same stimulus, it is very difficult to build a generic ERP classifier whose parameters fit all subjects. The classifier needs to be calibrated for each individual subject, using some labeled subject-specific data. This paper proposes both online and offline weighted adaptation regularization (wAR) algorithms to reduce this calibration effort, i.e., to minimize the amount of labeled subject-specific EEG data required in BCI calibration, and hence to increase the utility of the BCI system. We demonstrate using a visually-evoked potential oddball task and three different EEG headsets that both online and offline wAR algorithms significantly outperform several other algorithms. Moreover, through source domain selection, we can reduce their computational cost by about 50%, making them more suitable for real-time applications.