Event-related Potential Signals Research Articles

Signal alignment for cross-datasets in P300 brain-computer interfaces

Objective. Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment (SA) for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning. Approach. We proposed a linear SA that uses the P300’s latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm). Results. Although the standard approach without SA had an average precision (AP) score of 25.5%, the approach demonstrated a 35.8% AP score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets. Significance. We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.

The impact of risk preference on decision-making during evacuation: Behavioral and physiological analysis using event-related potential signals

Individuals with different cognitive and personality traits may be required to serve as evacuees during a fire or other emergencies. We investigated how the personality trait of risk preference influences the decision-making strategy of fire evacuees, and classified evacuees by different risk preferences to reveal behavioral and neurophysiological differences associated with different preference categories. Event-related potentials and reaction times were recorded as 49 healthy participants performed an emergency decision-making task that required them to choose between two escape routes by considering route distance and fire severity. Our simulated evacuation experiment revealed the participants’ risk-taking preferences during emergency evacuation. All participants were classified into three statistically different groups (risk-averse group; risk-neutral group; risk-seeking group). We found that evacuees determined their risk preference early and preferred to maintain their original decision-making strategy rather than change it frequently. Our findings indicate that, when participants selected an option against their risk preference, they demonstrated longer reaction time and used more cognitive and attentional resources. This difference is shown by the significant difference in reaction times, P300 amplitudes, and activity on topographic scalp maps. Our study is the first to identify physiological changes that correspond with different risk preferences during evacuation decision-making and explain more fully how evacuees make emergency decisions in the context of fire. Future research should examine these findings in a larger sample size.

MOCNN: A Multiscale Deep Convolutional Neural Network for ERP-Based Brain-Computer Interfaces.

Event-related potentials (ERPs) reflect neurophysiological changes of the brain in response to external events and their associated underlying complex spatiotemporal feature information is governed by ongoing oscillatory activity within the brain. Deep learning methods have been increasingly adopted for ERP-based brain-computer interfaces (BCIs) due to their excellent feature representation abilities, which allow for deep analysis of oscillatory activity within the brain. Features with higher spatiotemporal frequencies usually represent detailed and localized information, while features with lower spatiotemporal frequencies usually represent global structures. Mining EEG features from multiple spatiotemporal frequencies is conducive to obtaining more discriminative information. A multiscale feature fusion octave convolution neural network (MOCNN) is proposed in this article. MOCNN divides the ERP signals into high-, medium-and low-frequency components corresponding to different resolutions and processes them in different branches. By adding mid-and low-frequency components, the feature information used by MOCNN can be enriched, and the required amount of calculations can be reduced. After successive feature mapping using temporal and spatial convolutions, MOCNN realizes interactive learning among different components through the exchange of feature information among branches. Classification is accomplished by feeding the fused deep spatiotemporal features from various components into a fully connected layer. The results, obtained on two public datasets and a self-collected ERP dataset, show that MOCNN can achieve state-of-the-art ERP classification performance. In this study, the generalized concept of octave convolution is introduced into the field of ERP-BCI research, which allows effective spatiotemporal features to be extracted from multiscale networks through branch width optimization and information interaction at various scales.

Sensory Attenuation Deficit and Auditory Hallucinations in Schizophrenia: A Causal Mechanism or a Risk Factor? Evidence From Meta-Analyses on the N1 Event-Related Potential Component

BackgroundSensory attenuation (SA), the dampened perception of self-generated sensory information, is typically associated with reduced event-related potential signals, such as for the N1 component of auditory event-related potentials. SA, together with efficient monitoring of intentions and actions, should facilitate the distinction between self-generated and externally generated sensory events, thereby optimizing interaction with the world. According to many, SA is deficient in schizophrenia. The question arises whether altered SA reflects a sufficient mechanism to explain positive symptoms such as auditory hallucinations. A systematic association of reduced auditory SA in hallucinating patients would support this hypothesis. MethodsWe conducted a series of meta-analyses on 15 studies on auditory SA in which the N1 component of event-related potential–electroencephalogram signals was measured during talking (self-generated sensory signals condition) or when listening to prerecorded vocalizations (externally generated sensory signals condition). ResultsWe found that individuals with schizophrenia did show some auditory SA because their N1 signal was significantly attenuated in talking conditions compared with listening conditions. However, the magnitude of such attenuation was reduced in individuals with schizophrenia compared to healthy control participants. This phenomenon generalizes independently from the stage of the disease, the severity of positive symptoms, and whether patients have auditory hallucinations or not. ConclusionsThese findings suggest that reduced SA cannot be a sufficient mechanism for explaining positive symptoms such as auditory hallucinations in schizophrenia. Because reduced SA was also present in participants at risk of schizophrenia, reduced SA may represent a risk factor for the disorder. We discuss the implications of these results for clinical-cognitive models of schizophrenia.

Opinion on the event-related potential signature of automatic detection of violated regularity (visual mismatch negativity): non-perceptual but predictive.

Opinion on the event-related potential signature of automatic detection of violated regularity (visual mismatch negativity): non-perceptual but predictive.

Event-related potentials reveal visual episodic memory deficits in patients with temporal lobe epilepsy

ObjectiveTemporal lobe epilepsy (TLE) patients usually suffer from impaired episodic memory (EM), but its underlying electrophysiologic mechanism and impacted cognitive performance are unclear. We aim to investigate the association between episodic memory reserve and physiological measures of memory workload in TLE patients using Event-related potentials (ERP). MethodsA change detection task with image stimuli assesses visual episodic memory. During the memory encoding and decoding phases, the ERP signals were analyzed from twenty-nine TLE patients (twelve with left TLE patients, seventeen with TLE), and thirty healthy controls. Given that EM is a complex process involving many fundamental cognitive processes, the amplitudes and latencies of EM-related ERP (FN400, late positive potential (LPC), and late posterior negativity (LPN)), and the ERP reflecting the fundamental processes (P100, N100, P200, and P300) were calculated. Then we used a three-by-two factorial design on the ERP metrics for interaction and main effects. The correlation analysis among Wechsler Memory Scales-Chinese Revision (WMS-RC) results, behavioral data, and the ERPs was carried out. ResultsThe TLE patients performed worse in WMS-RC and the memory task. The increased P200 and decreased P300 amplitudes were observed in the TLE patients, and LPN was abnormal in only LTLE patients. For EM-related components, differences were observed in both the LTLE and RTLE patients: the lack of the FN400 effect, the lack of the reversed LPC effect, and the reduced FN400. No significant inter-group difference was detected for the latencies of all the ERPs. Additionally, there were significant correlations among WMS-RC scores, behaviors, and some ERP amplitudes. ConclusionsThe impaired EM is linked to the increased P200 and decreased P300 amplitudes. LPN seems to be sensitive to left temporal lobe dysfunction. More importantly, the abnormal old or new effects of the FN400 and LPC, and the reduced FN400 amplitude might be associated with the visual EM deficit in the TLE patients. These findings may assist in the deep understanding of the EM disorder and the evaluation of the side effects of antiepileptic drugs.

Low-rank Tensor Restoration for ERP extraction

Event-related potential (ERP) data is essentially multi-dimensional, with correlated data in some spaces. Therefore, matrix and vector analysis results in structural information loss. Tensor decomposition can be used to explore the shared structural information of ERP signal among related conditions. Perceiving the decaying trends of the singular value changes of unfolding matrices indicate that they are low-rank matrices. Based on this assumption, in this work, a low-rank tensor restoration (LTR) method is proposed. An operator splitting method known as the alternating direction method of multipliers (ADMM) is adapted to tackle the proposed optimization problem with orthogonality and sparsity constraints. Accordingly, the problem is solved in a sequential fashion by computing the unconstrained and orthogonality constrained quadratic sub-problems with closed-form solutions. The algorithm is examined under three application areas, namely, noise removal, feature extraction and subject-to-subject transfer learning. The empirical evaluations on real P300-based ERP dataset demonstrate the robustness and effectiveness of the proposed method.

Analysis and Classification of Event-Related Potentials During Image Observation.

In the field of cognitive neuroscience, researchers have conducted extensive studies on object categorization using Event-Related Potential (ERP) analysis, specifically by analyzing electroencephalographic (EEG) response signals triggered by visual stimuli. The most common approach for visual ERP analysis is to use a low presentation rate of images and an active task where participants actively discriminate between target and non-target images. However, researchers are also interested in understanding how the human brain processes visual information in real-world scenarios. To simulate real-life object recognition, this study proposes an analysis pipeline of visual ERPs evoked by images presented in a Rapid Serial Visual Presentation (RSVP) paradigm. Such an approach allows for the investigation of recurrent patterns of visual ERP signals across specific categories and subjects. The pipeline includes segmentation of the EEGs in epochs, and the use of the resulting features as inputs for Support Vector Machine (SVM) classification. Results demonstrate common ERP patterns across the selected categories and the ability to obtain discriminant information from single visual stimuli presented in the RSVP paradigm.

Single-trial extraction of event-related potentials (ERPs) and classification of visual stimuli by ensemble use of discrete wavelet transform with Huffman coding and machine learning techniques

BackgroundPresentation of visual stimuli can induce changes in EEG signals that are typically detectable by averaging together data from multiple trials for individual participant analysis as well as for groups or conditions analysis of multiple participants. This study proposes a new method based on the discrete wavelet transform with Huffman coding and machine learning for single-trial analysis of evenal (ERPs) and classification of different visual events in the visual object detection task.MethodsEEG single trials are decomposed with discrete wavelet transform (DWT) up to the 4{th} level of decomposition using a biorthogonal B-spline wavelet. The coefficients of DWT in each trial are thresholded to discard sparse wavelet coefficients, while the quality of the signal is well maintained. The remaining optimum coefficients in each trial are encoded into bitstreams using Huffman coding, and the codewords are represented as a feature of the ERP signal. The performance of this method is tested with real visual ERPs of sixty-eight subjects.ResultsThe proposed method significantly discards the spontaneous EEG activity, extracts the single-trial visual ERPs, represents the ERP waveform into a compact bitstream as a feature, and achieves promising results in classifying the visual objects with classification performance metrics: accuracies 93.60pm {6.5}, sensitivities 93.55pm {4.5}, specificities 94.85pm {4.2}, precisions 92.50pm {5.5}, and area under the curve (AUC) 0.93pm {0.3} using SVM and k-NN machine learning classifiers.ConclusionThe proposed method suggests that the joint use of discrete wavelet transform (DWT) with Huffman coding has the potential to efficiently extract ERPs from background EEG for studying evoked responses in single-trial ERPs and classifying visual stimuli. The proposed approach has O(N) time complexity and could be implemented in real-time systems, such as the brain-computer interface (BCI), where fast detection of mental events is desired to smoothly operate a machine with minds.

LDER: a classification framework based on ERP enhancement in RSVP task

Objective. Rapid serial visual presentation (RSVP) based on electroencephalography (EEG) has been widely used in the target detection field, which distinguishes target and non-target by detecting event-related potential (ERP) components. However, the classification performance of the RSVP task is limited by the variability of ERP components, which is a great challenge in developing RSVP for real-life applications. Approach. To tackle this issue, a classification framework based on the ERP feature enhancement to offset the negative impact of the variability of ERP components for RSVP task classification named latency detection and EEG reconstruction was proposed in this paper. First, a spatial-temporal similarity measurement approach was proposed for latency detection. Subsequently, we constructed a single-trial EEG signal model containing ERP latency information. Then, according to the latency information detected in the first step, the model can be solved to obtain the corrected ERP signal and realize the enhancement of ERP features. Finally, the EEG signal after ERP enhancement can be processed by most of the existing feature extraction and classification methods of the RSVP task in this framework. Main results. Nine subjects were recruited to participate in the RSVP experiment on vehicle detection. Four popular algorithms (spatially weighted Fisher linear discrimination-principal component analysis (PCA), hierarchical discriminant PCA, hierarchical discriminant component analysis, and spatial-temporal hybrid common spatial pattern-PCA) in RSVP-based brain–computer interface for feature extraction were selected to verify the performance of our proposed framework. Experimental results showed that our proposed framework significantly outperforms the conventional classification framework in terms of area under curve, balanced accuracy, true positive rate, and false positive rate in four feature extraction methods. Additionally, statistical results showed that our proposed framework enables better performance with fewer training samples, channel numbers, and shorter temporal window sizes. Significance. As a result, the classification performance of the RSVP task was significantly improved by using our proposed framework. Our proposed classification framework will significantly promote the practical application of the RSVP task.

Multi-channel EEG signals classification via CNN and multi-head self-attention on evidence theory

Electroencephalography (EEG) provides valuable physiological information to identify human activities. However, it can be difficult to analyze the EEG data in human patterns identification, because both subjective and objective factors can easily affect sensitivity. In this study, a novel multi-head self-attention convolutional neural networks (CNN) framework based on Dempster-Shafer (D-S) evidence theory, called ETNN, is proposed to classify the EEG signal. The ETNN model considers the multi-type networks and fuses multi-output with D-S evidence theory, which can handle the EEG data more reasonably. In particular, a classification algorithm for EEG signals is derived with information fusion. Finally, an application for event-related potential signal classification and sensitivity analysis is used to demonstrate the effectiveness of the proposed ETNN model compared with existing classification techniques.

Does MOED Rhyme with FRUIT? An event-related potential study of cross-language rhyming.

The study aimed to characterize the event-related potentials signature elicited by visual rhyme judgements across two alphabetic orthographies that differ in depth (shallow: Dutch; deep: English) and by spelling-sound consistency in a deep L2-English orthography. Twenty-four Dutch-English bilinguals who varied on measures of L2-English proficiency, made rhyme judgments of semantically unrelated Dutch-English word pairs presented sequentially in the visual modality, while behavioral and electrophysiological responses were recorded. The spelling-sound consistency of target words was varied systematically. Nonrhyming targets elicited a larger N450 amplitude than rhyming targets, indicating sensitivity to mismatching phonology across languages. English target words with consistent spelling-sound mappings elicited less negative N250 amplitudes when preceded by rhyming Dutch primes. Overall, event-related potentials revealed robust responses to phonological mismatch, but subtle responses to spelling-sound inconsistency in L2-English. Results suggest that bilingual readers of a shallow L1 orthography who are immersed in an L1-speaking environment may not tune into the degree of spelling-sound consistency of a deep L2 orthography.

Источники и значимость вариативности потенциалов мозга человека в интерфейсе мозг–компьютер

In the brain-computer interface based on the P300 wave (P300 BCI), the selection of the command by the user becomes possible due to focusing the user's attention on the external stimulus/command and extraction of the response to this stimulus in the form of the event-related potential (ERP) components from EEG. To obtain the ERP signal, stimuli should be repeated many times, however, in view of the existing variability in latency of the response to certain stimuli, the averaged ERPs may give a distorted view of the nature of such responses and reduce accuracy of the interface. The study was aimed to develop an effective method for identification of the effects of the ERP components' latency variability and for accounting these effects in the P300 BCI, as well as to identify the possible impact of psychophysiological factors on the nature of ERP variability. We have conducted a BCI-based study of 19 healthy subjects involving extraction and adjustment of latency in the N1 and P300 spatial components, which play a key role in the command classification in the P300 BCI, to explore the mechanisms underlying variability. Such an approach ensured higher accuracy compared to the use of conventional EEG leads, and the highest increase of 10% was observed when using the minimum number of the stimulus repetitions. Furthermore, modifications of the interface allowing one to ensure a higher level of the user's focus on the task and a more accurate visual fixation on the target objects contributed to the increase in the amplitude of the ERP components by reducing variability of the responses to single stimuli. The findings emphasize the important role of the processes underlying the ERP components' variability and provide an effective tool for scientific exploration of such processes and the development of advanced BCI systems.

Tactile perception of textile fabrics using event-related potentials and nonlinear methods

Tactile perception is processed in the somatosensory cortices of the brain. To study the electrical activity of the brain evoked by the tactile perceptions of fabric textures, electroencephalograph signals during tactile perception were evaluated using event-related potentials and recurrence quantification analysis methods. A support vector machine was used to realize the identification of electroencephalograph signals. The results showed that fabric surface with large surface roughness, critical buckling force, and fiber diameter, and low warp-weft density was preferentially perceived, need large attentional resources, and evoke strong nonstationary state of the electroencephalographic system during the tactile perception. The δ rhythm was the main rhythm of event-related potential signals evoked by tactile perception of fabrics and the main energy of the electroencephalograph rhythm was concentrated in the low frequency band. The electroencephalographic system was in a strong nonstationary and fluctuating state during the production stage of P200 and P300 components of the event-related potential wave. The combination of P200, P300, mean diagonal line length L, and trapping time TT can significantly improve the classification accuracy of the electroencephalograph signal of tactile perception to about 72%.

Intention affects fairness processing: Evidence from behavior and representational similarity analysis of event-related potential signals.

In an ultimatum game, the responder must decide between pursuing self-interest and insisting on fairness, and these choices are affected by the intentions of the proposer. However, the time course of this social decision-making process is unclear. Representational similarity analysis (RSA) is a useful technique for linking brain activity with rich behavioral data sets. In this study, electroencephalography (EEG) was used to measure the time course of neural responses to proposed allocation schemes with different intentions. Twenty-eight participants played an ultimatum game as responders. They had to choose between accepting and rejecting the fair or unfair money allocation schemes of proposers. The schemes were offered based on the proposer's selfish intention (monetary gain), altruistic intention (donation to charity), or ambiguous intention (unknown to the responder). We used a spatiotemporal RSA and inter-subject RSA (IS-RSA) to explore the connections between event-related potentials (ERPs) after offer presentation and intention presentation with four types of behavioral data (acceptance, response time, fairness ratings, and pleasantness ratings). The spatiotemporal RSA results revealed that only response time variation was linked with the difference in ERPs at 432-592 ms after offer presentation on the posterior parietal and prefrontal regions. Meanwhile, the IS-RSA results found a significant association between inter-individual differences in response time and differences in ERP activity at 596-812 ms after the presentation of ambiguous intention, particularly in the prefrontal region. This study expands the intention-based reciprocal model to the third-party context and demonstrates that brain activity can represent response time differences in social decision-making.

MCGRAM: Linking Multi-Scale CNN With a Graph-Based Recurrent Attention Model for Subject-Independent ERP Detection

Electroencephalogram (EEG)-based event-related potential (ERP) detection is expected to play an important role in many applications. However, limited by the low signal-to-noise ratio and large inter-subject variability of EEG signals, it is still challenging to design a high-precision ERP detection method in subject-independent scenarios. In this brief, a novel graph-based multi-scale convolutional recurrent attention model (MCGRAM) is proposed to extract the underlying invariant features of EEG signals for subject-independent ERP detection. Specifically, considering the frequency factors of ERP signals, the multi-scale convolution module is designed for learning the frequency representations. Then, the graph convolution module is developed by constructing the links between each EEG node and designing trainable weights of the links to extract the discriminative spatial features. Besides, two-layer LSTM combined with a self-attention model is utilized to extract temporal representations of different EEG slices. Compared to the existing state-of-the-art methods, MCGRAM achieves the best performance in terms of AUC and F1-Score on a rapid serial visual presentation benchmark dataset. Lastly, the ablation studies demonstrate the superiority of each component in MCGRAM.

Phase Analysis of Event-Related Potentials Based on Dynamic Mode Decomposition

Real-time detection of event-related potentials (ERPs) and exploration of ERPs generation mechanisms are vital to practical application of brain–computer interfaces (BCI). Traditional methods for ERPs analysis often fall into time domain, time–frequency domain, or spatial domain. Methods which can reveal spatiotemporal interactions by simultaneously analyzing multi-channel EEG signals may provide new insights into ERP research and is highly desired. Additionally, although phase information has been investigated to describe the phase consistency of a certain frequency component across different ERP trials, it is of research significance to analyze the phase reorganization across different frequency components that constitute a single-trial ERP signal. To address these problems, dynamic mode decomposition (DMD) was applied to decompose multi-channel EEG into a series of spatial–temporal coherent DMD modes, and a new metric, called phase variance distribution (PVD) is proposed as an index of the phase reorganization of DMD modes during the ERP in a single trial. Based on the PVD, a new error-related potential (ErrP) detection method based on symmetric positive defined in Riemann manifold is proposed to demonstrate the significant PVD differences between correct and error trials. By including the phase reorganization index, the 10-fold cross-validation results of an ErrP detection task showed that the proposed method is 4.98%, 27.99% and 7.98% higher than the counterpart waveform-based ErrP detection method in the terms of weighted accuracy rate, precision and recall of the ErrP class, respectively. The resulting PVD curve shows that with the occurrence of ERP peaks, the phases of different frequency rhythms are getting to aligned and yields a significant smaller PVD. Since the DMD modes of different frequencies characterize spatiotemporal coherence of multi-channel EEG at different functional regions, the new phase reorganization index, PVD, may indicate the instantaneous phase alignment of different functional networks and sheds light on a new interpretation of ERP generation mechanism.

Two inconsistent rounds of feedback enhance the framing effect: Coding two consecutive outcome evaluations

Previous experience often guides people's decision-making. How the corresponding consecutive outcomes in the gain-loss frame modulate feedback evaluation in dynamic situations remains unclear. In this study, we used event-related potentials (ERPs) with a Balloon Analogue Risk Task (BART) to investigate the effect of the consistency with previous feedback on current choices under the gain-loss frame and the neural mechanism by coding ERP signals evoked by consecutive outcomes among adults (N = 42). Results indicated that the framing effect on feedback-related negativity (FRN) amplitudes following current feedback in decision-making was insensitive to prior feedback. In contrast, the P300 amplitudes following two inconsistent consecutive rounds of feedback were greater than that following two consistent consecutive rounds of feedback in the gain frame, while this effect was absent in the loss frame, revealing a more significant valance-framing effect. These findings demonstrate that two inconsistent rounds of feedback enhance the framing effect, and suggest that the framing effect is sensitive to prior feedback in a late cognitive appraisal stage in decision-making under uncertainty, rather than in an early feedback processing stage. The present study provides novel insights into how consecutive outcomes in the gain-loss frame modulate feedback evaluation in dynamic situations.

Processing Gapped and Gapless Relative Clauses in Mandarin: Evidence from Event–Related Brain Potentials

This paper reports an event–related potential (ERP) study examining the processing of “gapless” relative clauses (RCs), normal (“gapped”) RCs, attributive clauses, Subject–Verb–Object (SVO) forms and the morphological markers de and le in Mandarin Chinese. The objectives were to (1) determine whether a processing cost signature can be identified for gap–filler processing by comparing gapped RCs with SVO forms; and (2) determine whether processing gapless RCs is more similar to processing gapped RCs that they resemble on surface or attributive clauses that they resemble structurally by comparing gapped RCs, gapless RCs, and attributive clauses. ERP data was collected from 27 native speakers as they read 160 sentences containing gapped, gapless, attributive, and SVO sentences, which differed only in the second word of the string (e.g., 教授设计的/烧菜的/助手的/设计了/那个广告… The professor designed Ø/cooked food/assistant’s/designed that advertisement…). Results indicate that (1) the head noun evoked larger N400 and P600 components and the determiner elicited a larger sustained anterior negativity for the gapped RCs than for the SVO forms, and (2) the word immediately following the head noun evoked a larger P600 and the determiner elicited a larger sustained anterior negativity for both gapped and gapless RCs than for attributive clauses. These results indicate that (1) gap–filler integration in Mandarin is both semantic (hence the N400) and syntactic (hence the P600) in nature, and (2) gapless RCs are neurophysiologically processed more like gapped RCs than attributive clauses. The ERP signatures to verb+le, verb+de, verbs with verb–object (VO)+de morphological structure, and noun+de revealed that the processing costs for VO verbs and noun modifications were higher than for simple verbs, and the morphological markers de and le induced similar processing costs, even though de triggered the positing of a syntactic gap whereas le did not.

PREFACE: A SPECIAL SELECTION ON EMERGING TECHNIQUES FOR BIOMECHANICS–PART II

PREFACE: A SPECIAL SELECTION ON EMERGING TECHNIQUES FOR BIOMECHANICS–PART II