Rapid Serial Visual Presentation Research Articles

The impact of fear of pain on the temporal processing of emotional faces: modulation of attentional resources.

Recent research has focused on the spatial attention bias toward pain-related cues in individuals with fear of pain (FOP), but our understanding of its temporal attention features, particularly regarding emotional faces, is limited. To address this gap, 39 individuals with high fear of pain (H-FOP) and 37 with low fear of pain (L-FOP) completed a rapid serial visual presentation dual task. Participants viewed a series of rapidly presented pictures, first identified neutral building images as the first target (T1) and then emotional faces (painful, neutral, and happy) as the second target (T2). T1 and T2 presented at time intervals of 232ms (lag 2) and 696ms (lag 6), respectively. Results indicate that H-FOP group members were less accurate in recognizing faces at lag 2 shortly after correctly identifying T1 compared to those L-FOP group members. Notably, all participants were more accurate in recognizing painful faces at lag 2 than in identifying neutral and happy faces, suggesting a negative bias toward pain stimuli when attentional resources were limited. In contrast, at lag 6, when attentional resources were more abundant, participants showed greater accuracy in recognizing neutral faces than emotional faces. These findings underscore the influence of fear of pain and available cognitive resources on the attentional processing of emotional faces over time, providing insights into how fear of pain affects the temporal dynamics of emotional face recognition and the role of attentional resources.

A temporal–spectral fusion transformer with subject-specific adapter for enhancing RSVP-BCI decoding

The Rapid Serial Visual Presentation (RSVP)-based Brain–Computer Interface (BCI) is an efficient technology for target retrieval using electroencephalography (EEG) signals. The performance improvement of traditional decoding methods relies on a substantial amount of training data from new test subjects, which increases preparation time for BCI systems. Several studies introduce data from existing subjects to reduce the dependence of performance improvement on data from new subjects, but their optimization strategy based on adversarial learning with extensive data increases training time during the preparation procedure. Moreover, most previous methods only focus on the single-view information of EEG signals, but ignore the information from other views which may further improve performance. To enhance decoding performance while reducing preparation time, we propose a Temporal-Spectral fusion transformer with Subject-specific Adapter (TSformer-SA). Specifically, a cross-view interaction module is proposed to facilitate information transfer and extract common representations across two-view features extracted from EEG temporal signals and spectrogram images. Then, an attention-based fusion module fuses the features of two views to obtain comprehensive discriminative features for classification. Furthermore, a multi-view consistency loss is proposed to maximize the feature similarity between two views of the same EEG signal. Finally, we propose a subject-specific adapter to rapidly transfer the knowledge of the model trained on data from existing subjects to decode data from new subjects. Experimental results show that TSformer-SA significantly outperforms comparison methods and achieves outstanding performance with limited training data from new subjects. This facilitates efficient decoding and rapid deployment of BCI systems in practical use.

Early perceptual locus of suppression during the attentional blink.

The attentional blink (AB) demonstrates that recognizing the second of two targets (T1 and T2) is difficult when they appear in close succession in a rapid serial visual presentation (RSVP) stream. The AB has been widely accepted as a suppression of T2 processing at the postperceptual stage. The current event-related potential study updates this view by demonstrating the existence of an early perceptual locus of suppression during the AB. Using line drawings of real-life objects as RSVP items, we required participants in Experiment 1 to either discriminate the exact identities or simply classify the object categories of T1 and T2, and in Experiment 2, we instructed participants to discriminate either T1 and T2 identities (dual-target task) or only T2 identity (single-target task) to invalidate the temporal expectation as an alternative account. The results of Experiments 1 and 2 showed that the T2-elicited first positive peak (P1) component was consistently decreased at Lag 3 whenever a dual-target, but not single-target, task was required, and the magnitude of this P1 suppression was significantly predictive of the behavioral AB magnitude in each dual-target task. When the RSVP items were substituted by classic but size-matched alphanumeric characters in Experiment 3, no P1 suppression was evident as expected, ruling out the large stimulus size as an alternative interpretation. These findings provide the strongest evidence to date that the AB can begin to suppress T2 processing at a very early perceptual stage, at least when observers encounter RSVP items of real-life objects, which calls for more flexible cognitive models for the AB. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Your Eyes on Speed: Using Pupil Dilation to Adaptively Select Speed-Reading Parameters in Virtual Reality

Rapid Serial Visual Presentation (RSVP) improves the reading speed for optimizing the user's information processing capabilities on Virtual Reality (VR) devices. Yet, the user's RSVP reading performance changes over time while the reading speed remains static. In this paper, we evaluate pupil dilation as a physiological metric to assess the mental workload of readers in real-time. We assess mental workload under different background lighting and RSVP presentation speeds to estimate the optimal color that discriminates the pupil diameter varying RSVP presentation speeds. We discovered that a gray background provides the best contrast for reading at various presentation speeds. Then, we conducted a second study to evaluate the classification accuracy of mental workload for different presentation speeds. We find that pupil dilation relates to mental workload when reading with RSVP. We discuss how pupil dilation can be used to adapt the RSVP speed in future VR applications to optimize information intake.

A MultiModal Vigilance (MMV) dataset during RSVP and SSVEP brain-computer interface tasks

Vigilance represents an ability to sustain prolonged attention and plays a crucial role in ensuring the reliability and optimal performance of various tasks. In this report, we describe a MultiModal Vigilance (MMV) dataset comprising seven physiological signals acquired during two Brain-Computer Interface (BCI) tasks. The BCI tasks encompass a rapid serial visual presentation (RSVP)-based target image retrieval task and a steady-state visual evoked potential (SSVEP)-based cursor-control task. The MMV dataset includes four sessions of seven physiological signals for 18 subjects, which encompasses electroencephalogram(EEG), electrooculogram (EOG), electrocardiogram (ECG), photoplethysmogram (PPG), electrodermal activity (EDA), electromyogram (EMG), and eye movement. The MMV dataset provides data from four stages: 1) raw data, 2) pre-processed data, 3) trial data, and 4) feature data that can be directly used for vigilance estimation. We believe this dataset will achieve flexible reuse and meet the various needs of researchers. And this dataset will greatly contribute to advancing research on physiological signal-based vigilance research and estimation.

RSVP-based BCI for inconspicuous targets: detection, localization, and modulation of attention

Objective.While brain-computer interface (BCI) based on rapid serial visual presentation (RSVP) is widely used in target detection, patterns of event-related potential (ERP), as well as the performance on detecting inconspicuous targets remain unknown. Moreover, participant-screening methods to excluded 'BCI-blind' users are still lacking.Approach.A RSVP paradigm was designed with targets of varied concealment, size, and location. ERPs (e.g. P300 and N2pc) and target detection accuracy were compared among these conditions. The relationship between participants' attention scores and target detection accuracy was also analyzed to test attention level as a criterion for participant screening.Main results.Statistical analysis showed that the conditions of target concealment and size significantly influenced ERP. In particular, ERP for inconspicuous targets, such as concealed and small targets, exhibited lower amplitudes and longer latencies. In consistent, the accuracy of detection in inconspicuous condition was significantly lower than that of conspicuous condition. In addition, a significant association was found between attention scores and target detection accuracy for camouflaged targets.Significance.The study was the first to address ERP features among multiple dimensions of concealment, size, and location. The conclusion provided insights into the relationship between ERP decoding and properties of targets. In addition, the association between attention scores and detection accuracy implied a promising method in screening well-behaved participants for camouflaged target detection.

Neural mechanisms for secondary suppression of emotional distractors: Evidence from concurrent electroencephalography-magnetoencephalography data.

Distractor suppression allows us to remain on-task in the presence of distractions by filtering task-irrelevant information from ongoing cognitive processing and responding. Electrophysiological studies have revealed that this key feature of selective attention is a dynamic process that involves at least two distinct stages of processing. Two important aspects of these processing stages remain unclear: Whether the processing of emotional distractors at an earlier stage is automatic, as reflected in the N2/early posterior negativity (EPN) component; and what functional-anatomical brain systems are recruited in each stage. The present study addresses these issues by measuring brain activity with concurrent electroencephalography-magnetoencephalography (MEG) recordings while participants performed a combined rapid serial visual presentation and motion tracking task. Event-related potentials (ERP) showed significant effects of attentional capture and attentional modulation during two time windows marked by the N2/EPN and P3b ERP components. Source reconstruction of concurrent MEG measurements revealed activation of the left visual association cortex and anterior cingulate cortex during the N2/EPN time window, activation of the insula during the early phase of the P3b and anterior cingulate cortex activity during the later phase of the P3b. The findings provide novel evidence establishing a connection between the increased N2 response to negative pictures and the activation of the cingulate gyrus, which facilitates the suppression of distractions during demanding cognitive tasks. In addition, distinct activation patterns were observed in the insula and anterior cingulate cortex during the P3b time window, indicating that attentional control mediated by the anterior cingulate cortex operates to suppress the processing of distracting emotional stimuli. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Endogenous attention enhances contrast appearance regardless of stimulus contrast.

There has been enduring debate on how attention alters contrast appearance. Recent research indicates that exogenous attention enhances contrast appearance for low-contrast stimuli but attenuates it for high-contrast stimuli. Similarly, one study has demonstrated that endogenous attention heightens perceived contrast for low-contrast stimuli, yet none have explored its impact on high-contrast stimuli. In this study, we investigated how endogenous attention alters contrast appearance, with a specific focus on high-contrast stimuli. In Experiment 1, we utilized the rapid serial visual presentation (RSVP) paradigm to direct endogenous attention, revealing that contrast appearance was enhanced for both low- and high-contrast stimuli. To eliminate potential influences from the confined attention field in the RSVP paradigm, Experiment 2 adopted the letter identification paradigm, deploying attention across a broader visual field. Results consistently indicated that endogenous attention increased perceived contrast for high-contrast stimuli. Experiment 3 employed equiluminant chromatic letters as stimuli in the letter identification task to eliminate potential interference from contrast adaption, which might have occurred in Experiment 2. Remarkably, the boosting effect of endogenous attention persisted. Combining the results from these experiments, we propose that endogenous attention consistently enhances contrast appearance, irrespective of stimulus contrast levels. This stands in contrast to the effects of exogenous attention, suggesting that mechanisms through which endogenous attention alters contrast appearance may differ from those of exogenous attention.

From lab to life: assessing the impact of real-world interactions on the operation of rapid serial visual presentation-based brain-computer interfaces

Objective. Brain-computer interfaces (BCI) have been extensively researched in controlled lab settings where the P300 event-related potential (ERP), elicited in the rapid serial visual presentation (RSVP) paradigm, has shown promising potential. However, deploying BCIs outside of laboratory settings is challenging due to the presence of contaminating artifacts that often occur as a result of activities such as talking, head movements, and body movements. These artifacts can severely contaminate the measured EEG signals and consequently impede detection of the P300 ERP. Our goal is to assess the impact of these real-world noise factors on the performance of a RSVP-BCI, specifically focusing on single-trial P300 detection. Approach. In this study, we examine the impact of movement activity on the performance of a P300-based RSVP-BCI application designed to allow users to search images at high speed. Using machine learning, we assessed P300 detection performance using both EEG data captured in optimal recording conditions (e.g. where participants were instructed to refrain from moving) and a variety of conditions where the participant intentionally produced movements to contaminate the EEG recording. Main results. The results, presented as area under the receiver operating characteristic curve (ROC-AUC) scores, provide insight into the significant impact of noise on single-trial P300 detection. Notably, there is a reduction in classifier detection accuracy when intentionally contaminated RSVP trials are used for training and testing, when compared to using non-intentionally contaminated RSVP trials. Significance. Our findings underscore the necessity of addressing and mitigating noise in EEG recordings to facilitate the use of BCIs in real-world settings, thus extending the reach of EEG technology beyond the confines of the laboratory.

Harnessing Few-Shot Learning for EEG signal classification: a survey of state-of-the-art techniques and future directions.

This paper presents a systematic literature review, providing a comprehensive taxonomy of Data Augmentation (DA), Transfer Learning (TL), and Self-Supervised Learning (SSL) techniques within the context of Few-Shot Learning (FSL) for EEG signal classification. EEG signals have shown significant potential in various paradigms, including Motor Imagery, Emotion Recognition, Visual Evoked Potentials, Steady-State Visually Evoked Potentials, Rapid Serial Visual Presentation, Event-Related Potentials, and Mental Workload. However, challenges such as limited labeled data, noise, and inter/intra-subject variability have impeded the effectiveness of traditional machine learning (ML) and deep learning (DL) models. This review methodically explores how FSL approaches, incorporating DA, TL, and SSL, can address these challenges and enhance classification performance in specific EEG paradigms. It also delves into the open research challenges related to these techniques in EEG signal classification. Specifically, the review examines the identification of DA strategies tailored to various EEG paradigms, the creation of TL architectures for efficient knowledge transfer, and the formulation of SSL methods for unsupervised representation learning from EEG data. Addressing these challenges is crucial for enhancing the efficacy and robustness of FSL-based EEG signal classification. By presenting a structured taxonomy of FSL techniques and discussing the associated research challenges, this systematic review offers valuable insights for future investigations in EEG signal classification. The findings aim to guide and inspire researchers, promoting advancements in applying FSL methodologies for improved EEG signal analysis and classification in real-world settings.

CRE-TSCAE: A Novel Classification Model Based on Stacked Convolutional Autoencoder for Dual-Target RSVP-BCI Tasks.

The Rapid Serial Visual Presentation (RSVP) paradigm facilitates target identification in a rapid picture stream, which is applied extensively in military target surveillance and police monitoring. Most researchers concentrate on the single target RSVP-BCI whereas the study of dual-target is scarcely conducted, limiting RSVP application considerably. This paper proposed a novel classification model named Common Representation Extraction-Targeted Stacked Convolutional Autoencoder (CRE-TSCAE) to detect two targets with one nontarget in RSVP tasks. CRE generated a common representation for each target class to reduce variability from different trials of the same class and distinguish the difference between two targets better. TSCAE aimed to control uncertainty in the training process while requiring less target training data. The model learned a compact and discriminative feature through the training from several learning tasks so as to distinguish each class effectively. It was validated on the World Robot Contest 2021 and 2022 ERP datasets. Experimental results showed that CRE-TSCAE outperformed the state-of-the-art RSVP decoding algorithms and the Average ACC was 71.25%, improving 6.5% at least over the rest. It demonstrated that CRE-TSCAE showed a strong ability to extract discriminative latent features in detecting the differences among two targets with nontarget, which guaranteed increased classification accuracy. CRE-TSCAE provided an innovative and effective classification model for dual-target RSVP-BCI tasks and some insights into the neurophysiological distinction between different targets.

Boosting brain-computer interface performance through cognitive training: A brain-centric approach

Previous efforts to boost the performance of brain-computer interfaces (BCIs) have predominantly focused on optimizing algorithms for decoding brain signals. However, the untapped potential of leveraging brain plasticity for optimization remains underexplored. In this study, we enhanced the temporal resolution of the human brain in discriminating visual stimuli by eliminating the attentional blink (AB) through color-salient cognitive training, and we confirmed that the mechanism was an attention-based improvement. Using the rapid serial visual presentation (RSVP)-based BCI, we evaluated the behavioral and electroencephalogram (EEG) decoding performance of subjects before and after cognitive training in high target percentage (with AB) and low target percentage (without AB) surveillance tasks, respectively. The results consistently demonstrated significant improvements in the trained subjects. Further analysis indicated that this improvement was attributed to the cognitively trained brain producing more discriminative EEG. Our work highlights the feasibility of cognitive training as a means of brain enhancement to boost BCI performance.

A novel hybrid decoding neural network for EEG signal representation

In this paper, we proposed a novel hybrid decoding model that combines the superiority of CNNs and multi-head self-attention mechanisms, called HCANN, to finely characterizing EEG features. Depthwise separable convolution with multi-scale factors efficiently decouples temporal relevant information between brain-computer interface (BCI) tasks and EEG signals. Multi-head mechanism adaptively modified for EEG focuses on brain spatial activation patterns and extracts complementary spatial representation information from multiple subspaces. The proposed HCANN decodes the intent information of EEG recorded by three BCI paradigms, including one active and two passive BCI paradigms: rapid serial visual presentation, motor imagery, and imagined speech. We evaluated HCANN by comparing with the current state-of-the-art methods. The experimental results demonstrated that HCANN can effectively decode EEG and improves classification performance for all three BCI tasks. In addition, the visualization of spatial-temporal features at different decoding stages demonstrated that the proposed HCANN gradually extracts effective features related to the BCI tasks. The code of HCANN is publicly available at https://github.com/youshuoji/HCANN.

A dissociation between the effects of expectations and attention in selective visual processing

It is often claimed that probabilistic expectations affect visual perception directly, without mediation by selective attention. However, these claims have been disputed, as effects of expectation and attention are notoriously hard to dissociate experimentally. In this study, we used a new approach to separate expectations from attention. In four experiments (N = 60), participants searched for a target in a rapid serial visual presentation (RSVP) stream and had to identify a digit or a letter defined by a low-level cue (colour or shape). Expectations about the target's alphanumeric category were probabilistically manipulated. Since category membership is a high-level feature and since the target was embedded among many distractors that shared its category, targets from the expected category should not attract attention more than targets from the unexpected category. In the first experiment, these targets were more likely to be identified relative to targets from the unexpected category. Importantly, in the following experiments, we also included behavioural and electrophysiological indices of attentional guidance and engagement. This allowed us to examine whether expectations also modulated these or earlier attentional processes. Results showed that category-based expectations had no modulatory effects on attention, and only affected processing at later encoding-related stages. Alternative interpretation of expectation effects in terms of repetition priming or response bias were also ruled out. These observations provide new evidence for direct attention-independent expectation effects on perception. We suggest that expectations can adjust the threshold required for encoding expectations-congruent information, thereby affecting the speed with which target objects are encoded in working memory.

Inattentional aftereffects: The role of attention on the strength of the motion aftereffect.

The way that attention affects the processing of visual information is one of the most intriguing fields in the study of visual perception. One way to examine this interaction is by studying the way perceptual aftereffects are modulated by attention. In the present study, we have manipulated attention during adaptation to translational motion generated by coherently moving random dots, in order to investigate the effect of the distraction of attention on the strength of the peripheral dynamic motion aftereffect (MAE). A foveal rapid serial visual presentation task (RSVP) of varying difficulty was introduced during the adaptation period while the adaptation and test stimuli were presented peripherally. Furthermore, to examine the interaction between the physical characteristics of the stimulus and attention, we have manipulated the motion coherence level of the adaptation stimuli. Our results suggested that the removal of attention through an irrelevant task modulated the MAE's magnitude moderately and that such an effect depends on the stimulus strength. We also showed that the MAE still persists with subthreshold and unattended stimuli, suggesting that perhaps attention is not required for the complete development of the MAE.

Rapid Serial Visual Presentation of transposed-word sequences in the grammatical decision task: an examination of the roles of temporal and spatial cues to word order

ABSTRACT Transposing two words in a sentence (e.g. “cat” and “was” in “the white cat was big”) creates a sequence that is harder to classify as ungrammatical than control sequences (e.g. “the white was cat slowly”), suggesting that word position coding is noisy and can be affected by syntactic expectations. In the present research, this transposed-word effect was examined more closely using Rapid Serial Visual Presentation (RSVP) formats which provided either clear temporal cues to word order but no spatial cues, or both types of cues. Compared to when all words were presented simultaneously, the two RSVP formats reduced the transposed-word effect to the same degree while having no parallel impact on another ungrammatical comparison condition involving no transposition. These results are discussed in the context of serial and parallel models of reading as well as models that propose a later processing stage for the locus of the transposed-word effect.

The Absence of Attentional Bias to Low-Calorie Food Stimuli in Restrictive Dieters: Differences in the Allocation of Attentional Resources to High-Calorie Foods.

Restrictive dieters are those who expect to achieve body shape and weight control through dieting. However, they often have difficulty suppressing the desire to consume food when confronted with it. It has been shown that when high- and low-calorie foods are presented together, the attention of restrictive eaters is preferentially directed to high-calorie foods. However, whether attentional bias occurs when low-calorie foods are present alone and whether the allocation of attentional resources is consistent with that for high-calorie foods has yet to be explored. The present study focused on the effects of high-/low-calorie foods on attentional bias in restrictive dieters. Seventy-eight participants were recruited to participate in the experiment via the Dutch Eating Behavior Questionnaire (DEBQ) scale, which is administered in a rapid serial visual presentation (RSVP) task. The results revealed that failed restrictive dieters had the lowest percentage of correct answers at the lag2 level, indicating attentional bias. Failed restrictive dieters allocated more attentional resources to high-calorie foods than to low-calorie foods. Restrictive dieters showed no attentional bias when low-calorie foods were presented alone. The results suggest that low-calorie foods do not elicit an attentional bias in restrictive dieters and that the allocation of attentional resources is not consistent when compared to that for high-calorie foods.

Evaluation of Different Types of Stimuli in an Event-Related Potential-Based Brain-Computer Interface Speller under Rapid Serial Visual Presentation.

Rapid serial visual presentation (RSVP) is currently a suitable gaze-independent paradigm for controlling visual brain-computer interfaces (BCIs) based on event-related potentials (ERPs), especially for users with limited eye movement control. However, unlike gaze-dependent paradigms, gaze-independent ones have received less attention concerning the specific choice of visual stimuli that are used. In gaze-dependent BCIs, images of faces-particularly those tinted red-have been shown to be effective stimuli. This study aims to evaluate whether the colour of faces used as visual stimuli influences ERP-BCI performance under RSVP. Fifteen participants tested four conditions that varied only in the visual stimulus used: grey letters (GL), red famous faces with letters (RFF), green famous faces with letters (GFF), and blue famous faces with letters (BFF). The results indicated significant accuracy differences only between the GL and GFF conditions, unlike prior gaze-dependent studies. Additionally, GL achieved higher comfort ratings compared with other face-related conditions. This study highlights that the choice of stimulus type impacts both performance and user comfort, suggesting implications for future ERP-BCI designs for users requiring gaze-independent systems.

Semantic image sorting method for RSVP presentation

Objective. The rapid serial visual presentation (RSVP) paradigm, which is based on the electroencephalogram (EEG) technology, is an effective approach for object detection. It aims to detect the event-related potentials (ERP) components evoked by target images for rapid identification. However, the object detection performance within this paradigm is affected by the visual disparity between adjacent images in a sequence. Currently, there is no objective metric to quantify this visual difference. Consequently, a reliable image sorting method is required to ensure the generation of a smooth sequence for effective presentation. Approach. In this paper, we propose a novel semantic image sorting method for sorting RSVP sequences, which aims at generating sequences that are perceptually smoother in terms of the human visual experience. Main results. We conducted a comparative analysis between our method and two existing methods for generating RSVP sequences using both qualitative and quantitative assessments. A qualitative evaluation revealed that the sequences generated by our method were smoother in subjective vision and were more effective in evoking stronger ERP components than those generated by the other two methods. Quantitatively, our method generated semantically smoother sequences than the other two methods. Furthermore, we employed four advanced approaches to classify single-trial EEG signals evoked by each of the three methods. The classification results of the EEG signals evoked by our method were superior to those of the other two methods. Significance. In summary, the results indicate that the proposed method can significantly enhance the object detection performance in RSVP-based sequences.

Nasal inhalation does not improve memory of visual repetitions.

Several studies suggest that breathing entrains neural oscillations and thereby improves visual detection and memory performance during nasal inhalation. However, the evidence for this association is mixed, with some studies finding no, minor, or opposite effects. Here, we tested whether nasal breathing phase influences memory of repeated images presented in a rapid serial visual presentation (RSVP) task. The RSVP task is ideal for studying the effects of respiratory-entrained oscillations on visual memory because it engages critical aspects of sensory encoding that depend on oscillatory activity, such as fast processing of natural images, repetition detection, memory encoding, and retrieval. It also enables the presentation of a large number of stimuli during each phase of the breathing cycle. In two separate experiments (n = 72 and n = 142, respectively) where participants were explicitly asked to breathe through their nose, we found that nasal breathing phase at target presentation did not significantly affect memory performance. An exploratory analysis in the first experiment suggested a potential benefit for targets appearing approximately 1 s after inhalation. However, this finding was not replicated in the pre-registered second experiment with a larger sample. Thus, in two large sample experiments, we found no measurable impact of breathing phase on memory performance in the RSVP task. These results suggest that the natural breathing cycle does not have a significant impact on memory for repeated images and raise doubts about the idea that visual memory is broadly affected by the breathing phase.