Time-varying networks of ERPs in P300-speller paradigms based on spatially and semantically congruent audiovisual bimodality

Abstract

ObjectiveIn the P300-speller paradigm using spatially and semantically congruent audiovisual (AV) stimuli, AV stimuli elicited significantly higher event-related potential (ERP) amplitudes than those for a single visual (V) stimulus. Nevertheless, it remains unclear whether ERPs in AV and V spelling paradigms are generated with identical brain network architecture.ApproachIn this study, we constructed time-varying networks for ERPs in AV and V spelling paradigms based on adaptive directed transfer function to investigate the dynamic processes underpinning the processing of stimuli in the two spelling paradigms.Main resultsOur analysis revealed that early AV integration, AV spatial and semantic information, and late AV integration enhanced activation in attention-related areas between 40–160 ms, areas associated with attention to target stimuli between 200–280 ms, and areas associated with decision-making between 320–520 ms, respectively. Left temporal areas were associated with AV spatial-semantic information-processing and late AV integration, the activation of which impacted the activation of brain areas associated with attention (P3a) and decision-making (P3b). In addition, the ability and efficiency of information transmission from brain networks in the AV spelling paradigm were significantly stronger than those in the V spelling paradigm between 40 and 160 ms, 200 and 280 ms, and 320 and 520 ms.SignificanceThis work provides a theoretical basis for deepening our understanding of the neural mechanisms underscoring ERPs production in the AV P300-speller paradigm as well as a theoretical reference for further optimization of the induced paradigm based on AV stimuli and improves the performance of BCI system.