Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time.

Ivo V Stuldreher,Jan B F Van Erp,Anne-Marie Brouwer,Nattapong Thammasan

doi:10.3389/fnins.2020.575521

Abstract

Interpersonal physiological synchrony (PS), or the similarity of physiological signals between individuals over time, may be used to detect attentionally engaging moments in time. We here investigated whether PS in the electroencephalogram (EEG), electrodermal activity (EDA), heart rate and a multimodal metric signals the occurrence of attentionally relevant events in time in two groups of participants. Both groups were presented with the same auditory stimulus, but were instructed to attend either to the narrative of an audiobook (audiobook-attending: AA group) or to interspersed emotional sounds and beeps (stimulus-attending: SA group). We hypothesized that emotional sounds could be detected in both groups as they are expected to draw attention involuntarily, in a bottom-up fashion. Indeed, we found this to be the case for PS in EDA or the multimodal metric. Beeps, that are expected to be only relevant due to specific “top-down” attentional instructions, could indeed only be detected using PS among SA participants, for EDA, EEG and the multimodal metric. We further hypothesized that moments in the audiobook accompanied by high PS in either EEG, EDA, heart rate or the multimodal metric for AA participants would be rated as more engaging by an independent group of participants compared to moments corresponding to low PS. This hypothesis was not supported. Our results show that PS can support the detection of attentionally engaging events over time. Currently, the relation between PS and engagement is only established for well-defined, interspersed stimuli, whereas the relation between PS and a more abstract self-reported metric of engagement over time has not been established. As the relation between PS and engagement is dependent on event type and physiological measure, we suggest to choose a measure matching with the stimulus of interest. When the stimulus type is unknown, a multimodal metric is most robust.

Highlights

Knowing what events in the external environment people attend to, and how their shared attentional engagement to events varies over time, can be useful in a range of settings, from evaluating educational or entertaining material, to real time adjustment of important instructions
For the audiobook-attending group (AA) group, we found that, as expected, only the occurrence of emotional sounds could be predicted, using physiological synchrony (PS) in electrodermal activity (EDA) (p < 0.001) or the multimodal metric p = 0.003
When stimuli were not differentiated according to stimulus type, detection performance was well above chance level for PS in EEG (p < 0.001), EDA (p < 0.001) and the multimodal metric (p < 0.001), but not for PS in heart rate

Summary

Introduction

Knowing what events in the external environment people attend to, and how their shared attentional engagement to events varies over time, can be useful in a range of settings, from evaluating educational or entertaining material, to real time adjustment of important instructions. Unlike explicit measures, such as questionnaires in which people are asked to specify their attentional engagement, physiological signals can provide continuous and implicit information on mental state (Zander and Kothe, 2011). A popular approach to uncover the complex links between physiology and mental state is the use of supervised learning algorithms. Attentional, emotional or cognitive state, cannot realistically be represented by a small number of discrete states, but are naturally of more continuous nature (Zehetleitner et al, 2012; Rosenberg et al, 2013)

Objectives

Methods

Results

Discussion

Conclusion