Parallels in Processing Boundary Cues in Speech and Action.

Matt Hilton,Romy Räling,Isabell Wartenburger,Birgit Elsner

doi:10.3389/fpsyg.2019.01566

Abstract

Speech and action sequences are continuous streams of information that can be segmented into sub-units. In both domains, this segmentation can be facilitated by perceptual cues contained within the information stream. In speech, prosodic cues (e.g., a pause, pre-boundary lengthening, and pitch rise) mark boundaries between words and phrases, while boundaries between actions of an action sequence can be marked by kinematic cues (e.g., a pause, pre-boundary deceleration). The processing of prosodic boundary cues evokes an Event-related Potentials (ERP) component known as the Closure Positive Shift (CPS), and it is possible that the CPS reflects domain-general cognitive processes involved in segmentation, given that the CPS is also evoked by boundaries between subunits of non-speech auditory stimuli. This study further probed the domain-generality of the CPS and its underlying processes by investigating electrophysiological correlates of the processing of boundary cues in sequences of spoken verbs (auditory stimuli; Experiment 1; N = 23 adults) and actions (visual stimuli; Experiment 2; N = 23 adults). The EEG data from both experiments revealed a CPS-like broadly distributed positivity during the 250 ms prior to the onset of the post-boundary word or action, indicating similar electrophysiological correlates of boundary processing across domains, suggesting that the cognitive processes underlying speech and action segmentation might also be shared.

Highlights

While still relatively rare, interdisciplinary examination of speech and action processing is vital given the striking parallels between the two domains: Both speech and action consist of sub-units that are sequentially and hierarchically organized, meaning that speech and action productions inflate over time and can be –in principle endlessly– concatenated
In two within-subjects conditions, the sequences either did or did not contain prosodic or kinematic boundary cues to signal a boundary between the critical second verb or action and the following subunit
A prosodic boundary within the auditory verb sequences evoked a broadly distributed positivity in the Event-related Potentials (ERP), which we interpret as a Closure Positive Shift (CPS) (e.g., Steinhauer et al, 1999; Bögels et al, 2011a; Holzgrefe et al, 2013; Holzgrefe-Lang et al, 2016), confirming that the CPS is a robust marker of prosodic boundary processing in speech

Summary

Introduction

Interdisciplinary examination of speech and action processing is vital given the striking parallels between the two domains: Both speech and action consist of sub-units that are sequentially and hierarchically organized, meaning that speech and action productions inflate over time and can be –in principle endlessly– concatenated. The listener or observer must process this continuous stream of information, encode it, and segment it into meaningful sub-units before being able to interpret it In both domains, bottom-up processes (analyzing perceptual cues) as well as top-down (contextual) processes support the segmentation of an utterance or an action sequence, allowing for the extraction of their underlying structure and meaning (Zacks, 2004; Goyet et al, 2016; Emberson, 2017). Major prosodic boundaries (so-called intonation phrase boundaries) often coincide with boundaries of syntactic clauses (Downing, 1970; Selkirk, 1984; Nespor and Vogel, 1986; for German, see Truckenbrodt, 2005) Infants track these bottom-up prosodic boundary cues, with the close prosody-syntax mapping supporting the infant’s developing understanding of syntactic structures (so-called prosodic bootstrapping, e.g., Gleitman and Wanner, 1982; Nazzi et al, 2000; for review, see Speer and Ito, 2009). Adult listeners make use of prosodic cues when syntactic and lexical structures do not provide sufficient information to guide segmentation (e.g., Marslen-Wilson et al, 1992; Warren et al, 1995; Schafer et al, 2000)

Objectives

Results

Conclusion