Beat gestures influence which speech sounds you hear.

Hans Rutger Bosker,David Peeters

doi:10.1098/rspb.2020.2419

Abstract

Beat gestures-spontaneously produced biphasic movements of the hand-are among the most frequently encountered co-speech gestures in human communication. They are closely temporally aligned to the prosodic characteristics of the speech signal, typically occurring on lexically stressed syllables. Despite their prevalence across speakers of the world's languages, how beat gestures impact spoken word recognition is unclear. Can these simple 'flicks of the hand' influence speech perception? Across a range of experiments, we demonstrate that beat gestures influence the explicit and implicit perception of lexical stress (e.g. distinguishing OBject from obJECT), and in turn can influence what vowels listeners hear. Thus, we provide converging evidence for a manual McGurk effect: relatively simple and widely occurring hand movements influence which speech sounds we hear.

Highlights

The human capacity to communicate evolved primarily to allow for efficient face-to-face interaction [1]
To what extent do we listen with our eyes? The classic McGurk effect illustrates that listeners take both visual and auditory information into account to arrive at a best possible approximation of the exact message a speaker intends to convey [10]
We observed across a range of experiments that beat gestures influence both the explicit and implicit perception of lexical stress, and in turn, may influence the perception of individual speech sounds

Summary

Background

The human capacity to communicate evolved primarily to allow for efficient face-to-face interaction [1]. ‘coat—code’ [41]; to vowel length contrasts in Dutch, e.g. tak /tɑk/ ‘branch’—taak /ta:k/ ‘task’ [42]) Based on this literature, prosodic expectations should influence the perceived identity of speech sounds in a contrastive fashion: a Dutch vowel midway between /ɑ/ and /a:/ may be perceived as relatively short /ɑ/ when presented in a syllable carrying audiovisual prominence (i.e. when paired with a beat gesture). Experiment 3 assesses whether and how beat gestures influence the implicit perception of lexical stress in Dutch, which—in turn—may affect whether human listeners perceive a short or a long vowel

Method

Results

Discussion

Findings

55. Golumbic EMZ et al 2013 Mechanisms underlying