Data_sheet_2.pdf

Abstract

Numerous studies have explored the benefit of iconic gestures in speech comprehension. However, only few studies have investigated how visual attention was allocated to these gestures in the context of clear versus degraded speech and the way information is extracted for enhancing comprehension. This study aimed to explore the effect of iconic gestures on comprehension and whether fixating the gesture is required for information extraction. Four types of gestures (i.e., semantically, and syntactically incongruent iconic gestures, meaningless configurations, and congruent iconic gestures) were presented in a sentence-context in three different listening conditions (i.e., clear, partly degraded or fully degraded speech). Using eye-tracking technology, participants’ gaze was recorded while they watched video clips after which they were invited to answer simple comprehension questions. Results first showed that different types of gestures differently attract attention, and that the more speech was degraded, the less participants would pay attention to gestures. Furthermore, semantically incongruent gestures appeared to particularly impair comprehension although not being fixated while congruent gestures appeared to improve comprehension despite also not being fixated. These results suggest that covert attention is sufficient to convey information that will be processed by the listener.