The facilitative effect of gestures on the neural processing of semantic complexity in a continuous narrative

Abstract

Gestures are elemental components of social communication and aid comprehension of verbal messages; however, little is known about the potential role of gestures in facilitating processing of semantic complexity in an ecologically valid setting. The goal of this study was to investigate whether cognitive load, as indexed by semantic complexity, is modulated by the presentation of gestures accompanying speech. Twenty healthy participants watched 16 video clips of a short narrative while instructed to carefully listen to and watch the narrator while functional magnetic resonance imaging (fMRI) data were acquired. The videos contained passages with and without various co-speech gestures, as well as passages where the semantic complexity was either low or high, as measured by the metric of idea density. Increasing semantic complexity led to reduced activation within the default mode network (DMN); whereas, presents of gestures decreased activation in language-related regions (left middle temporal gyrus and left inferior frontal gyrus) and increased activation in high-level visual and multimodal regions of occipitotemporal cortex. Most interestingly, an interaction between semantic complexity and gestures was observed in a language-related area in left anterior temporal cortex; specifically, increasing gestures led to a greater drop in activation with high vs. Low semantic complexity. These results provide evidence that the facilitation of gestures on semantic processing, particularly for complex narratives, is reflected in the neural substrates of language processing.