Abstract
Humans integrate information communicated by speech and gestures. Functional magnetic resonance imaging (fMRI) studies suggest that the posterior superior temporal sulcus (STS) and adjacent gyri are relevant for multisensory integration. However, a connectivity model representing this essential combinatory process is still missing. Here, we used dynamic causal modeling for fMRI to analyze the effective connectivity pattern between middle temporal gyrus (MTG), occipital cortex (OC) and STS associated with auditory verbal, visual gesture-related, and integrative processing, respectively, to unveil the neural mechanisms underlying integration of intrinsically meaningful gestures (e.g., “Thumbs-up gesture”) and corresponding speech.20 participants were presented videos of an actor either performing intrinsic meaningful gestures in the context of German or Russian sentences, or speaking a German sentence without gesture, while performing a content judgment task.The connectivity analyses resulted in a winning model that included bidirectional intrinsic connectivity between all areas. Furthermore, the model included modulations of both connections to the STS (OC→STS; MTG→STS), and non-linear modulatory effects of the STS on bidirectional connections between MTG and OC. Coupling strength in the occipital pathway (OC→STS) correlated with gesture related advantages in task performance, whereas the temporal pathway (MTG→STS) correlated with performance in the speech only condition. Coupling between MTG and OC correlated negatively with subsequent memory performance for sentences of the Gesture-German condition.Our model provides a first step towards a better understanding of speech-gesture integration on network level. It corroborates the importance of the STS during audio-visual integration by showing that this region inhibits direct auditory-visual coupling.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
