Using and Seeing Co-speech Gesture in a Spatial Task

Abstract

Visibility of speakers’ conversational hand gestures has been shown to facilitate listeners’ comprehension. Is this true for all types of gestures? In Experiment 1, speakers were videotaped describing apartment layouts from memory. When describing complex compared to simple layouts, speakers used more representational gestures. In particular, they used more iconic–deictic gestures, or hand movements that represent both an object or action, and direction or location. This suggests that the frequency of complex representational gestures increases as a function of task difficulty. To assess the extent to which these gestures facilitate speech comprehension, in Experiment 2 a new group of participants either watched the videos of the apartment descriptions (audio + video condition) or only heard the descriptions (audio-only condition) and drew each corresponding layout. Although drawing accuracy did not differ as a function of condition, the more iconic–deictic gestures produced during the apartment descriptions, the less accurate listeners’ drawings were. Together, findings from Experiments 1 and 2 suggest that iconic–deictic gestures reflect task difficulty for the speaker and do not necessarily facilitate comprehension for the listener.