Can co-speech hand gestures facilitate learning of non-native tones?

Abstract

Speech perception research has indicated that information from multiple input modalities (e.g., auditory, visual) facilitates second language (L2) speech learning. However, co-speech gestural information has shown mixed results. While L2 learners may benefit from this additional channel of information, it may also be inhibitory as learners may experience excessive cognitive load. This study examines the role of metaphoric hand gestures in L2 lexical tone learning using previously established laboratory training procedures. Training stimuli include Mandarin tones produced by native Mandarin speakers, with concurrent hand gestures mimicking pitch contours in space. Native Canadian English speakers are trained to perceive tones presented in one of three modalities: audio-visual (AV, speaker voice and face), audio-gesture (AG, speaker voice and hand gestures) and audio-visual-gesture (AVG). The effects of training are assessed by comparing the pre-training and post-training tone identification results. Greater improvements for the AVG compared to AV group would indicate the facilitative role of gestures. However, greater improvements for the AG or AV compared to AVG group would support the cognitive overload account. Findings are discussed in terms of how sensory-motor and cognitive domains cooperate functionally in speech perception and learning. [Equal contributions by KE, BH, and YW; work supported by SSHRC.]