Hearing and Seeing Tone Through Color: An Efficacy Study of Web‐Based, Multimodal Chinese Tone Perception Training

Abstract

AbstractMultimodal approaches have been shown to be effective for many learning tasks. In this study, we compared the effectiveness of five multimodal methods for second language (L2) Mandarin tone perception training: three single‐cue methods (number, pitch contour, color) and two dual‐cue methods (color and number, color and pitch contour). A total of 303 true novice learners of L2 Mandarin (native speakers of English) completed a 3‐week online training program. Results from pretests as well as immediate and delayed posttests indicated that multimodal training aided L2 learners’ tone perception, with a small, practical advantage for pitch contours and numbers over color coding. Dual‐cue methods did not yield better learning than single‐cue training. Thus, the additive benefits of multimodal input (i.e., auditory and visual) did not extend to instruction featuring doubled visual input (i.e., visual and visual). We argue for embedding color in visuals in a way that helps make abstract information concrete.