Developmental factors and the non-native speaker effect in auditory-visual speech perception

Abstract

The study investigated language and developmental factors in the use of visual information in audiovisual speech perception in speakers from different language backgrounds. Mandarin-Chinese and English adults and 8- to 9-year-old children were presented with /ba/, /da/, and /ga/ tokens spoken by two English and two Mandarin-Chinese speakers. A syllable identification task was presented in auditory-only, visual-only, and audiovisual (congruent and incongruent) conditions in clear and in noise. The results showed an increase in the use of visual information in adults relative to children in both the Chinese and English groups. In addition, a positive correlation between the total visual effect and speechreading performance was found, suggesting that the smaller visual influence in the bimodal condition for children might be accounted by their less sophisticated speechreading ability. In regard to the language factor, it was found that Chinese perceivers use visual information in their audiovisual speech processing to the same extent as English perceivers. Finally, there was evidence for a "non-native speaker effect" (i.e., stronger visual effect for non-native speech stimuli), but only for the English participants. Results from the current study suggest that the visual appearance of individual speakers and the acoustic-phonetic properties of specific languages should be considered in future cross-language studies.