Role of linguistic experience on audio-visual perception of English fricatives in quiet and noise backgrounds

Abstract

Previous research shows that for native perceivers, visual information enhances speech perception, especially when auditory distinctiveness decreases. This study examines how linguistic experience affects audio-visual (AV) perception of non-native (L2) speech. Native Canadian English perceivers and Mandarin perceivers with two levels of English exposure (early and late arrival in Canada) were presented with English fricative-initial syllables in a quiet and a caf-noise background in four ways: audio-only (A), visual-only (V), congruent AV, and incongruent AV. Identification results show that for all groups, performance was better in the congruent AV than A or V condition, and better in quiet than in caf-noise background. However, whereas Mandarin early arrivals approximate the native English patterns, the late arrivals showed poorer identification, more reliance on visual information, and greater audio-visual integration with the incongruent AV materials. These findings indicate that although non-natives were more attentive to visual information, they failed to use the linguistically significant L2 visual cues, suggesting language-specific AV processing. Nonetheless, these cues were adopted by the early arrivals who had more L2 exposure. Moreover, similarities across groups indicate possible perceptual universals involved. Together they point to an integrated network in speech processing across modalities and linguistic backgrounds. [Work supported by SSHRC.]