Effects of clear speech and language background on multimodal perception of English fricatives

Abstract

Research shows that acoustic modifications in clearly enunciated fricative consonants (relative to the plain, conversational productions) facilitate auditory fricative perception. However, clear-speech effects on visual fricative perception have received less attention. A comparison of auditory and visual (facial) clear-fricative perception is particularly interesting since sibilant fricatives in English are more auditorily salient while non-sibilants are more visually salient. This study thus examines clear-speech effects on audio-visual perception of English sibilant and non-sibilant fricatives. Native English perceivers and non-native perceivers with different L1 fricative inventories (Mandarin, Korean) identified clear and plain fricative-vowel syllables in audio-only (AO), visual-only (VO), and audio-visual (AV) modes. The results across perceiver groups and speech styles showed an overall visual benefit and auditory dominance (AV>AO>VO). Comparisons of styles revealed clear-speech benefits in AO across fricatives and groups, but different patterns were noted in the visual conditions. In VO and AV, clear speech helped the more visually salient non-sibilant identification for native perceivers; however, clear-speech benefits were less prominent in non-natives’ perception of the non-sibilants, which are non-existent in their L1s. These findings are discussed in terms of the relative audio-visual weighting that benefits perception in clear speech as a function of input saliency and perceiver experience.