Talker variability effects in auditory-visual speech perception

Abstract

Talker variability effects have been demonstrated for many aspects of syllable, word, and sentence recognition for stimuli presented in the auditory-only modality. In the current investigation, talker variability effects were examined in multiple modalities—visual-only, auditory-only, and auditory-visual. Auditory-visual presentations were of two types: consonant [auditory and visual were of the same token—/visual bi and auditory bi/] or disparate [/visual gi and auditory bi or visual bi and auditory gi/]. The disparate condition has elicited ‘‘the McGurk effect,’’ in which listeners may report a fused response that is neither the auditory or the visual component of the stimulus. Eleven talkers (five male and six female) of varying cultural backgrounds were videotaped. Their productions of CV syllables were presented to adult listeners who reported their percepts. Individual talkers elicited different degrees of syllable fusion among listeners, despite their equivalent auditory intelligibility. In addition, individual listeners varied considerably in their overall perceptions, from complete fusion for all talkers to almost no fusion for any talker. Results suggest that listeners may employ different types of processing strategies for bimodal stimuli. [Research supported by NIDCD.]