Asymmetric discrimination of phonetically incongruent audio-visual vowels

Abstract

Masapollo, Polka, and Ménard (2017) recently reported a robust directional asymmetry in visual vowel perception: perceivers discriminated a change from an English /u/ viseme to a French /u/ viseme significantly better than a change in the reverse direction. This asymmetry parallels a frequent pattern in auditory vowel perception that points to a universal bias favoring “focal” vowels, whose distal articulatory gestures result in the convergence of adjacent formant frequencies. In the present study, we investigated how the integration of acoustic and visual speech cues influences these directional effects in bimodal vowel perception. In AX discrimination tests in which acoustic and visual cues were phonetically-incongruent, subjects showed an asymmetry, comparable to that found with unimodal stimuli, when there was an acoustic change in focalization, with visual focalization held constant. These findings indicate that acoustic cues are capable of inducing an asymmetry even when there is no cross-modal correspondence between the acoustic and visual channels. We are currently examining whether a similar asymmetry also emerges when there is a visual change in focalization, with acoustic focalization held constant. By comparing different types of audio-visual incongruence, we will be able to examine whether one sensory channel has more perceptual potency than the other.