Confidence ratings in auditory–visual speech perception

Abstract

In an earlier report, Carney et al. [J. Acoust. Soc. Am. 106, 2270 (1999)] showed that the perception of the McGurk effect, the result of a visual syllable biasing the perception of a simultaneously presented auditory syllable, was affected by the use of multiple talkers. In addition, individual listeners could be strongly auditory in their perception, even when a visually biasing stimulus was present. During the original experiment, listeners were asked to rate each of their perceptions in the auditory-only (AO), visual-only (VO), and auditory–visual (AV) modes for syllabic stimuli. Listeners were very confident of their perceptions for bilabial stimuli in the VO mode but not for alveolar or velar stimuli. Confidence ratings were also high for all AO stimuli and for AV stimuli for matching visual–auditory tokens. However, confidence ratings varied for tokens with mismatched A and V stimuli. Some individual listeners were always confident in their perception regardless of speaker or condition. Other listeners varied in confidence with speaker and condition. Across listeners, certain talkers evoked different ratings of confidence as well. Results support the notion of a graded perception of bimodal stimuli. [Work supported by NIDCD.]