Auditory and visual cueing of the [+/- rounded] feature of vowels.

Abstract

That lipreading plays a role in phoneme recognition, even when the acoustic signal alone is phonologically unambiguous, has been concluded from experiments in the perception of discrepant combinations of acoustic and visual speech signals. Little is known about the effect of visual information on explicitly phonetic judgments, the kind of judgments made by trained observers that are the basis for describing the phonological pattern of a language. In this study some isolated vowels, most of them similar to vowels in standard French, were produced in ten random orders by an experienced phonetician. The acoustic signals and frontal views of the lower half of the speaker's face were recorded on video tape. By computer editing, audiovisual stimuli were prepared in which pairs of vowels supposed to differ primarily in rounding were variously combined. Twenty French-speaking speech researchers carried out three tasks: to decide on the rounding of each vowel by sound alone, by sight alone, and by sound when accompanied by matching or discrepant images of the talker. Their summed responses indicate that, despite the instruction to base decisions on the auditory signal, visual evidence of speech activity significantly "perturbed" subjects' rounding judgments. However, the lipreading effect varied greatly across both subjects and vowels. Most subjects judged most vowels strictly on the basis of the auditory information, while for others lipreading exerted paramount influence. Only a small minority responded so as to indicate any integration of discrepant rounding information registered by ear and eye.