Consequences of audiovisual asynchrony for speech perception: Implications for signal processing in aids to lipreading

Abstract

Audiovisual identification of sentences was measured as a function of audio delay in untrained listeners with normal hearing; the sound track was replaced by rectangular pulses originally synchronized to the closing of the talker's vocal folds and then subjected to delay. Although group‐mean performance declined monotonically with delay, systematic decrements occurred only when delay exceeded 80 ms. A similar tolerance of delay was found in judgments of audiovisual onset time when observers determined whether a 120‐Hz triangular wave started before or after the opening of a pair of liplike Lissajou figures. Group‐mean 70% DLs were − 78 ms (sound leading) and + 137 ms (sound lagging). This result suggests, first, that most observers possess insufficient sensitivity to intermodal timing cues in audiovisual speech for them to be used analogously to VOT in auditory speech perception, and, second, that the effects found in the first experiment derive from syllabic rather than phonemic interference. However, th...