When Lips and Voice Disagree: Determining the Practical Limits and Consequences of Visual-Auditory Asynchroity

Abstract

This talk explores the limits and costs of the visual-auditory asynchrony that occurs in video teleconferencing systems using separate transmission paths for the video and audio signals. After a brief review of video teleconferencing, the special problem of asynchrony in two-path systems is developed, and the small quantity of directly applicable research is reviewed. The two human factors questions which needed to be answered were: 1) What are the “just tolerable” limits of asynchrony?, and 2) What is the cost in terms of misperceptions of living with asynchrony? The experiment had nine participants determine their “just tolerable” asynchrony limits with video first and with audio first, and their “perfect synchrony” point. The average “just tolerable” limit with video preceding audio was 104 msec with a small variability. Very surprisingly, the “just tolerable” limit with audio first was at least 160 msec. Thus, common wisdom not withstanding, it is apparently easier to live with the audio preceding the video. Research is currently underway to measure the cost of living with asynchrony. MacDonald and McGurk, 1978, found that particular combinations of spoken and seen syllables led to the perception of completely different syllables. The present research extends MacDonald and McGurk' work to word pairs with first syllables from the corresponding special syllable pairs to determine if living with asynchrony necessarily means living with misperceptions in addition to just simple “annoyance”.