Cross-modal synchrony perception reveals aspects of categorical perception

Abstract

The phenomenon of "categorical perception" has played an important role in speech research. When a specific (combination of) feature(s) of a speech stimulus is varied along a physical dimension, categorical perception is reflected by two observations: (1) the percept of the sound changes abruptly from one category to another, e.g., from "ba" to "da" to "ga" for changes in formant transition frequencies, and (2) physical changes of a given amount lead to more easily perceivable differences for stimuli close to a category boundary, compared to stimuli in the center of a category. In this talk, I present data about audio-visual synchrony perception, which indicate that perceived synchrony also reveals properties of categorical perception. Depending on the physical delay between the auditory and visual component, the percept changes from "audio first," to "synchronous" to "video first." When measuring sensitivity to changes in audio-visual delays, we observe that the thresholds are small for stimuli at the transitions between the earlier determined categories, while they are larger for stimuli within a category.