The temporal resolution of binding brightness and loudness in dynamic random sequences

Abstract

Purpose. This study investigated the ways in which observers can combine dynamic visual and auditory information. Method. The stimuli were composed of gray disks accompanied by simultaneous bursts of auditory white noise. Three levels of disk brightness and of noise loudness were used to produce 9 different types of audiovisual tokens. On a given trial, 18 tokens (83ms per token) drawn from these 9 token types were presented in random order. Different conditions required participants to try to classify stimulus sequences using various target filters that gave differential weight to the 9 audiovisual token types. In each condition, a probit model was used to measure the attention filter achieved by the participant (the impact exerted on the observer’s judgments by each of the 9 token-types). The model also included terms reflecting potential non-simultaneous "misbindings" across time of auditory and visual components. Results. Participants demonstrated a high degree of strategic flexibility in achieving attention filters that varied widely across tasks, but these filters often deviated strongly from the corresponding target filters. Model fits revealed that misbindings of auditory and visual components that were 83 ms apart influenced judgments with half the strength of simultaneous components. Components that were 167 ms apart did not misbind. Conclusions. The temporal resolution of the binding achieved with these stimuli is higher than is found in tasks requiring the observer to judge the phase with which an alternating pair of visual stimuli matches up with an alternating pair of auditory stimuli (Fujisaki & Nishida, 2010). In correlating loudness and brightness, the space of attention filters that participants are able to achieve is at least three dimensional. Citation: Fujisaki, W & Nishida, S (2010). A common perceptual temporal limit of binding synchronous inputs across different sensory attributes and modalities, Proceedings of the Royal Society of London B: Biological Sciences,277(1692):2281-90. Meeting abstract presented at VSS 2012