Postdiction and the effects of spatial, temporal, and feature compatibility on sensory integration

Abstract

The brain continually integrates stimuli over multiple sensory modalities and reconciles often disparate information into a unified, consistent representation of the surrounding environment. This process must be robust to differential neural latencies and imperfect alignments of spatial reference frames between sensory modalities. Numerous studies have examined the perception of multisensory stimuli with the presumption that multisensory integration is categorically different from within-modality integration. We looked at a variety of issues related to the updating of sensory reference frames and the integration of unimodal and multimodal stimuli over temporal and spatial disparities. Study 1 found simultaneous, opposite gaze-dependent aftereffects at the same retinal location for both depth and color, demonstrating the degree to which visual-coordinate space is gaze-contingent, not merely retinotopic. Study 2 found that the flash-lag effect, in which a flashed target is perceived as lagging behind a smoothly moving target, generalizes to third-order motion perception of cyclopean stimuli. Study 3 introduced a novel motion illusion which we termed the turn-point phantom, wherein the position of an abrupt orthogonal direction change is mislocalized backwards along the object's subsequent trajectory. This effect, like flash-lag, can only be adequately explained with postdiction. Study 4 explored the effect of passive head or body turns on spatial perception of visual and auditory stimuli and found systematic mislocalization of pre-turn stimuli in the direction of the turn. This mislocalization decayed with added delay between target and turn onset. Study 5 examined spatial and temporal disparity in visual-motor ventriloquism and found that early visual distracters were essentially equivalent, whereas the influence of late visual distracters diminished with increasing asynchrony. Study 6 found suppression of saccade latency induced by stimulus repetition in certain multisensory experimental contexts. Together, these studies provide numerous examples supporting the idea that sensory perception, both unimodal and multimodal, is postdictive in nature, involving integration of sensory information over a time window that includes, but does not end with, task-relevant stimulus presentation. Additionally, these results provide clues to the character and relevant parameters of the integration process.