Size-disparity Correlation Research Articles

Size matters: Perceived depth magnitude varies with stimulus height

Both the upper and lower disparity limits for stereopsis vary with the size of the targets. Recently, Tsirlin, Wilcox, and Allison (2012) suggested that perceived depth magnitude from stereopsis might also depend on the vertical extent of a stimulus. To test this hypothesis we compared apparent depth in small discs to depth in long bars with equivalent width and disparity. We used three estimation techniques: a virtual ruler, a touch-sensor (for haptic estimates) and a disparity probe. We found that depth estimates were significantly larger for the bar stimuli than for the disc stimuli for all methods of estimation and different configurations. In a second experiment, we measured perceived depth as a function of the height of the bar and the radius of the disc. Perceived depth increased with increasing bar height and disc radius suggesting that disparity is integrated along the vertical edges. We discuss size–disparity correlation and inter-neural excitatory connections as potential mechanisms that could account for these results.

Size matters: Perceived depth magnitude varies with stimulus height

Both the upper and lower disparity limits for stereopsis vary with the size of the targets. Recently, Tsirlin, Wilcox, and Allison (2012) suggested that perceived depth magnitude from stereopsis might also depend on the vertical extent of a stimulus. To test this hypothesis we compared apparent depth in small discs to depth in long bars with equivalent width and disparity. We used three estimation techniques: a virtual ruler, a touch-sensor (for haptic estimates) and a disparity probe. We found that depth estimates were significantly larger for the bar stimuli than for the disc stimuli for all methods of estimation and different configurations. In a second experiment, we measured perceived depth as a function of the height of the bar and the radius of the disc. Perceived depth increased with increasing bar height and disc radius suggesting that disparity is integrated along the vertical edges. We discuss size-disparity correlation and inter-neural excitatory connections as potential mechanisms that could account for these results.

Spatial stereoresolution for depth corrugations may be set in primary visual cortex

Stereo depth perception has recently been modelled based on local cross-correlation between the left and the right eye’s images. This model, which is based on the known physiology of primary visual cortex (V1), has successfully explained many aspects of stereo vision. In particular, it has explained the low spatial stereoresolution for sinusoidal depth corrugations [1,2], suggesting that the limit on stereoresolution may be set in V1. In accordance with the properties of V1 neurons, the disparity detectors used in this model are tuned to locally uniform patches of disparity. Consequently, the model responds better to high amplitude square-wave corrugations than to high amplitude sine-waves, because the square-waves are locally flat while the sinusoidal corrugations are slanted almost everywhere and this slant is particularly large at large amplitudes. The model therefore predicts better performance at detecting square-wave than sine-wave disparity corrugations at high amplitudes. However, in contradiction with this prediction of the model we have recently shown that humans perform no better at detecting square-waves than sine-waves even at high amplitudes [3]. This failure of the model raised the question of whether stereoresolution is not set in V1 but at some later stage of cortical processing, for example involving neurons tuned to slant or curvature or whether a modified version of the model, incorporating more of the known physiology, may explain the new results with square-waves. We have tested a modified version of the local cross-correlation model which, based on psychophysical and physiological evidence that larger disparities are detected by neurons with larger receptive fields (a size-disparity correlation), uses larger windows to detect larger disparities. We show that the performance of this modified model is consistent with the human results, confirming that stereoresolution may indeed be limited by V1 receptive field sizes.

Stereo channels with different temporal frequency tunings

To investigate the spatial and temporal frequency tunings for stereopsis, we measured the contrast sensitivity for depth discrimination with variable spatiotemporal frequencies and disparities using drifting sinusoidal gratings. The results showed that the contrast sensitivity changed with the stimulus disparity and the disparity tuning function varied with the spatial frequency. The disparity in the peak sensitivity decreased proportionally with the spatial frequency (size-disparity correlation). Although the temporal frequency exhibited a limited influence on the peak disparity, the temporal frequency tuning varied with the spatial frequency. The shape of the temporal frequency tuning function was lowpass for higher spatial frequencies, whereas it was bandpass for low spatial frequencies. These results suggest that more than one channel with different temporal as well as spatial frequency tunings contribute to stereopsis.

Range and Mechanism of Encoding of Horizontal Disparity in Macaque V1

The responses of single cortical neurons were measured as a function of the binocular disparity of dynamic random dot stereograms for a large sample of neurons (n = 787) from V1 of the awake macaque. From this sample, we selected 180 neurons whose tuning curves were strongly tuned for disparity, well sampled and well described by one-dimensional Gabor functions. The fitted parameters of the Gabor functions were used to resolve three outstanding issues in binocular stereopsis. First, we considered whether tuning curves can be meaningfully divided into discrete tuning types. Careful examination of the distributions of the Gabor parameters that determine tuning shape revealed no evidence for clustering. We conclude that a continuum of tuning types is present. Second, we investigated the mechanism of disparity encoding for V1 neurons. The shape of the disparity tuning function can be used to distinguish between position-encoding (in which disparity is encoded by an interocular shift in receptive field position) and phase-encoding (in which disparity is encoded by a difference in the receptive field profile in the 2 eyes). Both position and phase encoding were found to be common. This was confirmed by an independent assessment of disparity encoding based on the measurement of disparity sensitivity for sinusoidal luminance gratings of different spatial frequencies. The contributions of phase and position to disparity encoding were compared by estimating a population average of the rate of change in firing rate per degree of disparity. When this was calculated separately for the phase and position contributions, they were found to be closely similar. Third, we investigated the range of disparity tuning in V1 as a function of eccentricity in the parafoveal range. We find few cells which are selective for disparities greater than +/-1 degrees even at the largest eccentricity of approximately 5 degrees. The preferred disparity was correlated with the spatial scale of the tuning curve, and for most units lay within a +/-pi radians phase limit. Such a size-disparity correlation is potentially useful for the solution of the correspondence problem.

Weighted directional energy model of human stereo correspondence

Previous work [Prince, S. J. D, & Eagle, R. A. (1999). Size-disparity correlation in human binocular depth perception. Proceedings of the Royal Society: Biological Sciences, 266, 1361–1365] has demonstrated that disparity sign discrimination performance in isolated bandpass patterns is supported at disparities much larger than a phase disparity model might predict. One possibility is that this extended performance relies on a separate second-order system [Hess, R. F., & Wilcox, L. M. (1994). Linear and non-linear filtering in stereopsis. Vision Research, 34, 2431–2438]. Here, a ‘weighted directional energy’ model is developed which explains a large body of crossed versus uncrossed disparity discrimination data with a single mechanism. This model assumes a population of binocular complex cells at every image point with a range of position disparity shifts. These cells sample a local energy function which is weighted so that energy at large disparities is relatively attenuated. Disparity sign is determined by summing and comparing energy at crossed and uncrossed disparities in the presence of noise. The model qualitatively predicts matching data for one-dimensional Gabor stimuli. This scheme also predicts DMax in Gabor stimuli and filtered noise. Moreover, a range of ‘non-linear’ phenomena, in which disparity is perceived from contrast envelope information alone, can be explained. The weighted directional energy model presents a biologically plausible, parsimonious explanation of matching behaviour in bandpass stimuli for both ‘first-order’ and ‘second-order’ stimuli which obviates the need for multiple mechanisms in stereo correspondence.

Stereo correspondence in one-dimensional Gabor stimuli

Previous data [Prince, S.J.D., & Eagle, R.A., (1999). Size-disparity correlation in human binocular depth perception. Proceedings of the Royal Society of London B, 266, 1361–1365] have demonstrated that the upper disparity limit for stereopsis (DMax) is considerably smaller in filtered noise stereograms than in isolated Gabor patches of the same spatial frequency. This discrepancy is not currently understood. Here, the solution of the correspondence problem for bandpass stereograms was further examined. On each trial observers were presented with two one-dimensional Gabor stimuli containing disparities of equal magnitude but opposite sign. Subjects were required to indicate which interval contained the crossed disparity stimulus. It was found that matching behaviour changed as a function of Gabor envelope size. As a function of disparity magnitude, performance cycled between mostly correct and mostly incorrect at large envelope sizes but was always correct at small envelope sizes. At intermediate envelope sizes performance was cyclical at small disparities but always correct at large disparities. The critical envelope size at which performance changed from mostly correct to mostly incorrect at 270° phase disparity was used as a measure of the matching performance as other parameters of the Gabor were varied. Both absolute and relative contrast were shown to influence the perceived sign of matches. Critical envelope size was also found to decrease as a function of spatial frequency, but more slowly than a phase-based limit would predict. These data cannot be predicted by current models of stereopsis, and can be used to constrain future models.

Size-disparity correlation in human binocular depth perception.

To use the small horizontal disparities between images projected to the eyes for the recovery of three-dimensional information, our visual system must first identify which feature in one eye's image corresponds with which in the other. The earliest level of disparity processing in primates (V1) contains cells that are spatial-frequency tuned. If such cells have a disparity range that covers only a single period of their mean tuning frequency, there will always be exactly one potential match within this range. Here, this 'size-disparity' hypothesis was tested by measuring the contrast sensitivity of stereopsis as a function of disparity for single bandpass-filtered items. It was found that thresholds were low and relatively constant up to disparities an order of magnitude larger than is predicted by this constraint. Furthermore, peak sensitivity was relatively independent of spatial frequency. A control experiment showed that binocular correlation of the carrier is necessary for this task. In a third experiment, the maximum disparity that supports threshold performance was compared for an isolated bandpass item and bandpass-filtered noise. This limit was found to be five times larger for the isolated stimuli. In summary, these findings show that the initial stage of disparity detection is not limited by the size-disparity constraint. For stimuli with multiple false targets, however, processes subsequent to this stage reduce the disparity range over which the correspondence problem can be solved.

Decreases in the critical disparity gradient with eccentricity may reflect the size-disparity correlation.

The interaction between dot separation and fusible disparity limits (critical disparity gradient) was assessed at 0, 2, 4, and 8 deg eccentricity by 19 subjects. The critical disparity gradient showed a significant decrease with eccentricity such that larger separations were required for fusion of a given disparity as eccentricity increased. An argument is put forth that this relationship supports a size-disparity correlation. The significant disparity-eccentricity interaction may be explained by both the progressive loss of higher-spatial-frequency channels with eccentricity and a reduction in the range of disparities processed by a channel as eccentricity is increased. Individual differences in the trend across eccentricity are noted and discussed.

Neuronal Mechanisms Underlying Stereopsis: How Do Simple Cells in the Visual Cortex Encode Binocular Disparity?

Binocular neurons in the visual cortex are thought to form the neural substrate for stereoscopic depth perception. How are the receptive fields of these binocular neurons organized to encode the retinal position disparities that arise from binocular parallax? The conventional notion is that the two receptive fields of a binocular neuron have identical shapes, but are spatially offset from the point of retinal correspondence (zero disparity). We consider an alternative disparity-encoding scheme, in which the two receptive fields may differ in shape (or phase), but are centered at corresponding retinal locations. Using a reverse-correlation technique to obtain detailed spatiotemporal receptive-field maps, we provide support for the latter scheme. Specifically, we show that receptive-field profiles for the left and right eyes are matched for cells that are tuned to horizontal orientations of image contours. However, for neurons tuned to vertical orientations, the left and right receptive fields are predominantly dissimilar in shape. These results show that the striate cortex possesses a specialized mechanism for processing vertical contours, which carry the horizontal-disparity information needed for stereopsis. Thus, in a major modification to the traditional notion of the neural basis of stereopsis, we propose that binocular simple cells encode horizontal disparities in terms of phase at multiple spatial scales. Implications of this scheme are discussed with respect to the size-disparity correlation observed in psychophysical studies.

Size–disparity correlation in stereopsis at contrast threshold

Contrast thresholds for 75% correct depth identification in narrow-band filtered random dot stereograms were determined for different center spatial frequencies and binocular disparities. Rigorous control over vergence was maintained during testing, and a forced-choice procedure was used. The resulting contrast sensitivity function for stereopsis revealed sensitivity over a greater range of disparities at low than at high spatial frequencies. Sensitivity peaked for large disparities at low spatial frequencies and for small disparities at high spatial frequencies. When disparities were converted to effective binocular phase differences, the variation of contrast sensitivity with phase followed a consistent pattern across spatial frequencies, with peak sensitivity occurring mainly for binocular phases of between 90 degrees and 180 degrees. These results have implications for the extent of spatial integration at the input to the disparity sensing mechanism. A model postulating a spread of positional disparities independent of the spatial frequency selectivity of disparity-sensitive units cannot account for the results. But the size-disparity correlation strongly evident in our data is predicted by certain models of stereopsis, such as phase disparity encoding. An ideal observer analysis is developed that demonstrates that our results were not forced by the nature of the stimulus employed; rather, the quantum efficiency for stereopsis at contrast threshold follows the size-disparity correlation.

Stereo boundary fusion by cortical complex cells: A system of maps, filters, and feedback networks for multiplexing distributed data

A neural network model of multiple-scale binocular fusion and rivalry in visual cortex is described and simulated on the computer. The model consists of three parts: a distributed spatial representation of binocular input patterns among simple cells that are organized into ocular dominance columns; an adaptive filter from simple cells to complex cells; and a nonlinear on-center off-surround shunting feedback network that joins together the complex cells. This data structure generates complex cell receptive fields which multiplex input position, orientation, spatial frequency, positional disparity, and orientational disparity, and which are insensitive to direction-of-contrast in the image. Multiple copies of this circuit are replicated in the model using receptive fields of different sizes. Within each such circuit, the simple cell and complex cell receptive field sizes covary. Together these circuits define a self-similar multiple-scale network. The self-similarity property across spatial scales enables the network to exhibit a size-disparity correlation, whereby simultaneous binocular fusion and rivalry can occur among the spatial scales corresponding to a given retinal region. It is shown that a laminar organization of the model interactions among the complex cells gives rise to conceptually simple growth rules for intercellular connections. The output patterns of the model complex cells are designed to feed into the model hypercomplex cells at the first competitive stage of a Boundary Contour System network, where they trigger a process of multiple-scale emergent binocular boundary segmentation. The modeling results are compared with psychophysical data about binocular fusion and rivalry, as well as with the cepstrum stereo model of Yeshurun and Schwartz. The results indicate that analogous self-similar multiple-scale neural networks may be used to carry out data fusion of many other types of spatially organized data structures.

Spatial tuning of static and dynamic local stereopsis.

The range of spatial tuning for channels that process static and dynamic disparities was investigated in the central visual field by measuring stereoscopic thresholds as a function of the difference in size of spatially filtered bar-like patterns presented to the two eyes. Spatial tuning functions were revealed by an elevation of stereothreshold as the difference between the widths of bar patterns increased. Functions tuned to low spatial frequencies (0.075-2 c/deg) were classified as transient since their stereosensitivity was greater for dynamic (1 Hz) than static disparities. Functions tuned to high spatial frequencies (2.4-19 c/deg) were classified as sustained since their stereosensitivity was equal for dynamic and static disparities. When equal width patterns were presented to the two eyes, stereothreshold increased with spatial periods greater than 0.4 deg according to a constant 6 deg phase disparity. This size-disparity correlation suggests that large disparities are processed by spatial filters tuned to disparities proportional to their receptive field dimensions.

Spatial organization of binocular disparity sensitivity

The range of sensitivity to spatial modulation of disparity in a vertical line was investigated. The function obtained was not greatly altered by conditions of fixed aperture, constant number of cycles, high resolution, foveal or eccentric presentation. The simplest hypothesis to account for the observed disparity scaling of the upper disparity limit was a correlation between preferred sizes and disparities of cortical receptive fields. Binocular fusion shows a disparity scaling effect for sinusoidal variations in a vertical line, but little change for sinusoidal variations in a horizontal line or for rectangular variations in either direction. Fusion thus exhibits vertical/horizontal anisotropy but the effects are not explained by a sizedisparity correlation in cortical receptive fields.