Models Of Stereopsis Research Articles

The mixed-polarity benefit of stereopsis arises in early visual cortex.

Depth perception is better when observers view stimuli containing a mixture of bright and dark visual features. It is currently unclear where in the visual system sensory processing benefits from the availability of different contrast polarity. To address this question, we applied transcranial magnetic stimulation to the visual cortex to modulate normal neural activity during processing of single- or mixed-polarity random-dot stereograms. In line with previous work, participants gave significantly better depth judgments for mixed-polarity stimuli. Stimulation of early visual cortex (V1/V2) significantly increased this benefit for mixed-polarity stimuli, and it did not affect performance for single-polarity stimuli. Stimulation of disparity responsive areas V3a and LO had no effect on perception. Our findings show that disparity processing in early visual cortex gives rise to the mixed-polarity benefit. This is consistent with computational models of stereopsis at the level of V1 that produce a mixed polarity benefit.

A Model for Stereopsis and Rivalry Based on Orientation Differences

A Model for Stereopsis and Rivalry Based on Orientation Differences

Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: A method for converting neural rate models into spiking models

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model shows how spiking neurons that interact in hierarchically organized laminar circuits of the visual cortex can generate analog properties of 3D visual percepts. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model suggests how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain how computationally complementary boundary and surface formation properties lead to a single consistent percept, eliminate redundant 3D boundaries, and trigger figure-ground perception. The model also shows how false binocular boundary matches may be eliminated by Gestalt grouping properties. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The 3D sLAMINART model simulates 3D surface percepts that are consciously seen in 18 psychophysical experiments. These percepts include contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and perceptual closure. The model hereby illustrates a general method of unlumping rate-based models that use the membrane equations of neurophysiology into models that use spiking neurons, and which may be embodied in VLSI chips that use spiking neurons to minimize heat production.

An Investigation of the Venetian Blind Effect

When a rectangular wave grating is binocularly viewed with a neutral density filter over one eye, an illusory rotation resembling that of a partially opened Venetian blind is perceived (Cibis and Haber, 1951). Using a binary classification task, in the first experiment, the probability of perceiving a rotation in a given direction was measured as a function of a factorial combination of inter-ocular contrast (see Note 1) and luminance ratios. The probability of a rotation in a given direction decreased monotonically with the luminance of the brighter bars when the grating contains a less than unity contrast. This result is inconsistent with (i) the model of the Venetian blind effect proposed by Cibis and Haber (1951), (ii) a mechanism based on irradiation with a compressive non-linearity (von Helmholtz, 1911/1924, pp. 186-193) and (iii) contemporary stereo-energy/cross-correlation models of stereopsis. In the second and third experiments, we tested the prediction that irradiation combined with an early compressive non-linearity in response implies a positive relationship between both the threshold contrast or average luminance disparity to perceive rotation and the magnitude of perceived rotation, and the blur width at the bar's edge. No support was found for the prediction. We propose an intensity difference model of the probability of perceiving a rotation in a given direction as a function of the interocular difference in luminance or contrast.

A laminar cortical model of stereopsis and 3D surface perception of complex natural scenes

A laminar cortical model of stereopsis and 3D surface perception of complex natural scenes

A computational model of stereopsis that produces depth from interocularly unpaired points as well as binocular rivalry

A computational model of stereopsis that produces depth from interocularly unpaired points as well as binocular rivalry

Interaction between binocular rivalry and depth in plaid patterns

Binocular rivalry was studied using plaids which were the sum of orthogonal diagonal gratings plus identical vertical gratings in the two eyes. The rivalry alternations sped up as the spatial frequency difference between the vertical and diagonal gratings was increased above about one octave, but slowed down for smaller differences. The interaction between depth and rivalry was studied using similar plaids but with depth introduced in the vertical components. Depth and rivalry coexisted when the spatial frequency difference between the vertical and diagonal gratings was greater than about one octave, but rivalry slowed down and depth perception was reduced for smaller differences. Plaids consisting of square wave gratings were used to compare: (1) added gratings; (2) vertical gratings superimposed on (i.e. occluding) diagonal gratings; (3) diagonal gratings superimposed on vertical gratings. Rivalry alternations were fastest in condition (3), indicating that grouping effects played a role. The final experiment indicated that depth and rivalry coexisted within a spatial frequency band if the orientation difference between the vertical and diagonal components was 60–70°. These results place constraints on models of stereopsis and rivalry, indicating that depth and rivalry can coexist in different spatial frequency and orientation bands but that each interferes with the other in the same band.

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.

The efficiency of depth discrimination for non-transparent and transparent stereoscopic surfaces

The perception of transparency in binocular vision presents a challenge for any model of stereopsis. We investigate here how well human observers cope with stereo transparency by comparing their efficiency between transparent and opaque depth judgments. In two experiments, the efficiency measure was computed relative to an ideal observer to take into account the larger correspondence ambiguity in the transparent condition. We found that thresholds for human and ideal observers were consistently higher in the transparent condition than in the opaque condition, across a range of dot densities (Experiment 1) and disparity ratios (Experiment 2). Efficiencies (the ratio of human to ideal performance) were approximately equivalent for the opaque and transparent conditions across all stimulus conditions. Therefore, the cost for stereoscopic transparency can be accounted for by the greater correspondence problem in that condition. Indeed, the fact that efficiencies were very low, around 1%, and decreased with increasing dot density demonstrates that human observers use far less information than is available to perform the task. This account contrasts with previous interpretations for the cost in stereoscopic transparency in terms of inhibitory interactions specific to transparent configurations. We relate our findings to a previous and comparable study of motion efficiency, and discuss our findings in terms of a physiologically plausible model.

A Laminar Cortical Model of Stereopsis and 3D Surface Perception: Closure and da Vinci Stereopsis

A Laminar Cortical Model of Stereopsis and 3D Surface Perception: Closure and da Vinci Stereopsis

The efficiency of depth discrimination for non-transparent and transparent stereoscopic surfaces

The perception of transparency in binocular vision presents a challenge for any model of stereopsis. We investigate here how well human observers cope with stereo transparency by comparing their efficiency between transparent and opaque depth judgments. In two experiments, the efficiency measure was computed relative to an ideal observer to take into account the larger correspondence ambiguity in the transparent condition. We found that thresholds for human and ideal observers were consistently higher in the transparent condition than in the opaque condition, across a range of dot densities (Experiment 1) and disparity ratios (Experiment 2). Efficiencies (the ratio of human to ideal performance) were approximately equivalent for the opaque and transparent conditions across all stimulus conditions. Therefore, the cost for stereoscopic transparency can be accounted for by the greater correspondence problem in that condition. Indeed, the fact that efficiencies were very low, around 1%, and decreased with increasing dot density demonstrates that human observers use far less information than is available to perform the task. This account contrasts with previous interpretations for the cost in stereoscopic transparency in terms of inhibitory interactions specific to transparent configurations. We relate our findings to a previous and comparable study of motion efficiency, and discuss our findings in terms of a physiologically plausible model.

A laminar cortical model of stereopsis and three-dimensional surface perception

A laminar cortical model of stereopsis and later stages of 3D surface perception is developed and simulated. The model describes how initial stages of monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the lateral geniculate nucleus and cortical areas V1, V2, and V4. In particular, it details how interactions between layers 4, 3B, and 2/3A in V1 and V2 contribute to stereopsis, and clarifies how binocular and monocular information combine to form 3D boundary and surface representations. Along the way, the model modifies and significantly extends the disparity energy model. Neural explanations are given for psychophysical data concerning: contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum’s limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and various lightness illusions. By relating physiology to psychophysics, the model provides new functional insights and predictions about laminar cortical architecture.

A Bayesian model of stereopsis depth and motion direction discrimination.

The extraction of stereoscopic depth from retinal disparity, and motion direction from two-frame kinematograms, requires the solution of a correspondence problem. In previous psychophysical work [Read and Eagle (2000) Vision Res 40: 3345-3358], we compared the performance of the human stereopsis and motion systems with correlated and anti-correlated stimuli. We found that, although the two systems performed similarly for narrow-band stimuli, broadband anti-correlated kinematograms produced a strong perception of reversed motion, whereas the stereograms appeared merely rivalrous. I now model these psychophysical data with a computational model of the correspondence problem based on the known properties of visual cortical cells. Noisy retinal images are filtered through a set of Fourier channels tuned to different spatial frequencies and orientations. Within each channel, a Bayesian analysis incorporating a prior preference for small disparities is used to assess the probability of each possible match. Finally, information from the different channels is combined to arrive at a judgement of stimulus disparity. Each model system--stereopsis and motion--has two free parameters: the amount of noise they are subject to, and the strength of their preference for small disparities. By adjusting these parameters independently for each system, qualitative matches are produced to psychophysical data, for both correlated and anti-correlated stimuli, across a range of spatial frequency and orientation bandwidths. The motion model is found to require much higher noise levels and a weaker preference for small disparities. This makes the motion model more tolerant of poor-quality reverse-direction false matches encountered with anti-correlated stimuli, matching the strong perception of reversed motion that humans experience with these stimuli. In contrast, the lower noise level and tighter prior preference used with the stereopsis model means that it performs close to chance with anti-correlated stimuli, in accordance with human psychophysics. Thus, the key features of the experimental data can be reproduced assuming that the motion system experiences more effective noise than the stereoscopy system and imposes a less stringent preference for small disparities.

Spatial interactions minimize relative disparity between adjacent surfaces

Computational models of stereopsis employ a number of algorithms that constrain stereo matches to produce the smallest absolute disparity and to minimize the relative disparity between nearby features. In some natural scenes, such as large slanted textured surfaces, these two constraints lead to different matching solutions. The current study utilized a stimulus in which there was a large discrepancy in both the magnitude and direction of matches that solved for minimum absolute and minimum relative disparity. This discrepancy revealed a dominance for the minimum relative disparity over the minimum absolute disparity matching solution that increased with spatial proximity, spatial frequency and width of adjacent features. The likelihood of a minimum-relative-disparity matching solution also increased when the difference between the amplitudes of the alternative relative disparities was large. When alternative relative disparity matching solutions had similar amplitudes but opposite signs (crossed vs. uncrossed), an idiosyncratic depth bias served as a tie-breaker. The present results show that absolute disparity matches are constrained to minimize relative disparity between adjacent features.

Stereo correspondence in one-dimensional Gabor stimuli

Previous data [Prince, S.J.D., & Eagle, R.A., (1999). Size-disparity correlation in human binocular depth perception. Proceedings of the Royal Society of London B, 266, 1361–1365] have demonstrated that the upper disparity limit for stereopsis (DMax) is considerably smaller in filtered noise stereograms than in isolated Gabor patches of the same spatial frequency. This discrepancy is not currently understood. Here, the solution of the correspondence problem for bandpass stereograms was further examined. On each trial observers were presented with two one-dimensional Gabor stimuli containing disparities of equal magnitude but opposite sign. Subjects were required to indicate which interval contained the crossed disparity stimulus. It was found that matching behaviour changed as a function of Gabor envelope size. As a function of disparity magnitude, performance cycled between mostly correct and mostly incorrect at large envelope sizes but was always correct at small envelope sizes. At intermediate envelope sizes performance was cyclical at small disparities but always correct at large disparities. The critical envelope size at which performance changed from mostly correct to mostly incorrect at 270° phase disparity was used as a measure of the matching performance as other parameters of the Gabor were varied. Both absolute and relative contrast were shown to influence the perceived sign of matches. Critical envelope size was also found to decrease as a function of spatial frequency, but more slowly than a phase-based limit would predict. These data cannot be predicted by current models of stereopsis, and can be used to constrain future models.

Cortical hypercolumn size determines stereo fusion limits.

The size of a pair of cortical ocular dominance columns determines a basic anatomical module of V-1 which Hubel and Wiesel have termed the hypercolumn. Does this correspond to a basic functional, or psychophysically measurable, module as well? This is the basic question addressed in the present paper. Since the ocular dominance column architecture is presumed to be related to stereo vision, it is natural to assume that hypercolumn size should provide a modular basis for basic phenomena of stereopsis. In previous work, we have suggested that local nonlinear filtering via the cepstral transform, operating on a local window of cortical tissue scaled by hypercolumn size, provides such a modular model of stereopsis. In the present paper, we review this model and then discuss a number of issues related to the biological plausibility and implementation of this algorithm. Then, we present the main result of this paper: we have analyzed a number of experiments related to stereo fusion limits (Panum's area) and to disparity gradient and disparity scaling, and demonstrate that there is a simple unifying explanation for these phenomena in terms of a constant cortical module whose size is determined by a pair of ocular dominance columns. As a corollary, Panum's area must increase according to (inverse) cortical magnification factor. We show that this is supported by all existing experimental data.

Cortical computation of stereo disparity.

Our ability to see the world in depth is a major accomplishment of the brain. Previous models of how positionally disparate cues to the two eyes are binocularly matched limit possible matches by invoking uniqueness and continuity constraints. These approaches cannot explain data wherein uniqueness fails and changes in contrast alter depth percepts, or where surface discontinuities cause surfaces to be seen in depth, although they are registered by only one eye (da Vinci stereopsis). A new stereopsis model explains these depth percepts by proposing how cortical complex cells binocularly filter their inputs and how monocular and binocular complex cells compete to determine the winning depth signals.

Birth of a learning law

Our ability to see the world in depth is a major accomplishment of the brain. Previous models of how positionally disparate cues to the two eyes are binocularly matched limit possible matches by invoking uniqueness and continuity constraints. These approaches cannot explain data wherein uniqueness fails and changes in contrast alter depth percepts, or where surface discontinuities cause surfaces to be seen in depth, although they are registered by only one eye (da Vinci stereopsis). A new stereopsis model explains these depth percepts by proposing how cortical complex cells binocularly filter their inputs and how monucular and binocular complex cells compete to determine the winning depth signals.

Cortical dynamics of three-dimensional surface perception: Binocular and half-occluded scenic images

Previous models of stereopsis have concentrated on the task of binocularly matching left and right eye primitives uniquely. A disparity smoothness constraint is often invoked to limit the number of possible matches. These approaches neglect the fact that surface discontinuities are both abundant in natural everyday scenes, and provide a useful cue for scene segmentation. da Vinci stereopsis refers to the more general problem of dealing with surface discontinuities and their associated unmatched monocular regions within binocular scenes. This study develops a mathematical realization of a neural network theory of biological vision, called FACADE theory, that shows how early cortical stereopsis processes are related to later cortical processes of three-dimensional surface representation. The mathematical model demonstrates through computer simulation how the visual cortex may generate three-dimensional boundary segmentations and use them to control filling-in of three-dimensional surface properties in response to visual scenes. Model mechanisms correctly match disparate binocular regions while filling-in monocular regions with the correct depth within a binocularly viewed scene. This achievement required the introduction of a new multiscale binocular filter for stereo matching which clarifies how cortical complex cells match image contours of like contrast polarity, while pooling signals from opposite contrast polarities. The filter also suggests how false binocular matches and unmatched monocular cues are automatically handled across multiple spatial scales. Pooling of signals from even- and odd-symmetric simple cells at complex cells helps to eliminate spurious activity peaks in matchable signals. Later stages of cortical processing by the blob and interblob streams, including refined models of cooperative boundary grouping and reciprocal stream interactions between boundary and surface representations, are modeled to provide a complete simulation of the da Vinci stereopsis percept.

Fine-scale processing in human binocular stereopsis.

Many studies have demonstrated that the human visual system is sensitive to very small differences in relative binocular disparity. It is not known over what monocular regions information is spatially integrated to mediate performance in such tasks. In this study we present psychophysical observations that define the smallest spatial scale involved in disparity processing, and we indicate the nature of the computations performed by the units mediating that disparity discrimination. We show that human observers can identify the sign of disparity of a single target dot when it is embedded in a row of identical dots, with these noise dots presented either in the fixation plane or with a proportion binocularly uncorrelated. In conjunction with the psychophysical data, we explore how a class of simple correlator models of stereopsis must be constrained in order to account for human performance for the same fine-scale tasks. Such models can perform the task only when the correlation is carried out over a very small region of the image, for a very small range of disparities. Our results demonstrate that there is a fine-scale input to the stereo system, mediated by foveal mechanisms that spatially integrate visual signals over a region as small as 4-6 arcmin in diameter.