Disparity Cues Research Articles

Investigating intermittent stereoscopy: its effects on perception and visual fatigue

In a context in which virtual reality making use of S3D is ubiquitous in certain industries, as well as the substantial amount of literature about the visual fatigue S3D causes, we wondered whether the presentation of intermittent S3D stimuli would lead to improved depth perception (over monoscopic) while reducing subjects’ visual asthenopia. In a between-subjects design, 60 individuals under 40 years old were tested in four different conditions, with head-tracking enabled: two intermittent S3D conditions (Stereo @ beginning, Stereo @ end) and two control conditions (Mono, Stereo). Several optometric variables were measured pre- and post-experiment, and a subjective questionnaire assessing discomfort was administered. Our results suggest a difference between simple scenes (containing few static objects, or slow, linear movement along one axis only), and more complex environments with more diverse movement. In the former case, Stereo @ beginning leads to depth perception which is as accurate as Stereo, and any condition involving S3D leads to more precision than Mono. We posit that the brain might build an initial depth map of the environment, which it keeps using after the suppression of disparity cues. In the case of more complex scenes, Stereo @ end leads to more accurate decisions: the brain might possibly need additional depth cues to reach an accurate decision. Stereo and Stereo @ beginning also significantly decrease response times, suggesting that the presence of disparity cues at task onset boosts the brain’s confidence in its initial evaluation of the environment’s depth map. Our results concerning fatigue, while not definitive, hint at it being proportional to the amount of exposure to S3D stimuli.

Modified Test Protocol Improves Sensitivity of the Stereo Fly Test

Background and PurposeStereoacuity measurement is a common element of pediatric ophthalmic examinations. Although the Stereo Fly Test is routinely used to establish the presence of coarse stereopsis (3000 arcsecs), it often yields a false negative “pass” due to learned responses and non-stereoscopic cues. We developed and evaluated a modified Stereo Fly Test protocol aimed at increasing sensitivity, thus reducing false negatives.Patients and MethodsThe Stereo Fly Test was administered according to manufacturer instructions to 321 children aged 3–12 years. Children with a “pass” outcome (n = 147) were re-tested wearing glasses fitted with polarizers of matching orientation for both eyes to verify that they were responding to stereoscopic cues (modified protocol). The response to the standard Stereo Fly Test was considered a false negative (pass) if the child still pinched above the plate after disparity cues were eliminated. Randot® Preschool Stereoacuity and Butterfly Tests were used as gold standards.Results and ConclusionsSensitivity was 81% (95% CI: 0.75 – 0.86) for standard administration of the Stereo Fly Test (19% false negative “pass”). The modified protocol increased sensitivity to 90% (95% CI: 0.85 – 0.94). The modified two-step protocol is a simple and convenient way to administer the Stereo Fly Test with increased sensitivity in a clinical setting.

The Effects of Disparity Cue and Blur Cue on the Relative Visual Comfort of Stereoscopic Content

Chinese Acad Sci, Inst Software, 4 South Fourth St, Beijing 100190, Peoples R China. Liaoning Normal Univ, Sch Comp & Informat Technol, Dalian, Peoples R China. Ctr Univ Finance & Econ, Dept Econ, Beijing, Peoples R China.

The (In)Effectiveness of Simulated Blur for Depth Perception in Naturalistic Images.

We examine depth perception in images of real scenes with naturalistic variation in pictorial depth cues, simulated dioptric blur and binocular disparity. Light field photographs of natural scenes were taken with a Lytro plenoptic camera that simultaneously captures images at up to 12 focal planes. When accommodation at any given plane was simulated, the corresponding defocus blur at other depth planes was extracted from the stack of focal plane images. Depth information from pictorial cues, relative blur and stereoscopic disparity was separately introduced into the images. In 2AFC tasks, observers were required to indicate which of two patches extracted from these images was farther. Depth discrimination sensitivity was highest when geometric and stereoscopic disparity cues were both present. Blur cues impaired sensitivity by reducing the contrast of geometric information at high spatial frequencies. While simulated generic blur may not assist depth perception, it remains possible that dioptric blur from the optics of an observer’s own eyes may be used to recover depth information on an individual basis. The implications of our findings for virtual reality rendering technology are discussed.

The effects of stereo disparity on the behavioural and electrophysiological correlates of perception of audio–visual motion in depth

Motion is represented by low-level signals, such as size-expansion in vision or loudness changes in the auditory modality. The visual and auditory signals from the same object or event may be integrated and facilitate detection. We explored behavioural and electrophysiological correlates of congruent and incongruent audio–visual depth motion in conditions where auditory level changes, visual expansion, and visual disparity cues were manipulated. In Experiment 1 participants discriminated auditory motion direction whilst viewing looming or receding, 2D or 3D, visual stimuli. Responses were faster and more accurate for congruent than for incongruent audio–visual cues, and the congruency effect (i.e., difference between incongruent and congruent conditions) was larger for visual 3D cues compared to 2D cues. In Experiment 2, event-related potentials (ERPs) were collected during presentation of the 2D and 3D, looming and receding, audio–visual stimuli, while participants detected an infrequent deviant sound. Our main finding was that audio–visual congruity was affected by retinal disparity at an early processing stage (135–160ms) over occipito-parietal scalp. Topographic analyses suggested that similar brain networks were activated for the 2D and 3D congruity effects, but that cortical responses were stronger in the 3D condition. Differences between congruent and incongruent conditions were observed between 140–200ms, 220–280ms, and 350–500ms after stimulus onset.

Shading Beats Binocular Disparity in Depth from Luminance Gradients: Evidence against a Maximum Likelihood Principle for Cue Combination.

Perceived depth is conveyed by multiple cues, including binocular disparity and luminance shading. Depth perception from luminance shading information depends on the perceptual assumption for the incident light, which has been shown to default to a diffuse illumination assumption. We focus on the case of sinusoidally corrugated surfaces to ask how shading and disparity cues combine defined by the joint luminance gradients and intrinsic disparity modulation that would occur in viewing the physical corrugation of a uniform surface under diffuse illumination. Such surfaces were simulated with a sinusoidal luminance modulation (0.26 or 1.8 cy/deg, contrast 20%-80%) modulated either in-phase or in opposite phase with a sinusoidal disparity of the same corrugation frequency, with disparity amplitudes ranging from 0’-20’. The observers’ task was to adjust the binocular disparity of a comparison random-dot stereogram surface to match the perceived depth of the joint luminance/disparity-modulated corrugation target. Regardless of target spatial frequency, the perceived target depth increased with the luminance contrast and depended on luminance phase but was largely unaffected by the luminance disparity modulation. These results validate the idea that human observers can use the diffuse illumination assumption to perceive depth from luminance gradients alone without making an assumption of light direction. For depth judgments with combined cues, the observers gave much greater weighting to the luminance shading than to the disparity modulation of the targets. The results were not well-fit by a Bayesian cue-combination model weighted in proportion to the variance of the measurements for each cue in isolation. Instead, they suggest that the visual system uses disjunctive mechanisms to process these two types of information rather than combining them according to their likelihood ratios.

Pose Estimation and Segmentation of Multiple People in Stereoscopic Movies.

We describe a method to obtain a pixel-wise segmentation and pose estimation of multiple people in stereoscopic videos. This task involves challenges such as dealing with unconstrained stereoscopic video, non-stationary cameras, and complex indoor and outdoor dynamic scenes with multiple people. We cast the problem as a discrete labelling task involving multiple person labels, devise a suitable cost function, and optimize it efficiently. The contributions of our work are two-fold: First, we develop a segmentation model incorporating person detections and learnt articulated pose segmentation masks, as well as colour, motion, and stereo disparity cues. The model also explicitly represents depth ordering and occlusion. Second, we introduce a stereoscopic dataset with frames extracted from feature-length movies "StreetDance 3D" and "Pina". The dataset contains 587 annotated human poses, 1,158 bounding box annotations and 686 pixel-wise segmentations of people. The dataset is composed of indoor and outdoor scenes depicting multiple people with frequent occlusions. We demonstrate results on our new challenging dataset, as well as on the H2view dataset from (Sheasby et al. ACCV 2012).

Flexible integration of visual cues in adolescents with autism spectrum disorder.

Although children with autism spectrum disorder (ASD) show atypical sensory processing, evidence for impaired integration of multisensory information has been mixed. In this study, we took a Bayesian model‐based approach to assess within‐modality integration of congruent and incongruent texture and disparity cues to judge slant in typical and autistic adolescents. Human adults optimally combine multiple sources of sensory information to reduce perceptual variance but in typical development this ability to integrate cues does not develop until late childhood. While adults cannot help but integrate cues, even when they are incongruent, young children's ability to keep cues separate gives them an advantage in discriminating incongruent stimuli. Given that mature cue integration emerges in later childhood, we hypothesized that typical adolescents would show adult‐like integration, combining both congruent and incongruent cues. For the ASD group there were three possible predictions (1) “no fusion”: no integration of congruent or incongruent cues, like 6‐year‐old typical children; (2) “mandatory fusion”: integration of congruent and incongruent cues, like typical adults; (3) “selective fusion”: cues are combined when congruent but not incongruent, consistent with predictions of Enhanced Perceptual Functioning (EPF) theory. As hypothesized, typical adolescents showed significant integration of both congruent and incongruent cues. The ASD group showed results consistent with “selective fusion,” integrating congruent but not incongruent cues. This allowed adolescents with ASD to make perceptual judgments which typical adolescents could not. In line with EPF, results suggest that perception in ASD may be more flexible and less governed by mandatory top‐down feedback. Autism Res 2016, 9: 272–281. © 2015 International Society for Autism Research, Wiley Periodicals, Inc.

The Influence of the Stereo Base on Blind and Sighted Reaches in a Virtual Environment

In virtual environments, perceived distances are frequently reported to be shorter than intended. One important parameter for spatial perception in a stereoscopic virtual environment is the stereo base—that is, the distance between the two viewing cameras. We systematically varied the stereo base relative to the interpupillary distance (IPD) and examined influences on distance and size perception. Furthermore, we tested whether an individual adjustment of the stereo base through an alignment task would reduce the errors in distance estimation. Participants performed reaching movements toward a virtual tennis ball either with closed eyes (blind reaches) or open eyes (sighted reaches). Using the participants' individual IPD, the stereo base was set to (a) the IPD, (b) proportionally smaller, (c) proportionally larger, or (d) adjusted according to the individual performance in an alignment task that was conducted beforehand. Overall, consistent with previous research, distances were underestimated. As expected, with a smaller stereo base, the virtual object was perceived as being farther away and bigger, in contrast to a larger stereo base, where the virtual object was perceived to be nearer and smaller. However, the manipulation of the stereo base influenced blind reaching estimates to a smaller extent than expected, which might be due to a combination of binocular disparity and pictorial depth cues. In sighted reaching, when visual feedback was available, presumably the use of disparity matching led to a larger effect of the stereo base. The use of an individually adjusted stereo base diminished the average underestimation but did not reduce interindividual variance. Interindividual differences were task specific and could not be explained through differences in stereo acuity or fixation disparity.

A functional link between MT neurons and depth perception based on motion parallax.

As an observer translates, objects lying at different distances from the observer have differential image motion on the retina (motion parallax). It is well established psychophysically that humans perceive depth rather precisely from motion parallax and that extraretinal signals may be used to correctly perceive the sign of depth (near vs far) when binocular and pictorial depth cues are absent or weak. However, the neural basis for this capacity remains poorly understood. We have shown previously that neurons in the macaque middle temporal (MT) area combine retinal image motion with smooth eye movement command signals to signal depth sign from motion parallax. However, those studies were performed in animals that were required simply to track a visual target, thus precluding direct comparisons between neural activity and behavior. Here, we examine the activity of MT neurons in rhesus monkeys that were trained to discriminate depth sign based on motion parallax, in the absence of binocular disparity and pictorial depth cues. We find that the most sensitive MT neurons approach behavioral sensitivity, whereas the average neuron is twofold to threefold less sensitive than the animal. We also find that MT responses are predictive of perceptual decisions (independent of the visual stimulus), consistent with a role for MT in providing sensory signals for this behavior. Our findings suggest that, in addition to its established roles in processing stereoscopic depth, area MT is well suited to contribute to perception of depth based on motion parallax.

Feature attention for binocular disparity in primate area MT depends on tuning strength.

Attending to a stimulus modulates the responses of sensory neurons that represent features of that stimulus, a phenomenon named "feature attention." For example, attending to a stimulus containing upward motion enhances the responses of upward-preferring direction-selective neurons in the middle temporal area (MT) and suppresses the responses of downward-preferring neurons, even when the attended stimulus is outside of the spatial receptive fields of the recorded neurons (Treue S, Martinez-Trujillo JC. Nature 399: 575-579, 1999). This modulation renders the representation of sensory information across a neuronal population more selective for the features present in the attended stimulus (Martinez-Trujillo JC, Treue S. Curr Biol 14: 744-751, 2004). We hypothesized that if feature attention modulates neurons according to their tuning preferences, it should also be sensitive to their tuning strength, which is the magnitude of the difference in responses to preferred and null stimuli. We measured how the effects of feature attention on MT neurons in rhesus monkeys (Macaca mulatta) depended on the relationship between features-in our case, direction of motion and binocular disparity-of the attended stimulus and a neuron's tuning for those features. We found that, as for direction, attention to stimuli containing binocular disparity cues modulated the responses of MT neurons and that the magnitude of the modulation depended on both a neuron's tuning preferences and its tuning strength. Our results suggest that modulation by feature attention may depend not just on which features a neuron represents but also on how well the neuron represents those features.

Are blur and disparity complementary cues to depth?

The image blur and binocular disparity of a 3D scene point both increase with distance in depth away from fixation. Perceived depth from disparity has been studied extensively and is known to be most precise near fixation. Perceived depth from blur is much less well understood. A recent experiment (Held, R. T, Cooper, E. A., & Banks, M. S. (2012). Current Biology, 22, 426–431) which used a volumetric stereo display found evidence that blur and disparity are complementary cues to depth, namely the disparity cue dominates over the blur cue near the fixation depth and blur dominates over disparity at depths that are far from fixation. Here we present a similar experiment but which used a traditional 3D display so that blur was produced by image processing rather than by the subjects’ optics. Contrary to Held et al., we found that subjects did not rely more on blur to discriminate depth at distances far from fixation, even though a sufficient level of blur was available to do so. The discrepancy between the findings of the two studies can be explained in at least two ways. First, Held et al.’s subjects received trial-to-trial feedback in a training phase and may have learned how to perform the task using blur discrimination. Second, Held et al.’s volumetric stereo display may have provided other optical cues that indicated that the blur was real rather than rendered. The latter possibility would have significant implications about how depth is perceived from blur under different viewing conditions.

Reliability-dependent contributions of visual orientation cues in parietal cortex.

Creating accurate 3D representations of the world from 2D retinal images is a fundamental task for the visual system. However, the reliability of different 3D visual signals depends inherently on viewing geometry, such as how much an object is slanted in depth. Human perceptual studies have correspondingly shown that texture and binocular disparity cues for object orientation are combined according to their slant-dependent reliabilities. Where and how this cue combination occurs in the brain is currently unknown. Here, we search for neural correlates of this property in the macaque caudal intraparietal area (CIP) by measuring slant tuning curves using mixed-cue (texture + disparity) and cue-isolated (texture or disparity) planar stimuli. We find that texture cues contribute more to the mixed-cue responses of CIP neurons that prefer larger slants, consistent with theoretical and psychophysical results showing that the reliability of texture relative to disparity cues increases with slant angle. By analyzing responses to binocularly viewed texture stimuli with conflicting texture and disparity information, some cells that are sensitive to both cues when presented in isolation are found to disregard one of the cues during cue conflict. Additionally, the similarity between texture and mixed-cue responses is found to be greater when this cue conflict is eliminated by presenting the texture stimuli monocularly. The present findings demonstrate reliability-dependent contributions of visual orientation cues at the level of the CIP, thus revealing a neural correlate of this property of human visual perception.

A cue integration method for anaglyph image partition

Image content analysis is important for automated image organization, labeling, and search. Partitioning an image into meaningful regions is one of the fundamental problems in image analysis. Anaglyph images and videos are more and more popular, such as in Flickr and YouTube. The anaglyph images provide disparity cue in a single image, which could be useful for image analysis. This paper exploits disparity cue for image partition. An image partition method for anaglyph is proposed. The disparity or depth cue is integrated with the traditional single-view image segmentation. A concept called dominant disparity is proposed, corresponding to each single-view image segment, which largely tolerates the disparity errors and image over-segmentations. A cue integration algorithm is developed. The integration is at the level of image segments rather than pixels, and object-level image segmentation is achieved. Experiments on both synthetic and real anaglyph images demonstrate the effectiveness of the proposed image partition method for anaglyph image analysis. To the best of our knowledge, our work is for the first time to perform anaglyph image partition.

The Reliability of Infant Accommodation and Vergence Responses in the Absence of Blur or Disparity Cues

The Reliability of Infant Accommodation and Vergence Responses in the Absence of Blur or Disparity Cues

The Perception of Surface Material from Disparity and Focus Cues

The� visual� properties� of� surfaces� reveal� many� things� including� a� floor�适s� cleanliness� and� a� car�适s� age.� These� judgments� of� material� are� based� on� the� spread� of� light� reflected� from� a� surface.� The� bidirectional� reflectance� distribution� function� (BRDF)� quantifies� the� pattern� of� spread� and� how� it� depends� on� the� direction� of� incident� light,� surface� shape,� and� surface� material.� Two� extremes� are� Lambertian� and� mirrored� surfaces,� which� respectively� have� uniform� and� delta�괂Ġ function� BRDFs.� Most� surfaces� have� more� complicated� BRDFs� and� we� examined� many� of� them� using� the� Ward� model� as� an� approximation� for� real� surfaces.� Reflections� are� generally� view� dependent.� This� dependence� creates� a� difference� between� the� binocular� disparities� of� a� reflection� and� the� surface� itself.� It� also� creates� focus� differences� between� the� reflection� and� physical� surface.� In� simulations� we� examined� how� material� type� affects� retinal� images.� We� calculated� point�괂Ġspread� functions� (PSFs)� for� reflections� off� different� materials� as� a� function� of� the� eye�适s� focus� state.� When� surface� roughness� is� zero,� the� reflection� PSF� changes� dramatically� with� focus� state.� With� greater� roughness,� the� PSF� change� is� reduced� until� there� is� no� effect� of� focus� state� with� sufficiently� rough� surfaces.� The� reflection� PSF� also� has� a� dramatic� effect� on� the� ability� to� estimate� disparity.� We� next� examined� people�适s� ability� to� distinguish� surface� markings� from� reflections� and� to� identify� different� types� of� material.� We� used� a� unique� volumetric� display� that� allows� us� to� present� nearly� correct� focus� cues� along� with� more� traditional� depth� cues� such� as� disparity.� With� binocular� viewing,� we� observed� a� clear� effect� of� the� disparity� of� reflections� on� these� judgments.� We� also� found� that� disparity� provided� less� useful� information� with� rougher� materials.� With� monocular� viewing,� we� observed� a� small� but� consistent� effect� of� the� reflection�适s� focal� distance� on� judgments� of� markings� vs.� reflections� and� on� identification� of� material.�

The retinal correlate of linear perspective in slant perception

Depicted slant is defined as slant based on linear-perspective assumptions about lines imaged on a flat surface. Slants of triangles and trapezoids were computed as a function of depicted slant and slant of the obliquely viewed picture plane. Computations were based on assumptions of parallelism and orthogonality. Perceived slant was measured during binocular viewing in a matching task. The matched slants were compared with computed slants. Matched and computed slants were highly correlated. Residual error analysis showed that both parallelism and orthogonality explained about 95% of the data. Contributions of the picture plane, signaled by binocular disparity and various monocular cues, were small for both trapezoids and triangles. The results imply that the claimed non-Euclidean nature of pictorial space is straightforwardly explained by linear-perspective assumptions. Further analysis indicated that the visual system derives slant from retinal angles alone without requiring knowledge of distance and orientation of the picture plane. Precision of the slant judgments requires a neural substrate that is able to make highly precise comparisons between orientations of lines imaged at different retinal locations. The neural basis of slant from linear perspective has not yet been clarified. Cells with long-range connections in V1, however, have features that suggest an involvement in slant perception. Meeting abstract presented at VSS 2014

Are blur and disparity complementary cues to depth?

The image blur and binocular disparity of a 3D scene point both increase with distance in depth away from fixation. Perceived depth from disparity has been studied extensively and is known to be most precise near fixation. Perceived depth from blur is much less well understood. A recent experiment (Held, R. T, Cooper, E. A., & Banks, M. S. (2012). Current Biology, 22, 426–431) which used a volumetric stereo display found evidence that blur and disparity are complementary cues to depth, namely the disparity cue dominates over the blur cue near the fixation depth and blur dominates over disparity at depths that are far from fixation. Here we present a similar experiment but which used a traditional 3D display so that blur was produced by image processing rather than by the subjects’ optics. Contrary to Held et al., we found that subjects did not rely more on blur to discriminate depth at distances far from fixation, even though a sufficient level of blur was available to do so. The discrepancy between the findings of the two studies can be explained in at least two ways. First, Held et al.’s subjects received trial-to-trial feedback in a training phase and may have learned how to perform the task using blur discrimination. Second, Held et al.’s volumetric stereo display may have provided other optical cues that indicated that the blur was real rather than rendered. The latter possibility would have significant implications about how depth is perceived from blur under different viewing conditions.

Aging and availability of binocular disparity and pictorial depth cues in 3D graphics contents

AbstractSome stereoblind observers do not perceive depth of 3D stimuli that depends on binocular disparity. These individuals, who have no disabilities, comprise over 5% of the general population. In addition, 17–30% of nonstereoblind young and young–middle people do not use disparity information in certain 3D environments, a phenomenon known as pseudo‐stereoblindness. This study aimed to investigate the relationship between aging and the proportion of pseudo‐stereoblindness in the general population. In an experiment, 134 nonstereoblind participants, ranging in age from 17 to 83 years, judged subjective depth of 3D stimuli containing binocular disparity and pictorial depth cues. Results showed that the proportion of pseudo‐stereoblindness among young (17–24 years old) and young–middle aged observers (25–39 years old) was 29%, in both cases. However, the proportion of pseudo‐stereoblind observers increased in older populations: 65% and 82% in the middle (40–54 years old) and senior (55–83 years old) age groups, respectively. These results suggest that a number of people, especially in elderly populations, have trouble perceiving depth from binocular disparity in 3D graphic contents despite their essential ability to use disparity information.

A trained perceptual bias that lasts for weeks

Classical (Pavlovian) conditioning procedures can be used to bias the appearance of physical stimuli. Under natural conditions this form of perceptual learning could cause perception to become more accurate by changing prior belief to be in accord with what is statistically likely. However, for learning to be of functional significance, it must last until similar stimuli are encountered again. Here, we used the apparent rotation direction of a revolving wire frame (Necker) cube to test whether a learned perceptual bias is long lasting. Apparent rotation direction was trained to have a different bias at two different retinal locations by interleaving the presentation of ambiguous cubes with presentation of cubes that were disambiguated by disparity and occlusion cues. Four groups of eight subjects were subsequently tested either 1, 7, 14, or 28 days after initial training, respectively, using a counter-conditioning procedure. All four groups showed incomplete re-learning of the reversed contingency relationship during their second session. One group repeated the counter-conditioning and showed an increase in the reverse bias, showing that the first counter-conditioning session also had a long-lasting effect. The fact that the original learning was still evident four weeks after the initial training is consistent with the operation of a mechanism that ordinarily would improve the accuracy and efficiency of perception.