Stereoscopic Cues Research Articles

Visual and haptic cues in processing occlusion.

Although shape is effective in processing occlusion, ambiguities in segmentation can also be addressed using depth discontinuity given visually and haptically. This study elucidates the contribution of visual and haptic cues to depth discontinuity in processing occlusion. A virtual reality experiment was conducted with 15 students as participants. Word stimuli were presented on a head-mounted display for recognition. The central part of the words was masked with a virtual ribbon placed at different depths so that the ribbon appeared as an occlusion. The visual depth cue was either present with binocular stereopsis or absent with monocular presentation. The haptic cue was either missing, provided consecutively, or concurrently, by actively tracing a real off-screen bar edge that was positionally aligned with the ribbon in the virtual space. Recognition performance was compared between depth cue conditions. We found that word recognition was better with the stereoscopic cue but not with the haptic cue, although both cues contributed to greater confidence in depth estimation. The performance was better when the ribbon was at the farther depth plane to appear as a hollow, rather than when it was at the nearer depth plane to cover the word. The results indicate that occlusion is processed in the human brain by visual input only despite the apparent effectiveness of haptic space perception, reflecting a complex set of natural constraints.

Perspective Cues Make Eye-specific Contributions to 3-D Motion Perception.

Robust 3-D visual perception is achieved by integrating stereoscopic and perspective cues. The canonical model describing the integration of these cues assumes that perspective signals sensed by the left and right eyes are indiscriminately pooled into a single representation that contributes to perception. Here, we show that this model fails to account for 3-D motion perception. We measured the sensitivity of male macaque monkeys to 3-D motion signaled by left-eye perspective cues, right-eye perspective cues, stereoscopic cues, and all three cues combined. The monkeys exhibited idiosyncratic differences in their biases and sensitivities for each cue, including left- and right-eye perspective cues, suggesting that the signals undergo at least partially separate neural processing. Importantly, sensitivity to combined cue stimuli was greater than predicted by the canonical model, which previous studies found to account for the perception of 3-D orientation in both humans and monkeys. Instead, 3-D motion sensitivity was best explained by a model in which stereoscopic cues were integrated with left- and right-eye perspective cues whose representations were at least partially independent. These results indicate that the integration of perspective and stereoscopic cues is a shared computational strategy across 3-D processing domains. However, they also reveal a fundamental difference in how left- and right-eye perspective signals are represented for 3-D orientation versus motion perception. This difference results in more effective use of available sensory information in the processing of 3-D motion than orientation and may reflect the temporal urgency of avoiding and intercepting moving objects.

Visual Search in 3D: Effects of Monoscopic and Stereoscopic Cues to Depth on the Validity of Feature Integration Theory and Perceptual Load Theory.

Previous research has successfully used feature integration theory to operationalise the predictions of Perceptual Load Theory, while simultaneously testing the predictions of both models. Building on this work, we test the extent to which these models hold up in a 3D world. In two experiments, participants responded to a target stimulus within an array of shapes whose apparent depth was manipulated using a combination of monoscopic and stereoscopic cues. The search task was designed to test the predictions of (a) feature integration theory, as the target was identified by a single feature or a conjunction of features and embedded in search arrays of varying size, and (b) perceptual load theory, as the task included congruent and incongruent distractors presented alongside search tasks imposing high or low perceptual load. Findings from both experiments upheld the predictions of feature integration theory, regardless of 2D/3D condition. Longer search times in conditions with a combination of monoscopic and stereoscopic depth cues suggests that binding features into three-dimensional objects requires greater attentional effort. This additional effort should have implications for perceptual load theory, yet our findings did not uphold its predictions; the effect of incongruent distractors did not differ between conjunction search trials (conceptualised as high perceptual load) and feature search trials (low perceptual load). Individual differences in susceptibility to the effects of perceptual load were evident and likely explain the absence of load effects. Overall, our findings suggest that feature integration theory may be useful for predicting attentional performance in a 3D world.

Natural binocular depth discrimination behavior in mice explained by visual cortical activity

In mice and other mammals, forebrain neurons integrate right and left eye information to generate a three-dimensional representation of the visual environment. Neurons in the visual cortex of mice are sensitive to binocular disparity,1-3 yet it is unclear whether that sensitivity is linked to the perception of depth.4-8 We developed a natural task based on the classic visual cliff and pole descent tasks to estimate the psychophysical range of mouse depth discrimination.5,9 Mice with binocular vision descended to a near (shallow) surface more often when surrounding far (deep) surfaces were progressively more distant. Occlusion of one eye severely impaired their ability to target the near surface. We quantified the distance at which animals make their decisions to estimate the binocular image displacement of the checkerboard pattern on the near and far surfaces. Then, we assayed the disparity sensitivity of large populations of binocular neurons in primary visual cortex (V1) using two-photon microscopy2 and quantitatively compared this information available in V1 to their behavioral sensitivity. Disparity information in V1 matches the behavioral performance over the range of depths examined and was resistant to changes in binocular alignment. These findings reveal that mice naturally use stereoscopic cues to guide their behavior and indicate a neural basis for this depth discrimination task.

Decoding Neural Responses to Motion-in-Depth Using EEG.

Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single “motion-in-depth” pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.

Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

Virtual and augmented reality is increasingly prevalent in industrial applications, such as remote control of industrial machinery, due to recent advances in head-mounted display technologies and low-latency communications via 5G. However, the influence of augmentations and camera placement-based viewing positions on operator performance in telepresence systems remains unknown. In this paper, we investigate the joint effects of depth-aiding augmentations and viewing positions on the quality of experience for operators in augmented telepresence systems. A study was conducted with 27 non-expert participants using a real-time augmented telepresence system to perform a remote-controlled navigation and positioning task, with varied depth-aiding augmentations and viewing positions. The resulting quality of experience was analyzed via Likert opinion scales, task performance measurements, and simulator sickness evaluation. Results suggest that reducing the reliance on stereoscopic depth perception via camera placement has a significant benefit to operator performance and quality of experience. Conversely, the depth-aiding augmentations can partly mitigate the negative effects of inferior viewing positions. However the viewing-position based monoscopic and stereoscopic depth cues tend to dominate over cues based on augmentations. There is also a discrepancy between the participants’ subjective opinions on augmentation helpfulness, and its observed effects on positioning task performance.

Optimized but Not Maximized Cue Integration for 3D Visual Perception.

Reconstructing three-dimensional (3D) scenes from two-dimensional (2D) retinal images is an ill-posed problem. Despite this, 3D perception of the world based on 2D retinal images is seemingly accurate and precise. The integration of distinct visual cues is essential for robust 3D perception in humans, but it is unclear whether this is true for non-human primates (NHPs). Here, we assessed 3D perception in macaque monkeys using a planar surface orientation discrimination task. Perception was accurate across a wide range of spatial poses (orientations and distances), but precision was highly dependent on the plane's pose. The monkeys achieved robust 3D perception by dynamically reweighting the integration of stereoscopic and perspective cues according to their pose-dependent reliabilities. Errors in performance could be explained by a prior resembling the 3D orientation statistics of natural scenes. We used neural network simulations based on 3D orientation-selective neurons recorded from the same monkeys to assess how neural computation might constrain perception. The perceptual data were consistent with a model in which the responses of two independent neuronal populations representing stereoscopic cues and perspective cues (with perspective signals from the two eyes combined using nonlinear canonical computations) were optimally integrated through linear summation. Perception of combined-cue stimuli was optimal given this architecture. However, an alternative architecture in which stereoscopic cues, left eye perspective cues, and right eye perspective cues were represented by three independent populations yielded two times greater precision than the monkeys. This result suggests that, due to canonical computations, cue integration for 3D perception is optimized but not maximized.

Motion-in-depth perception and prey capture in the praying mantisSphodromantis lineola

Perceiving motion-in-depth is essential to detecting approaching or receding objects, predators and prey. This can be achieved using several cues, including binocular stereoscopic cues such as changing disparity and interocular velocity differences, and monocular cues such as looming. Although these have been studied in detail in humans, only looming responses have been well characterized in insects and we know nothing about the role of stereoscopic cues and how they might interact with looming cues. We used our 3D insect cinema in a series of experiments to investigate the role of the stereoscopic cues mentioned above, as well as looming, in the perception of motion-in-depth during predatory strikes by the praying mantis Sphodromantis lineola Our results show that motion-in-depth does increase the probability of mantis strikes but only for the classic looming stimulus, an expanding luminance edge. Approach indicated by radial motion of a texture or expansion of a motion-defined edge, or by stereoscopic cues, all failed to elicit increased striking. We conclude that mantises use stereopsis to detect depth but not motion-in-depth, which is detected via looming.

Perceptual integration of perspective and stereoscopic cues in macaque monkeys

Perceptual integration of perspective and stereoscopic cues in macaque monkeys

Effect of age and stereopsis on a multiple-object tracking task.

3D vision develops during childhood and tends to diminish after 65 years of age. It is still relatively unknown how stereopsis is used in more complex/ecological contexts such as when walking about in crowds where objects are in motion and occlusions occur. One task that shares characteristics with the requirements for processing dynamic crowds is the multiple object-tracking task (MOT). In the present study we evaluated the impact of stereopsis on a MOT task as a function of age. A total of 60 observers consisting of three groups of 20 subjects (7–12 years old, 18–40 years old and 65 years and older) completed the task in both conditions (with and without stereoscopic effects). The adult group obtained the better scores, followed by the children and the older adult group. The performance difference between the stereoscopic and non-stereoscopic conditions was significant and similar for the adults and the children but was non significant for the older observers. These results show that stereopsis helps children and adults accomplish a MOT task, but has no impact on older adults’ performances. The present results have implications as to how populations differ in their efficiency of using stereoscopic cues for disambiguating complex dynamic scenes.

Early dynamics of stereoscopic surface slant perception.

Surface orientation is an important visual primitive that can be estimated from monocular or binocular (stereoscopic) signals. Changes in motor planning occur within about 200 ms after either type of signal is perturbed, but the time it takes for apparent (perceived) slant to develop from stereoscopic cues is not known. Apparent slant sometimes develops very slowly (Gillam, Chambers, & Russo, 1988; van Ee & Erkelens, 1996). However, these long durations could reflect the time it takes for the visual system to resolve conflicts between slant cues that inevitably specify different slants in laboratory displays (Allison & Howard, 2000). We used a speed–accuracy tradeoff analysis to measure the time it takes to discriminate slant, allowing us to report psychometric functions as a function of response time. Observers reported which side of a slanted surface was farther, with a temporal deadline for responding that varied block-to-block. Stereoscopic slant discrimination rose above chance starting at 200 ms after stimulus onset. Unexpectedly, observers discriminated slant from binocular disparity faster than texture, and for stereoscopic whole-field stimuli faster than stereoscopic slant contrast stimuli. However, performance after the initial deviation from chance increased more rapidly for slant-contrast stimuli than whole-field stimuli. Discrimination latencies were similar for slants about the horizontal and vertical axes, but performance increased faster for slants about the vertical axis. Finally, slant from vertical disparity was somewhat slower than slant from horizontal disparity, which may reflect cue conflict. These results demonstrate, in contradiction with the previous literature, that the perception of slant from disparity happens very quickly—in fact, more quickly than the perception of slant from texture—and in comparable time to the simple perception of brightness from luminance.

Accommodation-invariant computational near-eye displays

Although emerging virtual and augmented reality (VR/AR) systems can produce highly immersive experiences, they can also cause visual discomfort, eyestrain, and nausea. One of the sources of these symptoms is a mismatch between vergence and focus cues. In current VR/AR near-eye displays, a stereoscopic image pair drives the vergence state of the human visual system to arbitrary distances, but the accommodation, or focus, state of the eyes is optically driven towards a fixed distance. In this work, we introduce a new display technology, dubbed accommodation-invariant (AI) near-eye displays, to improve the consistency of depth cues in near-eye displays. Rather than producing correct focus cues, AI displays are optically engineered to produce visual stimuli that are invariant to the accommodation state of the eye. The accommodation system can then be driven by stereoscopic cues, and the mismatch between vergence and accommodation state of the eyes is significantly reduced. We validate the principle of operation of AI displays using a prototype display that allows for the accommodation state of users to be measured while they view visual stimuli using multiple different display modes.

VP146 A Comparative Assessment Of 3D/2D Laparoscopic Display Systems

INTRODUCTION:The purpose of the study was to gather evidence on safety and overall effectiveness of performing laparoscopic surgery by using 3D versus 2D display systems in a variety of pediatric surgical procedures in order to efficiently support the final investment decision on the video system to be acquired.METHODS:A new methodology, that is, Decision-oriented HTA (DoHTA) (1) was applied to assess the technology on clinical, technical, organizational, economic, social, ethical and safety domains. A decision-tree covering all the relevant assessment aspects of 3D systems has been derived and weighted following the Analytic Hierarchy Process. Afterwards, another pairwise comparison list was set up to compare both alternative technologies with respect to every lowest indicator.RESULTS:DoHTA results of the 3D system has mainly forecast its impact on clinical efficacy and productivity within the specific context of use. The 3D system is particularly suitable in reducing the mean error rate, thanks to the stereoscopic depth cues which are lost in 2D vision (2,3) From the technical perspective, the analyses have indicated the reduction in median instrument path length, an enhancement of median motion smoothness, and the decrease in grasper frequency with the 3D display. However, the comparative cost analysis has pointed out that the 3D procedure cost was higher that its comparator.CONCLUSIONS:The assessment of the 3D visual system seems to reasonably satisfy the criteria of feasibility, clinical effectiveness and safety. However, the adoption of the 3D display system in surgical practice could involve increased hospital costs, mainly because of the initial cost of the technology. Indeed, based on the appreciation of the results of DoHTA, especially taking into account the positive technical and clinical features, we conclude that the 3D system may be a good alternative to the 2D system.

Depth Perception and the History of Three-Dimensional Art: Who Produced the First Stereoscopic Images?

The history of the expression of three-dimensional structure in art can be traced from the use of occlusion in Palaeolithic cave paintings, through the use of shadow in classical art, to the development of perspective during the Renaissance. However, the history of the use of stereoscopic techniques is controversial. Although the first undisputed stereoscopic images were presented by Wheatstone in 1838, it has been claimed that two sketches by Jacopo Chimenti da Empoli (c. 1600) can be to be fused to yield an impression of stereoscopic depth, while others suggest that Leonardo da Vinci’s Mona Lisa is the world’s first stereogram. Here, we report the first quantitative study of perceived depth in these works, in addition to more recent works by Salvador Dalí. To control for the contribution of monocular depth cues, ratings of the magnitude and coherence of depth were recorded for both stereoscopic and pseudoscopic presentations, with a genuine contribution of stereoscopic cues revealed by a difference between these scores. Although effects were clear for Wheatstone and Dalí’s images, no such effects could be found for works produced earlier. As such, we have no evidence to reject the conventional view that the first producer of stereoscopic imagery was Sir Charles Wheatstone.

Perceptual Depth Quality in Distorted Stereoscopic Images.

Subjective and objective measurement of the perceptual quality of depth information in symmetrically and asymmetrically distorted stereoscopic images is a fundamentally important issue in stereoscopic 3D imaging that has not been deeply investigated. Here, we first carry out a subjective test following the traditional absolute category rating protocol widely used in general image quality assessment research. We find this approach problematic, because monocular cues and the spatial quality of images have strong impact on the depth quality scores given by subjects, making it difficult to single out the actual contributions of stereoscopic cues in depth perception. To overcome this problem, we carry out a novel subjective study where depth effect is synthesized at different depth levels before various types and levels of symmetric and asymmetric distortions are applied. Instead of following the traditional approach, we ask subjects to identify and label depth polarizations, and a depth perception difficulty index (DPDI) is developed based on the percentage of correct and incorrect subject judgements. We find this approach highly effective at quantifying depth perception induced by stereo cues and observe a number of interesting effects regarding image content dependency, distortion-type dependence, and the impact of symmetric versus asymmetric distortions. Furthermore, we propose a novel computational model for DPDI prediction. Our results show that the proposed model, without explicitly identifying image distortion types, leads to highly promising DPDI prediction performance. We believe that these are useful steps toward building a comprehensive understanding on 3D quality-of-experience of stereoscopic images.

Recovering stereo vision by squashing virtual bugs in a virtual reality environment.

Stereopsis is the rich impression of three-dimensionality, based on binocular disparity-the differences between the two retinal images of the same world. However, a substantial proportion of the population is stereo-deficient, and relies mostly on monocular cues to judge the relative depth or distance of objects in the environment. Here we trained adults who were stereo blind or stereo-deficient owing to strabismus and/or amblyopia in a natural visuomotor task-a 'bug squashing' game-in a virtual reality environment. The subjects' task was to squash a virtual dichoptic bug on a slanted surface, by hitting it with a physical cylinder they held in their hand. The perceived surface slant was determined by monocular texture and stereoscopic cues, with these cues being either consistent or in conflict, allowing us to track the relative weighting of monocular versus stereoscopic cues as training in the task progressed. Following training most participants showed greater reliance on stereoscopic cues, reduced suppression and improved stereoacuity. Importantly, the training-induced changes in relative stereo weights were significant predictors of the improvements in stereoacuity. We conclude that some adults deprived of normal binocular vision and insensitive to the disparity information can, with appropriate experience, recover access to more reliable stereoscopic information.This article is part of the themed issue 'Vision in our three-dimensional world'.

Cortical Dynamics of Figure-Ground Separation in Response to 2D Pictures and 3D Scenes: How V2 Combines Border Ownership, Stereoscopic Cues, and Gestalt Grouping Rules.

The FACADE model, and its laminar cortical realization and extension in the 3D LAMINART model, have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out 3D vision and figure-ground perception, and how these cortical mechanisms enable 2D pictures to generate 3D percepts of occluding and occluded objects. In particular, these models have proposed how border ownership occurs, but have not yet explicitly explained the correlation between multiple properties of border ownership neurons in cortical area V2 that were reported in a remarkable series of neurophysiological experiments by von der Heydt and his colleagues; namely, border ownership, contrast preference, binocular stereoscopic information, selectivity for side-of-figure, Gestalt rules, and strength of attentional modulation, as well as the time course during which such properties arise. This article shows how, by combining 3D LAMINART properties that were discovered in two parallel streams of research, a unified explanation of these properties emerges. This explanation proposes, moreover, how these properties contribute to the generation of consciously seen 3D surfaces. The first research stream models how processes like 3D boundary grouping and surface filling-in interact in multiple stages within and between the V1 interblob—V2 interstripe—V4 cortical stream and the V1 blob—V2 thin stripe—V4 cortical stream, respectively. Of particular importance for understanding figure-ground separation is how these cortical interactions convert computationally complementary boundary and surface mechanisms into a consistent conscious percept, including the critical use of surface contour feedback signals from surface representations in V2 thin stripes to boundary representations in V2 interstripes. Remarkably, key figure-ground properties emerge from these feedback interactions. The second research stream shows how cells that compute absolute disparity in cortical area V1 are transformed into cells that compute relative disparity in cortical area V2. Relative disparity is a more invariant measure of an object's depth and 3D shape, and is sensitive to figure-ground properties.

Modified Test Protocol Improves Sensitivity of the Stereo Fly Test

Background and PurposeStereoacuity measurement is a common element of pediatric ophthalmic examinations. Although the Stereo Fly Test is routinely used to establish the presence of coarse stereopsis (3000 arcsecs), it often yields a false negative “pass” due to learned responses and non-stereoscopic cues. We developed and evaluated a modified Stereo Fly Test protocol aimed at increasing sensitivity, thus reducing false negatives.Patients and MethodsThe Stereo Fly Test was administered according to manufacturer instructions to 321 children aged 3–12 years. Children with a “pass” outcome (n = 147) were re-tested wearing glasses fitted with polarizers of matching orientation for both eyes to verify that they were responding to stereoscopic cues (modified protocol). The response to the standard Stereo Fly Test was considered a false negative (pass) if the child still pinched above the plate after disparity cues were eliminated. Randot® Preschool Stereoacuity and Butterfly Tests were used as gold standards.Results and ConclusionsSensitivity was 81% (95% CI: 0.75 – 0.86) for standard administration of the Stereo Fly Test (19% false negative “pass”). The modified protocol increased sensitivity to 90% (95% CI: 0.85 – 0.94). The modified two-step protocol is a simple and convenient way to administer the Stereo Fly Test with increased sensitivity in a clinical setting.

Effects of 3D perspective on head gaze estimation with a multiview autostereoscopic display

Head gaze, or the orientation of the head, is a very important attentional cue in face to face conversation. Some subtleties of the gaze can be lost in common teleconferencing systems, because a single perspective warps spatial characteristics. A recent random hole display is a potentially interesting display for group conversation, as it allows multiple stereo viewers in arbitrary locations, without the restriction of conventional autostereoscopic displays on viewing positions. We represented a remote person as an avatar on a random hole display. We evaluated this system by measuring the ability of multiple observers with different horizontal and vertical viewing angles to accurately and simultaneously judge which targets the avatar is gazing at. We compared three perspective conditions: a conventional 2D view, a monoscopic perspective-correct view, and a stereoscopic perspective-correct views. In the latter two conditions, the random hole display shows three and six views simultaneously. Although the random hole display does not provide high quality view, because it has to distribute display pixels among multiple viewers, the different views are easily distinguished. Results suggest the combined presence of perspective-correct and stereoscopic cues significantly improved the effectiveness with which observers were able to assess the avatar's head gaze direction. This motivates the need for stereo in future multiview displays. HighlightsRepresented a remote person as an avatar on a random hole display (RHD).This RHD provides multiple users with perspective-correct stereo views from arbitrary positions.Evaluated a RHD for improving the head gaze perception for group teleconferencing.A perspective-corrected rendering improved viewers' ability in head gaze perception.Adding stereoscopic cues provided an additional improvement in head gaze perception.

Infants Perceive Three-Dimensional Illusory Contours as Occluding Surfaces.

The study assessed the contribution of stereoscopic depth cues to infants' perception of a Kanizsa rectangle as a surface that temporarily occludes a moving object. In Experiment 1, the Kanizsa figure was shifted into the foreground by enriching it with stereoscopic depth information. According to the results, perception of a three-dimensional Kanizsa figure as an occluding surface emerges between 5 (n = 16) and 7 (n = 16) months of age. Experiment 2 demonstrated that 7-month-old (n = 16) infants performed similarly to the 7-month-olds who participated in Experiment 1 if the moving object was shifted into the background. These findings suggest that 7-month-old infants respond to stereoscopic depth cues and that they exploit it to perceive subjective contours as occluders.