Monocular Depth Cues Research Articles

Comparison of Depth-Related Visuomotor Task Performance in Uniocular Individuals and in Binocular Controls With and Without Temporary Monocular Occlusion.

Do one-eyed (uniocular) humans use monocular depth cues differently from those with intact binocularity to perform depth-related visuomotor tasks that emulate complex activities of daily living? If so, does performance depend on the participant's age, duration of uniocularity and head movements? Forty-five uniocular cases (age range 6-37years; 2.4months-31.0years of uniocularity) and 46 age-similar binocular controls performed a task that required them to pass a hoop around an electrified wire convoluted in depth multiple times, while avoiding contact as indicated by auditory feedback. The task was performed with and without head restraint, in random order. The error rate and speed were calculated from the frequency of contact between the hoop and wire and the total task duration (adjusting for error time), respectively, all determined from video recordings of the task. Head movements were analyzed from the videos using face-tracking software. Error rate decreased with age (P < 0.001) until the late teen years while speed revealed no such trend. Across all ages, the error rate increased and speed decreased in the absence of binocularity (P < 0.001). There was no additional error reduction with duration of uniocularity (P = 0.16). Head movements provided no advantage to task performance, despite generating parallax disparities comparable to binocular viewing. Performance in a dynamic, depth-related visuomotor task is reduced in the absence of binocular viewing, independent of age-related performance level. This study finds no evidence for a prolonged experience with monocular depth cues being advantageous for such tasks over transient loss of binocularity.

Extraction of three-dimensional shapes in glaucoma patients in response to monocular depth cues.

To assess the impact of glaucoma on perceiving three-dimensional (3D) shapes based on monocular depth cues. Clinical observational study. Twenty glaucoma patients, subjected to binocular visual-field sensitivity (binocular-VFS) tests using a Humphrey Visual Field Analyzer, and 20 age-matched healthy volunteers, underwent two tasks: identifying the nearest vertex of a 3D shape using monocular shading (3D-SfS), texture (3D-SfT), or motion (3D-SfM) cues, and distinguishing elementary one-dimensional (1D) features of these cues. The association of the visual-field index (VFI) of binocular-VFS with 3D shape perception in glaucoma patients was also examined. Glaucoma patients demonstrated reduced accuracy in distinguishing 1D luminance brightness and a larger "error-in-depth" between the perceived and actual depths for 3D-SfM and 3D-SfS compared to healthy volunteers. Six glaucoma patients with a 100% VFI for binocular-VFS exhibited a similar error-in-depth to the other fourteen glaucoma patients; they had a larger error-in-depth for 3D-SfM compared to healthy volunteers. No correlation between the error-in-depth values and the VFI values of binocular-VFS was observed. The 3D shape perception in glaucoma patients varies based on the depth cue's characteristics. Impaired 1D discrimination and larger thresholds for 3D-SfM in glaucoma patients with a 100% VFI for binocular-VFS indicate more pronounced perceptual deficits of lower-level elementary features for 3D-SfS and higher-level visual processing of 3D shapes for 3D-SfM. The effects of the location and degree of binocular visual-field defects on 3D shape perception remain to be elucidated. Our research provides insights into the 3D shape extraction mechanism in glaucoma.

Peripheral target detection can be modulated by target distance but not attended distance in 3D space simulated by monocular depth cues

Most studies of visuo-spatial attention present stimuli on a 2D plane, and less is known about how attention varies in 3D space. Previous studies found better peripheral detection performance for targets at a near compared to a far depth, simulated by pictorial cues and optical flow. The current study examined whether target detectability is monotonically related to distance along the depth axis, and whether the attended distance modulates the effect of target distance. We investigated these questions in two experiments that measured how apparent distance and target eccentricity affects peripheral target detection when performed alone during passive simulated self-motion, or during a simultaneous, active central car-following task. Experiment 1 found that targets at an apparent distance of 18.5 virtual meters were detected faster and more accurately than targets at 9.25 and 37 virtual meters, and detectability declined with eccentricity. Experiment 2 examined the effect of the attended location by varying the distance between the viewer and the lead car on which participants were instructed to fixate (i.e. the headway) while equating target distances across headway conditions. Experiment 2 replicated the effects found in Experiment 1, and headway did not modulate the effect of target distance. These results are consistent with the hypothesis that target detection depends non-monotonically on the distance between the viewer and the target, and is not affected by the distance between the target and attended location. However, target detection may also have been affected by stimulus characteristics that co-varied with apparent depth, rather than depth per se.

Super multi-view near-eye display with a lightguide combiner.

We propose a lightguide-type super multi-view near-eye display that uses a digital micromirror device and a LED array. The proposed method presents three-dimensional images with a natural monocular depth cue using a compact combiner optics which consists of a thin lightguide and holographic optical elements (HOEs). Feasibility of the proposed method is verified by optical experiments which demonstrate monocular three-dimensional image presentation over a wide depth range. We also analyze the degradation of the image quality stemming from the spectral spread of the HOEs and show its reduction by a pre-compensation exploiting an adaptive moment estimation (Adam) optimizer.

Shedding Light on Cast Shadows: An Investigation of Perceived Ground Contact in AR and VR.

Virtual objects in augmented reality (AR) often appear to float atop real world surfaces, which makes it difficult to determine where they are positioned in space. This is problematic as many applications for AR require accurate spatial perception. In the current study, we examine how the way we render cast shadows-which act as an important monocular depth cue for creating a sense of contact between an object and the surface beneath it-impacts spatial perception. Over two experiments, we evaluate people's sense of surface contact given both traditional and non-traditional shadow shading methods in optical see-through augmented reality (OST AR), video see-through augmented reality (VST AR), and virtual reality (VR) head-mounted displays. Our results provide evidence that nontraditional shading techniques for rendering shadows in AR displays may enhance the accuracy of one's perception of surface contact. This finding implies a possible tradeoff between photorealism and accuracy of depth perception, especially in OST AR displays. However, it also supports the use of more stylized graphics like non-traditional cast shadows to improve perception and interaction in AR applications.

Structure‐aware dehazing of sewer inspection images based on monocular depth cues

AbstractIn sewer pipes, haze caused by the humid environment seriously impairs the quality of closed‐circuit television (CCTV) images, which leads to poor performance of subsequent pipe defects detection. Meanwhile, the complexity of sewer images, such as steep depth change and extensive textureless regions, brings great challenges to the performance or application of general dehazing algorithms. Therefore, this study estimates sewer depth maps first with the help of the water–pipewall borderlines to produce the paired dehazing dataset. Then a structure‐aware nonlocal network (SANL‐Net) is proposed with the detected borderlines and the dehazing result as two supervisory signals. SANL‐Net shows its superiority over other state‐of‐the‐art approaches with 147 in mean square error (MSE), 27.28 in peak signal to noise ratio (PSNR), 0.8963 in structural similarity index measure (SSIM), and 15.47M in parameters. Also, the outstanding performance in real image dehazing implies the accuracy of depth estimation. Experimental results indicate that SANL‐Net significantly improves the performance of defects detection tasks, such as an increase of 23.16% in mean intersection over union (mIoU) for semantic segmentation.

42‐1: Invited Paper: High‐Resolution Light‐Field AR at Comparable Computing Cost to Stereo 3D

Sequential Iight‐field by CREAL creates high‐resolution genuinely 3D imagery with correct monocular depth cues. It eliminates focal rivalry, vergence‐accommodation conflict, and provides correct ocular parallax. Contrary to widespread notion, Iight‐field can be equally or more computationally and data‐efficient as an AR display than conventional raster displays. This paper will introduce why and how.

Stereopsis Only: Validation of a Monocular Depth Cues Reduced Gamified Virtual Reality with Reaction Time Measurement.

The visual depth perception is composed of monocular and binocular depth cues. Studies show that in absence of binocular depth cues the performance of visuomotor tasks like pointing to or grasping objects is limited. Thus, binocular depth cues are of great importance for motor control required in everyday life. However, binocular depth cues like retinal disparity (basis for stereopsis) might be influenced due to developmental disorders of the visual system. For example, amblyopia in which one eye's visual input is not processed leads to loss of stereopsis. The primary amblyopia treatment is occlusion of the healthy eye to force the amblyopic eye to train. However, improvements in stereopsis are poor. Therefore, binocular treatments arose that equilibrate both eyes' visual input to enable binocular vision. However, most approaches rely on divided stimuli which do not account for loss of stereopsis. We created a Virtual Reality (VR) with reduced monocular depth cues in which a stereoscopic task is shown to both eyes simultaneously, consisting of two balls jumping towards the user. One ball appears closer to the user which must be identified. To evaluate the task performance the reaction time is measured. We validated our approach with 18 participants with stereopsis under three contrast settings including one leading to monocular vision. The number of correct responses reduces from 90% under binocular vision to 52% under monocular vision corresponding to random guessing. Our results indicate that it is possible to disable monocular depth cues and create a dynamic stereoscopic task inside a VR.

Binocular Viewing Facilitates Size Constancy for Grasping and Manual Estimation.

A prerequisite for efficient prehension is the ability to estimate an object’s distance and size. While most studies demonstrate that binocular viewing is associated with a more efficient grasp programming and execution compared to monocular viewing, the factors contributing to this advantage are not fully understood. Here, we examined how binocular vision facilitates grasp scaling using two tasks: prehension and manual size estimation. Participants (n = 30) were asked to either reach and grasp an object or to provide an estimate of an object’s size using their thumb and index finger. The objects were cylinders with a diameter of 0.5, 1.0, or 1.5 cm placed at three distances along the midline (40, 42, or 44 cm). Results from a linear regression analysis relating grip aperture to object size revealed that grip scaling during monocular viewing was reduced similarly for both grasping and estimation tasks. Additional analysis revealed that participants adopted a larger safety margin for grasping during monocular compared to binocular viewing, suggesting that monocular depth cues do not provide sufficient information about an object’s properties, which consequently leads to a less efficient grasp execution.

Monocular Depth Estimation (Literature Review)

Background. The physiological basis of spatial perception is traditionally attributed to the binocular system, which integrates the signals coming to the brain from each eye into a single image of the three-dimensional outside world. The perception of three-dimensionality, however, is also possible due to the evolutionarily older monocular system of spatial perception. Normally, the binocular mechanism plays the leading role in depth perception, and its violations lead to a shift towards the monocular. In this regard, one of the relevant areas of ophthalmology and neurophysiology is the study of the features of monocular depth estimation in normal conditions and cases of ophthalmic pathology.Purpose: to study the literature data on the monocular depth estimation mechanism, methods of its assessment, as well as the peculiarities of its manifestations in normal conditions and cases of ophthalmic pathology.Materials and methods. The literature analysis of publications on PubMed, eLibrary, Cyberleninka and crossref metadata search was carried out.Results. The review considers modern ideas regarding monocular depth cues that can ensure the effective operation of the monocular mechanism of spatial vision. The stereokinetic effect (SE) is considered in detail. The possibilities of using SE assessment methods to evaluate the state of spatial vision mechanisms in cases of ophthalmic and neurological pathology have been studied.Conclusion. There are a number of monocular depth cues that can ensure the effective operation of the monocular mechanism of spatial vision, such as: perspective, light and color effects, accommodation and knowledge of the true sizes of the objects acquired with experience. Stereokinetic effect caused by the successive displacement of projections of circular eccentric images on the retina, which allows to evaluate relationship of monocular and binocular mechanisms of spatial perception, has a particular importance for ophthalmology practice. In patients with binocular vision disorders (amblyopia and strabismus), a decrease in monocular and an increase in binocular SE indicators were observed, whereas only a decrease in monocular indicators is more typical for organic ocular fundus pathology. At the same time, changes in SE indicators can serve as additional criteria for evaluating the efficacy of functional treatment of binocular disorders.

Monocular Depth Perception on Microcontrollers for Edge Applications

Depth estimation is crucial in several computer vision applications, and a recent trend in this field aims at inferring such a cue from a single camera. Unfortunately, despite the compelling results achieved, state-of-the-art monocular depth estimation methods are computationally demanding, thus precluding their practical deployment in several application contexts characterized by low-power constraints. Therefore, in this paper, we propose a lightweight Convolutional Neural Network based on a shallow pyramidal architecture, referred to as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula> PyD-Net, enabling monocular depth estimation on microcontrollers. The network is trained in a peculiar self-supervised manner leveraging proxy labels obtained through a traditional stereo algorithm. Moreover, we propose optimization strategies aimed at performing computations with quantized 8-bit data and map the high-level description of the network to low-level layers optimized for the target microcontroller architecture. Exhaustive experimental results on standard datasets and an in-depth evaluation with a device belonging to the popular Arm Cortex-M family confirm that obtaining sufficiently accurate monocular depth estimation on microcontrollers is feasible. To the best of our knowledge, our proposal is the first one enabling such remarkable achievement, paving the way for the deployment of monocular depth cues onto the tiny end-nodes of distributed sensor networks.

Vision-based speedometer regulates human walking

SummaryCan we recover self-motion from vision? This basic issue remains unsolved since, while the human visual system is known to estimate the direction of self-motion from optic flow, it remains unclear whether it also estimates the speed. Importantly, the latter requires disentangling self-motion speed and depths of objects in the scene as retinal velocity depends on both. Here we show that our automatic regulator of walking speed based on vision, which estimates and maintains the speed to its preferred range by adjusting stride length, is robust to changes in the depths. The robustness was not explained by temporal-frequency-based speed coding previously suggested to underlie depth-invariant object-motion perception. Meanwhile, it broke down, not only when the interocular distance was virtually manipulated but also when monocular depth cues were deceptive. These observations suggest that our visuomotor system embeds a speedometer that calculates self-motion speed from vision by integrating monocular/binocular depth and motion cues.

3D light-field display with an increased viewing angle and optimized viewpoint distribution based on a ladder compound lenticular lens unit.

Three-dimensional (3D) light-field displays (LFDs) suffer from a narrow viewing angle, limited depth range, and low spatial information capacity, which limit their diversified application. Because the number of pixels used to construct 3D spatial information is limited, increasing the viewing angle reduces the viewpoint density, which degrades the 3D performance. A solution based on a holographic functional screen (HFS) and a ladder-compound lenticular lens unit (LC-LLU) is proposed to increase the viewing angle while optimizing the viewpoint utilization. The LC-LLU and HFS are used to create 160 non-uniformly distributed viewpoints with low crosstalk, which increases the viewpoint density in the middle viewing zone and provides clear monocular depth cues. The corresponding coding method is presented as well. The optimized compound lenticular lens array can balance between suppressing aberration and improving displayed quality. The simulations and experiments show that the proposed 3D LFD can present natural 3D images with the right perception and occlusion relationship within a 65° viewing angle.

Effect of immersive visualization technologies on cognitive load, motivation, usability, and embodiment

Virtual reality (VR) is a promising tool to promote motor (re)learning in healthy users and brain-injured patients. However, in current VR-based motor training, movements of the users performed in a three-dimensional space are usually visualized on computer screens, televisions, or projection systems, which lack depth cues (2D screen), and thus, display information using only monocular depth cues. The reduced depth cues and the visuospatial transformation from the movements performed in a three-dimensional space to their two-dimensional indirect visualization on the 2D screen may add cognitive load, reducing VR usability, especially in users suffering from cognitive impairments. These 2D screens might further reduce the learning outcomes if they limit users’ motivation and embodiment, factors previously associated with better motor performance. The goal of this study was to evaluate the potential benefits of more immersive technologies using head-mounted displays (HMDs). As a first step towards potential clinical implementation, we ran an experiment with 20 healthy participants who simultaneously performed a 3D motor reaching and a cognitive counting task using: (1) (immersive) VR (IVR) HMD, (2) augmented reality (AR) HMD, and (3) computer screen (2D screen). In a previous analysis, we reported improved movement quality when movements were visualized with IVR than with a 2D screen. Here, we present results from the analysis of questionnaires to evaluate whether the visualization technology impacted users’ cognitive load, motivation, technology usability, and embodiment. Reports on cognitive load did not differ across visualization technologies. However, IVR was more motivating and usable than AR and the 2D screen. Both IVR and AR rea ched higher embodiment level than the 2D screen. Our results support our previous finding that IVR HMDs seem to be more suitable than the common 2D screens employed in VR-based therapy when training 3D movements. For AR, it is still unknown whether the absence of benefit over the 2D screen is due to the visualization technology per se or to technical limitations specific to the device.

Is accommodation a confounder in pupillometry research?

Much psychological research uses pupil diameter measurements to investigate the cognitive and emotional effects of visual stimuli. A potential problem is that accommodating at a nearby point causes the pupil to constrict. This study examined to what extent accommodation is a confounder in pupillometry research. Participants solved multiplication problems at different distances (Experiment 1) and looked at line drawings with different monocular depth cues (Experiment 2) while their pupil diameter, refraction, and vergence angle were recorded using a photorefractor. Experiment 1 showed that the pupils dilated while performing the multiplications, for all presentation distances. Pupillary constriction due to accommodation was not strong enough to override pupil dilation due to cognitive load. Experiment 2 showed that monocular depth cues caused a small shift in refraction in the expected direction. We conclude that, for the young student sample we used, pupil diameter measurements are not substantially affected by accommodation.

Stereoscopic 3D geometric distortions analyzed from the viewer's point of view.

Stereoscopic 3D (S3D) geometric distortions can be introduced by mismatches among image capture, display, and viewing configurations. In previous work of S3D geometric models, geometric distortions have been analyzed from a third-person perspective based on the binocular depth cue (i.e., binocular disparity). A third-person perspective is different from what the viewer sees since monocular depth cues (e.g., linear perspective, occlusion, and shadows) from different perspectives are different. However, depth perception in a 3D space involves both monocular and binocular depth cues. Geometric distortions that are solely predicted by the binocular depth cue cannot describe what a viewer really perceives. In this paper, we combine geometric models and retinal disparity models to analyze geometric distortions from the viewer's perspective where both monocular and binocular depth cues are considered. Results show that binocular and monocular depth-cue conflicts in a geometrically distorted S3D space. Moreover, user-initiated head translations averting from the optimal viewing position in conventional S3D displays can also introduce geometric distortions, which are inconsistent with our natural 3D viewing condition. The inconsistency of depth cues in a dynamic scene may be a source of visually induced motions sickness.

Towards describing scenes by animals: Pigeons' ordinal discrimination of objects varying in depth.

The perception of a complex scene requires visual mechanisms that include identifying objects and their relative placement in depth. To examine apparent depth perception in birds, we tested four pigeons with a novel multiple-sequential-choice procedure. We created 3D-rendered scene stimuli containing three objects located at different apparent depths based on a variety of pictorial cues and placed small circular target response areas on them. The pigeons were trained to sequentially choose among the multiple response areas to report the object closest in apparent depth (ordinal position; front then middle object). After the pigeons learned this sequential depth discrimination, their use of three different monocular depth cues (occlusion, relative size, height in field) was tested, and their flexibility evaluated using three novel objects. In addition to the contribution to understanding apparent depth perception in birds, the use of more flexible open-ended choice discriminations, as employed here, has considerable promise for creating informative production-like tasks in nonverbal animals.

Extended geometric models for stereoscopic 3D with vertical screen disparity

Geometric models are tools used to analyze geometric distortions in S3D viewing. For standard geometric models, 3D perception is predicted by calculating the intersection between the two projection lines from the left and the right eye to two corresponding onscreen points. However, standard geometric models fail in two conditions: (1) S3D images are captured by converging cameras (toe-in) and displayed on a flat screen; (2) the viewer’s head is rotated about the vertical axis (yaw) or depth axis (roll) relative to the screen. In these two conditions, vertical screen disparity with respect to the viewer is introduced. In this paper, we extend standard geometric models to cover the conditions with vertical screen disparity. To test our models, we conduct experiments using random-dot stereograms (RDS) with vertical screen disparities introduced by rotating RDS images. Experimental results show that the perceived depth is well predicted by the proposed model when the rotation angle is smaller than 20°. However, this paper introduces a retinal eccentricity model to analyze geometric distortions from the viewer’s perspective, which indicates that the geometric distortions only affect the binocular depth cue and not monocular depth cues. The monocular-binocular depth cue conflicts with motions may induce motion sickness in S3D viewing.

Color for the perceptual organization of the pictorial plane: Victor Vasarely's legacy to Gestalt psychology

Victor Vasarely's (1906–1997) important legacy to the study of human perception is brought to the forefront and discussed. A large part of his impressive work conveys the appearance of striking three-dimensional shapes and structures in a large-scale pictorial plane. Current perception science explains such effects by invoking brain mechanisms for the processing of monocular (2D) depth cues. Here in this study, we illustrate and explain local effects of 2D color and contrast cues on the perceptual organization in terms of figure-ground assignments, i.e. which local surfaces are likely to be seen as “nearer” or “bigger” in the image plane. Paired configurations are embedded in a larger, structurally ambivalent pictorial context inspired by some of Vasarely's creations. The figure-ground effects these configurations produce reveal a significant correlation between perceptual solutions for “nearer” and “bigger” when other geometric depth cues are missing. In consistency with previous findings on similar, albeit simpler visual displays, a specific color may compete with luminance contrast to resolve the planar ambiguity of a complex pattern context at a critical point in the hierarchical resolution of figure-ground uncertainty. The potential role of color temperature in this process is brought forward here. Vasarely intuitively understood and successfully exploited the subtle context effects accounted for in this paper, well before empirical investigation had set out to study and explain them in terms of information processing by the visual brain.

Optical Metasurface‐Based Holographic Stereogram

AbstractHolographic stereography providing binocular depth cues is one of the most promising technologies for 3D displays. However, conventional holographic stereograms based on micrometer‐scale pixels suffer from multiple diffraction orders and narrow viewing angles. Optical metasurfaces with sub‐wavelength‐scale features have recently been leading amongst the state‐of‐the‐art technologies in 3D holograms but employing only monocular depth cues. Here, a novel method is presented based on optical metasurfaces for obtaining a binocular holographic stereopsis. The demonstrated optical metasurface is an ensemble of several hologram pieces, which produce the different 2D projections of the target 3D structure depending on the observation direction, and displays the holographic stereogram of 25 × 25 × 25 µm3 over a wide viewing angle of more than ±30°. A Gerchberg–Saxton algorithm modified with a spatial Fourier filter calculates the phase and amplitude distribution of meta‐atoms. The results will open avenues to advanced eyeglasses‐free 3D displays that can provide rich and well‐defined depth cues.