Monocular Cues Research Articles

Pictorial depth cues elicit the perception of tridimensionality in dogs

The perception of tridimensionality is elicited by binocular disparity, motion parallax, and monocular or pictorial cues. The perception of tridimensionality arising from pictorial cues has been investigated in several non-human animal species. Although dogs can use and discriminate bidimensional images, to date there is no evidence of dogs’ ability to perceive tridimensionality in pictures and/or through pictorial cues. The aim of the present study was to assess the perception of tridimensionality in dogs elicited by two pictorial cues: linear perspective and shading. Thirty-two dogs were presented with a tridimensional stimulus (i.e., a ball) rolling onto a planar surface until eventually falling into a hole (control condition) or until reaching and rolling over an illusory hole (test condition). The illusory hole corresponded to the bidimensional pictorial representation of the real hole, in which the pictorial cues of shading and linear perspective created the impression of tridimensionality. In a violation of expectation paradigm, dogs showed a longer looking time at the scene in which the unexpected situation of a ball rolling over an illusory hole occurred. The surprise reaction observed in the test condition suggests that the pictorial cues of shading and linear perspective in the bidimensional image of the hole were able to elicit the perception of tridimensionality in dogs.

Comparison of Depth-Related Visuomotor Task Performance in Uniocular Individuals and in Binocular Controls With and Without Temporary Monocular Occlusion.

Do one-eyed (uniocular) humans use monocular depth cues differently from those with intact binocularity to perform depth-related visuomotor tasks that emulate complex activities of daily living? If so, does performance depend on the participant's age, duration of uniocularity and head movements? Forty-five uniocular cases (age range 6-37years; 2.4months-31.0years of uniocularity) and 46 age-similar binocular controls performed a task that required them to pass a hoop around an electrified wire convoluted in depth multiple times, while avoiding contact as indicated by auditory feedback. The task was performed with and without head restraint, in random order. The error rate and speed were calculated from the frequency of contact between the hoop and wire and the total task duration (adjusting for error time), respectively, all determined from video recordings of the task. Head movements were analyzed from the videos using face-tracking software. Error rate decreased with age (P < 0.001) until the late teen years while speed revealed no such trend. Across all ages, the error rate increased and speed decreased in the absence of binocularity (P < 0.001). There was no additional error reduction with duration of uniocularity (P = 0.16). Head movements provided no advantage to task performance, despite generating parallax disparities comparable to binocular viewing. Performance in a dynamic, depth-related visuomotor task is reduced in the absence of binocular viewing, independent of age-related performance level. This study finds no evidence for a prolonged experience with monocular depth cues being advantageous for such tasks over transient loss of binocularity.

Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space.

Arthropods' eyes are effective biological vision systems for object tracking and wide field of view because of their structural uniqueness; however, unlike mammalian eyes, they can hardly acquire the depth information of a static object because of their monocular cues. Therefore, most arthropods rely on motion parallax to track the object in three-dimensional (3D) space. Uniquely, the praying mantis (Mantodea) uses both compound structured eyes and a form of stereopsis and is capable of achieving object recognition in 3D space. Here, by mimicking the vision system of the praying mantis using stereoscopically coupled artificial compound eyes, we demonstrated spatiotemporal object sensing and tracking in 3D space with a wide field of view. Furthermore, to achieve a fast response with minimal latency, data storage/transportation, and power consumption, we processed the visual information at the edge of the system using a synaptic device and a federated split learning algorithm. The designed and fabricated stereoscopic artificial compound eye provides energy-efficient and accurate spatiotemporal object sensing and optical flow tracking. It exhibits a root mean square error of 0.3 centimeter, consuming only approximately 4 millijoules for sensing and tracking. These results are more than 400 times lower than conventional complementary metal-oxide semiconductor-based imaging systems. Our biomimetic imager shows the potential of integrating nature's unique design using hardware and software codesigned technology toward capabilities of edge computing and sensing.

Extraction of three-dimensional shapes in glaucoma patients in response to monocular depth cues.

To assess the impact of glaucoma on perceiving three-dimensional (3D) shapes based on monocular depth cues. Clinical observational study. Twenty glaucoma patients, subjected to binocular visual-field sensitivity (binocular-VFS) tests using a Humphrey Visual Field Analyzer, and 20 age-matched healthy volunteers, underwent two tasks: identifying the nearest vertex of a 3D shape using monocular shading (3D-SfS), texture (3D-SfT), or motion (3D-SfM) cues, and distinguishing elementary one-dimensional (1D) features of these cues. The association of the visual-field index (VFI) of binocular-VFS with 3D shape perception in glaucoma patients was also examined. Glaucoma patients demonstrated reduced accuracy in distinguishing 1D luminance brightness and a larger "error-in-depth" between the perceived and actual depths for 3D-SfM and 3D-SfS compared to healthy volunteers. Six glaucoma patients with a 100% VFI for binocular-VFS exhibited a similar error-in-depth to the other fourteen glaucoma patients; they had a larger error-in-depth for 3D-SfM compared to healthy volunteers. No correlation between the error-in-depth values and the VFI values of binocular-VFS was observed. The 3D shape perception in glaucoma patients varies based on the depth cue's characteristics. Impaired 1D discrimination and larger thresholds for 3D-SfM in glaucoma patients with a 100% VFI for binocular-VFS indicate more pronounced perceptual deficits of lower-level elementary features for 3D-SfS and higher-level visual processing of 3D shapes for 3D-SfM. The effects of the location and degree of binocular visual-field defects on 3D shape perception remain to be elucidated. Our research provides insights into the 3D shape extraction mechanism in glaucoma.

Active Vision in Binocular Depth Estimation: A Top-Down Perspective.

Depth estimation is an ill-posed problem; objects of different shapes or dimensions, even if at different distances, may project to the same image on the retina. Our brain uses several cues for depth estimation, including monocular cues such as motion parallax and binocular cues such as diplopia. However, it remains unclear how the computations required for depth estimation are implemented in biologically plausible ways. State-of-the-art approaches to depth estimation based on deep neural networks implicitly describe the brain as a hierarchical feature detector. Instead, in this paper we propose an alternative approach that casts depth estimation as a problem of active inference. We show that depth can be inferred by inverting a hierarchical generative model that simultaneously predicts the eyes' projections from a 2D belief over an object. Model inversion consists of a series of biologically plausible homogeneous transformations based on Predictive Coding principles. Under the plausible assumption of a nonuniform fovea resolution, depth estimation favors an active vision strategy that fixates the object with the eyes, rendering the depth belief more accurate. This strategy is not realized by first fixating on a target and then estimating the depth; instead, it combines the two processes through action-perception cycles, with a similar mechanism of the saccades during object recognition. The proposed approach requires only local (top-down and bottom-up) message passing, which can be implemented in biologically plausible neural circuits.

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

Self-supervised monocular methods can efficiently learn depth information of weakly textured surfaces or reflective objects. However, the depth accuracy is limited due to the inherent ambiguity in monocular geometric modeling. In contrast, multi-frame depth estimation methods improve depth accuracy thanks to the success of Multi-View Stereo (MVS), which directly makes use of geometric constraints. Unfortunately, MVS often suffers from texture-less regions, non-Lambertian surfaces, and moving objects, especially in real-world video sequences without known camera motion and depth supervision. Therefore, we propose MOVEDepth, which exploits the MOnocular cues and VElocity guidance to improve multi-frame Depth learning. Unlike existing methods that enforce consistency between MVS depth and monocular depth, MOVEDepth boosts multi-frame depth learning by directly addressing the inherent problems of MVS. The key of our approach is to utilize monocular depth as a geometric priority to construct MVS cost volume, and adjust depth candidates of cost volume under the guidance of predicted camera velocity. We further fuse monocular depth and MVS depth by learning uncertainty in the cost volume, which results in a robust depth estimation against ambiguity in multi-view geometry. Extensive experiments show MOVEDepth achieves state-of-the-art performance: Compared with Monodepth2 and PackNet, our method relatively improves the depth accuracy by 20% and 19.8% on the KITTI benchmark. MOVEDepth also generalizes to the more challenging DDAD benchmark, relatively outperforming ManyDepth by 7.2%. The code is available at https://github.com/JeffWang987/MOVEDepth.

Monocular cues are superior to binocular cues for size perception when they are in conflict in virtual reality

Three-dimensional (3D) depth information is important to estimate object sizes. The visual system extracts 3D depth information using both binocular cues and monocular cues. However, how these different depth signals interact with each other to compute the object size in 3D space is unclear. Here, we aim to study the relative contribution of monocular and binocular depth information to size perception in a modified Ponzo context by manipulating their relations in a virtual reality environment. Specifically, we compared the amount of the size illusion in the following two conditions, in which monocular cues and binocular disparity in the Ponzo context can indicate the same depth sign (congruent) or opposite depth sign (incongruent). Our results show an increase in the amount of the Ponzo illusion in the congruent condition. In contrast, in the incongruent condition, we find that the two cues indicating the opposite depth signs do not cancel out the Ponzo illusion, suggesting that the effects of the two cues are not equal. Rather, binocular disparity information seems to be suppressed and the size judgment is mainly dependent on the monocular depth information when the two cues are in conflict. Our results suggest that monocular and binocular depth signals are fused for size perception only when they both indicate the same depth sign and top-down 3D depth information based on monocular cues contributes more to size perception than binocular disparity when they are in conflict in virtual reality.

Monocular Based Navigation System for Autonomous Ground Robots Using Multiple Deep Learning Models

In recent years, the development of ground robots with human-like perception capabilities has led to the use of multiple sensors, including cameras, lidars, and radars, along with deep learning techniques for detecting and recognizing objects and estimating distances. This paper proposes a computer vision-based navigation system that integrates object detection, segmentation, and monocular depth estimation using deep neural networks to identify predefined target objects and navigate towards them with a single monocular camera as a sensor. Our experiments include different sensitivity analyses to evaluate the impact of monocular cues on distance estimation. We show that this system can provide a ground robot with the perception capabilities needed for autonomous navigation in unknown indoor environments without the need for prior mapping or external positioning systems. This technique provides an efficient and cost-effective means of navigation, overcoming the limitations of other navigation techniques such as GPS-based and SLAM-based navigation.Graphical

Binocular portraiture

Pictorial portraits are viewed with two eyes despite the fact that they are mostly monocular: they have been produced from a single viewpoint (either by painters or photographers). The differences between the images on each eye are a consequence of the separation between them rather than differences in two pictorial images. Viewing with two eyes detracts from the monocular cues to depth within the singular portrait because of information for the flatness of the pictorial surface. Binocular portraits, on the other hand, incorporate differences between two pictorial images producing perceptual effects that cannot be seen by a single eye alone. The differences can consist of small disparities that yield stereoscopic depth or large ones that produce binocular rivalry. Binocular portraits require viewing with a stereoscope, many varieties of which exist. Those shown here are anaglyphs which can be observed through red/cyan filters. They are not conventional stereoscopic portraits where the sitter is imaged from two slightly different locations. Rather, the binocular processes of cooperation (stereoscopic depth perception) and competition (binocular rivalry) are manipulated in the binocular portraits. The subjects shown in the anaglyphic portraits have been involved in the science and art of binocular vision.

Mapping residual stereopsis in macular degeneration.

Individuals with macular degeneration typically lose vision in the central region of one or both eyes. A binocular scotoma occurs when vision loss occurs in overlapping locations in both eyes, but stereopsis is impacted even in the non-overlapping region wherever the visual field in either eye is affected. We used a novel stereoperimetry protocol to measure local stereopsis across the visual field (up to 25° eccentricity) to determine how locations with functional stereopsis relate to the scotomata in the two eyes. Participants included those with monocular or binocular scotomata and age-matched controls with healthy vision. Targets (with or without depth information) were presented on a random dot background. Depth targets had true binocular disparity of 20' (crossed), whereas non-depth targets were defined by monocular cues such as contrast and dot density. Participants reported target location and whether it was in depth or flat. Local depth sensitivity (d') estimates were then combined to generate a stereopsis map. This stereopsis map was compared to the union of the monocular microperimetry estimates that mapped out the functional extent of the scotoma in each eye. The "union" prediction aligned with residual stereopsis, showing impaired stereopsis within this region and residual stereopsis outside this region. Importantly, the stereoblind region was typically more extensive than the binocular scotoma defined by the intersection (overlap) of the scotomata. This explains why individuals may have intact binocular visual fields but be severely compromised in tasks of daily living that benefit from stereopsis, such as eye-hand coordination and navigation.

Peripheral target detection can be modulated by target distance but not attended distance in 3D space simulated by monocular depth cues

Most studies of visuo-spatial attention present stimuli on a 2D plane, and less is known about how attention varies in 3D space. Previous studies found better peripheral detection performance for targets at a near compared to a far depth, simulated by pictorial cues and optical flow. The current study examined whether target detectability is monotonically related to distance along the depth axis, and whether the attended distance modulates the effect of target distance. We investigated these questions in two experiments that measured how apparent distance and target eccentricity affects peripheral target detection when performed alone during passive simulated self-motion, or during a simultaneous, active central car-following task. Experiment 1 found that targets at an apparent distance of 18.5 virtual meters were detected faster and more accurately than targets at 9.25 and 37 virtual meters, and detectability declined with eccentricity. Experiment 2 examined the effect of the attended location by varying the distance between the viewer and the lead car on which participants were instructed to fixate (i.e. the headway) while equating target distances across headway conditions. Experiment 2 replicated the effects found in Experiment 1, and headway did not modulate the effect of target distance. These results are consistent with the hypothesis that target detection depends non-monotonically on the distance between the viewer and the target, and is not affected by the distance between the target and attended location. However, target detection may also have been affected by stimulus characteristics that co-varied with apparent depth, rather than depth per se.

Integration of ontology reasoning-based monocular cues in deep learning modeling for single image depth estimation in urban driving scenarios

Humans are able to estimate the depth of objects in their environment even using only one eye through the use of what are known as monocular cues. In this paper, we aim to integrate human knowledge and human-like reasoning used for monocular depth estimation within deep neural networks. The idea is to support the network in order to help it learn in an explicit and fast way the essential cues for the target task. For this purpose, we investigate the possibility of directly integrating geometric, semantic, and contextual information into the monocular depth estimation process. We propose exploiting an ontology model in a deep learning context to represent the urban environment as a structured set of concepts linked with semantic relationships. Monocular cues information are extracted through reasoning performed on the proposed ontology and are fed together with the RGB image in a multistream way into the deep neural network for depth estimation. Our approach is validated and evaluated on widespread benchmark datasets: KITTI, CityScapes, and AppolloScape. The obtained results show that the proposed method improves upon the state-of-the-art monocular depth estimation deep models and shows promising results regarding cross-evaluation, mainly for unseen driving scenarios.

Super multi-view near-eye display with a lightguide combiner.

We propose a lightguide-type super multi-view near-eye display that uses a digital micromirror device and a LED array. The proposed method presents three-dimensional images with a natural monocular depth cue using a compact combiner optics which consists of a thin lightguide and holographic optical elements (HOEs). Feasibility of the proposed method is verified by optical experiments which demonstrate monocular three-dimensional image presentation over a wide depth range. We also analyze the degradation of the image quality stemming from the spectral spread of the HOEs and show its reduction by a pre-compensation exploiting an adaptive moment estimation (Adam) optimizer.

Shedding Light on Cast Shadows: An Investigation of Perceived Ground Contact in AR and VR.

Virtual objects in augmented reality (AR) often appear to float atop real world surfaces, which makes it difficult to determine where they are positioned in space. This is problematic as many applications for AR require accurate spatial perception. In the current study, we examine how the way we render cast shadows-which act as an important monocular depth cue for creating a sense of contact between an object and the surface beneath it-impacts spatial perception. Over two experiments, we evaluate people's sense of surface contact given both traditional and non-traditional shadow shading methods in optical see-through augmented reality (OST AR), video see-through augmented reality (VST AR), and virtual reality (VR) head-mounted displays. Our results provide evidence that nontraditional shading techniques for rendering shadows in AR displays may enhance the accuracy of one's perception of surface contact. This finding implies a possible tradeoff between photorealism and accuracy of depth perception, especially in OST AR displays. However, it also supports the use of more stylized graphics like non-traditional cast shadows to improve perception and interaction in AR applications.

The impacts of lens and stereo camera separation on perceived slant in Virtual Reality head-mounted displays.

Stereoscopic AR and VR headsets have displays and lenses that are either fixed or adjustable to match a limited range of user inter-pupillary distances (IPDs). Projective geometry predicts a misperception of depth when either the displays or virtual cameras used to render images are misaligned with the eyes. However, misalignment between the eyes and lenses might also affect binocular convergence, which could further distort perceived depth. This possibility has been largely ignored in previous studies. Here, we evaluated this phenomenon in a VR headset in which the inter-lens and inter-axial camera separations are coupled and adjustable. In a baseline condition, both were matched to observers' IPDs. In two other conditions, the inter-lens and inter-axial camera separations were set to the maximum and minimum allowed by the headset. In each condition, observers were instructed to adjust a fold created by two intersecting, textured surfaces until it appeared to have an angle of 90°. The task was performed at three randomly interleaved viewing distances, monocularly and binocularly. In the monocular condition, observers underestimated the fold angle and there was no effect of viewing distance on their settings. In the binocular conditions, we found that when the lens and camera separation were less than the viewer's IPD, they exhibited compression of perceived slant relative to baseline. The reverse pattern was seen when the lens and camera separation were larger than the viewer's IPD. These results were well explained by a geometric model that considers shifts in convergence due to lens and display misalignment with the eyes, as well as the relative contribution of monocular cues.

SGM3D: Stereo Guided Monocular 3D Object Detection

Monocular 3D object detection aims to predict the object location, dimension and orientation in 3D space alongside the object category given only a monocular image. It poses a great challenge due to its ill-posed property, which is a critical lack of depth information in the 2D image plane. While existing approaches leverage off-the-shelf depth estimation or rely on LiDAR sensors to mitigate this problem, dependence on the additional depth model or expensive equipment severely limits their scalability to generic 3D perception. In this paper, we propose a stereo-guided monocular 3D object detection framework, dubbed SGM3D, which aligns the monocular feature to the stereo feature and exploits the network's ability of generating the cross-modal feature for accurate 3D object detection. We present a multi-granularity feature alignment (MG-FA) mechanism to exploit the network's ability to generate stereo-mimicking features given only monocular cues. Coarse feature-level, as well as fine anchor-level feature alignment, are both leveraged for monocular feature guidance. In addition, we introduce an IoU matching-based feature alignment (IoU-MA) method for object-level feature alignment between the stereo and monocular predictions to alleviate the mismatches while adopting the MG-FA. Extensive experiments on the KITTI datasets demonstrate the effectiveness of our method. Code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zhouzheyuan/sgm3d</uri> .

Analysis on the Change of Stereoscopic Depth Perception with Geometrical Monocular Cues

Analysis on the Change of Stereoscopic Depth Perception with Geometrical Monocular Cues

Distance estimation from monocular cues in an ethological visuomotor task.

In natural contexts, sensory processing and motor output are closely coupled, which is reflected in the fact that many brain areas contain both sensory and movement signals. However, standard reductionist paradigms decouple sensory decisions from their natural motor consequences, and head-fixation prevents the natural sensory consequences of self-motion. In particular, movement through the environment provides a number of depth cues beyond stereo vision that are poorly understood. To study the integration of visual processing and motor output in a naturalistic task, we investigated distance estimation in freely moving mice. We found that mice use vision to accurately jump across a variable gap, thus directly coupling a visual computation to its corresponding ethological motor output. Monocular eyelid suture did not affect gap jumping success, thus mice can use cues that do not depend on binocular disparity and stereo vision. Under monocular conditions, mice altered their head positioning and performed more vertical head movements, consistent with a shift from using stereopsis to other monocular cues, such as motion or position parallax. Finally, optogenetic suppression of primary visual cortex impaired task performance under both binocular and monocular conditions when optical fiber placement was localized to binocular or monocular zone V1, respectively. Together, these results show that mice can use monocular cues, relying on visual cortex, to accurately judge distance. Furthermore, this behavioral paradigm provides a foundation for studying how neural circuits convert sensory information into ethological motor output.

Application of Spatial Cues and Optical Distortions as Augmentations during Virtual Reality (VR) Gaming: The Multifaceted Effects of Assistance for Eccentric Viewing Training.

The present study investigates the effects of peripheral spatial cues and optically distorting augmentations over eccentric vision mechanisms in normally sighted participants with simulated scotoma. Five different augmentations were tested inside a virtual reality (VR)-gaming environment. Three were monocular spatial cues, and two were binocular optical distortions. Each was divided into three conditions: baseline with normal viewing, augmentation with one of the assistance methods positioned around the scotoma, and one with only the simulated central scotoma. The study found that the gaming scenario induced eccentric viewing for the cued augmentation groups, even when the peripheral assistance was removed, while for the optical distortions group, the eccentric behavior disappeared after the augmentation removal. Additionally, an upwards directionality of gaze relative to target during regular gaming was found. The bias was maintained and implemented during and after the cued augmentations but not after the distorted ones. The results suggest that monocular peripheral cues could be better candidates for implementing eccentric viewing training in patients. At the same time, it showed that optical distortions might disrupt such behavior. Such results are noteworthy since distortions such as zoom are known to help patients with macular degeneration see targets of interest.

Structure‐aware dehazing of sewer inspection images based on monocular depth cues

AbstractIn sewer pipes, haze caused by the humid environment seriously impairs the quality of closed‐circuit television (CCTV) images, which leads to poor performance of subsequent pipe defects detection. Meanwhile, the complexity of sewer images, such as steep depth change and extensive textureless regions, brings great challenges to the performance or application of general dehazing algorithms. Therefore, this study estimates sewer depth maps first with the help of the water–pipewall borderlines to produce the paired dehazing dataset. Then a structure‐aware nonlocal network (SANL‐Net) is proposed with the detected borderlines and the dehazing result as two supervisory signals. SANL‐Net shows its superiority over other state‐of‐the‐art approaches with 147 in mean square error (MSE), 27.28 in peak signal to noise ratio (PSNR), 0.8963 in structural similarity index measure (SSIM), and 15.47M in parameters. Also, the outstanding performance in real image dehazing implies the accuracy of depth estimation. Experimental results indicate that SANL‐Net significantly improves the performance of defects detection tasks, such as an increase of 23.16% in mean intersection over union (mIoU) for semantic segmentation.