Binocular Depth Research Articles

Calibration of random dot stereograms and correlograms free of monocular cues.

Dynamic random dot stereograms (DRDSs) and correlograms (DRDCs) are cyclopean stimuli containing binocular depth cues that are ideally, invisible by one eye alone. Thus, they are important tools in assessing stereoscopic function in experimental or ophthalmological diagnostic settings. However, widely used filter-based three-dimensional display technologies often cannot guarantee complete separation of the images intended for the two eyes. Without proper calibration, this may result in unwanted monocular cues in DRDSs and DRDCs, which may bias scientific or diagnostic results. Here, we use a simple mathematical model describing the relationship of digital video values and average luminance and dot contrast in the two eyes. We present an optimization algorithm that provides the set of digital video values that achieve minimal crosstalk at user-defined average luminance and dot contrast for both eyes based on photometric characteristics of a given display. We demonstrated in a psychophysical experiment with color normal participants that this solution is optimal because monocular cues were not detectable at either the calculated or the experimentally measured optima. We also explored the error by which a range of luminance and contrast combinations can be implemented. Although we used a specific monitor and red-green glasses as an example, our method can be easily applied for other filter based three-dimensional systems. This approach is useful for designing psychophysical experiments using cyclopean stimuli for a specific display.

Optical Metasurface‐Based Holographic Stereogram

AbstractHolographic stereography providing binocular depth cues is one of the most promising technologies for 3D displays. However, conventional holographic stereograms based on micrometer‐scale pixels suffer from multiple diffraction orders and narrow viewing angles. Optical metasurfaces with sub‐wavelength‐scale features have recently been leading amongst the state‐of‐the‐art technologies in 3D holograms but employing only monocular depth cues. Here, a novel method is presented based on optical metasurfaces for obtaining a binocular holographic stereopsis. The demonstrated optical metasurface is an ensemble of several hologram pieces, which produce the different 2D projections of the target 3D structure depending on the observation direction, and displays the holographic stereogram of 25 × 25 × 25 µm3 over a wide viewing angle of more than ±30°. A Gerchberg–Saxton algorithm modified with a spatial Fourier filter calculates the phase and amplitude distribution of meta‐atoms. The results will open avenues to advanced eyeglasses‐free 3D displays that can provide rich and well‐defined depth cues.

Blind Stereo Image Quality Assessment Based on Binocular Visual Characteristics and Depth Perception

The quality prediction of stereo images has great challenges without reference images. In this paper, we propose a novel no-reference stereo image quality assessment (NR-SIQA) model based on binocular visual characteristics and depth perception, which can effectively evaluate the quality of symmetric distortion and asymmetric distortion images. To be specific, we discriminate the different binocular behaviors by analyzing binocular visual characteristics, and construct the corresponding cyclopean view instead of single cyclopean view to simulate different binocular behaviors. Then, we extract monocular and binocular visual features from the left view, the right view and the synthetic cyclopean view. Furthermore, in order to evaluate the depth quality of the stereo image accurately, we extract the depth perception features from the weighted disparity map and the longitudinal correlation coefficient map. Finally, we construct the mapping relationship model from quality perception feature domain to quality score domain by training an adaptive enhancement algorithm based on support vector regression (SVR). We evaluate the performance of the proposed algorithm on four stereo image databases. The experimental results show that compared with the state-of-the-art full reference(FR), reduced reference(RR) and NR-SIQA algorithms, the proposed algorithm achieves highly competitive performance for both symmetric and asymmetric distortions.

Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy

Depth mapping from binocular endoscopy images plays an important role in stereoscopic surgical treatment. Owing to the development of deep convolutional neural networks (CNNs), binocular depth estimation models have achieved many exciting results in the fields of autonomous driving and machine vision. However, the application of these methods to endoscopic imaging is greatly limited by the fact that binocular endoscopic images not only are rare, but also have unsatisfying features such as no texture, no ground truth, bad contrast, and high gloss. Aiming at solving the above-mentioned problems, we have built a precise gastrointestinal environment by the open-source software blender to simulate abundant binocular endoscopy data and proposed a 23-layer deep CNNs method to generate real-time stereo depth mapping. An efficient scale-invariant loss function is introduced in this paper to accommodate the characteristics of endoscope images, which improves the accuracy of achieved depth mapping results. Regarding the considerable training data for typical CNNs, our method requires only a few images ($960\times 720$ resolution) at 45 frames per second on an NVIDIA GTX 1080 GPU module, then the depth mapping information is generated in real-time with satisfactory accuracy. The effectiveness of the developed method is validated by comparing with state-of-the-art methods on processing the same datasets, demonstrating a faster and more accurate performance than other model frames.

Comparison of Layer Operations and Optimization Methods for Light Field Display

A light-field display provides not only binocular depth sensation but also natural motion parallax with respect to head motion, which invokes a strong feeling of immersion. Such a display can be implemented with a set of stacked layers, each of which has pixels that can carry out light-ray operations (multiplication and addition). With this structure, the appearance of the display varies over the observed directions (i.e., a light field is produced) because the light rays pass through different combinations of pixels depending on both the originating points and outgoing directions. To display a specific 3-D scene, these layer patterns should be optimized to produce a light field that is as close as possible to that produced by the target three-dimensional scene. To deepen the understanding for this type of light field display, we focused on two important factors: light-ray operations carried out using layers and optimization methods for the layer patterns. Specifically, we compared multiplicative and additive layers, which are optimized using analytical methods derived from mathematical optimization or faster data-driven methods implemented as convolutional neural networks (CNNs). We compared combinations within these two factors in terms of the accuracy of light-field reproduction and computation time. Our results indicate that multiplicative layers achieve better accuracy than additive ones, and CNN-based methods perform faster than the analytical ones. We suggest that the best choice in terms of the balance between accuracy and computation speed is using multiplicative layers optimized using a CNN-based method.

Occluded Pedestrian Detection Based on Depth Vision Significance in Biomimetic Binocular

Pedestrian detection and tracking has become an important field in the field of computer vision research. However, the existing pedestrian detection algorithms have some problems, such as low accuracy and poor stability due to the similar background and overlapped occlusion interference. Therefore, an occluded pedestrian detection method based on binocular vision is proposed in this paper. We simulate the recognition of human brain and use the deep learning network MobileNet to detect and locate the initial pedestrians. Then, binocular depth is introduced as visual salience prior information, which solves the problem of identifying pedestrians with similar background and occlusion. The experimental results show that our pedestrian detection framework greatly improves the pedestrian error detection under similar background and occlusion conditions.

AB019. A proposed test of stereovision to identify local retinal abnormalities

Background: The human visual system extracts depth information from disparity in the images seen by the two eyes. The ability to calculate depth from disparity will be disrupted if local retinal abnormalities distort parts of those images, especially if these distortions are different in the two eyes. In its early stages, age-related macular degeneration (AMD) causes slight distortions in the central vision field which differ in the two eyes. AMD is the most common form of irreversible blindness in people over the age of 50. The goal of this project is to develop a stereoscopic perception test which leverages the sensitivity of binocular depth perception to detect the interocular differences symptomatic of either early-stage AMD or other diseases affecting the retina.

On the Aperture Problem of Binocular 3D Motion Perception.

Like many predators, humans have forward-facing eyes that are set a short distance apart so that an extensive region of the visual field is seen from two different points of view. The human visual system can establish a three-dimensional (3D) percept from the projection of images into the left and right eye. How the visual system integrates local motion and binocular depth in order to accomplish 3D motion perception is still under investigation. Here, we propose a geometric-statistical model that combines noisy velocity constraints with a spherical motion prior to solve the aperture problem in 3D. In two psychophysical experiments, it is shown that instantiations of this model can explain how human observers disambiguate 3D line motion direction behind a circular aperture. We discuss the implications of our results for the processing of motion and dynamic depth in the visual system.Dataset: osf.io/2j6sq.

A model of how depth facilitates scene-relative object motion perception.

Many everyday interactions with moving objects benefit from an accurate perception of their movement. Self-motion, however, complicates object motion perception because it generates a global pattern of motion on the observer’s retina and radically influences an object’s retinal motion. There is strong evidence that the brain compensates by suppressing the retinal motion due to self-motion, however, this requires estimates of depth relative to the object—otherwise the appropriate self-motion component to remove cannot be determined. The underlying neural mechanisms are unknown, but neurons in brain areas MT and MST may contribute given their sensitivity to motion parallax and depth through joint direction, speed, and disparity tuning. We developed a neural model to investigate whether cells in areas MT and MST with well-established neurophysiological properties can account for human object motion judgments during self-motion. We tested the model by comparing simulated object motion signals to human object motion judgments in environments with monocular, binocular, and ambiguous depth. Our simulations show how precise depth information, such as that from binocular disparity, may improve estimates of the retinal motion pattern due the self-motion through increased selectivity among units that respond to the global self-motion pattern. The enhanced self-motion estimates emerged from recurrent feedback connections in MST and allowed the model to better suppress the appropriate direction, speed, and disparity signals from the object’s retinal motion, improving the accuracy of the object’s movement direction represented by motion signals.

Unsupervised Monocular Depth Estimation Based on Residual Neural Network of Coarse–Refined Feature Extractions for Drone

To take full advantage of the information of images captured by drones and given that most existing monocular depth estimation methods based on supervised learning require vast quantities of corresponding ground truth depth data for training, the model of unsupervised monocular depth estimation based on residual neural network of coarse–refined feature extractions for drone is therefore proposed. As a virtual camera is introduced through a deep residual convolution neural network based on coarse–refined feature extractions inspired by the principle of binocular depth estimation, the unsupervised monocular depth estimation has become an image reconstruction problem. To improve the performance of our model for monocular depth estimation, the following innovations are proposed. First, the pyramid processing for input image is proposed to build the topological relationship between the resolution of input image and the depth of input image, which can improve the sensitivity of depth information from a single image and reduce the impact of input image resolution on depth estimation. Second, the residual neural network of coarse–refined feature extractions for corresponding image reconstruction is designed to improve the accuracy of feature extraction and solve the contradiction between the calculation time and the numbers of network layers. In addition, to predict high detail output depth maps, the long skip connections between corresponding layers in the neural network of coarse feature extractions and deconvolution neural network of refined feature extractions are designed. Third, the loss of corresponding image reconstruction based on the structural similarity index (SSIM), the loss of approximate disparity smoothness and the loss of depth map are united as a novel training loss to better train our model. The experimental results show that our model has superior performance on the KITTI dataset composed by corresponding left view and right view and Make3D dataset composed by image and corresponding ground truth depth map compared to the state-of-the-art monocular depth estimation methods and basically meet the requirements for depth information of images captured by drones when our model is trained on KITTI.

Wide visual field angle holographic display using compact electro-holographic projectors.

Electro-holography has the problem of having a narrow visual field angle, because the resolution of a spatial light modulator is insufficient for displaying a fringe pattern. To solve this problem, this paper proposes a projector-type electro-holographic compact display that achieves a wide visual field angle by using the combination of an optical system and calculation algorithms. The results of experiments show that the visual field angle is three times larger than that of a normal electro-holographic display. In addition, it is demonstrated that the system has the ability to display 3D reconstructed images with binocular, full-color, high-resolution, and accurate depth presentation.

Progressive Fusion for Unsupervised Binocular Depth Estimation Using Cycled Networks

Recent deep monocular depth estimation approaches based on supervised regression have achieved remarkable performance. However, they require costly ground truth annotations during training. To cope with this issue, in this paper we present a novel unsupervised deep learning approach for predicting depth maps. We introduce a new network architecture, named Progressive Fusion Network (PFN), that is specifically designed for binocular stereo depth estimation. This network is based on a multi-scale refinement strategy that combines the information provided by both stereo views. In addition, we propose to stack twice this network in order to form a cycle. This cycle approach can be interpreted as a form of data-augmentation since, at training time, the network learns both from the training set images (in the forward half-cycle) but also from the synthesized images (in the backward half-cycle). The architecture is jointly trained with adversarial learning. Extensive experiments on the publicly available datasets KITTI, Cityscapes and ApolloScape demonstrate the effectiveness of the proposed model which is competitive with other unsupervised deep learning methods for depth prediction.

Perceptual stereoscopic images quality assessment method based on visual adaptable characteristics

The quality assessment of stereoscopic images plays an important role in various three-dimensional (3-D) areas and encounters more problems than two-dimensional (2-D) image quality assessment. We propose a perceptual full-reference quality assessment model by considering human adaptable double visual channel. Human binocular combination characteristics and depth perception are both considered in this model, and an adaptable gain-control model is adopted to assign appropriate weights to the information of human visual channels. Experimental results indicate that the proposed algorithm can serve as an efficient predictive image quality feature, which delivers not only highly competitive prediction accuracy but also moderate computational complexity.

Disruptive coloration and binocular disparity: breaking camouflage.

Many species employ camouflage to disguise their true shape and avoid detection or recognition. Disruptive coloration is a form of camouflage in which high-contrast patterns obscure internal features or break up an animal's outline. In particular, edge enhancement creates illusory, or ‘fake’ depth edges within the animal's body. Disruptive coloration often co-occurs with background matching, and together, these strategies make it difficult for an observer to visually segment an animal from its background. However, stereoscopic vision could provide a critical advantage in the arms race between perception and camouflage: the depth information provided by binocular disparities reveals the true three-dimensional layout of a scene, and might, therefore, help an observer to overcome the effects of disruptive coloration. Human observers located snake targets embedded in leafy backgrounds. We analysed performance (response time) as a function of edge enhancement, illumination conditions and the availability of binocular depth cues. We confirm that edge enhancement contributes to effective camouflage: observers were slower to find snakes whose patterning contains ‘fake’ depth edges. Importantly, however, this effect disappeared when binocular depth cues were available. Illumination also affected detection: under directional illumination, where both the leaves and snake produced strong cast shadows, snake targets were localized more quickly than in scenes rendered under ambient illumination. In summary, we show that illusory depth edges, created via disruptive coloration, help to conceal targets from human observers. However, cast shadows and binocular depth information improve detection by providing information about the true three-dimensional structure of a scene. Importantly, the strong interaction between disparity and edge enhancement suggests that stereoscopic vision has a critical role in breaking camouflage, enabling the observer to overcome the disruptive effects of edge enhancement.

The effect of induced fusional demand on static and dynamic stereoacuity thresholds: the digital Synoptophore

Background/aimsThe ability to extract depth from disparity may be hindered under fusional stress, as alignment of the eyes may be more difficult to maintain consistently. Therefore we aim to determine the effect of fusional demand on stereoacuity in individuals with no known binocular vision impairments.MethodsA novel static and dynamic binocular depth detection task, capable of assessing many discrete levels of stereoacuity, was presented on digital displays attached to each tube of the Synoptophore. Stereoacuity was measured with any latent deviation fully corrected and compared to that measured at the ‘recovery’ angle. This recovery angle is where single vision is restored after decompensation to diplopia, during vergence range assessment.ResultsSeventy-two subjects (50 Female, 22 Male; mean (SD) age 22 (6) years) were assessed. The amount of fusional demand was between 1 and 26 prism dioptres (PD), with a mean (SD) of 8(6)PD. Under zero fusion demand the mean (SD) static and dynamic depth detection thresholds were 322(53)” and 69(23)”. Under fusional stress these were 224(40)” and 77(21)”. There was no significant difference between thresholds in stressed and zero demand fusion (p = 0.08). Dynamic depth detection thresholds were significantly lower than static (P < 0.01).ConclusionFusional stress does not appear to impact on stereoacuity. The numerical value of the recovery point varied amongst individuals, but this represents a common point, where single vision is easily restored and binocularity well established. Due to individual differences in the ability to control a certain amount of fusional stress (e.g. vergences stress of 10PD, when recovery is 8PD, will perturb binocularity more than a person with a recovery of 20PD), previous reports may not accurately represent the effect of fusional stress. Whilst our findings are contrary to previous reports, we did not stress fusion beyond the recovery point and used a more accurate/repeatable method to measure stereoacuity.

Toward a Quality Predictor for Stereoscopic Images via Analysis of Human Binocular Visual Perception

Perceptual stereoscopic image quality assessment (SIQA) has become a challenge research problem due to the poor understanding of human binocular visual characteristics. For the task of SIQA, an intuitive idea is to develop effective models on the basis of the image content and depth perception. In this paper, we propose a full-reference objective quality evaluator for stereoscopic images by simulating binocular behaviors of the human visual system (HVS): Binocular interaction and depth perception. This model is based on a cyclopean image from a novel binocular combination model as image content quality description and a depth binocular combination model from a depth synthesized procedure as depth perception description. The final quality score of the distorted stereoscopic images is calculated by integrating the above two perception indicators. The experimental results on two stereoscopic image quality databases demonstrate that our proposed metric works efficiently for both symmetric and asymmetric distortions and achieves high consistent alignment with subjective observations.

Enhancing Binocular Depth Estimation Based on Proactive Perception and Action Cyclic Learning for an Autonomous Developmental Robot

In humans, perception and action (PA) possess cyclically causal relations. In this paper, we propose a new PA-based cyclic learning framework to autonomously enhance the depth-estimation accuracy of a humanoid robot and perform given behavioral tasks. The proposed method integrates the concepts of sensory invariance-driven action and object-size invariance to autonomously enhance the depth-estimation accuracy. If the depth estimation is reliable, the reinforcement learning framework is used to generate goal-directed actions of a humanoid robot based on a perceived environment. Iterative PA cycles of a robot autonomously refine its depth-estimation. The proposed method is evaluated using a humanoid robot (NAO) with stereo cameras, and the experimental results demonstrate that the proposed framework is effective for autonomously enhancing both the depth-estimation accuracy and the action-generation performance.

Binocular Visual Function With Staged Implantation of Diffractive Multifocal Intraocular Lenses With Three Add Powers

To evaluate the binocular visual function and depth of focus after staged implantation of diffractive multifocal intraocular lenses (MIOLs) with 3 different add powers following cataract removal. Prospective comparative case series. Setting: Institutional. Both eyes of 56 patients that underwent staged implantation of bifocal IOLs. In the first eye, MIOLs with a+4.0-diopter (D) add power were implanted. Based on outcomes at 1week postoperatively and patient preference for near vision, 1 of 3 near add powers-+4.0 D (group MM),+3.25 D (group ML), or+2.75 D (group MK)-was implanted in the fellow eye. Three months postoperatively, the binocular uncorrected and distance-corrected visual acuities (BUCVA and BDCVA, respectively) at 5.0, 1.0, 0.5, 0.4, and 0.3 meters; binocular defocus curves; contrast sensitivity; and near stereopsis were measured. The MM, ML, and MK groups included 21, 18, and 17 patients, respectively. The mean BUCVA in the MM group was significantly (P= .003) worse than that in the MK group at 0.5 meter. The BDCVAs at 0.5 meter in the ML and MK groups were significantly (P < .02) better than that in the MM group. The defocus curves with the -2.0 D refraction differed significantly (P < .001) between the MM and MK groups. The binocular contrast sensitivity and stereopsis values did not differ significantly. Staged implantation of different add powers obtained better binocular intermediate visual acuity without degradation of visual function.

Discriminative representation combinations for accurate face spoofing detection

Three discriminative representations for face presentation attack detection are introduced in this paper. Firstly we design a descriptor called spatial pyramid coding micro-texture (SPMT) feature to characterize local appearance information. Secondly we utilize the SSD, which is a deep learning framework for detection, to excavate context cues and conduct end-to-end face presentation attack detection. Finally we design a descriptor called template face matched binocular depth (TFBD) feature to characterize stereo structures of real and fake faces. For accurate presentation attack detection, we also design two kinds of representation combinations. Firstly, we propose a decision-level cascade strategy to combine SPMT with SSD. Secondly, we use a simple score fusion strategy to combine face structure cues (TFBD) with local micro-texture features (SPMT). To demonstrate the effectiveness of our design, we evaluate the representation combination of SPMT and SSD on three public datasets, which outperforms all other state-of-the-art methods. In addition, we evaluate the representation combination of SPMT and TFBD on our dataset and excellent performance is also achieved.

Distinct timing of neurogenesis of ipsilateral and contralateral retinal ganglion cells.

In higher vertebrates, the circuit formed by retinal ganglion cells (RGCs) projecting ipsilaterally (iRGCs) or contralaterally (cRGCs) to the brain permits binocular vision and depth perception. iRGCs and cRGCs differ in their position within the retina and in expression of transcription, guidance and activity-related factors. To parse whether these two populations also differ in the timing of their genesis, a feature of distinct neural subtypes and associated projections, we used newer birthdating methods and cell subtype specific markers to determine birthdate and cell cycle exit more precisely than previously. In the ventrotemporal (VT) retina, i- and cRGCs intermingle and neurogenesis in this zone lags behind RGC production in the rest of the retina where only cRGCs are positioned. In addition, within the VT retina, i- and cRGC populations are born at distinct times: neurogenesis of iRGCs surges at E13, and cRGCs arise as early as E14, not later in embryogenesis as reported. Moreover, in the ventral ciliary margin zone (CMZ), which contains progenitors that give rise to some iRGCs in ventral neural retina (Marcucci et al., 2016), cell cycle exit is slower than in other retinal regions in which progenitors give rise only to cRGCs. Further, when the cell cycle regulator Cyclin D2 is missing, cell cycle length in the CMZ is further reduced, mirroring the reduction of both i- and cRGCs in the Cyclin D2 mutant. These results strengthen the view that differential regulation of cell cycle dynamics at the progenitor level is associated with specific RGC fates and laterality of axonal projection.