Object Depth Estimation Research Articles

Transparent Depth Completion Using Segmentation Features

Estimating the depth of transparent objects is one of the well-known challenges of RGB-D cameras due to the reflection and refraction effects. Previously, researchers propose to correct the depth of transparent objects by using their estimated segmentation masks, because it is possible to recover the internal depth of an object just from its boundary, as illustrated by those depth-from-silhouette methods. However, these algorithms only use segmentation masks. They ignore the internal structure information from the mask segmentation features, which we argue are more useful for transparent depth estimation. In this work, we demonstrate the effectiveness of segmentation features for transparent object depth estimation. We show that it is even possible to recover the depth map just from segmentation features, without any RGB or depth map as input. Based on this observation, we propose DualTransNet which uses segmentation features for transparent depth completion. In our DualTransNet, we feed segmentation features from an extra module to the main network for better depth completion quality. Extensive experiments have shown the superiority of segmentation features as well as the state-of-the-art performance of our network.

An Experimental Study on Deep Learning Technique Implemented on Low Specification OpenMV Cam H7 Device

This research aims to identify and recognize the OpenMV Camera H7. In this research, all tests were carried out using Deep Machine Learning and applied to several functions, including Face Recognition, Facial Expression Recognition, Detection and Calculation of the Number of Objects, and Object Depth Estimation. Face Expression Recognition was used in the Convolutional Neural Network to recognize five facial expressions: angry, happy, neutral, sad, and surprised. This allowed the use of a primary dataset with a 48MP resolution camera. Some scenarios are prepared to meet environment variability in the implementation, such as indoor and outdoor environments, with different lighting and distance. Most pre-trained models in each identification or recognition used mobileNetV2 since this model allows low computation cost and matches with low hardware specifications. The object detection and counting module compared two methods: the conventional Haar Cascade and the Deep Learning MobileNetV2 model. The training and validation process is not recommended to be carried out on OpenMV devices but on computers with high specifications. This research was trained and validated using selected primary and secondary data, with 1500 image data. The computing time required is around 5 minutes for ten epochs. On average, recognition results on OpenMV devices take around 0.3 - 2 seconds for each frame. The accuracy of the recognition results varies depending on the pre-trained model and the dataset used, but overall, the accuracy levels achieved tend to be very high, exceeding 96.6%.

Effect of expansive optic flow and lateral motion parallax on depth estimation with normal and artificially reduced acuity.

When an observer moves in space, the retinal projection of a stationary object either expands if the motion is toward the object or shifts horizontally if the motion contains a lateral component. This study examined the impact of expansive optic flow and lateral motion parallax on the accuracy of depth perception for observers with normal or artificially reduced acuity and asked whether any benefit is due to the continuous motion or to the discrete object image displacement. Stationary participants viewed a virtual room on a computer screen. They used an on-screen slider to estimate the depth of a target object relative to a reference object after seeing 2-second videos simulating five conditions: static viewing, expansive optic flow, and lateral motion parallax in either continuous motion or image displacement. Ten participants viewed the stimuli with normal acuity in Experiment 1 and 11 with three levels of artificially reduced acuity in Experiment 2. Linear regression models represented the relationship between the depth estimates of participants and the ground truth. Lateral motion parallax produced more accurate depth estimates than expansive optic flow and static viewing. Depth perception with continuous motion was more accurate than that with displacement under mild and moderate, but not severe, acuity reduction. For observers with both normal and artificially reduced acuity, lateral motion parallax was more helpful for object depth estimation than expansive optic flow, and continuous motion parallax was more helpful than object image displacement.

An Experimental Study of GPR Near-Field Problem in Depth Ranging of Buried Objects

We evaluate the often neglected ranging (or cover depth) error of an object located within the Fresnel Region (or called radiating near-field) and reactive near-field region in ground penetrating radar (GPR). The experiments were conducted in concrete specimens with various rebar cover depths from 10 to 80 mm, which imitates any object imaged by GPR from reactive near-field, Fresnel region to far-field region. Velocities of each individual data point of the hyperbolic reflections of the embedded reinforcement in radargram are calculated by developed algorithms based on common offset antenna and multiple trilaterated ray paths. Two different velocity algorithms, based on semi- and full-trilaterated ray paths, were applied to estimate the wave travelling velocity as a prerequisite of cover depth measurement. The algorithms were firstly validated using a high-frequency 2-GHz antenna to verify the travelling speed of radar waves in air, which is equal to the speed of light. Then, the same antenna was used to estimate the cover depth by measuring the time of flight and velocity based on the two velocity algorithms. The results reveal that the ranging accuracy is highly in doubt in reactive near-field and Fresnel regions but is largely improved when the object locates in far-field region where GPR wave propagation becomes plane-type. The measurements, riding on accurate modelling of ray-path’s trilateration models, provide evidence that the normal linear scale of GPR time-to-depth conversion overlooks the effects of near-field/Fresnel region. We therefore suggest that the GPR near-field/Fresnel region and far-field boundary must be taken into account before any attempt of time-to-depth conversion and depth estimation of objects by GPR are carried out.

3D integral imaging depth estimation of partially occluded objects using mutual information and Bayesian optimization.

Integral imaging (InIm) is useful for passive ranging and 3D visualization of partially-occluded objects. We consider 3D object localization within a scene and in occlusions. 2D localization can be achieved using machine learning and non-machine learning-based techniques. These techniques aim to provide a 2D bounding box around each one of the objects of interest. A recent study uses InIm for the 3D reconstruction of the scene with occlusions and utilizes mutual information (MI) between the bounding box in this 3D reconstructed scene and the corresponding bounding box in the central elemental image to achieve passive depth estimation of partially occluded objects. Here, we improve upon this InIm method by using Bayesian optimization to minimize the number of required 3D scene reconstructions. We evaluate the performance of the proposed approach by analyzing different kernel functions, acquisition functions, and parameter estimation algorithms for Bayesian optimization-based inference for simultaneous depth estimation of objects and occlusion. In our optical experiments, mutual-information-based depth estimation with Bayesian optimization achieves depth estimation with a handful of 3D reconstructions. To the best of our knowledge, this is the first report to use Bayesian optimization for mutual information-based InIm depth estimation.

GPR uncertainty modelling and analysis of object depth based on constrained least squares

The development of ground penetrating radar (GPR) in recent decades has promoted the role of this popular near-surface geophysical detection method. A step further involves its transition from use as a prospecting tool aimed at estimating the approximate location of buried objects to its application as an accurate piece of survey equipment [1], which then requires an understanding of measurement errors/uncertainties. This paper firstly discusses the sources of uncertainty affecting an object’s depth measurement with respect to host material, instrumentation, data collection method and signal processing. These error sources were modelled by formulating computation sets based on the application of a constrained least squares algorithm to hyperbolic reflections produced by buried objects. Based on the computation, uncertainty analysis was performed (1) through identification of errors on measurements of hyperbolic reflections and (2) by conducting error propagation to evaluate the combined uncertainty of the surveyed depth. At a geophysical test site at IFSTTAR, Nantes, France, sets of controlled experiments were conducted to validate the proposed uncertainty analysis method and to investigate the correlation between the evaluated uncertainty and the factors of host material, antenna centre frequency, target depth and the horizontal and vertical resolution in the radargram. Based on the experiments, it was possible to draw two main conclusions. Firstly, a centimetre-order of uncertainty can be achieved in the survey results for depth estimation of objects at several metres deep with a 95% confidence level at ± 2 standard deviations. Secondly, errors of time zero location at different GPR centre frequencies dominate the evaluation of uncertainty, whereas the resolution of radargrams and scattering noise do not explicitly affect the evaluated uncertainty.

Object Occlusion Detection Using Automatic Camera Calibration for a Wide-Area Video Surveillance System

This paper presents an object occlusion detection algorithm using object depth information that is estimated by automatic camera calibration. The object occlusion problem is a major factor to degrade the performance of object tracking and recognition. To detect an object occlusion, the proposed algorithm consists of three steps: (i) automatic camera calibration using both moving objects and a background structure; (ii) object depth estimation; and (iii) detection of occluded regions. The proposed algorithm estimates the depth of the object without extra sensors but with a generic red, green and blue (RGB) camera. As a result, the proposed algorithm can be applied to improve the performance of object tracking and object recognition algorithms for video surveillance systems.

Virtual Focus and Depth Estimation From Defocused Video Sequences

In this paper, we present a novel method for virtual focus and object depth estimation from defocused video captured by a moving camera. We use the term virtual focus to refer to a new approach for producing in-focus image sequences by processing blurred videos captured by out-of-focus cameras. Our method relies on the concept of Depth-from-Defocus (DFD) for virtual focus estimation. However, the proposed approach overcomes limitations of DFD by reformulating the problem in a moving-camera scenario. We introduce the interframe image motion model, from which the relationship between the camera motion and blur characteristics can be formed. This relationship subsequently leads to a new method for blur estimation. We finally rely on the blur estimation to develop the proposed technique for object depth estimation and focused video reconstruction. The proposed approach can be utilized to correct out-of-focus video sequences and can potentially replace the expensive apparatus required for auto-focus adjustments currently employed in many camera devices. The performance of the proposed algorithm is demonstrated through error analysis and computer simulated experiments.