Accuracy Of Depth Estimation Research Articles

Polarimetric 3D integral imaging profilometry under degraded environmental conditions

We propose polarimetric three-dimensional (3D) integral imaging profilometry and investigate its performance under degraded environmental conditions in terms of the accuracy of object depth acquisition. Integral imaging based profilometry provides depth information by capturing and utilizing multiple perspectives of the observed object. However, the performance of depth map generation may degrade due to light condition, partial occlusions, and object surface material. To improve the accuracy of depth estimation in these conditions, we propose to use polarimetric profilometry. Our experiments indicate that the proposed approach may result in more accurate depth estimation under degraded environmental conditions. We measure a number of metrics to evaluate the performance of the proposed polarimetric profilometry methods for generating the depth map under degraded conditions. Experimental results are presented to evaluate the robustness of the proposed method under degraded environment conditions and compare its performance with conventional integral imaging. To the best of our knowledge, this is the first report on polarimetric 3D integral imaging profilometry, and its performance under degraded environments.

Matched cross‐spectrum phase processing for source depth estimation in deep water

AbstractZhou et al. (2022) proposed a method for instantaneous source depth estimation in deep water using matched beam intensity processing (MBIP) that leverages depth‐related oscillatory features arising from Lloyd's mirror interference in the frequency domain (Appl. Acoust., 2022, 186, 108493). However, the efficacy of MBIP diminishes with a random source spectrum. To counteract the effect of a random source spectrum, a vertical line array (VLA) positioned near the sea bottom is employed and a cross‐spectrum is generated from the conventional beamforming output, which is achieved by dividing the VLA into upper‐half and lower‐half arrays. The authors find that the cross‐spectrum phase interference pattern is determined by the source depth and is unaffected by the source spectrum. Consequently, a matched cross‐spectrum phase processing method is proposed, which demonstrates appreciable depth estimation accuracy compared to that of the existing techniques, as evidenced by the results from both simulated and experimental data.

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos.

In minimally invasive surgery videos, label-free monocular laparoscopic depth estimation is challenging due to smoke. For this reason, we propose a self-supervised collaborative network-based depth estimation method with smoke-removal for monocular endoscopic video, which is decomposed into two steps of smoke-removal and depth estimation. In the first step, we develop a de-endoscopic smoke for cyclic GAN (DS-cGAN) to mitigate the smoke components at different concentrations. The designed generator network comprises sharpened guide encoding module (SGEM), residual dense bottleneck module (RDBM) and refined upsampling convolution module (RUCM), which restores more detailed organ edges and tissue structures. In the second step, high resolution residual U-Net (HRR-UNet) consisting of a DepthNet and two PoseNets is designed to improve the depth estimation accuracy, and adjacent frames are used for camera self-motion estimation. In particular, the proposed method requires neither manual labeling nor patient computed tomography scans during the training and inference phases. Experimental studies on the laparoscopic data set of the Hamlyn Centre show that our method can effectively achieve accurate depth information after net smoking in real surgical scenes while preserving the blood vessels, contours and textures of the surgical site. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art methods in effectiveness and achieves a frame rate of 94.45fps in real time, making it a promising clinical application.

Improving Accuracy and Efficiency of Monocular Depth Estimation in Power Grid Environments Using Point Cloud Optimization and Knowledge Distillation

Improving Accuracy and Efficiency of Monocular Depth Estimation in Power Grid Environments Using Point Cloud Optimization and Knowledge Distillation

Auditory perception based milling posture detection and depth control enhancement for orthopedic robots

The use of robots for bone milling is thought to improve surgical efficiency and reduce surgeon fatigue. Surgeons can use their hearing to perceive the milling state and adjust the position of surgical tools. Inspired by this, this paper proposes an automatic tool position control method based on acoustic signal feedback. Firstly, the mapping model between milling depth and sound energy was calibrated through experiments, and the influence of different angles on depth estimation accuracy was tested. Then, a network of CNN-GRU with an attention mechanism is designed to realize milling angle recognition using acoustic features as inputs. Finally, a milling depth automatic control framework with angle optimization was designed and validated through relevant experiments. The results of the experiment showed that the depth estimation error was larger when the milling angle of ball end milling tools (BMT) was within 70-90°. After adding the angle adjustment function, the milling depth accuracy in angle interval 1 (80-90°) and angle interval 2 (70°-80°) increased by about 30% and 10%, respectively.

Custom Anchorless Object Detection Model for 3D Synthetic Traffic Sign Board Dataset with Depth Estimation and Text Character Extraction

This paper introduces an anchorless deep learning model designed for efficient analysis and processing of large-scale 3D synthetic traffic sign board datasets. With an ever-increasing emphasis on autonomous driving systems and their reliance on precise environmental perception, the ability to accurately interpret traffic sign information is crucial. Our model seamlessly integrates object detection, depth estimation, deformable parts, and text character extraction functionalities, facilitating a comprehensive understanding of road signs in simulated environments that mimic the real world. The dataset used has a large number of artificially generated traffic signs for 183 different classes. The signs include place names in Japanese and English, expressway names in Japanese and English, distances and motorway numbers, and direction arrow marks with different lighting, occlusion, viewing angles, camera distortion, day and night cycles, and bad weather like rain, snow, and fog. This was done so that the model could be tested thoroughly in a wide range of difficult conditions. We developed a convolutional neural network with a modified lightweight hourglass backbone using depthwise spatial and pointwise convolutions, along with spatial and channel attention modules that produce resilient feature maps. We conducted experiments to benchmark our model against the baseline model, showing improved accuracy and efficiency in both depth estimation and text extraction tasks, crucial for real-time applications in autonomous navigation systems. With its model efficiency and partwise decoded predictions, along with Optical Character Recognition (OCR), our approach suggests its potential as a valuable tool for developers of Advanced Driver-Assistance Systems (ADAS), Autonomous Vehicle (AV) technologies, and transportation safety applications, ensuring reliable navigation solutions.

TSUDepth: Exploring temporal symmetry-based uncertainty for unsupervised monocular depth estimation

When faced with occlusions and non-rigid motions, machines often struggle with depth estimation, a task effortlessly performed by humans with just one eye. Continuous RGB images embody rich temporal features, such as symmetry and optical flow, which current deep-learning models fail to effectively leverage. In response to this limitation, we introduce an innovative framework known as Temporal Symmetry-based Uncertainty (TSU)-Depth, aimed at enhancing the accuracy of unsupervised monocular depth estimation. The Temporal Symmetry-based Occlusion Optimization (TSOO) component plays a pivotal role in robustly identifying occluded regions and comparable optimization across adjacent frames. Simultaneously, we propose Temporal Optical Flow Masking (TOFM) to effectively identify and exclude static pixels (such as out-of-range depths and non-rigid objects) between adjacent frames. Additionally, we introduce Cross-Resolution Distillation (CRED) to enhance depth estimation accuracy across various resolutions, especially in low input resolution scenarios. Furthermore, we designed a new depth estimation structure utilizing the DPT structure and incorporating a GRU module to enhance performance details. Through extensive experiments on benchmark datasets, including KITTI, Cityscapes, and Make3D, our TSUDepth framework has consistently demonstrated state-of-the-art performance. Code is available at https://github.com/BlueEg/TSUDepth/.

Monocular Depth Estimation via Self-Supervised Self-Distillation.

Self-supervised monocular depth estimation can exhibit excellent performance in static environments due to the multi-view consistency assumption during the training process. However, it is hard to maintain depth consistency in dynamic scenes when considering the occlusion problem caused by moving objects. For this reason, we propose a method of self-supervised self-distillation for monocular depth estimation (SS-MDE) in dynamic scenes, where a deep network with a multi-scale decoder and a lightweight pose network are designed to predict depth in a self-supervised manner via the disparity, motion information, and the association between two adjacent frames in the image sequence. Meanwhile, in order to improve the depth estimation accuracy of static areas, the pseudo-depth images generated by the LeReS network are used to provide the pseudo-supervision information, enhancing the effect of depth refinement in static areas. Furthermore, a forgetting factor is leveraged to alleviate the dependency on the pseudo-supervision. In addition, a teacher model is introduced to generate depth prior information, and a multi-view mask filter module is designed to implement feature extraction and noise filtering. This can enable the student model to better learn the deep structure of dynamic scenes, enhancing the generalization and robustness of the entire model in a self-distillation manner. Finally, on four public data datasets, the performance of the proposed SS-MDE method outperformed several state-of-the-art monocular depth estimation techniques, achieving an accuracy (δ1) of 89% while minimizing the error (AbsRel) by 0.102 in NYU-Depth V2 and achieving an accuracy (δ1) of 87% while minimizing the error (AbsRel) by 0.111 in KITTI.

SIM-MultiDepth: Self-Supervised Indoor Monocular Multi-Frame Depth Estimation Based on Texture-Aware Masking

Self-supervised monocular depth estimation methods have become the focus of research since ground truth data are not required. Current single-image-based works only leverage appearance-based features, thus achieving a limited performance. Deep learning based multiview stereo works facilitate the research on multi-frame depth estimation methods. Some multi-frame methods build cost volumes and take multiple frames as inputs at the time of test to fully utilize geometric cues between adjacent frames. Nevertheless, low-textured regions, which are dominant in indoor scenes, tend to cause unreliable depth hypotheses in the cost volume. Few self-supervised multi-frame methods have been used to conduct research on the issue of low-texture areas in indoor scenes. To handle this issue, we propose SIM-MultiDepth, a self-supervised indoor monocular multi-frame depth estimation framework. A self-supervised single-frame depth estimation network is introduced to learn the relative poses and supervise the multi-frame depth learning. A texture-aware depth consistency loss is designed considering the calculation of the patch-based photometric loss. Only the areas where multi-frame depth prediction is considered unreliable in low-texture regions are supervised by the single-frame network. This approach helps improve the depth estimation accuracy. The experimental results on the NYU Depth V2 dataset validate the effectiveness of SIM-MultiDepth. The zero-shot generalization studies on the 7-Scenes and Campus Indoor datasets aid in the analysis of the application characteristics of SIM-MultiDepth.

Multiple prior representation learning for self-supervised monocular depth estimation via hybrid transformer

Self-supervised monocular depth estimation aims to infer depth information without relying on labeled data. However, the lack of labeled information poses a significant challenge to the model’s representation, limiting its ability to capture the intricate details of the scene accurately. Prior information can potentially mitigate this issue, enhancing the model’s understanding of scene structure and texture. Nevertheless, solely relying on a single type of prior information often falls short when dealing with complex scenes, necessitating improvements in generalization performance. To address these challenges, we introduce a novel self-supervised monocular depth estimation model that leverages multiple priors to bolster representation capabilities across spatial, context, and semantic dimensions. Specifically, we employ a hybrid transformer and a lightweight pose network to obtain long-range spatial priors in the spatial dimension. Then, the context prior attention is designed to improve generalization, particularly in complex structures or untextured areas. In addition, semantic priors are introduced by leveraging semantic boundary loss, and semantic prior attention is supplemented, further refining the semantic features extracted by the decoder. Experiments on three diverse datasets demonstrate the effectiveness of the proposed model. It integrates multiple priors to comprehensively enhance the representation ability, improving the accuracy and reliability of depth estimation. Codes are available at: https://github.com/MVME-HBUT/MPRLNet.

MFPANet: Multi-Scale Feature Perception and Aggregation Network for High-Resolution Snow Depth Estimation

Accurate snow depth estimation is of significant importance, particularly for preventing avalanche disasters and predicting flood seasons. The predominant approaches for such snow depth estimation, based on deep learning methods, typically rely on passive microwave remote sensing data. However, due to the low resolution of passive microwave remote sensing data, it often results in low-accuracy outcomes, posing considerable limitations in application. To further improve the accuracy of snow depth estimation, in this paper, we used active microwave remote sensing data. We fused multi-spectral optical satellite images, synthetic aperture radar (SAR) images and land cover distribution images to generate a snow remote sensing dataset (SRSD). It is a first-of-its-kind dataset that includes active microwave remote sensing images in high-latitude regions of Asia. Using these novel data, we proposed a multi-scale feature perception and aggregation neural network (MFPANet) that focuses on improving feature extraction from multi-source images. Our systematic analysis reveals that the proposed approach is not only robust but also achieves high accuracy in snow depth estimation compared to existing state-of-the-art methods, with RMSE of 0.360 and with MAE of 0.128. Finally, we selected several representative areas in our study region and applied our method to map snow depth distribution, demonstrating its broad application prospects.

Optimizing Underwater Image Restoration and Depth Estimation with Light Field Images

Methods based on light field information have shown promising results in depth estimation and underwater image restoration. However, improvements are still needed in terms of depth estimation accuracy and image restoration quality. Previous work on underwater image restoration employed an image formation model (IFM) that overlooked the effects of light attenuation and scattering coefficients in underwater environments, leading to unavoidable color deviation and distortion in the restored images. Additionally, the high blurriness and associated distortions in underwater images make depth information extraction and estimation very challenging. In this paper, we refine the light propagation model and propose a method to estimate the attenuation and backscattering coefficients of the underwater IFM. We simplify these coefficients into distance-related functions and design a relationship between distance and the darkest channel to estimate the water coefficients, effectively suppressing color deviation and distortion in the restoration results. Furthermore, to increase the accuracy of depth estimation, we propose using blur cues to construct a cost for refocusing in the depth direction, reducing the impact of high signal-to-noise ratio environments on depth information extraction, and effectively enhancing the accuracy and robustness of depth estimation. Finally, experimental comparisons show that our method achieves more accurate depth estimation and image restoration closer to real scenes compared to state-of-the-art methods.

Efficiency-Accuracy Trade-Off in Light Field Estimation with Cost Volume Construction and Aggregation.

The Rich spatial and angular information in light field images enables accurate depth estimation, which is a crucial aspect of environmental perception. However, the abundance of light field information also leads to high computational costs and memory pressure. Typically, selectively pruning some light field information can significantly improve computational efficiency but at the expense of reduced depth estimation accuracy in the pruned model, especially in low-texture regions and occluded areas where angular diversity is reduced. In this study, we propose a lightweight disparity estimation model that balances speed and accuracy and enhances depth estimation accuracy in textureless regions. We combined cost matching methods based on absolute difference and correlation to construct cost volumes, improving both accuracy and robustness. Additionally, we developed a multi-scale disparity cost fusion architecture, employing 3D convolutions and a UNet-like structure to handle matching costs at different depth scales. This method effectively integrates information across scales, utilizing the UNet structure for efficient fusion and completion of cost volumes, thus yielding more precise depth maps. Extensive testing shows that our method achieves computational efficiency on par with the most efficient existing methods, yet with double the accuracy. Moreover, our approach achieves comparable accuracy to the current highest-accuracy methods but with an order of magnitude improvement in computational performance.

Flow rate influence on sediment depth estimation in sewers using temperature sensors.

Enhancing sediment accumulation monitoring techniques in sewers will enable a better understanding of the build-up processes to develop improved cleaning strategies. Thermal sensors provide a solution to sediment depth estimation by passively monitoring temperature fluctuations in the wastewater and sediment beds, which allows evaluation of the heat-transfer processes in sewer pipes. This study analyses the influence of the flow conditions on heat-transfer processes at the water-sediment interface during dry weather flow conditions. For this purpose, an experimental campaign was performed by establishing different flow, temperature patterns, and sediment depth conditions in an annular flume, which ensured steady flow and room-temperature conditions. Numerical simulations were also performed to assess the impact of flow conditions on the relationships between sediment depth and harmonic parameters derived from wastewater and sediment-bed temperature patterns. Results show that heat transfer between water and sediment occurred instantaneously for velocities greater than 0.1 m/s, and that sediment depth estimations using temperature-based systems were barely sensitive to velocities between 0.1 and 0.4 m/s. A depth estimation accuracy of ±7 mm was achieved. This confirms the ability of using temperature sensors to monitor sediment build-up in sewers under dry weather conditions, without the need for flow monitoring.

Deep Learning Based Monocular Depth Estimation for Object Distance Inference in 2D Images

Monocular depth estimation, a process of predicting depth from a single 2D image, has seen significant advancements due to the proliferation of deep learning techniques. This research focuses on leveraging deep learning for monocular depth estimation to infer object distances accurately in 2D images. We explore various convolutional neural network (CNN) architectures and transformer models to analyze their efficacy in predicting depth information. Our approach involves training these models on extensive datasets annotated with depth information, followed by rigorous evaluation using standard metrics. The results demonstrate substantial improvements in depth estimation accuracy, highlighting the potential of deep learning in enhancing computer vision tasks such as autonomous driving, augmented reality, and robotic navigation. This study not only underscores the importance of model architecture but also investigates the impact of training data diversity and augmentation strategies. The findings provide a comprehensive understanding of the current state-of-the-art in monocular depth estimation, paving the way for future innovations in object distance inference from 2D images. By providing a detailed analysis of various models and their performance, this research contributes to a better understanding of monocular depth estimation and its potential for real-world applications, paving the way for future advancements in object distance inference from 2D images.

End-to-end optimization of single-shot monocular metasurface camera for RGBD imaging

RGBD cameras that capture color and depth information have a wide range of applications such as robotics, autonomous driving, and cons umer electronics. Traditional RGBD imaging usually requires multi-cameras or extra active illumination which inevitably leads to bulky or complex imaging systems, hindering the growing demand for compact integrated optical devices. Optical metasurface have emerged as a powerful substitute to the traditional diffractive optical element due to the superior dispersion manipulation and extremely compact size. However, it is still a great challenge to realize a single-shot monocular metasurface camera, due to the strong and unignorable wavelength-dependent aberrations. In this work, we demonstrate an end-to-end compact single-shot monocular metasurface camera for RGBD imaging. By utilizing end-to-end joint optimization framework to learn metasurface physical structure in conjunction with deep neural networks reconstruction algorithm, the multidimensional light field information RGBD of a scene in a single shot can be reconstructed within a depth range of 0.5 m. Compared with traditional lens-based RGBD imaging, our proposed metasurface-based imaging achieves an improvement of about 2 dB in chromatic imaging and a 4 times increase in depth estimation accuracy. Our proposed scheme could facilitate the further development of the intelligent computational meta-optics in diverse fields ranging from machine vision to biomedical imaging.

Dense monocular depth estimation for stereoscopic vision based on pyramid transformer and multi-scale feature fusion

Stereoscopic display technology plays a significant role in industries, such as film, television and autonomous driving. The accuracy of depth estimation is crucial for achieving high-quality and realistic stereoscopic display effects. In addressing the inherent challenges of applying Transformers to depth estimation, the Stereoscopic Pyramid Transformer-Depth (SPT-Depth) is introduced. This method utilizes stepwise downsampling to acquire both shallow and deep semantic information, which are subsequently fused. The training process is divided into fine and coarse convergence stages, employing distinct training strategies and hyperparameters, resulting in a substantial reduction in both training and validation losses. In the training strategy, a shift and scale-invariant mean square error function is employed to compensate for the lack of translational invariance in the Transformers. Additionally, an edge-smoothing function is applied to reduce noise in the depth map, enhancing the model's robustness. The SPT-Depth achieves a global receptive field while effectively reducing time complexity. In comparison with the baseline method, with the New York University Depth V2 (NYU Depth V2) dataset, there is a 10% reduction in Absolute Relative Error (Abs Rel) and a 36% decrease in Root Mean Square Error (RMSE). When compared with the state-of-the-art methods, there is a 17% reduction in RMSE.

AMENet is a monocular depth estimation network designed for automatic stereoscopic display.

Monocular depth estimation has a wide range of applications in the field of autostereoscopic displays, while accuracy and robustness in complex scenes are still a challenge. In this paper, we propose a depth estimation network for autostereoscopic displays, which aims at improving the accuracy of monocular depth estimation by fusing Vision Transformer (ViT) and Convolutional Neural Network (CNN). Our approach feeds the input image as a sequence of visual features into the ViT module and utilizes its global perception capability to extract high-level semantic features of the image. The relationship between the losses is quantified by adding a weight correction module to improve robustness of the model. Experimental evaluation results on several public datasets show that AMENet exhibits higher accuracy and robustness than existing methods in different scenarios and complex conditions. In addition, a detailed experimental analysis was conducted to verify the effectiveness and stability of our method. The accuracy improvement on the KITTI dataset compared to the baseline method is 4.4%. In summary, AMENet is a promising depth estimation method with sufficient high robustness and accuracy for monocular depth estimation tasks.

Estimation of buried pipe depth in an artificial soil tank using ground-penetrating radar and moisture sensor

A method is required for accurate determination of the location (horizontal position and depth) of subsurface utilities with minimal digging. Ground-penetrating radars (GPRs) have been used at many sites as a nondestructive method for detecting underground structures. Previous studies have examined the accuracy and resolution of GPRs based on comparisons of exploration images and factors such as soil, pipe material, and environmental conditions. Although numerous studies have explored the influence of relative permittivity and water content on GPR surveys, only a limited number have conducted direct measurements of water content using a moisture sensor within a soil tank and applied this data to enhance the precision of depth estimation. In this study, the effects of pipe material, relative permittivity, and antenna frequency on the depth estimation accuracy were quantitatively investigated using an artificial soil tank made of silica sand with known physical properties. In addition to the GPR survey, relative permittivity measurements were conducted using time-domain reflectometry (TDR) sensors located at different water levels. The results show that the antenna frequency and pipe material affect the resolution but not the depth estimation accuracy. Furthermore, the GPR survey using the relative permittivity from the TDR sensors estimated the pipe depths with a relative error of <7.33% relative to the actual burial depth under the following conditions: a volumetric water content of 2.6–38.2%, the relative permittivity of 2.9–23.2, and buried pipes of 0.1 m in diameter and 1.5 m in depth. The study findings show that the selection of an appropriate frequency and the estimation of soil permittivity using a moisture sensor can improve the depth estimation accuracy of buried pipes by GPR survey.

OD-MVSNet: Omni-dimensional dynamic multi-view stereo network.

Multi-view stereo based on learning is a critical task in three-dimensional reconstruction, enabling the effective inference of depth maps and the reconstruction of fine-grained scene geometry. However, the results obtained by current popular 3D reconstruction methods are not precise, and achieving high-accuracy scene reconstruction remains challenging due to the pervasive impact of feature extraction and the poor correlation between cost and volume. In addressing these issues, we propose a cascade deep residual inference network to enhance the efficiency and accuracy of multi-view stereo depth estimation. This approach builds a cost-volume pyramid from coarse to fine, generating a lightweight, compact network to improve reconstruction results. Specifically, we introduce the omni-dimensional dynamic atrous spatial pyramid pooling (OSPP), a multiscale feature extraction module capable of generating dense feature maps with multiscale contextual information. The feature maps encoded by the OSPP module can generate dense point clouds without consuming significant memory. Furthermore, to alleviate the issue of feature mismatch in cost volume regularization, we propose a normalization-based 3D attention module. The 3D attention module aggregates crucial information within the cost volume across the dimensions of channel, spatial, and depth. Through extensive experiments on benchmark datasets, notably DTU, we found that the OD-MVSNet model outperforms the baseline model by approximately 1.4% in accuracy loss, 0.9% in completeness loss, and 1.2% in overall loss, demonstrating the effectiveness of our module.