Depth Estimation Error Research Articles

Auditory perception based milling posture detection and depth control enhancement for orthopedic robots

The use of robots for bone milling is thought to improve surgical efficiency and reduce surgeon fatigue. Surgeons can use their hearing to perceive the milling state and adjust the position of surgical tools. Inspired by this, this paper proposes an automatic tool position control method based on acoustic signal feedback. Firstly, the mapping model between milling depth and sound energy was calibrated through experiments, and the influence of different angles on depth estimation accuracy was tested. Then, a network of CNN-GRU with an attention mechanism is designed to realize milling angle recognition using acoustic features as inputs. Finally, a milling depth automatic control framework with angle optimization was designed and validated through relevant experiments. The results of the experiment showed that the depth estimation error was larger when the milling angle of ball end milling tools (BMT) was within 70-90°. After adding the angle adjustment function, the milling depth accuracy in angle interval 1 (80-90°) and angle interval 2 (70°-80°) increased by about 30% and 10%, respectively.

Edge-Enhanced Dual-Stream Perception Network for Monocular Depth Estimation

Estimating depth from a single RGB image has a wide range of applications, such as in robot navigation and autonomous driving. Currently, Convolutional Neural Networks based on encoder–decoder architecture are the most popular methods to estimate depth maps. However, convolutional operators have limitations in modeling large-scale dependence, often leading to inaccurate depth predictions at object edges. To address these issues, a new edge-enhanced dual-stream monocular depth estimation method is introduced in this paper. ResNet and Swin Transformer are combined to better extract global and local features, which benefits the estimation of the depth map. To better integrate the information from the two branches of the encoder and the shallow branch of the decoder, we designed a lightweight decoder based on the multi-head Cross-Attention Module. Furthermore, in order to improve the boundary clarity of objects in the depth map, a loss function with an additional penalty for depth estimation error on the edges of objects is presented. The results on three datasets, NYU Depth V2, KITTI, and SUN RGB-D, show that the method presented in this paper achieves better performance for monocular depth estimation. Additionally, it has good generalization capabilities for various scenarios and real-world images.

Analysis of error propagation: from raw light-field data to depth estimation.

In micro-lens-array-based light-field imaging, the micro-lens centers serve as the origins of local micro-lens coordinate systems. Each micro-lens receives angular/depth information coded according to its center location. Therefore, the errors in positioning the micro-lens centers will lead to errors in depth estimation. This paper proposes a method that resolves error propagation from raw light-field data to depth estimation based on analyzing large amounts of simulated images with various aperture sizes, noise levels, and object distance values. The simulation employs backward ray tracing and Monte Carlo sampling to improve computational efficiency. The errors are counted and accumulated stepwise from center positioning and generation of sub-aperture images to depth estimation. The disparity errors calculated during depth estimation are shown to be more apparent either with more significant center positioning errors or with a greater defocusing distance. An experiment using an industrial light-field camera is conducted, confirming that disparity errors at considerable object distances can be reduced significantly when the micro-lens centers are positioned with higher accuracy.

Sources depth estimation for a tonal source by matching the interference structure in the arrival angle domain.

A publication by McCargar and Zurk [J. Acoust. Soc. Am. 133(4), EL320-EL325 (2013)] introduced a passive source depth estimation method for a moving tonal source with a vertical line array (VLA), utilizing the depth-dependent modulation in the arrival angle domain caused by the interference between the direct and surface-reflected acoustic arrivals. Under the isovelocity approximation, this method can estimate the depth of sources at close ranges, but the depth estimation error will increase with the increase in source range, as the impact of the sound speed profile on sound propagation is ignored. This paper presents a theoretical formula for calculating the modeled interference structure in the arrival angle domain with the knowledge of the sound speed profile. By matching the measured interference structure obtained from the beamforming of the acoustic data received by the VLA with the modeled structure under different assumed source depths, the tonal source depth estimation is achieved, even for sources at the remote part of the direct arrival zone. The performance of this method is verified by simulation data, as well as experimental data radiated from a towed source and a non-cooperative passing ship.

A Monocular SLAM System Based on ResNet Depth Estimation

Currently, monocular SLAM (simultaneous location and mapping) systems cannot extract depth information from monocular cameras directly and require initialization to solve the problem of scale uncertainty. It is extremely difficult to reconstruct maps using such systems, and it is difficult to cope with scenarios that require navigation and obstacle avoidance. In order to solve the above problems, in this paper, a simple monocular depth estimation network framework is proposed. Transfer learning from a pre-trained ResNet is utilized for the encoding part of the framework and a CNN is used as the decoding part. Only a few training parameters and iterations are required to obtain fairly accurate depth information. At the same time, a similarity-based filter is used to denoise the surfels and improve the RGB-D SLAM system, which not only reduces the impact of the depth estimation error on the surfels, but also ensures the quality of the dense mapping. From the results of comparative experiments, it can be seen that the proposed monocular depth estimation network framework is better than current popular methods, and the associated SLAM system can achieve pose estimation and dense mapping tasks. As a monocular camera-based SLAM system, the proposed method is a promising and practical approach.

Multi-View Stereo Network With Gaussian Distribution Iteration

Multi-view stereo estimates the depth maps of multiple perspective images in a scene and then fuses them to generate a 3D point cloud of the scene, which is an essential technology of 3D reconstruction. In this paper, we propose a deep learning method GDINet, applying probabilistic methods to the pyramid framework, which can significantly improve reconstruction quality. In detail, we first establish a Gaussian distribution for each image’s pixel and iterate it in the pyramid framework. The mean value is the estimated depth, and the variance represents the depth estimation error. In addition, we design a novel loss function with excellent convergence to train our network. Finally, we present an initialization module to generate the coarse Gaussian distribution, controlling the parameters in a reasonable range. Our results rank <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2nd$ </tex-math></inline-formula> on both DTU and Tanks & Temples datasets, showing that our network has high accuracy, completeness, and robustness. We also make a visualization comparison on the BlendedMVS dataset (containing many aerial scene images) to demonstrate the generalization ability of our model.

Automated vehicle data pipeline for accident reconstruction: New insights from LiDAR, camera, and radar data

As automated vehicles are deployed across the world, it has become critically important to understand how these vehicles interact with each other, as well as with other conventional vehicles on the road. One such method to achieve a deeper understanding of the safety implications for Automated Vehicles (AVs) is to analyze instances where AVs were involved in crashes. Unfortunately, this poses a steep challenge to crash-scene investigators. It is virtually impossible to fully understand the factors that contributed to an AV involved crash without taking into account the vehicle’s perception and decision making. Furthermore, there is a tremendous amount of data that could provide insight into these crashes that is currently unused, as it also requires a deep understanding of the sensors and data management of the vehicle. To alleviate these problems, we propose a data pipeline that takes raw data from all on-board AV sensors such as LiDAR, radar, cameras, IMU’s, and GPS’s. We process this data into visual results that can be analyzed by crash scene investigators with no underlying knowledge of the vehicle’s perception system. To demonstrate the utility of this pipeline, we first analyze the latest information on AV crashes that have occurred in California and then select two crash scenarios that are analyzed in-depth using high-fidelity synthetic data generated from the automated vehicle simulator CARLA. The data visualization procedure is demonstrated on the real-world Kitti dataset by using the YOLO object detector and a monocular depth estimator called AdaBins. Depth from LIDAR is used as ground truth to calibrate and assess the effect of noise and errors in depth estimation. The visualization and data analysis from these scenarios clearly demonstrate the vast improvement in crash investigations that can be obtained from utilizing state-of-the-art sensing and perception systems used on AVs.

Building-scale flood loss estimation through vulnerability pattern characterization: application to an urban flood in Milan, Italy

Abstract. The vulnerability of flood-prone areas is determined by the susceptibility of the exposed assets to the hazard. It is a crucial component in risk assessment studies, both for climate change adaptation and disaster risk reduction. In this study, we analyse patterns of vulnerability for the residential sector in a frequently hit urban area of Milan, Italy. The conceptual foundation for a quantitative assessment of the structural dimensions of vulnerability is based on the modified source–pathway–receptor–consequence model. This conceptual model is used to improve the parameterization of the flood risk analysis, describing (i) hazard scenario definitions performed by hydraulic modelling based on past event data (source estimation) and morphological features and land-use evaluation (pathway estimation) and (ii) the exposure and vulnerability assessment which consists of recognizing elements potentially at risk (receptor estimation) and event losses (consequence estimation). We characterized flood hazard intensity on the basis of variability in water depth during a recent event and spatial exposure also as a function of a building's surroundings and buildings' intrinsic characteristics as a determinant vulnerability indicator of the elements at risk. In this sense the use of a geographic scale sufficient to depict spatial differences in vulnerability allowed us to identify structural vulnerability patterns to inform depth–damage curves and calculate potential losses from mesoscale (land-use level) to microscale (building level). Results produces accurate estimates of the flood characteristics, with mean error in flood depth estimation in the range 0.2–0.3 m and provide a basis to obtain site-specific damage curves and damage mapping. Findings show that the nature of flood pathways varies spatially, is influenced by landscape characteristics and alters vulnerability spatial distribution and hazard propagation. At the mesoscale, the “continuous urban fabric” Urban Atlas 2018 land-use class with the occurrence of at least 80 % of soil sealing shows higher absolute damage values. At microscale, evidence demonstrated that even events with moderate magnitude in terms of flood depth in a complex urbanized area may cause more damage than one would expect.

Structured Light Field by Two Projectors Placed in Parallel for High-Speed and Precise 3D Feedback

In recent years, it is required to acquire three-dimensional information at high speed in various fields. Previously, a structured light field (SLF) method for high-speed three dimensional measurement in 1 ms was proposed by our group. However, the SLF method has a drawback of worse depth estimation error by several tens millimeters. In this paper, a novel method to generate SLF with two projectors placed in parallel is proposed. This arrangement could produce bigger pattern change depending on the depth and made more precise estimation possible. The depth estimation experiments for precision evaluation and dynamic projection mapping experiment successfully demonstrated precise depth estimation with the error of several millimeters and high-speed estimation within 1 ms, though the measurement range was limited to approximately 100 mm.

Recognition and Depth Estimation of Ships Based on Binocular Stereo Vision

To improve the navigation safety of inland river ships and enrich the methods of environmental perception, this paper studies the recognition and depth estimation of inland river ships based on binocular stereo vision (BSV). In the stage of ship recognition, considering the computational pressure brought by the huge network parameters of the classic YOLOv4 model, the MobileNetV1 network was proposed as the feature extraction module of the YOLOv4 model. The results indicate that the mAP value of the MobileNetV1-YOLOv4 model reaches 89.25%, the weight size of the backbone network was only 47.6 M, which greatly reduced the amount of computation while ensuring the recognition accuracy. In the stage of depth estimation, this paper proposes a feature point detection and matching algorithm based on the ORB algorithm at sub-pixel level, that is, firstly, the FSRCNN algorithm was used to perform super-resolution reconstruction of the original image, to further increase the density of image feature points and detection accuracy, which was more conducive to the calculation of the image parallax value. The ships’ depth estimation results indicate that when the distance to the target is about 300 m, the depth estimation error is less than 3%, which meets the depth estimation needs of inland ships. The ship target recognition and depth estimation technology based on BSV proposed in this paper makes up for the shortcomings of the existing environmental perception methods, improves the navigation safety of ships to a certain extent, and greatly promotes the development of intelligent ships in the future.

Egocentric scene reconstruction from an omnidirectional video

Omnidirectional videos capture environmental scenes effectively, but they have rarely been used for geometry reconstruction. In this work, we propose an egocentric 3D reconstruction method that can acquire scene geometry with high accuracy from a short egocentric omnidirectional video. To this end, we first estimate per-frame depth using a spherical disparity network. We then fuse per-frame depth estimates into a novel spherical binoctree data structure that is specifically designed to tolerate spherical depth estimation errors. By subdividing the spherical space into binary tree and octree nodes that represent spherical frustums adaptively, the spherical binoctree effectively enables egocentric surface geometry reconstruction for environmental scenes while simultaneously assigning high-resolution nodes for closely observed surfaces. This allows to reconstruct an entire scene from a short video captured with a small camera trajectory. Experimental results validate the effectiveness and accuracy of our approach for reconstructing the 3D geometry of environmental scenes from short egocentric omnidirectional video inputs. We further demonstrate various applications using a conventional omnidirectional camera, including novel-view synthesis, object insertion, and relighting of scenes using reconstructed 3D models with texture.

Monocular 3D object detection with Pseudo-LiDAR confidence sampling and hierarchical geometric feature extraction in 6G network

The high bandwidth and low latency of 6G network technology enable the successful application of monocular 3D object detection on vehicle platforms. Monocular 3D-object-detection-based Pseudo-LiDAR is a low-cost, low-power solution compared to LiDAR solutions in the field of autonomous driving. However, this technique has some problems, i.e., (1) the poor quality of generated Pseudo-LiDAR point clouds resulting from the nonlinear error distribution of monocular depth estimation and (2) the weak representation capability of point cloud features due to the neglected global geometric structure features of point clouds existing in LiDAR-based 3D detection networks. Therefore, we proposed a Pseudo-LiDAR confidence sampling strategy and a hierarchical geometric feature extraction module for monocular 3D object detection. We first designed a point cloud confidence sampling strategy based on a 3D Gaussian distribution to assign small confidence to the points with great error in depth estimation and filter them out according to the confidence. Then, we present a hierarchical geometric feature extraction module by aggregating the local neighborhood features and a dual transformer to capture the global geometric features in the point cloud. Finally, our detection framework is based on Point-Voxel-RCNN (PV-RCNN) with high-quality Pseudo-LiDAR and enriched geometric features as input. From the experimental results, our method achieves satisfactory results in monocular 3D object detection.

Feasibility of Estimating Parameters and Enhancing Reflections of Dipping Moho From Teleseismic P Wave Coda Autocorrelation

AbstractTeleseismic P wave coda autocorrelation can retrieve reflections from subsurface structural boundaries, which can be used in lithospheric imaging. However, dipping structures cause difficulty in extraction of reflections and errors in depth estimation. Based on a previously proposed velocity analysis method, we formulated a method for estimating the dipping parameters and correcting the effects of dipping structures. First, we derived an arrival time formula for the PP reflection of the dipping Moho, which includes the effects of the ray parameters, dipping structures, and azimuthal distribution of available events. Second, the formula was used for move out correction and velocity analysis. Finally, by scanning the peak of the Moho energy group in the velocity spectrum, which varies with the azimuth and dip angles, a method was proposed to estimate the dipping parameters of the Moho. We referred to the scanning map as a dipping spectrum. A field data test at station NE44 revealed the prospects of the practical applications of this method. The dip angle of the Moho below the station was 3°, and exhibited an azimuth angle of 25°NE. The stacked waveform of the Moho reflection was significantly enhanced. This method was successfully applied to more stations adjacent to station NE44. Our results provide new insight on the Moho in this area, which are important for imaging and interpretation in areas that contain a dipping Moho such as plate subduction zones.

Effects of stormwater infrastructure data completeness and model resolution on urban flood modeling

The accuracy of hydrologic and hydrodynamic models, used to study urban hydrology and predict urban flooding, depends on the availability of high-resolution terrain and infrastructure data. Unfortunately, cities often do not have or cannot release complete infrastructure data, and high-resolution terrain data products are not available everywhere. In this study, we quantify how the accuracy and precision of urban hydrologic-hydrodynamic models vary as a function of data completeness and model resolution. For this aim, we apply the one-dimensional (1D) and coupled one- and two-dimensional (1D-2D) versions of the U.S. Environmental Protection Agency’s Storm Water Management Model (SWMM) in an urban catchment in the city of Phoenix, Arizona. Here, we have collected detailed infrastructure data, a high-resolution 0.3-m LiDAR-based digital elevation model, and catchment properties data. We tested several model configurations assuming different levels of (i) availability of stormwater infrastructure data (ranging from 5% to 75% of attribute-values missing) and (ii) terrain aggregation (i.e., 4.6 m and 9.7 m). These configurations were generated through random Monte Carlo sampling for SWMM 1D and selective sampling with four cases for SWMM 1D-2D. We ran simulations under the 50-year return period design storm and compared simulated flood metrics assuming the highest-resolution and complete data model configuration as a reference. The study found that the model may over or underestimate flood volume and duration with different levels of missing data depending on the parameters — roughness, diameter or depth, and that model performance is more sensitive to missing data that is downstream and closer to the outfall as opposed to missing data upstream. Errors in flood depth, area and volume estimation are functions of both the data completeness and model resolution. Missing feature data leads to overestimation of flood depth, while lower model resolution results in underestimating flood depth and overestimating flood extent and volume.

Processing and Interpretation of UAV Magnetic Data: A Workflow Based on Improved Variational Mode Decomposition and Levenberg–Marquardt Algorithm

Unmanned aerial vehicles (UAVs) have become a research hotspot in the field of magnetic exploration because of their unique advantages, e.g., low cost, high safety, and easy to operate. However, the lack of effective data processing and interpretation method limits their further deployment. In view of this situation, a complete workflow of UAV magnetic data processing and interpretation is proposed in this paper, which can be divided into two steps: (1) the improved variational mode decomposition (VMD) is applied to the original data to improve its signal-to-noise ratio as much as possible, and the decomposition modes number K is determined adaptively according to the mode characteristics; (2) the parameters of target position and magnetic moment are obtained by Euler deconvolution first, and then used as the prior information of the Levenberg–Marquardt (LM) algorithm to further improve its accuracy. Experiments are carried out to verify the effectiveness of the proposed method. Results show that the proposed method can significantly improve the quality of the original data; by combining the Euler deconvolution and LM algorithm, the horizontal positioning error can be reduced from 15.31 cm to 4.05 cm, and the depth estimation error can be reduced from 16.2 cm to 5.4 cm. Moreover, the proposed method can be used not only for the detection and location of near-surface targets, but also for the follow-up work, such as the clearance of targets (e.g., the unexploded ordnance).

Event-based tracking of human hands

Purpose This paper proposes a novel method for human hands tracking using data from an event camera. The event camera detects changes in brightness, measuring motion, with low latency, no motion blur, low power consumption and high dynamic range. Captured frames are analysed using lightweight algorithms reporting three-dimensional (3D) hand position data. The chosen pick-and-place scenario serves as an example input for collaborative human–robot interactions and in obstacle avoidance for human–robot safety applications. Design/methodology/approach Events data are pre-processed into intensity frames. The regions of interest (ROI) are defined through object edge event activity, reducing noise. ROI features are extracted for use in-depth perception. Findings Event-based tracking of human hand demonstrated feasible, in real time and at a low computational cost. The proposed ROI-finding method reduces noise from intensity images, achieving up to 89% of data reduction in relation to the original, while preserving the features. The depth estimation error in relation to ground truth (measured with wearables), measured using dynamic time warping and using a single event camera, is from 15 to 30 millimetres, depending on the plane it is measured. Originality/value Tracking of human hands in 3 D space using a single event camera data and lightweight algorithms to define ROI features (hands tracking in space).

Accurate depth determination of defects in composite materials using pulsed thermography

The determination of defect depth is one of the important advantages of pulsed thermography. The existing methods for depth estimation in pulsed thermography are generally established based on the one-dimension heat conduction along the thickness of the sample. However, most of the methods neglect the effects of defect size and sample thickness, which induces additional systematic errors in depth estimation. In this study, simulations are conducted to demonstrate that the accuracy of depth determination of defects is greatly influenced by the defect size and sample thickness in some simulations. By establishing the relationship between the characteristic time and these factors, an accurate depth determination method based on the peak slope time is proposed. The proposed method has been applied to characterize the defect depths of a composite panel with flat-bottomed holes. The experimental results show that the use of the proposed method is able to identify the depths of defects with high accuracy.

Adaptive LiDAR Sampling and Depth Completion Using Ensemble Variance.

This work considers the problem of depth completion, with or without image data, where an algorithm may measure the depth of a prescribed limited number of pixels. The algorithmic challenge is to choose pixel positions strategically and dynamically to maximally reduce overall depth estimation error. This setting is realized in daytime or nighttime depth completion for autonomous vehicles with a programmable LiDAR. Our method uses an ensemble of predictors to define a sampling probability over pixels. This probability is proportional to the variance of the predictions of ensemble members, thus highlighting pixels that are difficult to predict. By additionally proceeding in several prediction phases, we effectively reduce redundant sampling of similar pixels. Our ensemble-based method may be implemented using any depth-completion learning algorithm, such as a state-of-the-art neural network, treated as a black box. In particular, we also present a simple and effective Random Forest-based algorithm, and similarly use its internal ensemble in our design. We conduct experiments on the KITTI dataset, using the neural network algorithm of Ma et al. and our Random Forest-based learner for implementing our method. The accuracy of both implementations exceeds the state of the art. Compared with a random or grid sampling pattern, our method allows a reduction by a factor of 4-10 in the number of measurements required to attain the same accuracy.

Rain Rendering for Evaluating and Improving Robustness to Bad Weather

Rain fills the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, quantifying its impact is difficult. In this context, we present a rain rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain. We present three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. We validate our rendering methods with a user study, demonstrating our rain is judged as much as 73% more realistic than the state-of-theart. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection, 60% for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of 21% on object detection, 37% on semantic segmentation, and 8% on depth estimation.

Detection and localization of rebar in concrete by deep learning using ground penetrating radar

Ground penetrating radar (GPR) has been widely used for detection and localization of reinforced steel bar (rebar) in concrete in a non-destructive way. However, manual interpretation of a large number of GPR images is time-consuming, and the results highly depend on practitioner experience and the available priori information. This paper proposes an automatic detection and localization method using deep learning and migration. Firstly, a Single Shot Multibox Detector (SSD) model is established to identify regions of interest containing hyperbolas in a GPR image. This deep learning model is trained using a real GPR dataset, which contains 13,026 rebar targets in 3992 images, collected on residential buildings under construction. Secondly, each target region is migrated and transformed into a binary image to locate the rebar. After the binarization, the apex of the focused cluster is obtained and used to estimate both the horizontal position and the depth of the rebar. The testing results show that the detection accuracy of the proposed artificial intelligence method is 90.9%. The computation time needed for processing a GPR image with a size of 300 × 300 pixels is only 0.47 s. The depth estimation error in a laboratory experiment is <1.5 mm (5%), and the lateral position error is <0.7 cm. Therefore, it is concluded that the proposed method can automatically detect the rebar from GPR images in real time when a handheld GPR system is operated at a walking speed and the depth estimation accuracy is acceptable in practice.