Depth Interpolation Research Articles

CR-LDSO: Direct Sparse LiDAR-Assisted Visual Odometry With Cloud Reusing

LiDAR-assisted visual odometry (VO) is a widely-used solution for pose estimation and mapping. However, most existing LiDAR-assisted VO systems could suffer from the problems of 1) lacking distinctive and evenly distributed pixels for tracking due to the sparsity of LiDAR points and limited FOV overlap between a camera and LiDAR, and 2) nontrivial errors when processing LiDAR point clouds. To address above problems, we present CR-LDSO, a direct sparse LiDAR-assisted VO with the core parts being: 1) a novel cloud reusing method with point extraction/re-extraction to increase both the camera-LiDAR FOV overlap and the number of high-quality tracking pixels and 2) an occlusion removal method to exclude mismatching pixels due to occluded 3D object from sliding-window optimization and a point extraction strategy without depth interpolation. Extensive experimental results on public datasets demonstrates the superiority of our method to the existing state-of-the-art methods.

Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities From Visual SLAM

Feature-based visual simultaneous localization and mapping (SLAM) methods only estimate the depth of extracted features, generating a sparse depth map. To solve this sparsity problem, depth completion tasks that estimate a dense depth from a sparse depth have gained significant importance in robotic applications like exploration. Existing methodologies that use sparse depth from visual SLAM mainly employ point features. However, point features have limitations in preserving structural regularities owing to texture-less environments and sparsity problems. To deal with these issues, we perform depth completion with visual SLAM using line features, which can better contain structural regularities than point features. The proposed methodology creates a convex hull region by performing constrained Delaunay triangulation with depth interpolation using line features. However, the generated depth includes low-frequency information and is discontinuous at the convex hull boundary. Therefore, we propose a mesh depth refinement (MDR) module to address this problem. The MDR module effectively transfers the high-frequency details of an input image to the interpolated depth and plays a vital role in bridging the conventional and deep learning-based approaches. The Struct-MDC outperforms other state-of-the-art algorithms on public and our custom datasets, and even outperforms supervised methodologies for some metrics. In addition, the effectiveness of the proposed MDR module is verified by a rigorous ablation study.

Comparative analysis of groundwater conditions on rain-fed and irrigated agriculture in the upper Blue Nile basin, Ethiopia

Study regionUpper Blue Nile Basin, Ethiopia. Study focusThis study was conducted at the Koga Irrigation Scheme in the upper Blue Nile basin (Ethiopia) to investigate the influence of irrigation activity on the groundwater system. Representative samples of six shallow wells in the irrigated fields and four shallow wells in the adjacent non-irrigated (rain-fed agricultural lands) were selected. Dipmeter and hand GPS (GARMIN 60) were used for the monitoring of water table depth and fixing the location of the wells respectively. Arc GIS 10.5 has been applied for interpolation of water table depth (WTD). Water samples were collected and analyzed in the laboratory following international standards. New hydrological insights for the regionThe result shows that irrigation contributed to the water table rises of wells located in the irrigated fields which range from 0.77 m to 0.52 m and water table drops were observed in wells located in the non-irrigated fields. Investigations on the groundwater quality (GWQ) indicated that no significant effect was observed in the irrigated and non-irrigated fields. The study shows that good irrigation water management, continuous follow-up of the groundwater table and its physico-chemical properties are essential for the sustainability of the Koga Irrigation Scheme.

Learning Steering Kernels for Guided Depth Completion.

This paper addresses the guided depth completion task in which the goal is to predict a dense depth map given a guidance RGB image and sparse depth measurements. Recent advances on this problem nurture hopes that one day we can acquire accurate and dense depth at a very low cost. A major challenge of guided depth completion is to effectively make use of extremely sparse measurements, e.g., measurements covering less than 1% of the image pixels. In this paper, we propose a fully differentiable model that avoids convolving on sparse tensors by jointly learning depth interpolation and refinement. More specifically, we propose a differentiable kernel regression layer that interpolates the sparse depth measurements via learned kernels. We further refine the interpolated depth map using a residual depth refinement layer which leads to improved performance compared to learning absolute depth prediction using a vanilla network. We provide experimental evidence that our differentiable kernel regression layer not only enables end-to-end training from very sparse measurements using standard convolutional network architectures, but also leads to better depth interpolation results compared to existing heuristically motivated methods. We demonstrate that our method outperforms many state-of-the-art guided depth completion techniques on both NYUv2 and KITTI. We further show the generalization ability of our method with respect to the density and spatial statistics of the sparse depth measurements.

Variational Densification and Refinement of Registration Maps

Local patch-based algorithms for image registration fail to accurately match points in areas not discriminative enough, mainly textureless regions. These methods normally involve a validation process and provide a non-completely dense solution. In this paper, we propose a novel refinement and completion approach for registration. The proposed model combines single image nonlocal densification with classical variational image registration. We associate a total variation regularization with a nonlocal term to provide a smooth solution leveraging the image geometry. We show experiments on public stereo and optical flow datasets to filter and densify incomplete depth maps and motion fields. Extensive comparisons against existing and state-of-the-art depth/motion fields densification approaches demonstrate the competitive performance of the introduced method. Additionally, we illustrate how our method can deal with other tasks, such as filtering and interpolation of depth maps from RGBD data and depth upsampling.

Peruvian Subduction Surface Model for Seismic Hazard Assessments

Throughout the years seismic hazard calculations in Peru have been developed using area sources models, having to date a great variety of models, however, since they are discretized planar models, they cannot adequately represent the continuity and subduction characteristics of the Nazca Plate. The main objective of this work is the developing of a surface subduction model (SSM), useful for seismic hazard assessments as well as the revision and control of previous models used in this sort of assessments. In this study a spatial interpolation was performed employing the Local Polynomial Interpolation method to capture short-range variation in addition to long-range trends. The data base is based on the compilation of seismic catalogs from Peruvian and international institutions such as the IGP, the USGS, the ISC and others, subsequently, in order to have independent events the elimination of duplicate events, aftershocks and foreshocks was carried out. Then, by interpolation of the focal depths of the independent events, a subduction surface model (SSM) was generated as well as a Standard Error Surface which supports a good correlation of the model. Furthermore, 14 transversal sections of the SSM was employed to compare with the hypocenter’s distributions, evidencing a good correlation with the spatial distribution of the events, in addition to adequately capturing the subduction characteristics of the Nazca Plate. Finally, a comparison was made between 2 Peruvian area models for seismic hazard and SSM developed in the present research, evidencing that seismic source models of the area type have deficiencies mainly in the depths they consider, thus is recommended the use of the present model for future seismic hazard assessments.

Optimal combination of illusory and luminance-defined 3-D surfaces: A role for ambiguity.

The shape of the illusory surface in stereoscopic Kanizsa figures is determined by the interpolation of depth from the luminance edges of adjacent inducing elements. Despite ambiguity in the position of illusory boundaries, observers reliably perceive a coherent three-dimensional (3-D) surface. However, this ambiguity may contribute additional uncertainty to the depth percept beyond what is expected from measurement noise alone. We evaluated the intrinsic ambiguity of illusory boundaries by using a cue-combination paradigm to measure the reliability of depth percepts elicited by stereoscopic illusory surfaces. We assessed the accuracy and precision of depth percepts using 3-D Kanizsa figures relative to luminance-defined surfaces. The location of the surface peak was defined by illusory boundaries, luminance-defined edges, or both. Accuracy and precision were assessed using a depth-discrimination paradigm. A maximum likelihood linear cue combination model was used to evaluate the relative contribution of illusory and luminance-defined signals to the perceived depth of the combined surface. Our analysis showed that the standard deviation of depth estimates was consistent with an optimal cue combination model, but the points of subjective equality indicated that observers consistently underweighted the contribution of illusory boundaries. This systematic underweighting may reflect a combination rule that attributes additional intrinsic ambiguity to the location of the illusory boundary. Although previous studies show that illusory and luminance-defined contours share many perceptual similarities, our model suggests that ambiguity plays a larger role in the perceptual representation of illusory contours than of luminance-defined contours.

Temporal Upsampling of Depth Maps Using a Hybrid Camera.

In recent years, consumer-level depth cameras have been adopted for various applications. However, they often produce depth maps at only a moderately high frame rate (approximately 30 frames per second), preventing them from being used for applications such as digitizing human performance involving fast motion. On the other hand, low-cost, high-frame-rate video cameras are available. This motivates us to develop a hybrid camera that consists of a high-frame-rate video camera and a low-frame-rate depth camera and to allow temporal interpolation of depth maps with the help of auxiliary color images. To achieve this, we develop a novel algorithm that reconstructs intermediate depth maps and estimates scene flow simultaneously. We test our algorithm on various examples involving fast, non-rigid motions of single or multiple objects. Our experiments show that our scene flow estimation method is more precise than a tracking-based method and the state-of-the-art techniques.

Spatial distribution of snow depth based on geographically weighted regression kriging in the Bayanbulak Basin of the Tianshan Mountains, China

Snow depth is a general input variable in many models of agriculture, hydrology, climate and ecology. This study makes use of observational data of snow depth and explanatory variables to compare the accuracy and effect of geographically weighted regression kriging (GWRK) and regression kriging (RK) in a spatial interpolation of regional snow depth. The auxiliary variables are analyzed using correlation coefficients and the variance inflation factor (VIF). Three variables, Height, topographic ruggedness index (TRI), and land surface temperature (LST) , are used as explanatory variables to establish a regression model for snow depth. The estimated spatial distribution of snow depth in the Bayanbulak Basin of the Tianshan Mountains in China with a spatial resolution of 1 km is obtained. The results indicate that 1) the result of GWRK's accuracy is slightly higher than that of RK (R2 = 0.55 vs. R2 = 0.50, RMSE (root mean square error) = 0.102 m vs. RMSE = 0.077 m); 2) for the subareas, GWRK and RK exhibit similar estimation results of snow depth. Areas in the Bayanbulak Basin with a snow depth greater than 0.15 m are mainly distributed in an elevation range of 2632.00–3269.00 m and the snow in this area comprises 45.00–46.00% of the total amount of snow in this basin. However, the GWRK resulted in more detailed information on snow depth distribution than the RK. The final conclusion is that GWRK is better suited for estimating regional snow depth distribution.

Correction and interpolation of depth maps from structured light infrared sensors

Infrared structured light sensors are widely employed for control applications, gaming, acquisition of dynamic and static 3D scenes. Recent developments have lead to the availability on the market of low-cost sensors, like Kinect devices, which prove to be extremely sensitive to noise, light conditions, and the geometry of the scene.The paper presents a quality enhancement strategy for Kinect-acquired depth maps that corrects depth values via a set of local differential equations and interpolates the missing depth samples. The approach improves both density and accuracy of the generated 3D model under different light conditions and in the presence of cross-talk noise derived from other devices. Moreover, the paper introduces a new experimental reference dataset for Kinect denoising algorithms consisting in multiple acquisition under different noise conditions associated to a laser scan model of the scene (ground truth).

3D light fields based foreground segmentation in static scenes

To address the problem of extracting foreground objects from complex scenes,a 3D light field based method for foreground segmentation of static scenes was presented. Firstly,the method relies on densely sampled 3D light field formed by stacking a sequence of images captured at different viewpoints. These viewpoints were equally spaced along a linear path so that object trace was smooth in epipolar plan image( EPI).Secondly,line segment detection( LSD) method was performed to extract edges and compute the corresponding depth in EPI,based on which a depth map of the scene was obtained through a fast local depth interpolation algorithm. Piecewise cubic Hermite interpolating polynomial( PCHIP) algorithm proved to have desirable results in this phase. Finally,the recovered dense depth information was exploited to facilitate foreground segmentation. A threshold approach was used to separate different objects in scenes. Preliminary experimental results show that our method is able to estimate the correct relative spatial relation of multiple objects and our proposed foreground segmentation method reduces over-segmentation effects existed in traditional methods based on region clustering and mathematical morphology.

Fast Minimax Path-Based Joint Depth Interpolation

We propose a fast minimax path-based depth interpolation method. The algorithm computes for each target pixel varying contributions from reliable depth seeds, and weighted averaging is used to interpolate missing depths. Compared with state-of-the-art joint geodesic upsampling method which selects the $K$ nearest seeds to interpolate missing depths with $O(Kn)$ complexity, our method does not need to limit the number of seeds to $K$ and reduces the computational complexity to $O(n)$ . In addition, the minimax path chooses a path with the smallest maximum immediate pairwise pixel difference on it, so it tends to preserve sharp depth discontinuities better. In contrast to the results of previous depth upsampling algorithms, our approach can provide accurate depths with fewer artifacts.

WITHDRAWN: Joint depth map interpolation using minimax paths

We propose a minimax path-based depth interpolation method. The algorithm computes for each target pixel varying contributions from reliable depth seeds, and weighted averaging is used to interpolate missing depths. Compared with state-of-the-art joint geodesic upsampling method which selects the K nearest seeds to interpolate missing depths with O(Kn) complexity, our method does not need to limit the number of seeds to K and reduces the computational complexity to O(n). In addition, the minimax path chooses a path with the smallest maximum immediate pairwise pixel difference on it, so it tends to preserve sharp depth discontinuities better. In contrast to the results of previous depth upsampling algorithms, our approach provides accurate depths with fewer artifacts.

Snow hazard estimation and mapping for a province in northeast China

The estimation of annual maximum snow load is important for designing light-weight structures experiencing severe winter climate. The specified (basic) snow load in the Chinese design code is based on statistics of the return period values of ground snow load. The code tabulates the values for a few locations in a region. For example, values are only available at 31 sites for Heilongjiang Province with an area of more than 470,000 km2, China. The snow load needs to be spatially interpolated for sites far away from the tabulated locations. However, the statistical justification of the selected probability distribution to model snow depth or load hazard is unclear and the preferred spatial interpolation technique is unknown. This study focuses on the extreme value analysis and spatial interpolation of the annual maximum snow depth and ground snow load using the records at 83 stations in Heilongjiang Province from 1981 to 2010. The statistical analysis results show that the use of the lognormal distribution rather than the Gumbel distribution for the annual maximum snow depth suggested in the code is preferred for most sites, and the application of the ordinary co-kriging is adequate for spatial interpolation of extreme snow depth. The results also show that the uncertainty in snowpack bulk density should not be neglected in estimating the extreme (ground) snow load for updating the snow load in Chinese design code.

Comparison of deterministic and stochastic methods to predict spatial variation of groundwater depth

Accurate and reliable interpolation of groundwater depth over a region is a pre-requisite for efficient planning and management of water resources. The performance of two deterministic, such as inverse distance weighting (IDW) and radial basis function (RBF) and two stochastic, i.e., ordinary kriging (OK) and universal kriging (UK) interpolation methods was compared to predict spatio-temporal variation of groundwater depth. Pre- and post-monsoon groundwater level data for the year 2006 from 110 different locations over Delhi were used. Analyses revealed that OK and UK methods outperformed the IDW method, and UK performed better than OK. RBF also performed better than IDW and OK. IDW and RBF methods slightly underestimated and both the kriging methods slightly overestimated the prediction of water table depth. OK, RBF and UK yielded 27.52, 27.66 and 51.11 % lower RMSE, 27.49, 35.34 and 51.28 % lower MRE, and 14.21, 16.12 and 21.36 % higher R2 over IDW. The isodepth-area curves indicated the possibility of exploitation of groundwater up to a depth of 20 m.

Quantifying Tropical Wetlands Using Field Surveys, Spatial Statistics and Remote Sensing

Tropical wetlands support high biodiversity and ecological services, but in most areas they suffer from a paucity of baseline data to support management. We demonstrate how modern technology can be used to develop ecological baseline data including, landuse/landcover, water depth, water quality, lake-level fluctuation, and normalized difference vegetation index (NDVI). For the first time we quantified and mapped these metrics for the Paya Indah Wetlands, Malaysia using the new high-spatial-resolution World View 2 imagery. Landuse/landcover classifications were validated by field visits and visual interpretation of the imagery. NDVI was extracted based on red and near infra-red 2 bands. Topo to Raster method was used for interpolation of water depths. Annual mean of a water-quality index and annual water-level fluctuation of lakes were interpolated across lakes using the inverse-distance weighting method. Qualitative and quantitative accuracy assessment of classification (75 % overall accuracy, user’s accuracies ranged from 60 % to 90 % and producer’s accuracy ranged from 60 % to 97 %) was promising and clearly illustrated that World View 2 imagery can yield fast and reasonably precise identification of ecosystem characteristics for ecological baselines.

Tensor-Guided Fitting of Subducting Slab Depths

Abstract Geophysical measurements are often acquired at scattered locations in space. Therefore, interpolating or fitting the sparsely sampled data as a uniform function of space (a procedure commonly known as gridding) is a ubiquitous problem in geophysics. Most gridding methods require a model of spatial correlation for data. This spatial correlation model can often be inferred from some sort of secondary information, which may also be sparsely sampled in space. In this paper, we present a new method to model the geometry of a subducting slab in which we use a data‐fitting approach to address the problem. Earthquakes and active‐source seismic surveys provide estimates of depths of subducting slabs but only at scattered locations. In addition to estimates of depths from earthquake locations, focal mechanisms of subduction zone earthquakes also provide estimates of the strikes of the subducting slab on which they occur. We use these spatially sparse strike samples and the Earth’s curved surface geometry to infer a model for spatial correlation that guides a blended neighbor interpolation of slab depths. We then modify the interpolation method to account for the uncertainties associated with the depth estimates.

High resolution local Moho determination using gravity inversion: A case study in Sri Lanka

The seismic data incorporated in global Moho models are sparse and therefore the interpolation of global Moho depths on a local area may give unrealistic results, especially in regions without adequate seismic information. Gravity inversion is a useful tool that can be used to determine Moho depths in the mentioned regions. This paper describes an interactive way of local Moho depth determination using the gravity inversion method constrained with available seismic data. Before applying inversion algorithms, the Bouguer gravity data is filtered in various stages that reduce the potential bias usually expected in Moho depth determination using gravity methods with constant density contrast assumption. A test area with reliable seismic data is used to validate the results of Moho computation, and subsequently the same computation procedure is applied to the Sri Lankan region. The results of the test area are in better agreement with seismically determined Moho depths than those obtained by global Moho models. In the Sri Lankan region, Moho determination reveals a fairly uniform thin crust of average thickness around 20km. The overall result suggests that our gravity inversion method is robust and may be suitable for local Moho determination in virgin regions, especially those without sufficient seismic data.

Low‐Probability Flood Risk Modeling for New York City

The devastating impact by Hurricane Sandy (2012) again showed New York City (NYC) is one of the most vulnerable cities to coastal flooding around the globe. The low-lying areas in NYC can be flooded by nor'easter storms and North Atlantic hurricanes. The few studies that have estimated potential flood damage for NYC base their damage estimates on only a single, or a few, possible flood events. The objective of this study is to assess the full distribution of hurricane flood risk in NYC. This is done by calculating potential flood damage with a flood damage model that uses many possible storms and surge heights as input. These storms are representative for the low-probability/high-impact flood hazard faced by the city. Exceedance probability-loss curves are constructed under different assumptions about the severity of flood damage. The estimated flood damage to buildings for NYC is between US$59 and 129 millions/year. The damage caused by a 1/100-year storm surge is within a range of US$2 bn-5 bn, while this is between US$5 bn and 11 bn for a 1/500-year storm surge. An analysis of flood risk in each of the five boroughs of NYC finds that Brooklyn and Queens are the most vulnerable to flooding. This study examines several uncertainties in the various steps of the risk analysis, which resulted in variations in flood damage estimations. These uncertainties include: the interpolation of flood depths; the use of different flood damage curves; and the influence of the spectra of characteristics of the simulated hurricanes.

NONLOCAL VARIATIONAL MODELS FOR INPAINTING AND INTERPOLATION

In this paper we study some nonlocal variational models for different image inpainting tasks. Nonlocal methods for denoising and inpainting have gained considerable attention due to their good performance on textured images, a known weakness of classical local methods which are performant in recovering the geometric structure of the image. We first review a general variational framework for the problem of nonlocal inpainting that exploits the self-similarity of natural images to copy information in a consistent way from the known parts of the image. We single out two particular methods depending on the information we copy: either the gray level (or color) information or its gradient. We review the main properties of the corresponding energies and their minima. Then we discuss three other applications: we consider the problem of stereo inpainting, some simple cases of video inpainting, and the problem of interpolation of incomplete depth maps knowing a reference image. Incomplete depth maps can be obtained as a result of stereo algorithms, or given for instance by Time-of-Flight cameras (in that case the interpolated result can be used to generate the images of the stereo pair). We discuss the basic algorithms to minimize the energies and we display some numerical experiments illustrating the main properties of the proposed models.