Depth Edges Research Articles

DepthCut: improved depth edge estimation using multiple unreliable channels

In the context of scene understanding, a variety of methods exists to estimate different information channels from mono or stereo images, including disparity, depth, and normals. Although several advances have been reported in the recent years for these tasks, the estimated information is often imprecise particularly near depth discontinuities or creases. Studies have however shown that precisely such depth edges carry critical cues for the perception of shape, and play important roles in tasks like depth-based segmentation or foreground selection. Unfortunately, the currently extracted channels often carry conflicting signals, making it difficult for subsequent applications to effectively use them. In this paper, we focus on the problem of obtaining high-precision depth edges (i.e., depth contours and creases) by jointly analyzing such unreliable information channels. We propose DepthCut, a data-driven fusion of the channels using a convolutional neural network trained on a large dataset with known depth. The resulting depth edges can be used for segmentation, decomposing a scene into depth layers with relatively flat depth, or improving the accuracy of the depth estimate near depth edges by constraining its gradients to agree with these edges. Quantitatively, we compare against 18 variants of baselines and demonstrate that our depth edges result in an improved segmentation performance and an improved depth estimate near depth edges compared to data-agnostic channel fusion. Qualitatively, we demonstrate that the depth edges result in superior segmentation and depth orderings. (Code and datasets will be made available.)

Hanging ability in climbing: an approach by finger hangs on adjusted depth edges in advanced and elite sport climbers

ABSTRACTHanging ability on small depth edges is one of the most limiting factors in climbing. The aim of this study was to assess the reliability and validity of a hanging ability indicator measured on an adjusted depth edge. Forty voluntary sport climbers (34 men) were divided into an advanced group (AG; n = 22) and an elite group (EG; n = 18). Climbers performed three sustained finger tests following a test–retest design: (a) maximum hanging time on a 14-mm edge depth (MHT_14), (b) minimum edge depth in which climbers could hang for 40 s exactly (MED_40) and (c) maximum added weight test on the MED_40 edge depth (MAW_5). EG performed better than AG in all tests. The regression analyses showed that the MHT_14 test and MAW_5 test explained a 35% and 69% of the climbing sport level in AG and EG, respectively. All the tests were reliable (ICC3,1 values ranging from 0.89 to 1.00). The MAW_5 and MHT_14 tests demonstrated to be valid and reliable hanging ability indicators for EG and AG, respectively. The measurement of hanging ability on adjusted depth edges might be a key factor in elite climbers, but not necessary in lower level climbers.

Use of local image information in depth edge classification by humans and neural networks

Use of local image information in depth edge classification by humans and neural networks

Minimum Spanning Forest with Embedded Edge Inconsistency Measurement Model for Guided Depth Map Enhancement.

Guided depth map enhancement based on Markov Random Field (MRF) normally assumes edge consistency between the color image and the corresponding depth map. Under this assumption, the low-quality depth edges can be refined according to the guidance from the high-quality color image. However, such consistency is not always true, which leads to texture-copying artifacts and blurring depth edges. In addition, the previous MRF-based models always calculate the guidance affinities in the regularization term via a non-structural scheme which ignores the local structure on the depth map. In this paper, a novel MRF-based method is proposed. It computes these affinities via the distance between pixels in a space consisting of the Minimum Spanning Trees (Forest) to better preserve depth edges. Furthermore, inside each Minimum Spanning Tree, the weights of edges are computed based on explicit edge inconsistency measurement model, which significantly mitigates texture-copying artifacts. To further tolerate the effects caused by noise and better preserve depth edges, a bandwidth adaption scheme is proposed. Our method is evaluated for depth map super-resolution and depth map completion problems on synthetic and real datasets including Middlebury, ToF-Mark and NYU. A comprehensive comparison against 16 state-of-the-art methods is carried out. Both qualitative and quantitative evaluation present the improved performances.

Explicit Edge Inconsistency Evaluation Model for Color-Guided Depth Map Enhancement

Color-guided depth enhancement is used to refine depth maps according to the assumption that the depth edges and the color edges at the corresponding locations are consistent. In methods on such low-level vision tasks, the Markov random field (MRF), including its variants, is one of the major approaches that have dominated this area for several years. However, the assumption above is not always true. To tackle the problem, the state-of-the-art solutions are to adjust the weighting coefficient inside the smoothness term of the MRF model. These methods lack an explicit evaluation model to quantitatively measure the inconsistency between the depth edge map and the color edge map, so they cannot adaptively control the efforts of the guidance from the color image for depth enhancement, leading to various defects such as texture-copy artifacts and blurring depth edges. In this paper, we propose a quantitative measurement on such inconsistency and explicitly embed it into the smoothness term. The proposed method demonstrates promising experimental results compared with the benchmark and state-of-the-art methods on the Middlebury ToF-Mark, and NYU data sets.

Determining the Critical Depth of Excavation and Investigating the Effect of Fault on the Resistance of Tehran Alluvial

In this paper, the effect of the burden of neighboring buildings on the stability of holes in a part of coarse-grained alluvial deposits of northern Tehran has been investigated. For this purpose two numerical and partial equilibrium methods have been used with FLAC and SLIDE software. As, the holes in the study area to the depth of 20 m in the without load state of the neighboring building and the building load of the 5-floor building in a distance of 3 meters from the depth edge were analyzed, and the results were compared and finally using the SLIDE software the guardian structure for deep recessed excavations was suggested which with this proposed guardian structure a high reliability coefficient of 1.5 can be achieved.

Snapshot difference imaging using correlation time-of-flight sensors

Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences. An intuitive approach to computing such differences is to capture several images sequentially and then process them jointly. In this paper, we introduce a snapshot difference imaging approach that is directly implemented in the sensor hardware of emerging time-of-flight cameras. With a variety of examples, we demonstrate that the proposed snapshot difference imaging technique is useful for direct-global illumination separation, for direct imaging of spatial and temporal image gradients, for direct depth edge imaging, and more.

RGB-D SLAM in Dynamic Environments Using Static Point Weighting

We propose a real-time depth edge based RGB-D SLAM system for dynamic environment. Our visual odometry method is based on frame-to-keyframe registration, where only depth edge points are used. To reduce the influence of dynamic objects, we propose a static weighting method for edge points in the keyframe. Static weight indicates the likelihood of one point being part of the static environment. This static weight is added into the intensity assisted iterative closest point (IAICP) method to perform the registration task. Furthermore, our method is integrated into a SLAM (Simultaneous Localization and Mapping) system, where an efficient loop closure detection strategy is used. Both our visual odometry method and SLAM system are evaluated with challenging dynamic sequences from the TUM RGB-D dataset. Compared to state-of-the-art methods for dynamic environment, our method reduces the tracking error significantly.

Edge‐guided with gradient‐assisted depth up‐sampling

Most of depth up-sampling algorithms are based on the consistent hypothesis, i.e. the object boundaries in the colour image are consistent with depth discontinuity regions in the depth map. However, the hypothesis is not always correct. Under the combined guidance of high-resolution (HR) depth edge map and HR colour image gradient map, a simple and efficient depth up-sampler is presented. Firstly, the consistent regions are distinguished from the other regions and more accurate depth edge points are found. Then, the initial up-sampled depth map from traditional bilinear interpolation is refined by an effective depth-assignment scheme. Extensive experiments demonstrate that the proposed method outperforms conventional interpolation algorithms and some other edge-based depth up-sampling methods.

Non-convex joint bilateral guided depth upsampling

Blurring depth edges and texture copy artifacts are challenging issues for guided depth map upsampling. They are caused by the inconsistency between depth edges and corresponding color edges. In this paper, we extend the well-known Joint Bilateral Upsampling (JBU) (Kopf et al. 2007) with a novel non-convex optimization framework for guided depth map upsampling, which is denoted as Non-Convex JBU (NCJBU). We show that the proposed NCJBU can well handle the edge inconsistency by making use of the property of both the guidance color image and the depth map. Through comprehensive experiments, we show that our NCJBU can preserve sharp depth edges and properly suppress texture copy artifacts. In addition, we present a data driven scheme to properly determine the parameter in our model such that fine details and sharp depth edges are well preserved even for a large upsampling factor (e.g., 8 ×). Experimental results on both simulated and real data show the effectiveness of our method.

Robust depth enhancement and optimization based on advanced multilateral filters

Stereo matching of two distanced cameras and structured-light RGB-D cameras are the two common ways to capture the depth map, which conveys the per-pixel depth information of the image. However, the results with mismatched and occluded pixels would not provide accurately well-matched depth and image information. The mismatched depth-image relations would degrade the performances of view syntheses seriously in modern-day three-dimension video applications. Therefore, how to effectively utilize the image and depth to enhance themselves becomes more and more important. In this paper, we propose an advanced multilateral filter (AMF), which refers spatial, range, depth, and credibility information to achieve their enhancements. The AMF enhancements could sharpen the image, suppress noisy depth, filling depth holes, and sharpen the depth edges simultaneously. Experimental results demonstrate that the proposed method provides a superior performance, especially around the object boundary.

Haze Removal via Edge Weighted Pixel-to-Patch Fusion

One of the key issues in image dehazing is how to accurately estimate the transmission map using strong priors or assumptions. By far the most common prior adopted by existing haze removal approaches is dark channel prior (DCP). Despite the remarkable progress, the existing DCP-based methods induce misestimation of the transmission while preventing halo artifacts, and may cause distortions of the recovered haze-free images in both chromaticity and contrast. This because they fail to observe the fact that the edges in a color image do not necessarily correspond to the changes in depth; instead, they try to preserve all types of edges indiscriminately in the estimated transmission map, which does not accord with the property of a realistic transmission map. In order to address this issue, we propose an efficient method for haze removal under the guidance of the depth edges. The main contribution of our study is that we present a depth edge prior to obtain the depth edges from the hazy image, and then employ a pixel-to-patch fusion scheme weighted by the depth edges to estimate the transmission directly, which can preserve the sharp discontinuity at depth edges but smooth the surface texture in the rest regions of the transmission map. The experimental results show that our approach can obtain more accurate estimation of the transmission, and consequently restore better quality haze-free images in terms of less color distortion and high contrast.

Depth Edge Filtering Using Parameterized Structured Light Imaging.

This research features parameterized depth edge detection using structured light imaging that exploits a single color stripes pattern and an associated binary stripes pattern. By parameterized depth edge detection, we refer to the detection of all depth edges in a given range of distances with depth difference greater or equal to a specific value. While previous research has not properly dealt with shadow regions, which result in double edges, we effectively remove shadow regions using statistical learning through effective identification of color stripes in the structured light images. We also provide a much simpler control of involved parameters. We have compared the depth edge filtering performance of our method with that of the state-of-the-art method and depth edge detection from the Kinect depth map. Experimental results clearly show that our method finds the desired depth edges most correctly while the other methods cannot.

Learning to identify depth edges in real-world images with 3D ground truth

Learning to identify depth edges in real-world images with 3D ground truth

Improved depth coding for HEVC focusing on depth edge approximation

The latest High Efficiency Video Coding (HEVC) standard has greatly improved the coding efficiency compared to its predecessor H.264. An important share of which is the adoption of hierarchical block partitioning structures and an extended number of modes. The structure of existing inter-modes is appropriate mainly to handle the rectangular and square aligned motion patterns. However, they could not be suitable for the block partitioning of depth objects having partial foreground motion with irregular edges and background. In such cases, the HEVC reference test model (HM) normally explores finer level block partitioning that requires more bits and encoding time to compensate large residuals. Since motion detection is the underlying criteria for mode selection, in this work, we use the energy concentration ratio feature of phase correlation to capture different types of motion in depth object. For better motion modeling focusing at depth edges, the proposed technique also uses an extra pattern mode comprising a group of templates with various rectangular and non-rectangular object shapes and edges. As the pattern mode could save bits by encoding only the foreground areas and beat all other inter-modes in a block once selected, the proposed technique could improve the rate-distortion performance. It could also reduce encoding time by skipping further branching using the pattern mode and selecting a subset of modes using innovative pre-processing criteria. Experimentally it could save 29% average encoding time and improve 0.10dB Bjontegaard Delta peak signal-to-noise ratio compared to the HM.

Effectiveness of Conventional and Herbal Treatment on Diabetic Foot Ulcer Using Bates-Jensen Wound Assessment Tool.

Diabetic mellitus type 2 is s metabolic disease and a silent killer disease which leads to systemic complications especially in major blood vessels, nerves, kidneys, and eyes. In uncontrolled diabetic the foot ulcers are common complication which ends up with gangrene of the foot and amputation. The aim of this study is to compare the effectiveness of conventional and herbal treatment in diabetic foot ulcer. A total of 160 diabetic patients with foot ulcer were selected randomly and divided into conventional and herbal treatment groups (n=80 each). In conventional treatment group and in herbal treatment group the wound was assessed by using of Bates Jensen wound assessment tool, in pre-test and post test 1 after 15 days and post test 2 after 30 days. There are significant changes Jansen wound assessment tool was used to study the type of wound which consists of 13 parameters and 10 parameters were taken in this study i.e. (1) size (2) depth (3) edges (4) exudates type (5) exudates amount (6) skin colour surrounding wound (7) peripheral tissue edema (8) peripheral tissue induration (9) granulation tissue and (10) epithelialisation. In all the parameters the χ2 value changes and the P value is <0.001 which is significant. The herbal treatment had more healing effect while comparing with the conventional treatment.

Kinect depth image enhancement with adaptive joint multi-lateral discrete filters

One of the major problems of Kinect depth image is the existence of noise and holes. Thus, it is necessary to enhance depth images for post processing. In this paper, we propose an adaptive joint bilateral/trilateral filter, considering both the depth image and the aligned colour image, to refine the depth edges and remove the noise. Specifically, the pixels around the common boundary of depth edges and colour edges are taken as unreliable pixels due to the perturbations around the depth edges, then a support region and kernel-width adaptive joint bilateral filter considering the colour edge is applied to refine the marked unreliable pixels and the depth edges. For the depth pixels except unreliable pixels, we apply a kernel-width adaptive joint trilateral filter to remove the noise. Experimental results show that our approach outperforms the compared methods in terms of finer depth edges and noise removal.

Fast robust detection of edges in noisy depth images

Depth edges play an important role in depth image upsampling. Many recent upsampling methods rely on the prior images of depth edges to preserve sharp depth edges in restored depth images. However, recent depth edge detection methods are not robust against the noise in depth images. Some methods are also too time-consuming. We propose a method to efficiently detect edges in depth images. The proposed method is very simple but very robust against the noise in depth images. It is also fast and has near O(1) implementation. We apply the proposed method to the existing edge guided depth image upsampling. Experimental results on both simulated and real data show the effectiveness of the proposed method.

Joint processing and fast encoding algorithm for multi-view depth video

The multi-view video plus depth format is the main representation of a three-dimensional (3D) scene. In the 3D extension of high-efficiency video coding (3D-HEVC), the main framework for depth video is similar to that of color video. However, because of the limitation of the mainstream capture technologies, depth video is inaccurate and inconsistent. In addition, the depth video coding method in the current 3D-HEVC software implementation is highly complex. In this paper, we introduce a joint processing and fast coding algorithm for depth video. The proposed algorithm utilizes the depth and color features to extract depth discontinuous regions, depth edge regions, and motion regions as masks for efficient processing and fast coding. The processing step includes spatial and temporal enhancement. The fast coding method mainly limits the traversal of the CU partition and mode decision. Experimental results demonstrate that the proposed algorithm reduces the coding time and the depth video coding bitrate; the proposed algorithm reduces overall coding time by 44.24 % and depth video coding time by 72.00 % on average. In addition, there is a 24.07 % depth video coding bitrate reduction with an average of 1.65 % Bjontegaard delta bitrate gains.

Depth map compression via 3D region-based representation

In 3D video, view synthesis is used to create new virtual views between encoded camera views. Errors in the coding of the depth maps introduce geometry inconsistencies in synthesized views. In this paper, a new 3D plane representation of the scene is presented which improves the performance of current standard video codecs in the view synthesis domain. Two image segmentation algorithms are proposed for generating a color and depth segmentation. Using both partitions, depth maps are segmented into regions without sharp discontinuities without having to explicitly signal all depth edges. The resulting regions are represented using a planar model in the 3D world scene. This 3D representation allows an efficient encoding while preserving the 3D characteristics of the scene. The 3D planes open up the possibility to code multiview images with a unique representation.