Enhanced Depth Maps Research Articles

Scale-Aware Multi-View Reconstruction Using an Active Triple-Camera System.

This paper presents an active wide-baseline triple-camera measurement system designed especially for 3D modeling in general outdoor environments, as well as a novel parallel surface refinement algorithm within the multi-view stereo (MVS) framework. Firstly, the pre-processing module converts the synchronized raw triple images from one single-shot acquisition of our setup to aligned RGB-Depth frames, which are then used for camera pose estimation using iterative closest point (ICP) and RANSAC perspective-n-point (PnP) approaches. Afterwards, an efficient dense reconstruction method, mostly implemented on the GPU in a grid manner, takes the raw depth data as input and optimizes the per-pixel depth values based on the multi-view photographic evidence, surface curvature and depth priors. Through a basic fusion scheme, an accurate and complete 3D model can be obtained from these enhanced depth maps. For a comprehensive test, the proposed MVS implementation is evaluated on benchmark and synthetic datasets, and a real-world reconstruction experiment is also conducted using our measurement system in an outdoor scenario. The results demonstrate that (1) our MVS method achieves very competitive performance in terms of modeling accuracy, surface completeness and noise reduction, given an input coarse geometry; and (2) despite some limitations, our triple-camera setup in combination with the proposed reconstruction routine, can be applied to some practical 3D modeling tasks operated in outdoor environments where conventional stereo or depth senors would normally suffer.

EF-Net: A novel enhancement and fusion network for RGB-D saliency detection

Salient object detection (SOD) has gained tremendous attention in the field of computer vision. Multi-modal SOD based on the complementary information from RGB images and depth maps has shown remarkable success, making RGB-D saliency detection an active research topic. In this paper, we propose a novel multi-modal enhancement and fusion network (EF-Net) for effective RGB-D saliency detection. Specifically, we first utilize a color hint map module with RGB images to predict a hint map, which encodes the coarse information of salient objects. The resulting hint map is then utilized to enhance the depth map with our depth enhancement module, which suppresses the noise and sharpens the object boundary. Finally, we propose an effective layer-wise aggregation module to fuse the features extracted from the enhanced depth maps and RGB images for the accurate detection of salient objects. Our EF-Net utilizes an enhancement-and-fusion framework for saliency detection, which makes full use of the information from RGB images and depth maps. In addition, our depth enhancement module effectively resolves the low-quality issue of depth maps, which boosts the saliency detection performance remarkably. Extensive experiments on five widely-used benchmark datasets demonstrate that our method outperforms 12 state-of-the-art RGB-D saliency detection approaches in terms of five key evaluation metrics.

How depth estimation in light fields can benefit from super-resolution?

With the development of consumer light field cameras, the light field imaging has become an extensively used method for capturing the three-dimensional appearance of a scene. The depth estimation often requires a dense sampled light field in the angular domain or a high resolution in the spatial domain. However, there is an inherent trade-off between the angular and spatial resolutions of the light field. Recently, some studies for super-resolving the trade-off light field have been introduced. Rather than the conventional approaches that optimize the depth maps, these approaches focus on maximizing the quality of the super-resolved light field. In this article, we investigate how the depth estimation can benefit from these super-resolution methods. Specifically, we compare the qualities of the estimated depth using (a) the original sparse sampled light fields and the reconstructed dense sampled light fields, and (b) the original low-resolution light fields and the high-resolution light fields. Experiment results evaluate the enhanced depth maps using different super-resolution approaches.

Depth maps and high-dynamic range image generation from alternating exposure multiview images

For stereo matching, it is hard to find accurate correspondence for saturated regions, such as too dark or too bright regions, because there is rarely reliable information to match. In this situation, conventional high-dynamic range (HDR) imaging techniques combining multiple exposures for each viewpoint can be adopted to generate well-exposed stereo images. This approach is, however, time-consuming and needs much memory to store multiple exposures for each viewpoint. We propose an efficient method to generate HDR multiview images as well as corresponding accurate depth maps. First, we take a single exposure for each viewpoint with alternating exposure setting, such as short and long exposure, as functions of viewpoint changes. Then, we compute an initial depth map for each view only using neighboring images that have the same exposure. To reduce the error of the initial depth maps for the saturated regions, we adopt the fusion move algorithm fusing neighboring depth maps that have different error regions. Finally, using the enhanced depth maps, we generate artifact-free and sharp HDR images using the joint bilateral filtering and a detail-transfer technique. Experimental results show that our method produces both consistent HDR images and accurate depth maps for various indoor and outdoor multiview images.

Interactive 3D Segmentation of Rock-Art by Enhanced Depth Maps and Gradient Preserving Regularization

Petroglyphs (rock engravings) have been pecked and engraved by humans into natural rock surfaces thousands of years ago and are among the oldest artifacts that document early human life and culture. Some of these rock engravings have survived until the present and serve today as a unique document of ancient human life. Since petroglyphs are pecked into the surface of natural rocks, they are threatened by environmental factors such as weather and erosion. To document and preserve these valuable artifacts of human history, the 3D digitization of rock surfaces has become a suitable approach due to the development of powerful 3D reconstruction techniques in recent years. The results of 3D reconstruction are huge 3D point clouds which represent the local surface geometry in high resolution. In this article, we present an automatic 3D segmentation approach that is able to extract rock engravings from reconstructed 3D surfaces. To solve this computationally complex problem, we transfer the task of segmentation to the image-space in order to efficiently perform segmentation. Adaptive learning is applied to realize interactive segmentation and a gradient preserving energy minimization assures smooth boundaries for the segmented figures. Our experiments demonstrate the efficiency and the strong segmentation capabilities of the approach. The precise segmentation of petroglyphs from 3D surfaces provides the foundation for compiling large petroglyph databases which can then be indexed and searched automatically.

Real-time shading-based refinement for consumer depth cameras

We present the first real-time method for refinement of depth data using shape-from-shading in general uncontrolled scenes. Per frame, our real-time algorithm takes raw noisy depth data and an aligned RGB image as input, and approximates the time-varying incident lighting, which is then used for geometry refinement. This leads to dramatically enhanced depth maps at 30Hz. Our algorithm makes few scene assumptions, handling arbitrary scene objects even under motion. To enable this type of real-time depth map enhancement, we contribute a new highly parallel algorithm that reformulates the inverse rendering optimization problem in prior work, allowing us to estimate lighting and shape in a temporally coherent way at video frame-rates. Our optimization problem is minimized using a new regular grid Gauss-Newton solver implemented fully on the GPU. We demonstrate results showing enhanced depth maps, which are comparable to offline methods but are computed orders of magnitude faster, as well as baseline comparisons with online filtering-based methods. We conclude with applications of our higher quality depth maps for improved real-time surface reconstruction and performance capture.

Generation of ROI Enhanced Depth Maps Using Stereoscopic Cameras and a Depth Camera

In this paper, we propose a new scheme to generate region-of-interest (ROI) enhanced depth maps combining one low-resolution depth camera with high-resolution stereoscopic cameras. Basically, the hybrid camera system produces four synchronized images at each frame: left and right images from the stereoscopic cameras, a color image and its associated depth map from the depth camera. In the hybrid camera system, after estimating initial depth information for the left image using a stereo matching algorithm, we project depths obtained from the depth camera onto ROI of the left image using three-dimensional (3-D) image warping. Then, the warped depths are linearly interpolated to fill depth holes occurred in ROI. Finally, we merge the ROI depths with background ones extracted from the initial depth information to generate the ROI enhanced depth map. Experimental results show that the proposed depth acquisition system provides more accurate depth information for ROI than previous stereo matching algorithms. Besides, the proposed scheme minimizes inherent problems of the current depth camera, such as limitation of its measuring distance and production of low-resolution depth maps.