High-resolution Depth Image Research Articles

Depth Image Super Resolution Based on Edge-Guided Method

Depth image super-resolution (SR) is a technique which can reconstruct a high-resolution (HR) depth image from a low-resolution (LR) depth image. Its purpose is to obtain HR details to meet the needs of various applications in computer vision. In general, conventional depth image SR methods often cause edges in the final HR image to be blurred or ragged. To solve this problem, an edge-guided method for depth image SR is presented in this paper. To get high-quality edge information, a pair of sparse dictionaries was applied to reconstruct edges of depth image. Then, with the guidance of these high-quality edges, a depth image was interpolated by using a modified joint bilateral filter. Edge-guided method can preserve the sharpness of edges and effectively avoid generating blurry and ragged edges when SR is performed. Experiments showed that the proposed method can get better results on both subjective and objective evaluation, and the reconstructed performance was superior to conventional depth image SR methods.

Large-volume, low-cost, high-precision FMCW tomography using stitched DFBs.

Optical frequency-modulated continuous-wave (FMCW) reflectometry is a ranging technique that allows for high-resolution distance measurements over long ranges. Similarly, swept-source optical coherence tomography (SS-OCT) provides high-resolution depth imaging over typically shorter distances and higher scan speeds. In this work, we demonstrate a low-cost, low-bandwidth 3D imaging system that provides the high axial resolution imaging capability normally associated with SS-OCT over typical FMCW ranging depths. The imaging system combines 12 distributed feedback laser (DFB) elements from a single butterfly module to provide an axial resolution of 27.1 μm over 6 m of depth and up to 14 cubic meters of volume. Active sweep linearization is used, greatly reducing the signal processing overhead. Various sub-surface, OCT-style tomograms of semi-transparent objects are shown, as well as 3D maps of various objects over depths ranging from sub-millimeter to several meters. Such imaging capability would make long-distance, high-resolution surface interrogation possible in a low-cost, compact package.

Edge-Preserving Depth Map Upsampling by Joint Trilateral Filter.

Compared to the color images, their associated depth images captured by the RGB-D sensors are typically with lower resolution. The task of depth map super-resolution (SR) aims at increasing the resolution of the range data by utilizing the high-resolution (HR) color image, while the details of the depth information are to be properly preserved. In this paper, we present a joint trilateral filtering (JTF) algorithm for depth image SR. The proposed JTF first observes context information from the HR color image. In addition to the extracted spatial and range information of local pixels, our JTF further integrates local gradient information of the depth image, which allows the prediction and refinement of HR depth image outputs without artifacts like textural copies or edge discontinuities. Quantitative and qualitative experimental results demonstrate the effectiveness and robustness of our approach over prior depth map upsampling works.

Towards Topological Correct Segmentation of Macular OCT from Cascaded FCNs.

Optical coherence tomography (OCT) is used to produce high resolution depth images of the retina and is now the standard of care for in-vivo ophthalmological assessment. In particular, OCT is used to study the changes in layer thickness across various pathologies. The automated image analysis of these OCT images has primarily been performed with graph based methods. Despite the preeminence of graph based methods, deep learning based approaches have begun to appear within the literature. Unfortunately, they cannot currently guarantee the strict biological tissue order found in human retinas. We propose a cascaded fully convolutional network (FCN) framework to segment eight retina layers and preserve the topological relationships between the layers. The first FCN serves as a segmentation network which takes retina images as input and outputs the segmentation probability maps of the layers. We next perform a topology check on the segmentation and those patches that do not satisfy the topology criterion are passed to a second FCN for topology correction. The FCNs have been trained on Heidelberg Spectralis images and validated on both Heidelberg Spectralis and Zeiss Cirrus images.

색상정보와 깊이정보 가중치를 이용한 깊이영상 업샘플러

In this paper, we present an upsampling technique for depth map image using color and depth weights. First, we construct a high-resolution image using the bilinear interpolation technique. Next, we detect a common edge region using RGB color space, HSV color space, and depth image. If an interpolated pixel belongs to the common edge region, we calculate weighting values of color and depth in neighboring pixels and compute the cost value to determine the boundary pixel value. Finally, the pixel value having minimum cost is determined as the pixel value of the high-resolution depth image. Simulation results show that the proposed algorithm achieves good performance in terns of PSNR comparison and subjective visual quality.

Visual Face Recognition Using Bag of Dense Derivative Depth Patterns

A novel biometric face recognition algorithm using depth cameras is proposed. The key contribution is the design of a novel and highly discriminative face image descriptor called bag of dense derivative depth patterns (Bag-D3P). This descriptor is composed of four different stages that fully exploit the characteristics of depth information: 1) dense spatial derivatives to encode the 3-D local structure; 2) face-adaptive quantization of the previous derivatives; 3) multibag of words that creates a compact vector description from the quantized derivatives; and 4) spatial block division to add global spatial information. The proposed system can recognize people faces from a wide range of poses, not only frontal ones, increasing its applicability to real situations. Last, a new face database of high-resolution depth images has been created and made it public for evaluation purposes.

Joint interpretation of high-resolution velocity and resistivity models from the Barents Sea

To realize the full potential of a geophysical data set, and resolve interpretation ambiguities, the data must be integrated with other geological and complementary geophysical data. Seismic velocity and electrical resistivity are rock properties that both depend on lithology and fluid content: we show that valuable insight can be gained by integrated interpretation of high-resolution velocity and resistivity models produced by inversion and imaging of broadband dual-sensor (GeoStreamer) and Towed Streamer EM data respectively. Broadband dual-sensor seismic data enables high-resolution velocity model building and depth imaging using reflections, refractions and sea-surface multiples. As a result of advances in algorithms, workflows and high performance computing it is fast becoming routine to produce high resolution and accurate velocity models for large-scale 3D broadband dual-sensor data-sets as part of the depth imaging workflow. Thus seismic velocity is a rock property that can now be determined with sufficient resolution and precision to be of use to an interpreter in an exploration setting, before more detailed quantitative interpretation studies are undertaken. Marine Controlled Source EM (CSEM) data has been used extensively to improve the chance of success in the search for hydrocarbons given that accumulations of oil and gas can be characterized by increased resistivity. CSEM data have been used mostly to de-risk prospects. By using a Towed Streamer EM system it is possible to acquire CSEM data efficiently to determine the sub-surface resistivity reliably at both regional and prospect scales.

Edge-Guided Single Depth Image Super Resolution.

Recently, consumer depth cameras have gained significant popularity due to their affordable cost. However, the limited resolution and the quality of the depth map generated by these cameras are still problematic for several applications. In this paper, a novel framework for the single depth image superresolution is proposed. In our framework, the upscaling of a single depth image is guided by a high-resolution edge map, which is constructed from the edges of the low-resolution depth image through a Markov random field optimization in a patch synthesis based manner. We also explore the self-similarity of patches during the edge construction stage, when limited training data are available. With the guidance of the high-resolution edge map, we propose upsampling the high-resolution depth image through a modified joint bilateral filter. The edge-based guidance not only helps avoiding artifacts introduced by direct texture prediction, but also reduces jagged artifacts and preserves the sharp edges. Experimental results demonstrate the effectiveness of our method both qualitatively and quantitatively compared with the state-of-the-art methods.

Multiscale Grid Method for Detection and Reconstruction of Building Roofs from Airborne LiDAR Data

This study proposes a multiscale grid method to detect and reconstruct building roofs from airborne LiDAR data. The method interpolates unorganized LiDAR point cloud into two sets of grids with different spatial scales. In the large-scale grid, building seed regions are obtained, including detection of initial building seed regions and refinement of building seed regions. In the small-scale grid, to detect the detailed features of building roofs with complicated top structures, a high-resolution depth image is generated by a new iterative morphological interpolation using gradually increasing scales, and then segmented by using a full <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\lambdab$</tex> </formula> -schedule algorithm. Based on the building seed regions, detailed roof features are detected for each building and 3-D building roof models are then reconstructed according to the elevation of these features. Experiments are analyzed from several aspects: the correctness and completeness, the elevation accuracy of building roof models, and the influence of elevation to 3-D roof reconstruction. The experimental results demonstrate promising correctness, completeness, and elevation accuracy, with a satisfactory 3-D building roof models. The strategy of hierarchical spatial scale (from large scale to small scale) obtains the complementary advantage between technical applicability in a large urban environment and high quality in 3-D reconstruction of building roofs with fine details.

Learning-Based Filter Selection Scheme for Depth Image Super Resolution

Depth images that have the same spatial resolution as color images are required in many applications, such as multiview rendering and 3-D texture modeling. Since a depth sensor usually has poorer spatial resolution compared with a color image sensor, many depth image super-resolution methods have been investigated in the literature. With an assumption that no one super-resolution method can universally outperform the other methods, in this paper we introduce a learning-based selection scheme for different super-resolution methods. In our case study, three distinctive mean-type, max-type, and median-type filtering methods are selected as candidate methods. In addition, a new frequency-domain feature vector is designed to enhance the discriminability of the methods. Given the candidate methods and feature vectors, a classifier is trained such that the best method can be selected for each depth pixel. The effectiveness of the proposed scheme is first demonstrated using the synthetic data set. The noise-free and noisy low-resolution depth images are constructed, and the quantitative performance evaluation is performed by measuring the difference between the ground-truth high-resolution depth images and the resultant depth images. The proposed algorithm is then applied to real color and time-of-flight depth cameras. The experimental results demonstrate that the proposed algorithm outperforms the conventional algorithms both quantitatively and qualitatively.

Kilometer-range, high resolution depth imaging via 1560 nm wavelength single-photon detection

This paper highlights a significant advance in time-of-flight depth imaging: by using a scanning transceiver which incorporated a free-running, low noise superconducting nanowire single-photon detector, we were able to obtain centimeter resolution depth images of low-signature objects in daylight at stand-off distances of the order of one kilometer at the relatively eye-safe wavelength of 1560 nm. The detector used had an efficiency of 18% at 1 kHz dark count rate, and the overall system jitter was ~100 ps. The depth images were acquired by illuminating the scene with an optical output power level of less than 250 µW average, and using per-pixel dwell times in the millisecond regime.

Depth image interpolation using confidence-based markov random field

Depth images are essential data for high-quality three-dimensional (3D) video services, but the resolution of depth images captured by commercially available depth cameras is lower than that of the corresponding color images, owing to technical limitations. A depth image up-sampling method that uses a confidence-based Markov random field is proposed for enhancing this resolution. An initial high-resolution depth image and confidence values are generated with consideration of boundaries and textures in the corresponding color images. These are used as the base for a new likelihood and prior model design. The energy function derived from this model is optimized by using a graph cut algorithm, and subsequent experiments show that the proposed algorithm provides sufficiently good up-sampled depth images compared to other state-of-the-art algorithms.

An infrared range camera-based approach for three-dimensional locomotion tracking and pose reconstruction in a rodent

We herein introduce an automated three-dimensional (3D) locomotion tracking and pose reconstruction system for rodents with superior robustness, rapidity, reliability, resolution, simplicity, and cost. An off-the-shelf composite infrared (IR) range camera was adopted to grab high-resolution depth images (640×480×2048pixels at 20Hz) in our system for automated behavior analysis. For the inherent 3D structure of the depth images, we developed a compact algorithm to reconstruct the locomotion and body behavior with superior temporal and solid spatial resolution. Since the range camera operates in the IR spectrum, interference from the visible light spectrum did not affect the tracking performance. The accuracy of our system was 98.1±3.2%. We also validated the system, which yielded strong correlation with automated and manual tracking. Meanwhile, the system replicates a detailed dynamic rat model in virtual space, which demonstrates the movements of the extremities of the body and locomotion in detail on varied terrain.

High-resolution seismic imaging in deep sea from a joint deep-towed/OBH reflection experiment: application to a Mass Transport Complex offshore Nigeria

We assess the feasibility of high-resolution seismic depth imaging in deep water based on a new geophysical approach involving the joint use of a deep-towed seismic device (SYSIF) and ocean bottom hydrophones (OBHs). Source signature measurement enables signature deconvolution to be used to improve the vertical resolution and signal-to-noise ratio. The source signature was also used to precisely determine direct traveltimes that were inverted to relocate source and receiver positions. The very high accuracy of the positioning that was obtained enabled depth imaging and a stack of the OBH data to be performed. The determination of the P-wave velocity distribution was realized by the adaptation of an iterative focusing approach to the specific acquisition geometry. This innovative experiment combined with advanced processing succeeded in reaching lateral and vertical resolution (2.5 and 1 m) in accordance with the objectives of imaging fine scale structures and correlation with in situ measurements. To illustrate the technological and processing advances of the approach, we present a first application performed during the ERIG3D cruise offshore Nigeria with the seismic data acquired over NG1, a buried Mass Transport Complex (MTC) interpreted as a debris flow by conventional data. Evidence for a slide nature of a part of the MTC was provided by the high resolution of the OBH depth images. Rigid behaviour may be inferred from movement of coherent material inside the MTC and thrust structures at the base of the MTC. Furthermore, a silt layer that was disrupted during emplacement but has maintained its stratigraphic position supports a short transport distance.

Spatial and temporal mapping of water content across Nafion membranes under wetting and drying conditions

Water transport and water management are fundamental to polymer electrolyte membrane fuel cell operation. Accurate measurements of water content within and across the Nafion layer are required to elucidate water transport behavior and validate existing numerical models. We report here a direct measurement of water content profiles across a Nafion layer under wetting and drying conditions, using a novel magnetic resonance imaging methodology developed for this purpose. This method, multi-echo double half k-space spin echo single point imaging, based on a pure phase encode spin echo, is designed for high resolution 1D depth imaging of thin film samples. The method generates high resolution (<8 μm) depth images with an SNR greater than 20, in an image acquisition time of less than 2 min. The high temporal resolution permits water content measurements in the transient states of wetting and drying, in addition to the steady state.

High-resolution depth electrode localization and imaging in patients with pharmacologically intractable epilepsy

Localization and targeting of depth electrodes in specific regions of the human brain is critical for accurate clinical diagnoses and treatment as well as for neuroscientific electrophysiological research. By using high-resolution magnetic resonance imaging combined with 2D computational unfolding, the authors present a method that improves electrode localization in the medial temporal lobe. This method permits visualization of electrode placements in subregions of the hippocampus and parahippocampal gyrus, allowing for greater specificity in relating electrophysiological and anatomical features in the human medial temporal lobe. Such methods may be extended to therapeutic procedures targeting specific neuronal circuitry in subfields of structures deep in the human brain.

Combining Time-Of-Flight depth and stereo images without accurate extrinsic calibration

We combine a low resolution Time-Of-Flight (TOF) depth image camera based on Photonic Mixer Devices with two standard cameras in a stereo configuration. We show that this approach is useful even without accurate calibration. In a graph cut approach, we use depth information from the low resolution TOF camera to initialise the domain, and colour information for accurate depth discontinuities in the high resolution depth image. The system is promising as it is low cost, and naturally extends to the setting of dynamic scenes, providing high frame rates.

Increasing depth lateral resolution based on sensor fusion

Technologies for measuring range data were intensively developed in recent years. The Time-Of-Flight matrix range sensor is one of the most active range acquisition devices, due to its effective suppression of background illumination, relative low cost and high portability. However, the resolution of such 3D range sensor is much lower than that of a modern colour image. This paper presents a solution to the problem of deriving a high-resolution depth image from a low-resolution sensor and a high-resolution colour image. The proposed algorithm is a modification of the Markov Random Fields fusion method that was presented by Diebel and Thrun (2005). Two extensions are suggested to improve the optimisation criteria.

Thin film MRI—high resolution depth imaging with a local surface coil and spin echo SPI

A multiple echo, single point imaging technique, employing a local surface coil probe, is presented for examination of thin film samples. Depth images with a nominal resolution of 5 μm were acquired with acquisition times on the order of 10 min. The method may be used to observe dynamic phenomenon such as polymerization, wetting, and drying in thin film samples. It is readily adapted to spatially resolved diffusion coefficient and T 2 relaxation time mapping.