Depth Acquisition Research Articles

Efficient Shape Reconstruction of Microlens Using Optical Microscopy

The imaging properties of a microlens are highly related to its 3-D profile; therefore, it is of fundamental importance to measure its 3-D geometrical characteristics with high accuracy after industrial fabrication. However, common 3-D measurement tools are difficult to use for fast, noninvasive, and precise 3-D measurement of a microlens. Depth acquisition is a direct way to understand the 3-D properties of objects in computer vision, and shape from defocus (SFD) has been demonstrated to be effective for 3-D reconstruction. In this paper, a depth reconstruction method from blurring using optical microscopy and optical diffraction is proposed to reconstruct the global shape of a microlens. First, the relationship between the intensity distribution and the depth information is introduced. Second, a blurring imaging model with optical diffraction is formulated through curve fitting, accounting for relative blurring and heat diffusion, and a new SFD method with optical diffraction and defocused images is proposed. Finally, a polydimethylsiloxane (PDMS) microlens is used to validate the proposed SFD method, and the results show that its global shape can be reconstructed with high precision. The average estimation error is 77 nm, and the cost time is reduced by 92.5% compared with atomic force microscopy scanning.

Interfered depth map recovery with texture guidance for multiple structured light depth cameras

In depth acquisition systems with multiple structured light depth cameras (SLDCs), interference between the devices causes interfered regions, which degrade depth quality and impair applications. This paper proposes a novel approach to recover the depth maps. Under the guidance of texture segments, interfered regions are categorized into flat regions and boundary regions. After that, different strategies are applied to the two kinds of regions because of their property differences. For flat regions, a Markov random field (MRF) model is utilized to get the optimal gradient solution. With the gradients, discrete Poisson equation (DPE) is applied to calculate the final depth solution. In boundary regions, another texture-guided MRF is utilized to peruse depth directly. Experiment results demonstrate that the proposed method works well on both synthetic data and captured data. In flat regions, we get smooth-varied depth, and boundaries between interfered and non-interfered regions are seamless. In boundary regions, sharp depth edges between objects are well preserved. Moreover, our method improves the PSNR of synthetic data by 4–14dB.

Hand fine-motion recognition based on 3D Mesh MoSIFT feature descriptor

The times of Big Data promotes increasingly higher demands for fine-motion analysis, such as hand activity recognition. However, in real-world scenarios, hand activity recognition suffers huge challenges with variations of illumination, poses and occlusions. The depth acquisition provides an effective way to solve the above issues. In this paper, a complete framework of hand activity recognition combined depth information is presented for fine-motion analysis. First, the improved graph cuts method is introduced to hand location and tracking over time. Then, combined with 3D geometric characteristics and hand behavior prior information, 3D Mesh MoSIFT feature descriptor is proposed to represent the discriminant property of hand activity. Simulation orthogonal matching pursuit (SOMP) is used to encode the visual codewords. Experiments are based on the public available depth datasets (ChaLearn gesture dataset and our captured RGB-D dataset). Compared with the previous popular approaches, our framework has a consistently better performance for real-world applications with fine-motion analysis in terms of effectiveness, robustness and universality.

One-shot depth acquisition with a random binary pattern.

In this paper, we propose a new spatial encoding method that integrates the random binary pattern and the improved phase-difference-matching method to acquire a dense and precise depth map. The adopted binary pattern can simplify pattern projecting devices compared with the patterns that use color. The density of speckles in the pattern is periodic and the positions of them are random. Based on these two properties, we propose an improved phase-difference corresponding method, which is divided into two steps: the coarse matching step to estimate the approximate coordinates of pixels in the pattern via analyzing the phase values of the image, and the fine matching step to compensate errors of the coarse matching results and to achieve subpixel accuracy. This matching method does not require an extra optimization method with high computational complexity. In the experiment, we show the effectiveness of the proposed method. We also evaluate this method in actual experiments. The results show that this method has advantages over the time-of-flight camera and Kinect, particularly in terms of precision.

3D scanning of cultural heritage with consumer depth cameras

Three dimensional reconstruction of cultural heritage objects is an expensive and time-consuming process. Recent consumer real-time depth acquisition devices, like Microsoft Kinect, allow very fast and simple acquisition of 3D views. However 3D scanning with such devices is a challenging task due to the limited accuracy and reliability of the acquired data. This paper introduces a 3D reconstruction pipeline suited to use consumer depth cameras as hand-held scanners for cultural heritage objects. Several new contributions have been made to achieve this result. They include an ad-hoc filtering scheme that exploits the model of the error on the acquired data and a novel algorithm for the extraction of salient points exploiting both depth and color data. Then the salient points are used within a modified version of the ICP algorithm that exploits both geometry and color distances to precisely align the views even when geometry information is not sufficient to constrain the registration. The proposed method, although applicable to generic scenes, has been tuned to the acquisition of sculptures and in this connection its performance is rather interesting as the experimental results indicate.

Gradient-domain-based enhancement of multi-view depth video

Multi-view depth is an emerging and attractive 3D representation in recent years, which acts a significant role in rendering numerous viewing angles from a small number of given input views. The performance of those depth-based 3D video applications is strongly dependent on the quality of multi-view depth. However, because of the limitations of depth acquisition and estimation, the quality of multi-view depth suffers from artifacts in spatial dimension and inconsistency in both the temporal and inter-view dimensions. In this paper, we propose a gradient-domain based enhancement method for multi-view depth. Being different from the traditional enhancement methods which intended to process one or two above dimensions, our proposal exploits the coherence of both temporal and inter-view dimensions in addition of the spatial one. It is very challenging to obtain the stable characteristics in multi-dimensions. To solve this problem, we propose to investigate the characteristics in the gradient domain rather than the intensity domain. The enhanced multi-view depth is obtained through the minimization of energy function under the constraint of a joint gradient field (JGF), which is estimated from multiple dimensions through motion estimation and geometric mapping. Therefore, the enhanced multi-view depth is a global optimization in multiple dimensions that ensures the consistency in temporal, inter-view and spatial domains. Furthermore, we also propose an enhancement structure to indicate the process order of depth frames. The experimental results suggest that the proposed method can enhance multi-view depth with desired sharpness and consistency.

Edge-preserving interpolation of depth data exploiting color information

The extraction of depth information associated to dynamic scenes is an intriguing topic, because of its perspective role in many applications, including free viewpoint and 3D video systems. Time-of-flight (ToF) range cameras allow for the acquisition of depth maps at video rate, but they are characterized by a limited resolution, specially if compared with standard color cameras. This paper presents a super-resolution method for depth maps that exploits the side information from a standard color camera: the proposed method uses a segmented version of the high-resolution color image acquired by the color camera in order to identify the main objects in the scene and a novel surface prediction scheme in order to interpolate the depth samples provided by the ToF camera. Effective solutions are provided for critical issues such as the joint calibration between the two devices and the unreliability of the acquired data. Experimental results on both synthetic and real-world scenes have shown how the proposed method allows to obtain a more accurate interpolation with respect to standard interpolation approaches and state-of-the-art joint depth and color interpolation schemes.

Real-Time Depth Image Acquisition and Restoration for Image Based Rendering and Processing Systems

Depth information is an important ingredient in image-based rendering (IBR) systems. Traditional depth acquisition is mainly based on computer vision or depth sensing devices. With the advent of el...

Depth from Refraction Using a Transparent Medium with Unknown Pose and Refractive Index

In this paper, we introduce a novel method for depth acquisition based on refraction of light. A scene is captured directly by a camera and by placing a transparent medium between the scene and the camera. A depth map of the scene is then recovered from the displacements of scene points in the images. Unlike other existing depth from refraction methods, our method does not require prior knowledge of the pose and refractive index of the transparent medium, but instead can recover them directly from the input images. By analyzing the displacements of corresponding scene points in the images, we derive closed form solutions for recovering the pose of the transparent medium and develop an iterative method for estimating the refractive index of the medium. Experimental results on both synthetic and real-world data are presented, which demonstrate the effectiveness of the proposed method.

Structure guided fusion for depth map inpainting

Depth acquisition becomes inexpensive after the revolutionary invention of Kinect. For computer vision applications, depth maps captured by Kinect require additional processing to fill up missing parts. However, conventional inpainting methods for color images cannot be applied directly to depth maps as there are not enough cues to make accurate inference about scene structures. In this paper, we propose a novel fusion based inpainting method to improve depth maps. The proposed fusion strategy integrates conventional inpainting with the recently developed non-local filtering scheme. The good balance between depth and color information guarantees an accurate inpainting result. Experimental results show the mean absolute error of the proposed method is about 20mm, which is comparable to the precision of the Kinect sensor.

乌苏里江水深遥感反演

Based on analyses of theoretical models of the fathoming method by using satellite remote sensing, we determined the most suitable spectral band for water depth retrieval using the SVC HR-1024 spectroradiometer. A water depth remote sensing model for the Wusuli River was established and its accuracy improved by analyzing the sensitivity to water of di?erent bands of the HJ-1a CCD camera output. Our results show that the acquisition of water depth from remote sensing could supply a new and more rapid way for obtaining the talweg, river terrain information and information on river pattern change for the Wusuli River.

Performance of reduced bit-depth acquisition for optical frequency domain imaging

High-speed optical frequency domain imaging (OFDI) has enabled practical wide-field microscopic imaging in the biological laboratory and clinical medicine. The imaging speed of OFDI, and therefore the field of view, of current systems is limited by the rate at which data can be digitized and archived rather than the system sensitivity or laser performance. One solution to this bottleneck is to natively digitize OFDI signals at reduced bit depths, e.g., at 8-bit depth rather than the conventional 12-14 bit depth, thereby reducing overall bandwidth. However, the implications of reduced bit-depth acquisition on image quality have not been studied. In this paper, we use simulations and empirical studies to evaluate the effects of reduced depth acquisition on OFDI image quality. We show that image acquisition at 8-bit depth allows high system sensitivity with only a minimal drop in the signal-to-noise ratio compared to higher bit-depth systems. Images of a human coronary artery acquired in vivo at 8-bit depth are presented and compared with images at higher bit-depth acquisition.

Clermont‐Ferrand dental school’ curriculum: an appraisal by last‐year students and graduates

Last-year students and young graduates of the Clermont-Ferrand dental school (France) assessed their professional skills in order to shed light on those which are perfectly mastered and those which are not, with the aim of improving the undergraduate training given by the school. A questionnaire was handed out to students in their last two years and to young graduates (n = 170). They were asked to appraise their self-assessed level concerning 52 skills using a 0-10 visual analogue scale. The respondents also had to highlight the five skills which, according to them, needed to be improved during the programme. Thirteen skills obtained a score lower than 5/10 showing a need of more thorough training, notably in the fields of implantology, paradontal surgery, temporomandibular joint disorders and dental practice financial management. Among these, five skills were found that demand in depth acquisition according to respondents. However, these five skills are not necessarily the ones which obtained the lowest scores. Our school needs to improve the quality of its training, notably by reorganising some of its courses and by intensifying student' training in outreach programmes, particularly in private practice, where skills like the financial management of a dental practice could be more easily acquired than in a hospital environment.

Generation of ROI Enhanced Depth Maps Using Stereoscopic Cameras and a Depth Camera

In this paper, we propose a new scheme to generate region-of-interest (ROI) enhanced depth maps combining one low-resolution depth camera with high-resolution stereoscopic cameras. Basically, the hybrid camera system produces four synchronized images at each frame: left and right images from the stereoscopic cameras, a color image and its associated depth map from the depth camera. In the hybrid camera system, after estimating initial depth information for the left image using a stereo matching algorithm, we project depths obtained from the depth camera onto ROI of the left image using three-dimensional (3-D) image warping. Then, the warped depths are linearly interpolated to fill depth holes occurred in ROI. Finally, we merge the ROI depths with background ones extracted from the initial depth information to generate the ROI enhanced depth map. Experimental results show that the proposed depth acquisition system provides more accurate depth information for ROI than previous stereo matching algorithms. Besides, the proposed scheme minimizes inherent problems of the current depth camera, such as limitation of its measuring distance and production of low-resolution depth maps.

Trinocular Depth Acquisition

This contribution presents a new concept for depth acquisition in post-production and visual effects (vfx) creation. The main idea is to mount two satellite cameras to the left and the right of an electronic D-cinema camera such that the whole rig forms a trinocular camera setup. Based on this system, depth is estimated by robust stereo matching and the trifocal constraint is exploited to determine the accuracy of the depth values. The information is calculated almost in realtime and can be directly previewed at the set. This paper explains the concept in more detail and describes the basic image processing algorithms. Finally, the theoretically achievable depth accuracy is discussed and results of the first experiments based on computer simulations are presented.

Near-infrared composite pattern projection for continuous motion hand–computer interaction

Traditional structured light techniques require multiple patterns to be projected onto a scene and captured for three-dimensional (3D) reconstruction without ambiguity and resistance to variation of albedo. However, multi-pattern techniques can be corrupted by the object movement during the sequential projection. We have introduced a single projected pattern which efficiently combines multiple patterns into a single composite pattern projection—allowing for video rate three-dimensional data acquisition. Attaining low cost 3D video acquisition would have a profound impact on applications such as human–computer interaction and assistive technology. So far the composite pattern technique has only been performed with visible light which would cause user annoyances for these applications. To solve the problem, we use near-infrared to project the pattern. But this raises another more general issue of spatial skin response. In this study, near-infrared illumination is used for imperceptible measurement of hand pose. In particular, we are studying continuous motion depth acquisition for tracking hand motion and rotation as an interface to a virtual reality. Applications include advanced cockpit controls and computer interfacing for the disabled.

Real-time acquisition of depth and color images using structured light and its application to 3D face recognition

In this paper, a novel real-time 3D and color sensor for the mid-distance range (0.1–3 m) based on color-encoded structured light is presented. The sensor is integrated using low-cost of-the-shelf components and allows the combination of 2D and 3D image processing algorithms, since it provides a 2D color image of the scene in addition to the range data. Its design is focused on enabling the system to operate reliably in real-world scenarios, i.e. in uncontrolled environments and with arbitrary scenes. To that end, novel approaches for encoding and recognizing the projected light are used, which make the system practically independent of intrinsic object colors and minimize the influence of the ambient light conditions. The system was designed to assist and complement a face authentication system integrating both 2D and 3D images. Depth information is used for robust face detection, localization and 3D pose estimation. To cope with illumination and pose variations, 3D information is used for the normalization of the input images. The performance and robustness of the proposed system is tested on a face database recorded in conditions similar to those encountered in real-world applications.

Surgical modification of the hyperplastic anterior maxillary ridge

H yperplastic tissue that replaces the bone of the residual alveolar ridge is incompatible with the demand for healthy, denture-supporting tissues (Figs 1 and 2>.1,* Its presence, particularly in the anterior part of the edentulous maxillae, usually necessitates surgical excision.3-6 The surgical intervention may be accompanied by a vestibulopiasty, which may also involve a grafting procedure. Injections of Silastic (Dow Corning, Midland, Mich.) or sclerosing solutions have also have proposed.7-9 The objectives of most reported techniques include (1) firm, healthy denture-supporting tissues, (2) maintenance or acquisition of maximum vestibular depth, and (3) avoidance of scars at the depth of the vestibule that might undermine the development of a denture’s correct peripheral seal. The objective of this report is to describe a surgical technique that we have used over the last 3 years in an effort to fulfill these objectives.