3D Shape Reconstruction Method Research Articles

Evaluation of hybrid deep learning and optimization method for 3D human pose and shape reconstruction in simulated depth images

In this paper, we address the problem of capturing both the shape and the pose of a human character using a single depth sensor. Some previous works proposed to fit a parametric generic human template into the depth image, while others developed deep learning (DL) approaches to find the correspondence between depth pixels and vertices of the template. We designed a hybrid approach, combining the advantages of both methods, and conducted extensive experiments on the SURREAL Varol et al. (2017), DFAUST datasets Bogo etal (2017) and a subset of AMASS Mahmood et al. (2019). Results show that this hybrid approach enables us to enhance pose and shape estimation compared to using DL or model fitting separately. We also evaluated the ability of the DL-based dense correspondence method to segment also the background — not only the body parts. We also evaluated 4 different methods to perform the model fitting based on a dense correspondence, where the number of available 3D points differs from the number of corresponding template vertices. These two results enabled us to better understand how to combine DL and model fitting, and the potential limits of this approach to deal with real-depth images. Future works could explore the potential of taking temporal information into account, which has proven to increase the accuracy of pose and shape reconstruction based on a unique depth or RGB image.

A New Method for 3D Shape Reconstruction with a High Dynamic Range Surface

A New Method for 3D Shape Reconstruction with a High Dynamic Range Surface

Shape from focus based on 3D structure tensor using optical microscopy.

Shape from focus (SFF) is a widely used passive optical method for 3D shape reconstruction. In SFF, a focus measure, which is used to estimate the relative focus level, plays a critical role in depth estimation. In this article, we present a new focus measure for accurate 3D shape estimation in optical microscopy based on the analysis of 3D structure tensor. First, the 3D tensors are computed from the input image sequence for each pixel. Then, each tensor is decomposed into point, curve, and surface tensors by decomposing tensors into eigenvalues and eigenvectors. Finally, the surfaceness is used to measure the quality of sharpness. The proposed focus measure provides accurate focus values and better resistance against noise. The proposed measure is evaluated by conducting experiments using image sequences of simulated and microscopic real objects. The comparative analysis demonstrates the effectiveness of the proposed focus measure in recovering 3D shape.

Robust Multiview Photometric Stereo Using Planar Mesh Parameterization.

We propose a robust uncalibrated multiview photometric stereo method for high quality 3D shape reconstruction. In our method, a coarse initial 3D mesh obtained using a multiview stereo method is projected onto a 2D planar domain using a planar mesh parameterization technique. We describe methods for surface normal estimation that work in the parameterized 2D space that jointly incorporates all geometric and photometric cues from multiple viewpoints. Using an estimated surface normal map, a refined 3D mesh is then recovered by computing an optimal displacement map in the same 2D planar domain. Our method avoids the need of merging view-dependent surface normal maps that is often required in conventional methods. We conduct evaluation on various real-world objects containing surfaces with specular reflections, multiple albedos, and complex topologies in both controlled and uncontrolled settings and demonstrate that accurate 3D meshes with fine geometric details can be recovered by our method.

A Robust Inversion Method for Quantitative 3D Shape Reconstruction from Coaxial Eddy Current Measurements

This work is motivated by the monitoring of conductive clogging deposits in steam generator at the level of support plates. One would like to use multistatic measurements from coaxial coils in order to obtain estimates on the clogging volume. We propose a 3D shape optimization technique based on simplified shape parametrization of the deposit. This parametrization is adapted to the measurement nature and resolution. The direct problem is modeled by the eddy current approximation of time-harmonic Maxwell's equations in the low frequency regime. A potential formulation is adopted in order to easily handle the complex topology of the industrial problem setting. We first characterize the shape derivatives of the deposit impedance signal using an adjoint field technique. For the inversion procedure, the direct and adjoint problems have to be solved for each vertical probe position which is excessively time- and memory-consuming. To overcome this difficulty, we propose and discuss a steepest descent method based on a invariant mesh. Numerical experiments are presented to illustrate the convergence and the efficiency of the method.

3D shape reconstruction of teeth by shadow speckle correlation method

This paper presents a new method for 3D shape reconstruction in computer-aided dental prosthetics. A specklegram is projected onto the tooth to be measured. This shadow speckle image is recorded and then processed by a digital image correlation method, which enables the computation of 2D shapes based on the similar principle of shadow moiré method. By repeating the procedure for all the sides, i.e., one crown and several side surfaces, local 2D shapes can be measured precisely. Afterwards, these local 2D profiles are merged to form a 3D model, during which certain constraints such as the widths along perpendicular directions are introduced to guide the process. As the height information within an entire image field is recorded instantly, it has the potential to be employed in an intra-oral environment, which would make the patient feel more comfortable during the restoration process. In vitro experiments were carried out on gypsum teeth models and the results proved the effectiveness of the proposed method.