Shape Reconstruction Technique Research Articles

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches.

The field of computer vision has been focusing on achieving accurate three-dimensional (3D) object representations from a single two-dimensional (2D) image through deep artificial neural networks. Recent advancements in 3D shape reconstruction techniques that combine structured light and deep learning show promise in acquiring high-quality geometric information about object surfaces. This paper introduces a new single-shot 3D shape reconstruction method that uses a nonlinear fringe transformation approach through both supervised and unsupervised learning networks. In this method, a deep learning network learns to convert a grayscale fringe input into multiple phase-shifted fringe outputs with different frequencies, which act as an intermediate result for the subsequent 3D reconstruction process using the structured-light fringe projection profilometry technique. Experiments have been conducted to validate the practicality and robustness of the proposed technique. The experimental results demonstrate that the unsupervised learning approach using a deep convolutional generative adversarial network (DCGAN) is superior to the supervised learning approach using UNet in image-to-image generation. The proposed technique's ability to accurately reconstruct 3D shapes of objects using only a single fringe image opens up vast opportunities for its application across diverse real-world scenarios.

PA-Net: Plane Attention Network for real-time urban scene reconstruction

Traditional urban reconstruction methods can only output incomplete 3D models, which depict the scene regions that are visible to the moving camera. While learning-based shape reconstruction techniques make single-view 3D reconstruction possible, they are designed to handle single objects that are well-presented in the training datasets. This paper presents a novel learning-based approach for reconstructing complete 3D meshes for large-scale urban scenes in real-time. The input video sequences are fed into a localization module, which segments different objects and determines their relative positions. Each object is then reconstructed under their local coordinates to better approximate models in the training datasets. The reconstruction module is adopted from BSP-Net (Chen et al., 2020), which is capable of producing compact polygon meshes. However, major changes have been made so that unoriented objects in large-scale scenes can be reconstructed efficiently using only a small number of planes. Experimental results demonstrate that our approach can reconstruct urban scenes with buildings and vehicles using 400∼800 convex parts in 0.1∼0.5 s.

An adaptive refined grid search strategy for assessing operational flexibility and application on refrigerant selection

AbstractA novel adaptive refined grid search strategy is developed for representative characterization of process feasible region boundaries and accurate estimation of its hypervolume. In particular, a linked list data structure adopted from the field of computer science is used to maintain the grid connectivity information. A uniform perturbation scheme is used to refine the search only near the boundaries. The volumetric flexibility index (FIV) is calculated directly from a summation of feasible hypercubes in the grid, without the need to apply shape reconstruction techniques. The proposed adaptive grid search strategy can capture complex region shapes with reduced sampling costs and without randomness, increasing reproducibility. Operational flexibility is optimized traditionally at a process scale. A case study on refrigerant selection is presented to demonstrate that the developed strategy can be combined within a computer‐aided molecular design framework for operational flexibility optimization at the molecular scale.

Accurate 3D Shape Reconstruction from Single Structured-Light Image via Fringe-to-Fringe Network

Accurate three-dimensional (3D) shape reconstruction of objects from a single image is a challenging task, yet it is highly demanded by numerous applications. This paper presents a novel 3D shape reconstruction technique integrating a high-accuracy structured-light method with a deep neural network learning scheme. The proposed approach employs a convolutional neural network (CNN) to transform a color structured-light fringe image into multiple triple-frequency phase-shifted grayscale fringe images, from which the 3D shape can be accurately reconstructed. The robustness of the proposed technique is verified, and it can be a promising 3D imaging tool in future scientific and industrial applications.

MIMONet: Structured-light 3D shape reconstruction by a multi-input multi-output network.

Reconstructing 3D geometric representation of objects with deep learning frameworks has recently gained a great deal of interest in numerous fields. The existing deep-learning-based 3D shape reconstruction techniques generally use a single red-green-blue (RGB) image, and the depth reconstruction accuracy is often highly limited due to a variety of reasons. We present a 3D shape reconstruction technique with an accuracy enhancement strategy by integrating the structured-light scheme with deep convolutional neural networks (CNNs). The key idea is to transform multiple (typically two) grayscale images consisting of fringe and/or speckle patterns into a 3D depth map using an end-to-end artificial neural network. Distinct from the existing autoencoder-based networks, the proposed technique reconstructs the 3D shape of target using a refinement approach that fuses multiple feature maps to obtain multiple outputs with an accuracy-enhanced final output. A few experiments have been conducted to verify the robustness and capabilities of the proposed technique. The findings suggest that the proposed network approach can be a promising 3D reconstruction technique for future academic research and industrial applications.

HNet: Single-shot 3D shape reconstruction using structured light and h-shaped global guidance network

Retrieving three-dimensional (3D) shape information from a single two-dimensional (2D) image has recently gained enormous attention in a variety of fields. In spite of recent advancements in algorithms and hardware developments, the easy-to-use characteristics and the accuracy of the 3D shape reconstruction are always of great interest. This paper presents a robust 3D shape reconstruction technique that integrates structured-light 3D imaging scheme with deep convolutional neural network (CNN) learning. The structured-light patterns facilitate the featuring process while the CNN modeling surpasses the complexity of the traditional 3D shape reconstructions. In the supervised learning pipeline, the input is either a single fringe-pattern or a single speckle-pattern image, and the output is its corresponding high-accuracy 3D shape label. Unlike the well-received autoencoder-based CNN model, a global guidance network path with multi-scale feature fusion is introduced into the CNN model to improve the accuracy of the 3D shape reconstruction. Experimental evaluations have been conducted to demonstrate the validity and robustness of the proposed technique, which provides a promising tool for ever-increasing scientific research and engineering applications.

Three-dimensional Shape Reconstruction from Single-shot Speckle Image Using Deep Convolutional Neural Networks

Three-dimensional (3D) shape reconstruction from a monocular two-dimensional (2D) image has emerged as a highly demanded tool in many applications. This paper presents a novel 3D shape reconstruction technique that employs an end-to-end deep convolutional neural network (CNN) to transform a single speckle-pattern image into its corresponding 3D point cloud. In the proposed approach, three CNN models are explored for comparison to find the best capable network. To train the models with reliable datasets in the learning process, a multi-frequency fringe projection profilometry technique is adopted to prepare high-accuracy ground-truth 3D labels. Unlike the conventional 3D imaging and shape reconstruction techniques which often involve complicated algorithms and intensive computation, the proposed technique is simple, yet very fast and robust. A few experiments have been conducted to assess and validate the proposed approach, and its capability provides promising solutions to the ever-increasing scientific research and engineering applications.

STEREO IMAGE ANALYSIS OF AERIAL IMAGES FROM TWO UAVS FOR WAVE MONITORING IN SURF ZONE

UAVs have been considered as one of the most powerful tools for coastal monitoring. 3D Topography of sandy beaches can be obtained with automatic flight of single UAV and 3D shape reconstruction technique. Furthermore, UAVs have been utilized for wave monitoring (Holman et al., 2017) as the video images or a series of still images from aerial cameras enable us to observe wave propagations from higher viewpoints. Although there are some proposed techniques to extract wave celerity and direction from the aerial images by UAV, there doesn't seem to be enough trials to obtain wave height, which is one of the most important parameters to consider coastal problems such as beach morphological changes. In this study, stereo image analysis for 3D shape reconstruction was applied to stereo image pairs by two UAVs as a test of the applicability of the existing stereo measurement technique and also possibility of 3D wave deformation measurement in field.Recorded Presentation from the vICCE (YouTube Link): https://youtu.be/QZBAwj5RIi4

Dark-field ghost imaging

Ghost imaging is a promising technique for shape reconstruction using two spatially correlated beams: one beam interacts with a target and is collected with a bucket detector, and the other beam is measured with a pixelated detector. However, orthodox ghost imaging always provides unsatisfactory results for unstained samples, phase objects, or highly transparent objects. Here we present a dark-field ghost imaging technique that can work well for these "bad" targets. The only difference from orthodox ghost imaging is that the bucket signals rule out the target's unscattered beam. As experimental proof, we demonstrate images of fine copper wires, quartz fibers, scratched and damaged glass plates, a pure phase object, and biospecimens.

Development of a deep learning-based image processing technique for bubble pattern recognition and shape reconstruction in dense bubbly flows

This work presents a Convolutional Neural Network (CNN) based method for the shape reconstruction of bubbles in bubbly flows using high-speed camera images. The bubble identification and shape reconstruction adopted a methodology based on a set of anchor points and boxes, where a single anchor point is used for different anchor boxes with various sizes. These anchor points are determined, based on the internal features of the bubble images, which are more easily identifiable, in particular, in regions of the images with high bubble overlapping. This makes possible the application of the procedure to high void fraction bubbly flows. For a given anchor point, different ellipsoidal shapes are suggested as bubble shape candidates and are then correctly chosen by a trained CNN. The CNN training used labeled images from air–water system data set and a hyper-parameter analysis was performed to find the best configuration of the CNN architecture. From this optimal CNN architecture, different high-speed camera acquisitions of bubbly flows were analyzed by the CNN-based bubble shape reconstruction method. In order to gain a better comprehension of the method, experiments were conducted in two gas–liquid systems, air–water and air-aqueous glycerol solution, which resulted in different image parameters, such as brightness, contrast and edge definition. The CNN method trained only with air–water data, showed excellent performance in the cases with air-aqueous glycerol, demonstrating its generalization capability. In addition, the results showed that the deep learning method used in this work is able to detect most of the bubbles present in the high-speed camera images, even in dense bubbly flow configurations. The method developed in this work can be used to further analyze bubbly flows and generate experimental data for the implementation and validation of CFD models.

Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks.

Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.

3DMMS: robust 3D Membrane Morphological Segmentation of C. elegans embryo

BackgroundUnderstanding the cellular architecture is a fundamental problem in various biological studies. C. elegans is widely used as a model organism in these studies because of its unique fate determinations. In recent years, researchers have worked extensively on C. elegans to excavate the regulations of genes and proteins on cell mobility and communication. Although various algorithms have been proposed to analyze nucleus, cell shape features are not yet well recorded. This paper proposes a method to systematically analyze three-dimensional morphological cellular features.ResultsThree-dimensional Membrane Morphological Segmentation (3DMMS) makes use of several novel techniques, such as statistical intensity normalization, and region filters, to pre-process the cell images. We then segment membrane stacks based on watershed algorithms. 3DMMS achieves high robustness and precision over different time points (development stages). It is compared with two state-of-the-art algorithms, RACE and BCOMS. Quantitative analysis shows 3DMMS performs best with the average Dice ratio of 97.7% at six time points. In addition, 3DMMS also provides time series of internal and external shape features of C. elegans.ConclusionWe have developed the 3DMMS based technique for embryonic shape reconstruction at the single-cell level. With cells accurately segmented, 3DMMS makes it possible to study cellular shapes and bridge morphological features and biological expression in embryo research.

A level set method for shape reconstruction in seismic full waveform inversion using a linear elastic model in 2D

We present a novel shape reconstruction technique for seismic full waveform inversion that uses a linear elastic system in 2D for modeling wave propagation in the Earth and a level set technique for performing shape evolution. A cost functional is formulated and minimized by a gradient based descent technique with backtracking line search and an Armijo condition. The three distributed elastic parameter profiles are in our model assumed to follow a given reference profile inside an unknown region D, and a different reference profile outside of D. The task is to reconstruct the unknown region D from seismic data obtained at or close to the surface. We present numerical experiments that address the imaging of salt domes buried in the Earth. The results demonstrate that our new algorithm is able to reconstruct shapes with non-trivial topology from a simpler starting guess in few iterations.

Construction of pixel-level resolution DEMs from monocular images by shape and albedo from shading constrained with low-resolution DEM

Lunar Digital Elevation Model (DEM) is important for lunar successful landing and exploration missions. Lunar DEMs are typically generated by photogrammetry or laser altimetry approaches. Photogrammetric methods require multiple stereo images of the region of interest and it may not be applicable in cases where stereo coverage is not available. In contrast, reflectance based shape reconstruction techniques, such as shape from shading (SfS) and shape and albedo from shading (SAfS), apply monocular images to generate DEMs with pixel-level resolution. We present a novel hierarchical SAfS method that refines a lower-resolution DEM to pixel-level resolution given a monocular image with known light source. We also estimate the corresponding pixel-wise albedo map in the process and based on that to regularize the shape reconstruction with pixel-level resolution based on the low-resolution DEM. In this study, a Lunar–Lambertian reflectance model is applied to estimate the albedo map. Experiments were carried out using monocular images from the Lunar Reconnaissance Orbiter Narrow Angle Camera (LRO NAC), with spatial resolution of 0.5–1.5m per pixel, constrained by the Selenological and Engineering Explorer and LRO Elevation Model (SLDEM), with spatial resolution of 60m. The results indicate that local details are well recovered by the proposed algorithm with plausible albedo estimation. The low-frequency topographic consistency depends on the quality of low-resolution DEM and the resolution difference between the image and the low-resolution DEM.

Robust technique for 3D shape reconstruction

In this paper a robust and simple scheme is presented for three dimensional (3D) shape reconstruction of real object. A novel composite pattern technique is proposed for projecting the light pattern on the object of interest. The proposed scheme reduces the number of patterns by combining the primary color coded channels into one composite format. Our approach uses both spatial and temporal intensity variation for calibration and construction phase. Gamma calibration is considered with the propose scheme. High quality depth map is obtained from the linear light reflected by the shape of object without complex calculations. Experimental results demonstrated that proposed technique is fast and exhibit high level of precision. In addition hardware cost is minimized as compare to current calibration procedures used in structured light scanning system. Our scheme requires a digital camera, flashlight and mask of pattern only.

Polygonal damage shape reconstruction in plates using guided Lamb wave

Tomographic reconstruction of the damaged region using guided Lamb waves provides important information about the health condition of structures. Two general types of shape reconstruction techniques are discussed before: transform based methods and algebraic iterative methods. There are some limitations associated with each method such as high sensitivity to noise and incomplete data sets, limitations for damage size and contrast, bulky setup for scanning the damaged region, or low speed of reconstruction. In this paper, the concepts of a novel method for reconstructing the contour of damage based on the Polygon Reconstruction Technique are introduced. First, the projections (Radon transform) of the damaged region are generated from a small number of angles with the aid of beamforming method. Then, the damaged region is modeled by a polygon, which its optimal number of vertices is estimated using the minimum description length (MDL) principle. Finally, using the polygon reconstruction technique, the coordinates of the vertices are determined. As the proposed method is at its preliminary stage, two finite element models of an aluminum plate with two different holes as the damage are designed in ABAQUS to examine the feasibility and performance of the method. Results show that the introduced technique based on polygonal damage shape reconstruction is fast and accurate. Copyright © 2016 John Wiley & Sons, Ltd.

Optimization of the longitudinal resolution of a structured light shape reconstruction system, a DOE approach

Shape reconstruction using structured light has become one of the most widely used optical 3D scanning methods, especially in the field of reverse engineering. Longitudinal resolution is a main parameter in a shape reconstruction technique. An experimental assessment technique is proposed to evaluate the longitudinal resolution of a structured light measurement system. The proposed approach is based on incrementally moving a flat board, and reconstructing its shape at each position. The phase shift between the board wavefront at each position and the reference position wavefront is evaluated. The unwrapped phase is then used to evaluate the smallest board movement detectable by the measurement system. Taguchi method and ANOVA test are used to investigate the effect of various measurement system parameters on the estimated resolution.

A non-parametric approach to shape reconstruction from planar point sets through Delaunay filtering

In this paper, we present a fully automatic Delaunay based sculpting algorithm for approximating the shape of a finite set of points S in R2. The algorithm generates a relaxed Gabriel graph (RGG) that consists of most of the Gabriel edges and a few non-Gabriel edges induced by the Delaunay triangulation. Holes are characterized through a structural pattern called as body-arm formed by the Delaunay triangles in the void regions. RGG is constructed through an iterative removal of Delaunay triangles subjected to circumcenter (of triangle) and topological regularity constraints in O(nlogn) time using O(n) space.We introduce the notion of directed boundary samples which characterizes the two dimensional objects based on the alignment of their boundaries in the cavities. Theoretically, we justify our algorithm by showing that under given sampling conditions, the boundary of RGG captures the topological properties of objects having directed boundary samples. Unlike many other approaches, our algorithm does not require tuning of any external parameter to approximate the geometric shape of point set and hence human intervention is completely eliminated. Experimental evaluations of the proposed technique are done using L2 error norm measure, which is the symmetric difference between the boundaries of reconstructed shape and the original shape. We demonstrate the efficacy of our automatic shape reconstruction technique by showing several examples and experiments with varying point set densities and distributions.

Radar Imaging of Impenetrable and Penetrable Targets From Finite-Duration Pulsed Signatures

A straightforward time-domain imaging technique for shape reconstruction of both impenetrable and penetrable objects is described. For a class of finite-duration incident waveforms, the ramp cross-sectional area function is recovered from the radar signature of a target through the stable integration procedure. The results presented in this paper can be readily applied to identification of targets from their radar signatures directly in time domain. A number of illustrative examples is presented.

Technique for 3D shape reconstruction of spherical and aspheric surfaces using deflectometric principle

We provide a description and analysis of a deflectometric technique for 3D measurements of optically smooth surfaces. It is presented that a surface reconstruction problem leads to a theoretical description by a nonlinear partial differential equation. Then, a surface shape can be calculated by solution of a derived equation. The presented method is noncontact and no reference surface is needed as in interferometry.