Shape Measurement Technique Research Articles

Composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation

Fourier transform profilometry (FTP) is a classic three-dimensional (3D) shape measurement technique that can retrieve the wrapped phase from a single fringe pattern. However, suffering from the spectral leakage and overlapping problems, it generally yields a coarse phase map with low spatial resolution and precision. Recently, deep learning has been introduced to the field of Fringe projection profilometry (FPP), revealing promising results in fringe analysis, phase unwrapping, depth constraint and system calibration. However, for absolute shape measurement of general objects, the inherent depth ambiguity problem of a single fringe is still insurmountable. In this work, we propose a composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation. Our method combines the advantages of FPP techniques for high-resolution phase retrieval and speckle correlation approaches for robust unambiguous depth measurement. The proposed deep learning framework comprises two paths: one is a U-net-structured network, which is used to extract the wrapped phase maps from a single fringe pattern with high accuracy (but with depth ambiguities). The other stereo matching network produces the initial absolute (but with low resolution) disparity map from an additional speckle pattern. The initial disparity map is refined by exploiting the wrapped phase maps as an additional constraint and finally, a high-accuracy high-resolution disparity map for absolute 3D measurement can be obtained. Experimental results demonstrated that the proposed deep-learning-based method could realize high-precision absolute 3D measurement with an accuracy of 50 µm for measuring objects with complex surfaces.

Improve Temporal Fourier Transform Profilometry for Complex Dynamic Three-Dimensional Shape Measurement.

The high-speed three-dimensional (3-D) shape measurement technique has become more and more popular recently, because of the strong demand for dynamic scene measurement. The single-shot nature of Fourier Transform Profilometry (FTP) makes it highly suitable for the 3-D shape measurement of dynamic scenes. However, due to the band-pass filter, FTP method has limitations for measuring objects with sharp edges, abrupt change or non-uniform reflectivity. In this paper, an improved Temporal Fourier Transform Profilometry (TFTP) algorithm combined with the 3-D phase unwrapping algorithm based on a reference plane is presented, and the measurement of one deformed fringe pattern producing a new 3-D shape of an isolated abrupt objects has been achieved. Improved TFTP method avoids band-pass filter in spatial domain and unwraps 3-D phase distribution along the temporal axis based on the reference plane. The high-frequency information of the measured object can be well preserved, and each pixel is processed separately. Experiments verify that our method can be well applied to a dynamic 3-D shape measurement with isolated, sharp edges or abrupt change. A high-speed and low-cost structured light pattern sequence projection has also been presented, it is capable of projection frequencies in the kHz level. Using the proposed 3-D shape measurement algorithm with the self-made mechanical projector, we demonstrated dynamic 3-D reconstruction with a rate of 297 Hz, which is mainly limited by the speed of the camera.

Investigation of Phase Pattern Modulation for Digital Fringe Projection Profilometry

Abstract The fringe projection profilometry with sinusoidal patterns based on phase-shifting algorithms is commonly distorted by the nonlinear intensity response of commercial projector. In order to solve this issue, sinusoidal width modulation is presented to generate binary sinusoidal patterns for defocusing the projection. However, the residual errors in the phase maps are usually notable for highly accurate three-dimensional shape measurements. In this paper, we propose the fringe patterns of the sinusoidal, square, and triangular periodic waveforms with seven-step phase-shifting algorithm to further improve the accuracy of three-dimensional profile reconstruction. The absolute phase values are calculated by using quality guided path unwrapping. We learn that by properly selecting fringe patterns according to the target shape, the undesired harmonics of the measured surface have negligible effect on the phase values. The experiments are presented to verify the imaging performances of three fringe patterns for different testing targets. The triangular fringe patterns are suitable for the shape measurements of complex targets with curved surfaces. The results provide a great possibility for high-accuracy shape measurement technique with wider measuring depth range.

高反光表面三维形貌测量技术

高反光表面三维形貌测量技术

Calibration method for panoramic 3D shape measurement with plane mirrors.

High-speed panoramic three-dimensional (3D) shape measurement can be achieved by introducing plane mirrors into the traditional fringe projection profilometry (FPP) system because such a system simultaneously captures fringe patterns from three different perspectives (i.e., by a real camera and two virtual cameras in the plane mirrors). However, calibrating such a system is nontrivial due to the complicated setup. This work introduces a flexible new technique to calibrate such a system. We first present the mathematical representation of the plane mirror, and then mathematically prove that it only requires the camera to observe a set of feature point pairs (including real points and virtual points) to generate a solution to the reflection matrix of a plane mirror. By calibrating the virtual and real camera in the same world coordinate system, 3D point cloud data obtained from real and virtual perspectives can be automatically aligned to generate a panoramic 3D model of the object. Finally, we developed a system to verify the performance of the proposed calibration technique for panoramic 3D shape measurement.

Three-dimensional shape measurement technique for large-scale objects based on line structured light combined with industrial robot

Three-dimensional(3-D) shape measurement for large-scale objects based on line structured light combined with industrial robot has been increasingly important because of its attributes of high-flexibility and high-robustness. To acquire the shape of large-scale objects, the line structured light obtained 3-D lines are stitched together by using robot hand-eye calibration results. However, in the line structured light technique, the traditional light plane calibration is complex and the result 3-D measurement accuracy is relatively low. Besides, because the light plane equation of the line structured light is arbitrary, the traditional standard ball hand-eye calibration method cannot be directly used. In this paper, a light plane calibration method is proposed by using binocular cameras, which simplifies the calibration process and improves the 3-D measurement accuracy. Furthermore, a modified space circle fitting method is proposed for the standard ball hand-eye calibration, which can be flexibly combined with the line structured light for arbitrary light plane. Based on the above improvement of the light plane calibration and the robot hand-eye calibration, the proposed measurement technique can accurately measure 3-D shape for large-scale objects. Theoretical analysis and experiments are provided to evaluate its performance.

Multi-objective strategy to optimize dithering technique for high-quality three-dimensional shape measurement**Project supported by the Zhejiang Provincial Welfare Technology Applied Research Project, China (Grant No. 2017C31080).

Dithering optimization techniques can be divided into the phase-optimized technique and the intensity-optimized technique. The problem with the former is the poor sensitivity to various defocusing amounts, and the problem with the latter is that it cannot enhance phase quality directly nor efficiently. In this paper, we present a multi-objective optimization framework for three-dimensional (3D) measurement by utilizing binary defocusing technique. Moreover, a binary patch optimization technique is used to solve the time-consuming issue of genetic algorithm. It is demonstrated that the presented technique consistently obtains significant phase performance improvement under various defocusing amounts.

Direct structural analysis of domains defined by point clouds

This contribution presents a method that aims at the numerical analysis of solids represented by oriented point clouds. The proposed approach is based on the Finite Cell Method, a high-order immersed boundary technique that computes on a regular background grid of finite elements and requires only inside–outside information from the geometric model. It is shown that oriented point clouds provide sufficient information for these point-membership classifications. Further, we address a tessellation-free formulation of contour integrals that allows to apply Neumann boundary conditions on point clouds without having to recover the underlying surface. Two-dimensional linear elastic benchmark examples demonstrate that the method is able to provide the same accuracy as those computed with conventional, continuous surface descriptions, because the associated error can be controlled by the density of the cloud. Three-dimensional examples computed on point clouds of historical structures show how the method can be employed to establish seamless connections between digital shape measurement techniques and numerical analyses.

Real-time 3D shape measurement using 3LCD projection and deep machine learning.

For 3D imaging and shape measurement, simultaneously achieving real-time and high-accuracy performance remains a challenging task in practice. In this paper, a fringe-projection-based 3D imaging and shape measurement technique using a three-chip liquid-crystal-display (3LCD) projector and a deep machine learning scheme is presented. By encoding three phase-shifted fringe patterns into the red, green, and blue (RGB) channels of a color image and controlling the 3LCD projector to project the RGB channels individually, the technique can synchronize the projector and the camera to capture the required fringe images at a fast speed. In the meantime, the 3D imaging and shape measurement accuracy is dramatically improved by introducing a novel phase determination approach built on a fully connected deep neural network (DNN) learning model. The proposed system allows performing 3D imaging and shape measurement of multiple complex objects at a real-time speed of 25.6fps with relative accuracy of 0.012%. Experiments have shown great promise for advancing scientific and engineering applications.

A Cost-Effective Single-Shot Structured Light System for 3D Shape Measurement

Single-shot three-dimensional (3D) shape measurement techniques have attracted extensive researches, as they are suitable for dynamic measurement. In this study, a cost-effective single-shot structured light system (CES-SLS) is proposed for 3D shape measurement by using a color camera and a normal projector. With the aid of two planar mirrors and an isosceles right-angle mirror, a single color camera functions as a binocular vision. A single-shot color random speckle pattern (CRSP) is used to encode the object by the normal projector. The dense corresponding points (DCP) are derived by the spatial-temporal correlation of RGB image subsets instead of the spatial correlation of gray image subsets in the conventional method. An inverse compositional Gaussian Newton (IC-GN) iteration method with the second-order shape function is then introduced to find the sub-pixel corresponding points (CP). Furthermore, a geometrical 3D recovery method is presented to calculate the 3D point by minimizing the re-projection error. The experimental results demonstrate the comparative advantages of the proposed CES-SLS against the system using a single-shot random speckle pattern (RSP) and the curve fitting sub-pixel method in the aspects of measurement accuracy and noise robustness. In addition, the re-projection error of each 3D point from the geometrical method is smaller than that of the conventional mid-point.

A flexible and easy-to-implement single-camera microscopic 3D digital image correlation technique

We propose a novel, flexible and easy-to-implement single-camera microscopic three-dimensional digital image correlation (3D-DIC) technique for accurate full-field shape, motion and deformation measurements at the microscale. The established microscopic 3D-DIC system comprises a single 3CCD color camera, a high-magnification zoom lens, and a specially designed color separation device using a beam splitter, two optical bandpass filters and three plane mirrors. To perform accurate stereo measurement, color images of a test sample surface are recorded by the 3CCD camera, which observes the test sample via the microscopic imaging system from two different optical paths with the aid of the color separation device. The recorded color images can be subsequently separated to red and green channel sub-images which constitute stereo pairs corresponding to different perspectives. The separated stereo pairs can be subsequently analyzed using regular 3D-DIC to extract full-field 3D shape and deformation of the test object surface. The method was first validated by a series of benchmark tests, involving the 3D shape reconstruction of steel balls with different diameters, in-plane and out-of-plane translation tests. Surface 3D deformation measurement of a planar acrylic plate with micro-features was also performed to show the practicality of the proposed method. This method provides new simple and practical avenues for performing shape and deformation measurement at microscale, showing great potential in both material testing and bio-tissue characterization.

Active phase error suppression for color phase-shifting fringe projection based on hue pre-correction

Color phase-shifting fringe projection is one of the single-shot three-dimensional shape measurement techniques based on fringe projection profilometry. In practice, the nonlinearity and color crosstalk of the projector-camera system yield undesired phase errors when using phase-shifting algorithm. In this paper, an active phase error suppression method based on hue pre-correction is proposed. Theoretical analysis of phase error is performed taking account of the gamma nonlinearity of the camera. Based on the analysis, the distortion of hue component of the acquired color fringe pattern is employed to represent the overall phase error induced by the combined effect of nonlinearity and color crosstalk. In this method, the hue is pre-corrected to be linear by measuring the function of the combined effect and modifying the computer-generated hue according to the source-distorted relationship. Then, the new modified color fringe pattern can be generated and saved for the further measurement of objects. Experimental results demonstrate the feasibility of the proposed method and it can effectively suppress the induced overall phase error caused by the nonlinearity and color crosstalk of the projector-camera system.

Calibration-free single camera stereo-digital image correlation for small-scale underwater deformation measurement

Stereo-digital image correlation (stereo-DIC) has been routinely used as a practical and powerful optical technique for surface 3D full-field shape and deformation measurements in various scenarios. However, it is challenging to perform accurate stereo-DIC measurements for submerged objects due to the significant refraction presented at the interfaces of air and water. In this paper, a novel underwater full-field 3D profile and deformation measurements method using the single camera stereo-DIC technique that combines single bilateral telecentric lens imaging and bi-prism-assisted pseudo stereovision is proposed. In using this technique, an immersed surface projects through the (semi-) submerged bi-prism and the bilateral telecentric lens, forming two virtual images on left and right parts of the camera sensor. Matching the virtual left and right images using DIC and substituting the matched image points into a set of newly derived linear equations, accurate 3D profiles and further 3D deformation fields can be readily obtained. The effectiveness and accuracy of the proposed method are successfully validated by a set of real experiments including underwater 3D shape reconstruction, in-plane and out-of-plane translation, and membrane inflation experiments. Because of the distinctive advantages of simple and compact optical configuration, without the need of stereo calibration, and strong robustness against water fluctuation and ambient light variation, the proposed method is expected to be a simple yet effective method for many underwater applications like in vitro biological tissues deformation measurements and submerged materials characterization.

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system.

In this paper, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

High-speed three-dimensional shape measurement based on cyclic complementary Gray-code light.

The binary defocusing technique has been widely used in high-speed three-dimensional (3D) shape measurement because it breaks the bottlenecks in high-speed fringe projection and the projector's nonlinear response. However, it is challenging for this method to realize a two- or multi-frequency phase-shifting algorithm because it is difficult to simultaneously generate high-quality sinusoidal fringe patterns with different periods under the same defocusing degree. To bypass this challenge, we proposed a high-speed 3D shape measurement technique for dynamic scenes based on cyclic complementary Gray-code (CCGC) patterns. In this proposed method, the projected phase-shifting sinusoidal fringes kept one same frequency, which is beneficial to ensure the optimum defocusing degree for binary dithering technique. The wrapped phase can be calculated by phase-shifting algorithm and unwrapped with the aid of complementary Gray-code (CGC) patterns in a simple and robust way. Then, the cyclic coding strategy further extends the unambiguous phase measurement range and improves the measurement accuracy compared with the traditional Gray-coding strategy under the condition of the same number of projected patterns. High-quality 3D results of three complex dynamic scenes-including a cooling fan and a standard ceramic ball with a free-falling table tennis, collapsing building blocks, and impact of the Newton's cradle-were demonstrated at a frame rate of 357 fps. This verified the proposed method's feasibility and validity.

A Survey on Dimensions and Shape Measurement Techniques of 3D Nano Structures

A Survey on Dimensions and Shape Measurement Techniques of 3D Nano Structures

High-speed three-dimensional shape measurement using geometry-constraint-based number-theoretical phase unwrapping

In this paper, we propose a high-speed three-dimensional (3-D) shape measurement technique for dynamic scenes using geometry-constraint-based number-theoretical phase unwrapping. As a classical algorithm for temporal phase unwrapping (TPU), the number-theoretical approach is suitable for the binary defocusing fringe projection system since it can retrieve an absolute phase without using low-frequency fringe patterns. However, the conventional number-theoretical TPU approach cannot provide sufficient stability to unwrap a high-frequency phase since it requires the two fringe frequencies to be coprime within the global range of the projector coordinate. In contrast, using low-frequency fringe patterns tends to make phase unwrapping more reliable, but at the expense of the measurement precision. By introducing depth constraint into the traditional number-theoretical TPU, we only need to eliminate the phase ambiguity of each pixel within a small period range defined by the depth range, which means that our method just requires the two fringe frequencies to be coprime within the local period range instead of the conventional global range. Due to the reduction of fringe order candidates and the unambiguous phase range, the reliability of phase unwrapping can be significantly improved compared with the traditional number-theoretical TPU approach even when high-frequency fringe patterns are used. The proposed method has been successfully implemented on a high-frame-rate fringe projection system, achieving high-precision, robust, and absolute 3-D shape measurement at 3333 frames per second.

Optimized dithering technique for three-dimensional shape measurement with projector defocusing

Abstract There have been active studies on optimized dithering techniques to improve 3D shape measurement quality with image sensor and defocusing projector. These techniques can be classified into intensity-based optimization technology and phase-based optimization technology. However, those phase-based optimization methods are sensitive to the amount of defocusing while intensity-based optimization methods cannot reduce phase errors efficiently. This paper presents an optimization framework. This framework combines structural similarity index measurement and intensity residual error optimization. By applying this optimization framework to patches and tiling the best patch, high quality fringe pattern can be generated for three-dimensional measurement. Both simulation and experimental results show that this proposed algorithm can achieve phase quality improvements and it is robust to various defocusing levels.

Identification of SIPS J2045-6332 as a Partially Resolved Binary

We show that SIPS J2045-6332, a late M/early L object previously identified as a candidate spectral mix binary, shows an elongated image shape. Using shape measurement techniques originally developed for cosmological weak lensing surveys on VISTA VHS images we show that this likely blended binary has an implied position angle of ~290 degrees East of North with a secondary companion that is likely to be a late L dwarf. This object would be a good follow-up target for high resolution imaging studies

Double-pattern triangular pulse width modulation technique for high-accuracy high-speed 3D shape measurement.

Using 1-bit binary patterns for three-dimensional (3D) shape measurement has been demonstrated as being advantageous over using 8-bit sinusoidal patterns in terms of achievable speeds. However, the phase quality generated by binary pattern(s) typically are not high if only a small number of phase-shifted patterns are used. This paper proposes a method to improve the phase quality by representing each pattern with the difference of two binary patterns: the first binary pattern is generated by triangular pulse width modulation (TPWM) technique, and the second being π shifted from the first pattern that is also generated by TPWM technique. The phase is retrieved by applying a three-step phase-shifting algorithm to the difference patterns. Through optimizing the modulation frequency of the triangular carrier signal, we demonstrate that a high-quality phase can be generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) with only six binary patterns. Since only 1-bit binary patterns are required for 3D shape measurement, this paper will present a real-time 3D shape measurement system that can achieve 30 Hz.