3D Shape Measurement Method Research Articles

3D shape measurement method for high-reflective surface based on a color camera

Fringe projection profilometry (FPP) has been widely utilized in many fields due to its non-contact, high accuracy, high resolution and full-field measurement capabilities. However, the limited dynamic range of the camera sensor can result in overexposure of high-reflective parts in industrial production measurement. To effectively solve the above issue, this paper proposes a 3D shape measurement method for the high-reflective surface based on a color camera. Firstly, the optimal exposure time for the low-reflective region is estimated using the relationship between the standard variance of the phase error and the modulation. Twelve blue phase-shifted fringe patterns and a uniform blue image are utilized to obtain 3D shape of high-reflective surface. Secondly, captured images are separated into red, green and blue components. The fused Gaussian low-pass filtering method with different cut-off frequencies is used to filter and denoise the green or red components and the true intensity of the blue component in the saturated regions is calculated by the unsaturated intensity of the green or red components based on the calibrated crosstalk matrix. Then the image in saturated regions is fused and normalized with the unsaturated region. The absolute phase obtained from the fused normalized fringe patterns is converted into 3D data. Finally, experiments have been carried out on measuring. The experimental results show that the proposed method is capable of obtaining the 3D shape of the surface of a high-reflective object with fewer patterns and high measurement accuracy.

Optimal multiple fringe pattern composition by weighted complex amplitude fusion in fringe projection profilometry

This paper presents an optimal multiple fringe pattern composition method for 3D shape measurement of high-dynamic-range (HDR) objects using fringe projection profilometry (FPP). With the inverse variance weighting theory, we take the square of the modulation intensities of the fringe pattern images with different intensity levels as the weights to obtain the composited phase of fringe patterns by weighted complex amplitude fusion, which improves the measurement precision of HDR objects. Additionally, we integrate HDR 3D shape measurement and temporal noise reduction into a unified framework by utilizing weighted complex amplitude fusion to completely measure translucent objects with specular reflections. Simulations and experiments demonstrate that our method can achieve higher measurement precision and is resistant to the time-varying ambient light.

High-precision composite 3D shape measurement of aeroengine blade based on parallel single-pixel imaging and high-dynamic range N-step fringe projection profilometry

Aeroengine blade is a core component of aircrafts. Accurate measurement of its profile has a great significance for flight safety. However, obtaining a high-quality 3D model of blade is difficult because traditional methods are limited in simultaneously meeting the requirements of large-scale measurement of smooth areas and high-precision measurement of fine structures. We develop a high-precision composite 3D shape measurement method of aeroengine blade. The proposed system consists of two sensors: a global sensor for large-range measurement and a local sensor for high-precision measurement. The global sensor based on parallel single-pixel imaging is used for smooth areas such as blade body and back, which undertakes most of the measurement tasks. The local sensor based on high-dynamic range N-step fringe projection profilometry has a high resolution to measure fine structures such as blade edge and top. We also use the local sensor to complete the missing point cloud of strong reflective areas. A phase-based data alignment method is developed to obtain coordinate transformation between the global and local sensors. Experiment results demonstrate that the proposed method can achieve accurate measurement and finally reconstruct the high-quality 3D model of aeroengine blade.

Automatic exposure control method for 3D shape measurement of specular surface based on digital fringe projection

Digital fringe projection (DFP) is widely applied in three-dimensional (3D) shape measurements. However, its performance is severely disturbed while measuring complex surfaces with an extensive range of reflectivity. In this paper, to enhance the underexposed regions and reduce the saturated regions, an automatic and effective method is proposed for the multi-exposure determination of the DFP system. During the whole process, only one unsaturated image must be captured preliminarily, which could establish an adaptive strategy based on the intensity value distribution function. Raw absolute phase maps captured with different exposure times are synthesized at the pixel level for phase retrieval and 3D reconstruction. To evaluate the effectiveness of our method, a high dynamic range method and a global optimal exposure method are introduced for comparison. Verification experiments demonstrate that our method performs better for the 3D reconstruction results of complex surfaces with a large reflectivity range.

A 3D shape measurement method for high-reflective surface based on dual-view multi-intensity projection

Phase shifting profilometry has been commonly used in three-dimensional shape measurement with the advantages of high-precision and non-contact. However, it is still challenging to measure high-reflective surface because image saturation will lead to absolute phase errors and reconstruction errors. In this paper, a dual-view multi-intensity projection method was proposed. Compared with the single-view method, the proposed method can reconstruct more points at each projection intensity especially for pixels around the specular angle and reduce the number of projections to reduce the time consumption. First, we established the dual-view structured light system consisting of two monocular systems that share the same projector. Subsequently, a dual-view saturated pixel judging method was proposed that enables the reconstruction results under two views to be combined without duplicate points. The multi-intensity projection method was adopted by reducing the input projection intensity step by step and reconstructing the remaining pixels around the specular angle. Finally, the reconstruction result can be obtained by stitching point clouds at each projection intensity. Experiments verified that the proposed method could improve the integrity of reconstructed point clouds and measurement efficiency.

Lightweight and edge-preserving speckle matching network for precise single-shot 3D shape measurement

Single-shot three-dimensional (3D) shape measurement method based on digital speckle correlation (DSC) has high measurement precision and only requires one pair of speckle images. Nevertheless, the matching of speckles is time-consuming and may lead to some data defects at large gradient variations and edge regions. This paper proposes an end-to-end speckle matching network to achieve fast, precise, and edge-preserved 3D measurement. Compared with other networks, the proposed network reduces the redundant channels in the Siamese feature-extraction subnetwork and constructs cost volume by group-wise correlation, making it possible to provide efficient representations at low computational consumption, thus being suitable for high-precision 3D measurement in large resolution. A lightweight 3D stacked hourglass subnetwork is used to aggregate cost information and furthermore, an edge refinement module is integrated by leveraging the left speckle image as a guide to regular edge details. A reliable speckle binocular dataset is prepared with a specialized device in a simple procedure and the labels are produced from the DSC method. Experiments demonstrate that the proposed network can produce dense disparity maps with subpixel precision and reduce single-shot inference time to 0.13 s with less GPU occupation, which is a significant improvement compared with the traditional DSC method and other learning-based methods. Moreover, the experiment validates that measurement precision of the reconstructed plane can reach less than 0.03 mm, meeting the requirements of most industrial applications.

An Efficient Polarization Code-Multiplexed 3-D Shape Measurement Method for High-Dynamic Range Objects

An Efficient Polarization Code-Multiplexed 3-D Shape Measurement Method for High-Dynamic Range Objects

Three-dimensional shape measurement method based on composite cyclic phase coding.

Phase coding is widely used in 3D measurement due to its good anti-interference and robustness. However, the measurement accuracy is affected by the limitation of the number of codewords. To solve this problem, we propose a 3D shape measurement method based on composite cyclic phase coding. The traditional phase coding is quantized cyclic without adding extra patterns, further adopting composite coding, using the composite cyclic phase coding grayscale values to distinguish the same cyclic codewords, and finally integrating them into a new fringe order sequentially for phase unwrapping to achieve effective expansion of codewords. The related experimental results show that the proposed method stably achieves high accuracy 3D reconstruction, which overcomes the misjudgment of codewords caused by traditional phase coding under high-frequency fringes due to system nonlinearity and noise. Meanwhile, compared with the improved phase coding method of temporal domain combined with spatial domain information, such as the method of quantized phase coding and connected region labeling, it can effectively avoid the phenomenon of error propagation, with high robustness and low algorithm complexity.

Omnidirectional 3D shape measurement using image outlines reconstructed from Gabor digital holography

We propose an omnidirectional 3D shape measurement method using the contour information of an object from Gabor holograms captured in omnidirectional measurements. To obtain the correct contour information from Gabor holograms, an edge-enhanced image reconstruction method is developed. The 3D shape can be restored by the rigid-body transformation of the contour information acquired from all directions. Omnidirectional 3D shape measurements are conducted using an aspheric lens and a spiral-shaped wire. The proposed method can successfully restore the complex 3D shapes of objects.

High-efficiency and robust binary fringe optimization for superfast 3D shape measurement.

By utilizing 1-bit binary fringe patterns instead of conventional 8-bit sinusoidal patterns, binary defocusing techniques have been successfully applied for high-speed 3D shape measurement. However, simultaneously achieving high accuracy and high speed remains challenging. To overcome this limitation, we propose a high-efficiency and robust binary fringe optimization method for superfast 3D shape measurement, which consists of 1D optimization and 2D modulation. Specifically, for 1D optimization, the three-level OPWM technique is introduced for high-order harmonics elimination, and an optimization framework is presented for generating the 'best' three-level OPWM pattern especially for large fringe periods. For 2D modulation, a single-pattern three-level OPWM strategy is proposed by utilizing all the dimensions for intensity modulation to decrease the required projection patterns. Thus, the proposed method essentially belongs to the 2D modulation technique, yet iterative optimization is carried out along one dimension, which drastically improves the computational efficiency while ensuring high accuracy. With only one set of optimized patterns, both simulations and experiments demonstrate that high-quality phase maps can be consistently generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) and different amounts of defocusing, and it can achieve superfast and high-accuracy 3D shape measurement.

A 3D shape measurement method for high-reflective surface based on accurate adaptive fringe projection

Fringe projection profilometry (FPP) is a popular three-dimensional (3D) shape measurement technique with advantages of high-precision and non-contact. However, measuring high-reflective surface is still a challenging task for conventional FPP because image saturation leads to absolute phase errors and reconstruction errors. In this paper, a new adaptive fringe projection (AFP) method is proposed. Compared with existing AFP methods, we simplify the experimental process and get more accurate coordinate mapping. First, we built the projection intensity model for each projector pixel rather than camera pixel. Then, a uniform gray pattern was projected and two images were captured under high and low exposures respectively to calculate the quantitative low projection intensity. In coordinate mapping, sub-pixel coordinates were mapped from camera image onto projector image. They were filtered with absolute phase to deal with resolution difference between camera and projector and get accurate coordinate correspondence. Finally, adaptive fringe patterns were generated. Besides, we proposed a novel quantitative criterion to evaluate the effectiveness of AFP methods called point cloud integrality (PCI), which is calculated based on the number of 3D points of high-reflective areas. The experiments demonstrate that the proposed method increases both the measurement accuracy and PCI compared with conventional FPP and some existing AFP methods.

Advances and Prospects of Vision-Based 3D Shape Measurement Methods

Vision-based three-dimensional (3D) shape measurement techniques have been widely applied over the past decades in numerous applications due to their characteristics of high precision, high efficiency and non-contact. Recently, great advances in computing devices and artificial intelligence have facilitated the development of vision-based measurement technology. This paper mainly focuses on state-of-the-art vision-based methods that can perform 3D shape measurement with high precision and high resolution. Specifically, the basic principles and typical techniques of triangulation-based measurement methods as well as their advantages and limitations are elaborated, and the learning-based techniques used for 3D vision measurement are enumerated. Finally, the advances of, and the prospects for, further improvement of vision-based 3D shape measurement techniques are proposed.

Improved dual-frequency phase-coding fringe projection method for 3D shape measurement

The phase-coding method is useful in 3D shape measurement due to its low sensitivity to surface contrast and camera noise. However, as the number of fringe periods increases, the wrapped phase recovered from the sinusoidal phase-shifting patterns is always misaligned with the fringe order calculated by the stair-coding patterns. To solve this, an improved dual-frequency phase-coding fringe projection method is designed to realize absolute phase retrieval. Specifically, the auxiliary fringe order determined by phase-shifting is used to get the absolute phase, which can eliminate fringe-order errors. Moreover, to increase speed and the number of codewords, this paper presents a scheme that only requires seven patterns, including a uniform gray image, two high-frequency sinusoidal fringe patterns, two low-frequency sinusoidal fringe patterns and two phase-coding fringe patterns. Therefore, this method requires fewer patterns to realize high-resolution measurement than existing methods. The success of the proposed method can be proved by experiments.

High-speed three-dimensional shape measurement with inner shifting-phase fringe projection profilometry

Fringe projection profilometry (FPP) has been extensively studied in the field of three-dimensional (3D) measurement. Although FPP always uses high-frequency fringes to ensure high measurement accuracy, too many patterns are projected to unwrap the phase, which affects the speed of 3D reconstruction. We propose a high-speed 3D shape measurement method using only three high-frequency inner shifting-phase patterns (70 periods), which satisfies both high precision and high measuring speed requirements. Besides, our proposed method obtains the wrapped phase and the fringe order simultaneously without any other information and constraints. The proposed method has successfully reconstructed moving objects with high speed at the camera’s full frame rate (1700 frames per second).

3D shape measurement method for high-reflection surface based on fringe projection.

3D measurement methods based on fringe projection have attracted extensive research. However, it is a challenge to deal with overshooting on a high-reflection or specular surface. To eliminate the saturated pixels caused by overshooting, we propose a projection intensity adaptive adjustment method. First, we project three uniform gray-level images and estimate the projection intensity of the measured surface through the captured uniform gray-level images. Then we can obtain the optimal projection fringes in the camera coordinate system. Second, a set of horizontal and vertical gray-coded patterns are used to establish a coordinate matching relationship between the projected image and the captured image. To check the decoding result of the gray-coded patterns, a set of horizontal and vertical sinusoidal fringes are used to calculate the high-reflection mapping area (HRMA) in the projector coordinate system. Through the distribution of HRMA, we can check whether the decoding is reliable or not. Finally, we project the optimal intensity fringes and obtain the measurement results. We develop a measurement system to verify the validity of the proposed method. Experimental results show that the proposed method can effectively avoid overshooting and obtain measurement results with a minimum rms error.

High-efficiency dynamic three-dimensional shape measurement based on misaligned Gray-code light

The combination of Gray-code method with phase-shifting algorithm has been widely used in three-dimensional (3D) shape measurement. However, the non-uniform surface reflectance, the defocus of the projector and the object motion will cause order jump error at the boundary of adjacent Gray-code. This paper proposes a 3D shape measurement method based on misaligned Gray-code (MGC) light to avoid this kind of error. In this method, all the traditional Gray-code patterns are pre-moved by half fringe period before projection so that the edges of the decoding order and the wrapped phase are staggered exactly, and then each period of decoding order is divided into four sub-regions to construct correct phase order and unwrap the wrapped phase. In addition, the fringe period number can be further expanded by introducing the virtual plane to improve the measurement efficiency and accuracy simultaneously. Two sets of comparative experiments on static scenes confirm that our method can successfully avoid the order jump error without projecting additional Gray-code pattern. And the dynamic experimental results demonstrate that proposed method can especially realize high-efficiency and high-speed 3D shape measurement at a rate of 2381 fps by labeling 32-period fringe patterns with only 4 misaligned Gray-code patterns.

PMENet: phase map enhancement for Fourier transform profilometry using deep learning

Fringe projection profilometry (FPP) is a three-dimensional (3D) shape measurement method that involves projecting fringe patterns onto the object. A phase map retrieved from these fringe images is used for reconstructing the 3D surface of the object. Fourier transform and phase-shifting are two of the widely used fringe analysis techniques for performing 3D shape measurement using FPP. Fourier transform profilometry (FTP) has the advantage of performing high-speed measurement due to its single-shot nature. However, the reconstructed 3D surface has artifacts and high noise, specifically on the edges. On the other hand, phase-shifting profilometry (PSP) has the advantage of higher accuracy (relatively less noise level) but compromises on measurement speed. In this research, we propose a deep learning method to enhance the quality of the FTP phase maps using an efficient deep learning model called phase map enhancement net (PMENet). PMENet takes an FTP phase map as input and predicts a high-quality phase map in a supervised manner by using the phase maps obtained from 18-step PSP as ground truth. The training dataset was generated using a virtual FPP system. Validations were conducted on both the simulated data (generated by virtual FPP system) and the real-world data. The experimental results demonstrate that the trained neural network model can successfully improve the quality of the 3D geometric reconstruction with FTP and reduced the mean and root-mean-square error by 66% and 43%, respectively.

Three-dimensional shape measurement based on spatial-temporal binary-coding method

Binary coding methods have been widely used for three-dimensional (3D) shape measurement. However, traditional binary coding methods always require a sequence of binary patterns to encode the fringe order. This paper presents a spatial-temporal binary coding (STBC) method for 3D shape measurement with three binary patterns, including a spatial pattern and two temporal patterns. Specifically, the spatial pattern is used to encode a 4-bit spatial code, and the temporal patterns are used to encode the 2-bit temporal code. By combining the spatial code and the temporal code, 6-bit code words can be obtained that distinguish fringe periods of sinusoidal phase-shifting patterns. A robust dual-region labeling decoding scheme is also developed to extract the 6-bit code word for fringe order determination, and then absolute-phase unwrapping can be carried out. Experiments have been conducted, and their results confirm that the STBC method can reliably reconstruct the 3D shapes of measured objects with only three binary patterns.

A Binocular Vision-Based 3D Sampling Moiré Method for Complex Shape Measurement

As a promising method for moiré processing, sampling moiré has attracted significant interest for binocular vision-based 3D measurement, which is widely used in many fields of science and engineering. However, one key problem of its 3D shape measurement is that the visual angle difference between the left and right cameras causes inconsistency of the fringe image carrier fields, resulting in the phase mismatch of sampling moiré. In this paper, we developed a phase correction method to solve this problem. After epipolar rectification and carrier phase introduction and correction, the absolute phase of the fringe images was obtained. A more universal 3D sampling moiré measurement can be achieved based on the phase match and binocular vision model. Our numerical simulation and experiment showed the high robustness and anti-noise ability of this new 3D sampling moiré method for high-precision 3D shape measurement. As an application, cantilever beams are fabricated by directed energy deposition (DED) using different process parameters, and their 3D deformation caused by residual stresses is measured, showing great potential for residual stress analyses during additive manufacturing.

Accurate and dynamic 3D shape measurement with digital image correlation-assisted phase shifting

Phase-shifting profilometry (PSP) has been widely used in structured-light (SL) systems for three-dimensional (3D) shape measurements, but the speed of the PSP technique is limited by increased phase-shifting patterns. This paper proposes an accurate and dynamic 3D shape measurement method by projecting only four patterns, including three-step phase-shifting patterns and one speckle pattern. Three-step phase-shifting images are used to obtain the initial unwrapped phase map with phase ambiguity. Based on the principle of digital image correlation and multi-view geometry, the absolute phase can be recovered reliably without requiring any embedded features or pre-defined information of the object. To improve the measurement accuracy, the projector coordinate is used as the measuring coordinate to establish a novel stereo SL system model. By solving a least square solution using the triple-view information, accurate 3D surface data can be reconstructed. The experimental results indicate that the proposed method can perform high-speed and accurate 3D shape measurements with an accuracy of 10.64 μm, which is superior to conventional methods and has certain instructive significance for 3D profilometry and measurement engineering.