Absolute Phase Map Research Articles

Spatial coding strategy for dual-frequency phase-shifting profilometry

Dual-frequency fringe projection techniques have been broadly utilized for three-dimensional (3D) shape recovery. However, a pervasive challenge across these methodologies lies in the precise implementation of high-speed measurements. This paper explores a spatial coding strategy designed for the dual-frequency phase unwrapping method to tackle this. The strategy mandates the utilization of only four patterns, encapsulating the phase information of both high and low frequencies. The codewords are strategically embedded within the low-frequency phase, facilitating the acquisition of an accurate absolute phase map. A robust decoding algorithm engineered to ensure dependable phase unwrapping, particularly in scenarios characterized by a substantial number of high-frequency cycles. The proposed method exhibits superior noise immunity and enhances accuracy when compared with the previous dual-frequency phase-shifting methodology. Simulations and experiments demonstrate the robustness and efficiency of the proposed strategy.

Deep learning-based frequency-multiplexing composite-fringe projection profilometry technique for one-shot 3D shape measurement

The pursuit of precise and efficient 3D shape measurement has long been a focal point within the fringe projection profilometry (FPP) domain. However, achieving precise 3D reconstruction for isolated objects from a single fringe image remains a challenging task in the field. In this paper, a deep learning-based frequency-multiplexing (FM) composite-fringe projection profilometry (DFCP) is proposed, where an end-to-end absolute phase retrieval network (APR-Net) is trained to directly recover the absolute phase map from a FM composite fringe pattern. The obtained absolute phase map exhibits exceptional precision and is devoid of spectrum crosstalk or leakage disturbance typically encountered in traditional FM techniques. APR-Net is intricately crafted, incorporating a nested strategy along with the concept of centralized information interaction. A diverse dataset encompassing various scenarios and free from spectrum aliasing is assembled to guide the training process. A seven-map loss calculation scheme is employed to guide the training process, of which the efficacy is proved through ablation experiments. In the first qualitative experiment, DFCP demonstrates comparable phase accuracy to ground truths with 47 fewer projected images, outperforming other three methods with mean absolute phase errors of 0.0052 rad, 0.1761 rad, 0.0169 rad, and 0.0139 rad. The second qualitative experiment and the quantitative evaluation respectively prove DFCP’s capability in high dynamic range 3D measurement and in precise 3D measurement, with sphere diameter errors of 0.0780 mm and 0.0726 mm, and a spherical centroid distance error of 0.0555 mm.

Single-shot 3D measurement of highly reflective objects with deep learning.

Three-dimensional (3D) measurement methods based on fringe projection profilometry (FPP) have been widely applied in industrial manufacturing. Most FPP methods adopt phase-shifting techniques and require multiple fringe images, thus having limited application in dynamic scenes. Moreover, industrial parts often have highly reflective areas leading to overexposure. In this work, a single-shot high dynamic range 3D measurement method combining FPP with deep learning is proposed. The proposed deep learning model includes two convolutional neural networks: exposure selection network (ExSNet) and fringe analysis network (FrANet). The ExSNet utilizes self-attention mechanism for enhancement of highly reflective areas leading to overexposure problem to achieve high dynamic range in single-shot 3D measurement. The FrANet consists of three modules to predict wrapped phase maps and absolute phase maps. A training strategy directly opting for best measurement accuracy is proposed. Experiments on a FPP system showed that the proposed method predicted accurate optimal exposure time under single-shot condition. A pair of moving standard spheres with overexposure was measured for quantitative evaluation. The proposed method reconstructed standard spheres over a large range of exposure level, where prediction errors for diameter were 73 µm (left) and 64 µm (right) and prediction error for center distance was 49 µm. Ablation study and comparison with other high dynamic range methods were also conducted.

High dynamic range 3D measurement technique based on adaptive fringe projection and curve fitting.

This paper proposes a technique combining an adaptive fringe projection algorithm and a curve fitting algorithm for the three-dimensional (3D) reconstruction of highly reflective objects. First, an adaptive projection algorithm is proposed to avoid image saturation. The phase information obtained by projecting vertical and horizontal fringes is used to establish the pixel coordinate mapping between the camera image and the projected image, and the highlight area in the camera image is found and linearly interpolated. By modifying the mapping coordinates of the highlight area, the optimal light intensity coefficient template of the projection image is calculated, the coefficient template is mapped to the projector image, and it is multiplied by the standard projection fringes to obtain the adaptive projection fringes we need. Secondly, after obtaining the absolute phase map, the phase at the hole is calculated by fitting the correct phase values at both ends of the data hole, and the phase value closest to the actual surface of the object is obtained by performing a fitting in the horizontal and vertical directions. Multiple experimental results show that the algorithm can reconstruct high-quality 3D shapes for highly reflective objects, with high adaptability and reliability in high dynamic range measurements.

Single-Shot Three-Dimensional Measurement by Fringe Analysis Network

Fringe projection profilometry (FPP) has been broadly applied in three-dimensional (3D) measurements, but the existing multi-shot methods, which mostly utilize phase-shifting techniques, are heavily affected by the disturbance of vibration and cannot be used in dynamic scenes. In this work, a single-shot 3D measurement method using a deep neural network named the Fringe Analysis Network (FrANet) is proposed. The FrANet is composed of a phase retrieval subnetwork, phase unwrapping subnetwork, and refinement subnetwork. The combination of multiple subnetworks can help to recover long-range information that is missing for a single U-Net. A two-stage training strategy in which the FrANet network is pre-trained using fringe pattern reprojection and fine-tuned using ground truth phase maps is designed. Such a training strategy lowers the number of ground truth phase maps in the data set, saves time during data collection, and maintains the accuracy of supervised methods in real-world setups. Experimental studies were carried out on a setup FPP system. In the test set, the mean absolute error (MAE) of the refined absolute phase maps was 0.0114 rad, and the root mean square error (RMSE) of the 3D reconstruction results was 0.67 mm. The accuracy of the proposed method in dynamic scenes was evaluated by measuring moving standard spheres. The measurement of the sphere diameter maintained a high accuracy of 84 μm at a speed of 0.759 m/s. Two-stage training only requires 8800 fringe images in data acquisition, while supervised methods require 96,000 fringe images for the same number of iterations. Ablation studies verified the effectiveness of two training stages and three subnetworks. The proposed method achieved accurate single-shot 3D measurements comparable to those obtained using supervised methods and has a high data efficiency. This enables the accurate 3D shape measurement of moving or vibrating objects in industrial manufacturing and allows for further exploration of network architecture and training strategy with few training samples for single-shot 3D measurement.

Absolute phase recovery with multiple-wavelength fringes

In a recent work, a technique was proposed to recover the absolute phase maps of fringe patterns with two or three selected fringe wavelengths with the aim to improve anti-error capability. To further increase the anti-error capability associated with the phase unwrapping operation, multiple fringe patterns with different wavelengths may also be used. In this paper, we present a general rule to select the wavelengths and the strategy to increase phase error tolerance bound. Based on this rule, it is concluded that to maximize the error tolerance, four-wavelength method should be selected, where the maximum upper bound of the tolerable phase error is π/6. To minimize the required number of fringe patterns, three-wavelength method with three-step PSP should be selected, that is exactly the case the previous paper has demonstrated. These theoretical conclusions are verified by experimental results.

Iteration of B-spline surface based deflectometric method for discontinuous specular surface

There is an over-growing demand for accurate three-dimension model of industrial components. In the measurement of specular surface, deflectometry has been widely used for its convenience and high accuracy. However, it is a challenge to measure discontinuous specular surface by the traditional deflectometry. In order to solve this problem, this paper proposes a deflectometric method based on iteration of control points of B-spline surface. In this method, a screen at two positions displays fringe patterns which will be reflected by specular surface under test and captured by a camera. According to the undistorted and absolute unwrapped phase map, a continuous zone containing all discontinuous area which is distinguished by a direct deflectometric method, is fitted by a B-spline surface with an estimated depth on the focus plane of the camera. Meanwhile, empty points and basis functions bridging isolated surface are removed. The control points of the fitted surface are solved by iterative calculation. With the solved control points, discontinuous specular surface is reconstructed. Simulated and actual experiments are carried out and the results show high accuracy and stability of the proposed method.

Absolute Phase Unwrapping for Objects With Large Depth Range

This paper presents a method for unwrapping the phase using only the geometric constraints and photometric information of the structured light system, which can cope with objects in a large depth range without the need for additional image acquisition or other cameras. Following the minimum phase method, this paper also establishes an artificial plane and generates an initial unwrapped phase map from the absolute phase map of the plane. Starting from the reference plane, we divide the space into several 2π intervals pixel by pixel. The boundaries at phase discontinuities in the initial unwrapped phase map are then used to segment the object into independent regions. Regarding the projector as a point light source, synthetic images of each region corresponding to different spatial 2π intervals are generated sequentially according to Lambert’s cosine law. The interval in which the region corresponds is identified by comparing the similarity of these synthetic images with the modulation image, leading to the acquisition of an entire absolute phase map of the object. Experiments are conducted to evaluate the performance of the proposed method.

Adaptive horizontal scaling method for speckle-assisted fringe projection profilometry.

Phase-shifting method is widely used in fringe projection profilometry to obtain high-precision wrapped phase maps. The wrapped phase map needs to be converted to an absolute phase map to recover 3D information. The speckle pattern based phase unwrapping method requires only one additional auxiliary pattern, showing great potential for fast 3D measurements. In this paper, a speckle assisted four-steps phase-shifting method was proposed for 3D measurements. This method requires five structured light patterns to complete 3D measurements, including four-steps phase-shifting fringe patterns and a speckle pattern which is used to remove phase ambiguity. The main challenge of speckle based phase unwrapping method is to overcome the mismatch problem which often occurs in some very steep surfaces. In order to improve the speckle matching accuracy, an adaptive horizontal scaling method was proposed. A voting strategy based on phase-connected regions was proposed to reduce the computational overhead. The experiments demonstrate its superior performance, and an accuracy of 0.21 mm was achieved.

Single-frequency and accurate phase unwrapping method using deep learning

Phase unwrapping is an important part of fringe projection profilometry(FPP), which greatly affects the efficiency and accuracy of reconstruction. Phase unwrapping methods with deep learning achieve single-frequency phase unwrapping without additional cameras. However, existing methods have low accuracy in the real complex scene, and can not process data whose resolution is greater than the resolution of training data. This paper introduces a neural convolutional network named as VRNet which achieves accurate and single-frequency phase unwrapping without extra cameras. VRNet with encoder-decoder structure gets multi-scale feature maps through feeding the wrapped phase map into the encoder, then fuses the feature maps recursively by using the proposed feature fusion module to accomplish precise prediction. In order to further improve the accuracy of phase unwrapping, this paper presents a phase correction method based on the distribution characteristics of the absolute phase. The method divides the cross-section of the absolute phase map into several curves and identifies a misclassified pixel by comparing its absolute phase value with the value of neighboring curves. In contrast to existing methods, the method is row-independent and does not require segmentation of image. Moreover, this paper accomplishes the prediction of high-resolution data through the phase stitching strategy and fine-tuning the phase correction method. Extensive experiments show that the proposed method is able to achieve high-accuracy and single-frequency phase unwrapping in real scenes which consist of at least one complex object, and is also effective for wrapped phase maps with a resolution larger than the training data.

Stereoscopic deflectometry with a curved screen.

Deflectometry has been widely used in topography measurement of specular surface. In deflectometry with a curved screen, the range of the gradient and height field of the measured specular surface can be effectively expanded compared to deflectometry with a plane screen. As stereo deflectometry measures gradient with high accuracy, the specular surface is reconstructed by integrating the gradient. In this paper, a stereo deflectometry with a curved screen is proposed in the aspect of system calibration and measuring principle. A pair of cameras and deflectometric system are calibrated simultaneously to obtain the camera parameters and relationship between two cameras and the curved screen. Then, pairs of rectified fringe patterns are demodulated to obtain two pairs of rectified absolute phase maps and homologous points are searched along epipolar line with the principle of the same normal direction on specular surface. Finally, simulated and actual experiments are carried out and the results show high accuracy and stability.

Phase unwrapping algorithm based on phase edge tracking for dynamic measurement.

Phase unwrapping is an essential procedure for fringe projection profilometry (FPP). To improve measurement efficiency and reduce phase unwrapping errors (PUEs) in dynamic measurement, a phase unwrapping algorithm based on phase edge tracking is proposed, which unwraps the current wrapped phase map with the aid of the previously unwrapped one. The phase edges are accurately tracked and their trajectories are used to divide the phase map into several regions, each of which is unwrapped either temporally or spatially according to its properties. It doesn't require extra patterns for phase unwrapping once the initial unwrapped phase map is obtained, thus significantly increasing the frame rate of the 3D result. Meanwhile, it greatly reduces the PUEs caused by noise amplification and motion-induced misalignment of phase edges. Experiments prove that it is capable of retrieving the absolute phase maps of complex dynamic scenes with high unwrapping accuracy and efficiency.

A general phase ambiguity suppression algorithm combining geometric constraints and temporal phase unwrapping

In fringe projection profilometry, various sophisticated temporal phase unwrapping (TPU) methods are conceived to recover a continuous absolute phase map. However, phase ambiguity is a thorny problem that affects measurement accuracy, which is deeply troubled by researchers. Dealing with the challenge, a universal algorithm (G-TPU) uniting geometric constraints phase unwrapping (GCPU) and TPU is proposed to comb out the phase ambiguity. Firstly, The invalid phase map or continuous relative phase map is unwrapped by GCPU with the minimum artifact. Then the unwrapped phas map is divided into several object regions, and the effective phase range (EPR) and invalid phase range (IPR) of each object region are determined by the proposed edge determination and image filling algorithm. Finally, through the statistical analysis of the decoded codewords, the relative orders are added to each EPR and IPR, and the absolute phase map is obtained. The proposed method inherits the complementary advantages of absolute position determination and strong robustness in TPU and GCPU respectively. Since GCPU does not need additional fringe patterns, it can be perfectly integrated into all TPU methods. The experimental results demonstrate the success of the proposed method combining GCPU and TPU.

Quasi-Periodic Phase Coding for Long-Depth-Range 3-D Measurements of Moving Objects

Fringe projection profilometry (FPP) requires an auxiliary projection sequence or spatial landmark to remove phase ambiguity, thereby severely limiting the measurement speed or depth range for moving objects. In this article, we propose a novel quasi-periodic phase coding method for long-depth-range 3-D measurements of moving objects without temporal or spatial limitations. This article mainly focuses on the construction of the encoding schema for quasi-periodic fringe patterns and the execution of the phase decoding framework for long-depth-range 3-D measurements of moving surfaces. First, a set of stripes with similar periods is designed on the basis of a preset equivalent period, and the various combinations of these stripes constitute quasi-periodic fringe patterns. Second, we present a progressive phase recovery strategy for decoding quasi-periodic fringe images, in which the wrapped phase is converted first into a continuous phase distribution and then into the absolute phase. In this strategy, the absolute phase map is effectively screened via the piecewise linear transformation from the quasi-periodic phase to the equivalent phase. Compared with the existing FPP methods, the proposed method avoids the low-frame-rate constraint and the need for landmark configuration. Simulation and experimental results verify that the proposed method is applicable for dynamic 3-D measurements of moving objects over a large-depth range.

Model-free method for intrinsic camera parameters calibration and lens distortion correction

Although various camera calibration methods have been proposed, most of these methods cannot deliver accurate determination of the intrinsic camera parameters, because of the coupling errors existing between intrinsic and extrinsic camera parameters and the use of explicit distortion models. Here, we propose a model-free method for accurate camera calibration, which utilizes phase-shifted fringe patterns shown on a liquid crystal display (LCD) screen for estimating the intrinsic parameters of a camera and correcting lens distortion. Horizontal and vertical fringe patterns are consecutively displayed on the LCD screen, which is placed at two positions parallel to the camera sensor plane. These fringe patterns are captured by the camera from the front viewpoint for calculating the absolute phase maps. Then, the images with lens distortion can be transformed to the distortion-free images using an inverse mapping operation. Subsequently, the principal point coordinates are calculated according to the geometric imaging relationship between the positions of the two LCD screens. Finally, the focal length can be estimated using the similarity of triangles formed by the obtained principal point coordinates and the obtained phase maps at the two positions. Both simulated and real experiments are performed to verify the validity of the proposed method. The results demonstrate that this method not only successfully eliminates coupling errors but also perfectly corrects lens distortion.

Enhanced geometric constraint-based phase unwrapping algorithm in binocular stereo vision fringe projection system

Phase unwrapping plays an important and central role in phase-based digital fringe projection profilometry. The unwrapping quality directly influences the three-dimensional measurement accuracy. Recently, an effective geometric constraint-based phase unwrapping algorithm has been proposed to obtain the continuous absolute phase map and the unwrapped phase accuracy was found to be high. However, in this technique the virtual depth plane at z = zmin is often created empirically, which increases the manual measurement error. For this reason, this paper proposes a method for accurately constructing the virtual plane and further applies it to phase unwrapping of objects with a larger depth range. In this method, a binocular stereo vision system is used as the measurement set-up for the virtual depth plane construction and a series of virtual depth planes at z = zimin (i ≥ 2) is automatically built using a computational framework. Then, the phase is unwrapped for each region according to the continuity of the unwrapped phase and a complete absolute phase map is obtained by merging the unwrapped phases in all regions for 3D reconstruction. In this process, the virtual depth planes are created automatically and quantitatively by the measurement system. No human intervention is required and it greatly reduces the manual measurement error. Experiments show that the artificial virtual planes can be built accurately and the phase is unwrapped correctly and readily.

A phase unwrapping method suitable for high frequency fringe based on edge feature

Phase unwrapping (PU) technology is an important component of fringe projection profilometry (FPP). The key of PU is to obtain accurate fringe order. The existing PU methods are hard to make a tradeoff between efficiency and accuracy. To reduce the number of stripes, a phase unwrapping method based on edge feature (EFPU) is proposed. For this method, we make full use of the properties of wrapped phase map rather than auxiliary fringe patterns. We firstly analyze an edge detection operator suitable for wrapped phase map. Furthermore, in order to improve the stability of the proposed method, the edge correction method is proposed to correct edge point and the reverse correction method is presented to further optimize the fringe order. The experimental results show that the absolute phase map obtained by EFPU is similar to that of multi-wavelength phase-shifting (MWPS) method, but the number of fringe patterns is only one-third of MWPS.

High-speed and accurate 3D shape measurement using DIC-assisted phase matching and triple-scanning

Structured-light (SL) three-dimensional (3D) shape measurements technique has been extensively studied and applied in various fields, but the real-time capability of SL system is limited by the number of projected SL patterns. This paper presents a digital image correlation (DIC) assisted phase matching and triple-scanning method for high-speed and accurate 3D shape measurements based on the combined projection of three-step phase-shifting patterns and one speckle pattern. Benefiting from the triple-view geometric constraints and DIC technique, stereo correspondence and absolute fringe order of each valid pixel in the left wrapped phase map can be explored without requiring spatial phase unwrapping process or any priori information, the initial disparity map and absolute phase map can be generated. Subsequently, an efficient disparity refinement scheme based on connectivity area segmentation and neighbor disparity check is applied to the initial disparity map and absolute phase map to compensate the pixels with wrong disparities. Several efficient strategies are developed in the pipeline of data processing to speed up the algorithm. To improve the measurement accuracy, stereo SL model is proposed to make adequate use of triple-view information to calculate 3D coordinates using the disparity map and absolute phase map, and the measurement error has been reduced by 33% compared with the conventional methods. The experimental results demonstrate that the proposed method can perform high-speed and accurate 3D shape measurements for arbitrary surfaces and dynamic scenes.

Single-shot 3D shape measurement using an end-to-end stereo matching network for speckle projection profilometry.

Speckle projection profilometry (SPP), which establishes the global correspondences between stereo images by projecting only a single speckle pattern, has the advantage of single-shot 3D reconstruction. Nevertheless, SPP suffers from the low matching accuracy of traditional stereo matching algorithms, which fundamentally limits its 3D measurement accuracy. In this work, we propose a single-shot 3D shape measurement method using an end-to-end stereo matching network for SPP. To build a high-quality SPP dataset for training the network, by combining phase-shifting profilometry (PSP) and temporal phase unwrapping techniques, high-precision absolute phase maps can be obtained to generate accurate and dense disparity maps with high completeness as the ground truth by phase matching. For the architecture of the network, a multi-scale residual subnetwork is first leveraged to synchronously extract compact feature tensors with 1/4 resolution from speckle images for constructing the 4D cost volume. Considering that the cost filtering based on 3D convolution is computationally costly, a lightweight 3D U-net network is proposed to implement efficient 4D cost aggregation. In addition, because the disparity maps in the SPP dataset should have valid values only in the foreground, a simple and fast saliency detection network is integrated to avoid predicting the invalid pixels in the occlusions and background regions, thereby implicitly enhancing the matching accuracy for valid pixels. Experiment results demonstrated that the proposed method improves the matching accuracy by about 50% significantly compared with traditional stereo matching methods. Consequently, our method achieves fast and absolute 3D shape measurement with an accuracy of about 100µm through a single speckle pattern.

Shading-based absolute phase unwrapping.

Absolute phase unwrapping in the phase-shifting profilometry (PSP) is significant for dynamic 3-D measurements over a large depth range. Among traditional phase unwrapping methods, spatial phase unwrapping can only retrieve a relative phase map, and temporal phase unwrapping requires auxiliary projection sequences. We propose a shading-based absolute phase unwrapping (SAPU) framework for in situ 3-D measurements without additional projection patterns. First, the wrapped phase map is calculated from three captured images. Then, the continuous relative phase map is obtained using the phase histogram check (PHC), from which the absolute phase map candidates are derived with different fringe orders. Finally, the correct absolute phase map candidate can be determined without additional patterns or spatial references by applying the shading matching check (SMC). The experimental results demonstrate the validity of the proposed method.