Fringe Projection Profilometry Research Articles

Comparative study of variant shifting-phase coding method based on noise error model

Phase shifting algorithm, which is well-known for its high-accuracy and high-resolution, plays a crucial role in fringe projection profilometry. A class of variant shifting-phase coding method (VSPCM) has been proposed for high-speed 3D measurement. The three kinds of VSPCM include the inner shifting-phase method (ISPM), the intensity-coded method (ICM) and the modified Carré algorithm (MCA). These methods project only four patterns avoiding the need for extra images, yet achieve accuracy comparable to the classical three-frequency heterodyne method. In VSPCM, the shifting-phase, as a key code-word for phase unwrapping, is encoded as a variable value. In practice, the noise and the encoding of shifting-phase have varying degrees of impact on the performance of these methods. But the coded ranges of the shifting-phase were previouly empirical values. Besides, there are no studies related to these factors, and also a lack of comparative researches on such methods. This paper compares three kinds of VSPCM for the first time. We derive the noise error models for shifting-phase and wrapped phase in three methods and examine the impact of variant shifting-phase on fringe orders and wrapped phase in these methods. We find that the abnormal shifting-phase errors predominantly cause fringe orders errors. Fortunately, median filtering can effectively handle shifting-phase errors to ensure successful phase unwrapping. To improve their accuracy, we also deduce that the optimal shifting-phase encoding range is [−77°, 77°] for ISPM, and is [−61°, 61°] for ICM, and is [34°, 74°] for MCA. For wrapped phase errors, the distributions of ICM and MCA exhibit a U-shaped distribution, while ISPM demonstrates a uniform distribution. Based on theoretical analysis and experiment results, this paper gives recommendations for better use of these methods to boost their accuracy and offers valuable guidance for selecting an appropriate VSPCM under the specific measurement requirements.

A robust phase unwrapping method for phase shifting coding high-frequency color fringe projection profilometry

Phase unwrapping is one of the key problems in fringe projection profilometry (FPP). How to achieve unambiguous fringe step order to perform phase unwrapping with few patterns still face a major challenge. We propose a robust phase unwrapping method by using a pair of encoded color fringe patterns, one is a conventional three-step phase-shifting color fringe pattern with equal phase shifts, another is added an additional phase shift valve t(k) to the former, in which t(k) is related to the order of fringe periods. A single-valued solution function for t(k) is constructed and used to determine the unique fringe order. Besides, multiplexing this pair of fringes can help denoise the extracted wrapped phase to improve the accuracy. The simulation and experimental results proved that the proposed method can achieve unambiguous phase unwrapping for high-frequency fringe patterns. The proposed method is also applicable for isolated objects.

Direct fringe projection profilometry based on the spatial frequency distribution function in optical 3D imaging

Phase unwrapping has been a big hurdle in conventional optical three-dimensional (3D) measurement technologies, badly affecting speed and accuracy. To skip the complicated step of phase unwrapping, a direct fringe projection profilometry based on the spatial frequency distribution function is presented in this paper. First, the spatial frequency distribution function is defined according to the physical meaning of spatial frequency. Then, direct phase-shift method and direct Fourier transform profilometry are developed respectively with reference to the conventional fringe projection profilometry. Finally, both numerical simulations and experiments were carried out to test the proposed direct fringe projection profilometry. The direct phase-shift method without phase unwrapping combines the advantages of the conventional in terms of high accuracy, while the direct Fourier transform method does not get rid of the drawbacks of the conventional in spectral aliasing although it skips the phase unwrapping procedure.

Color crosstalk compensation method for color phase-shifting fringe projection profilometry based on the phase correction matrix.

Color phase-shifting fringe projection profilometry is one of the single-shot three-dimensional shape measurement techniques. The color crosstalk of the projector-camera system yields undesired phase errors when using phase-shifting method. In this paper, a color crosstalk compensation method based on phase correction matrix is proposed. In this method, the phase correction matrix is established to compensate the deviations between the actual phase-shift values in the acquired fringes and the standard ones in the ideal fringes. Only two fringe patterns are utilized to obtain the phase correction matrix. The quadratic equations for calculating the actual phase-shift values of the fringes in the three color channels are derived. The actual phase-shift values and the corresponding standard ones are employed to form the equilibrium equations for computing the phase correction coefficients in the matrix. Experimental results demonstrate the feasibility of the proposed method and it can effectively reduce the induced overall phase error caused by the color crosstalk.

Phase correction strategy based on structured light fringe projection profilometry.

Fringe projection profilometry based on structured light has been widely used in 3-D vision due to its advantages of simple structure, good robustness, and high speed. The principle of this technique is to project multiple orders of stripes on the object, and the camera captures the deformed stripe map. Phase unwrapping and depth map calculation are important steps. Still, in actual situations, phase ambiguity is prone to occur at the edges of the object. In this paper, an adaptive phase segmentation and correction (APSC) method after phase unwrapping is proposed. In order to effectively distinguish the stable area and unstable area of the phase, a boundary identification method is proposed to obtain the structural mask of the phase. A phase compensation method is proposed to improve the phase accuracy. Finally, we obtain the 3-D reconstruction result based on the corrected phase. Specific experimental results verify the feasibility and effectiveness of this method.

Weakly supervised phase unwrapping for single-camera fringe projection profilometry

Phase unwrapping is essential in fringe projection profilometry (FPP). FPP's primary focus is to simultaneously realize high-speed, high-precision three-dimensional (3D) reconstruction. The dual-frequency hierarchical temporal phase unwrapping method (DF-TPU) technique is one prominent technique for accomplishing this goal. However, phase errors are inevitable and typically limit the DF-TPU approach's maximal period number, reducing reconstruction accuracy. The fully supervised approaches based on deep learning can unwrap dual-frequency phase maps of single-camera FPP systems but require accurate labels of high-frequency absolute phase. This study proposes a novel deep learning phase unwrapping scheme suited to single-camera FPP setups. Without the need for accurate labels corresponding to the high-frequency phase maps, supervisory signals are extracted from the high-frequency phase map, the one-period phase map, and the geometry constraint to guide the training of the phase unwrapping network. This training is a weakly supervised learning, which makes the time-consuming and expertise-demanding precise labeling unnecessary. Data collected in multiple real-world circumstances, such as motion blur, isolated objects, non-uniform reflectivity, and phase discontinuities, are used to validate the effectiveness of this weakly supervised method. Experimental results demonstrate that the proposed method outperforms the fully supervised and conventional DF-TPU methods in terms of depth accuracy and measurement efficiency in terms of fringe pattern number.

Depth acquisition from dual-frequency fringes based on end-to-end learning

The end-to-end networks have been successfully applied in fringe projection profilometry in recent years for their high flexibility and fast speed. Most of them can predict the depth map from a single fringe. But the depth map inherits the fringe fluctuation and loses the local details of the measured object. To address this issue, an end-to-end network based on double spatially frequency fringes (dual-frequency based depth acquisition network) is proposed. To release the periodic error of the predicted depth map, a dual-branch structure is designed to learn the global contour and local details of the measured object from dual-frequency patterns. To fully exploit the contextual information of the fringe patterns, five novel modules are proposed to accomplish feature extraction, down-sampling/up-sampling, and information feeding. Ablation experiments verify the effectiveness of the presented modules. Competitive experiments demonstrate that the proposed lightweight network presents higher accuracy compared to the existing end-to-end learning algorithms. Noise immunity test and physical validation demonstrate the generalization of the network.

Flexible and accurate system calibration method in microscopic fringe projection profilometry.

The three-dimensional (3D) measurement task of complex microstructures holds paramount significance in the domains of precision manufacturing and inspection. The calibration of the 3D system heavily determines the final reconstruction accuracy. The widely adopted system calibration method is phase-height mapping (PHM) and stereo vision (SV) based. The former can be applied directly to the calculation without considering the imaging model of the system, but it relies on highly precise and expensive translation stages or standard blocks. The latter's accuracy cannot be guaranteed because it is difficult to accurately calibrate the projector. In this paper, we establish an optically coupled microscopic fringe projection profilometry system that consists of a Scheimpflug pinhole projector and a super-low distortion bi-telecentric camera. We introduce a simplified 3D system calibration approach that combines phase modulation transfer and ray propagation. Our method enables the simultaneous calibration of the system, including the calibration of the projector, camera, and the phase to a 3D coordinates relationship, using only a 2D target. The calibrated projector's external parameters are used to obtain the target's complete poses, and then the direct mapping coefficients of the phase to the 3D coordinates can be obtained through the optical geometry structure and phase labels. Comparable experiments verify the feasibility of the proposed method.

Defocus-Driven Variable-Shape Fringe for Accuracy Improvement in Fringe Projection Profilometry

Defocus-Driven Variable-Shape Fringe for Accuracy Improvement in Fringe Projection Profilometry

Generalized Technique for Separating Nonsinusoidal Errors in Fringe Projection Profilometry With Arbitrary Phase Shifts

Generalized Technique for Separating Nonsinusoidal Errors in Fringe Projection Profilometry With Arbitrary Phase Shifts

Novel calibration technique for hybrid structured-light three-dimensional measurement system

Hybrid structured-light measurement technique is used for form measurement of structured composite surfaces. A hybrid structured-light measurement system contains a phase measuring deflectometry (PMD) subsystem and a fringe projection profilometry (FPP) subsystem. Each subsystem measures specular surfaces and rough surfaces based on structured-light reflection and projection principle, respectively. Calibration’s accuracy extremely effects data stitching precision between the subsystems. A novel calibration technique is explored for the hybrid structured-light system to complete reliable measurement accuracy. Calibration algorithms are developed based on designed calibration targets. Information of the calibration procedure are discussed and presented. Effectiveness of the proposed calibration technique has been conducted and verified through experiments by measuring structured composite samples. Experimental results demonstrate that the proposed technique can significantly improve data fusion accuracy of a hybrid structured-light measurement system.

A Method for Suppressing Saturation-Induced Errors in Fringe Projection Profilometry by Fringe Restoration Assisted via Euler Formula-Based Method and Intersection Points

A Method for Suppressing Saturation-Induced Errors in Fringe Projection Profilometry by Fringe Restoration Assisted via Euler Formula-Based Method and Intersection Points

Accurate 3D Measurement of Complex Texture Objects by Height Compensation Using a Dual-Projector Structure.

Fringe projection profilometry is a widely used technique for 3D measurement due to its high accuracy and speed. However, the accuracy significantly decreases when measuring complex texture objects, especially in the junction of different colors. This paper analyzes the causes of errors resulting from complex textures and proposes a height compensation method to revise the error by employing a dual-projector structure. Moreover, the dual-projector is capable of acquiring a pair of errors with opposite signs, which can be utilized to calculate the accurate 3D information after determining the ratio of this pair of errors. Experiments provide significant improvement in measuring complex texture objects, demonstrating the proposed method's ability.

Error Model and Concise Temporal Network for Indirect Illumination in 3D Reconstruction.

3D reconstruction is a fundamental task in robotics and AI, providing a prerequisite for many related applications. Fringe projection profilometry is an efficient and non-contact method for generating 3D point clouds out of 2D images. However, during the actual measurement, it is inevitable to experiment with translucent objects, such as skin, marble, and fruit. Indirect illumination from these objects has substantially compromised the precision of 3D reconstruction via the contamination of 2D images. This paper presents a fast and accurate approach to correct for indirect illumination. The essential idea is to design a highly suitable network architecture founded on a precise error model that facilitates accurate error rectification. Initially, our method transforms the error generated by indirect illumination into a sine series. Based on this error model, the multilayer perceptron is more effective in error correction than traditional methods and convolutional neural networks. Our network was trained solely on simulated data but was tested on authentic images. Three sets of experiments, including two sets of comparison experiments, indicate that the designed network can efficiently rectify the error induced by indirect illumination.

High-Speed Phase Structured Light Integrated Architecture on FPGA

Phase-shifting fringe projection profilometry (FPP) in three-dimension (3D) structured light measurement is widely applied in high-accuracy 3D shape measurement scenarios. But motion objects or other dynamic scenarios make the traditional phase-shifting FPP hard to deal with due to its sizable CPU-based execution time. Therefore, an embedded high-speed and high-accuracy phase-shifting FPP system is designed and implemented on FPGAs. The system implements a three-frequency four-step phase-shifting hardware algorithm with RTL-level caching, multi-frame image ping-pong buffering, and data handling, etc. The system exhibits fully-parallel and pipeline processing while behaving with lower resource and power consumption. Measurement experiments indicate that the proposed method can reach high-accuracy real-time 3D shape measurement with a 3D decoding speed of 40.15 fps when the clock frequency is 200MHz, and the image size is 2432×2048.

A Novel In Situ Calibration Method Based on Virtual Planes for Fringe Projection Profilometry

A Novel In Situ Calibration Method Based on Virtual Planes for Fringe Projection Profilometry

Deep-Learning-Based Single-Shot Fringe Projection Profilometry Using Spatial Composite Pattern

Deep-Learning-Based Single-Shot Fringe Projection Profilometry Using Spatial Composite Pattern

High-precision point cloud registration method based on volume image correlation

With the rapid development of binocular reconstruction, fringe projection profilometry, and time of flight, 3D imaging technology has been widely applied in the field of 3D measurement. However, due to limited measurement range and self-occlusion, point cloud registration methods are often used to obtain larger or more complete 3D contours. Although many scholars have proposed various point cloud registration methods, the accuracy and efficiency of point cloud registration still need to be further improved, especially for point clouds with different density or non-rigid transformation. Image registration technology based on image correlation has been developed for many years and has achieved great success in fields such as computer vision, photomechanics, and photogrammetry. Therefore, a simple and direct idea in this paper is to transform the point cloud registration problem into volume image correlation problem. By this, an efficient image registration method based on fast Fourier transform and an inverse compositional Gaussian Newton optimization method that only needs to calculate the Hessian matrix once can be introduced into the point cloud registration field, which can greatly improve the speed and accuracy of point cloud registration. Comparative experiments have shown that our method has doubled the accuracy and efficiency compared to the iterative closest point (ICP) method, and its practicality has also been verified in impeller reconstruction experiments.

Two-dimensional Hanning self-convolution window for enhancing Moiré fringe alignment in lithography

The current image processing techniques heavily rely on classic windows for suppressing spectral leakage. However, the limited capability of these classic windows to suppress spectral leakage restricts their application in the field of high-precision measurement. This paper proposes a novel family of windows capable of suppressing spectral leakage associated with the Moiré fringes pattern, thereby enhancing the accuracy of lithography alignment. Specifically, a novel two-dimensional Hanning self-convolution window is introduced, which is derived from the Hanning window to inherit its advantage of ultra-rapid sidelobe decay. The behaviors of the main lobe and desirable sidelobes of the 1st- to 4th-order are examined. Additionally, a novel fringe phase extraction algorithm is established based on the proposed window, which demonstrates high accuracy and efficiency. Simulation and experimental results validate the feasibility and practicality of the proposed window. In particular, the proposed window outperforms classic windows in suppressing spectral leakage, thereby successfully enhancing the phase extraction accuracy to achieve high-precision misalignment measurement at the sub-2-nm level (1.47 nm @ 3σ criterion). The proposed method can be easily extended to other real engineering applications utilizing fringe to achieve high-precision measurements, such as fringe projection profilometry, damage detection, and visual mechanical vibration monitoring.

Dual-stage hybrid network for single-shot fringe projection profilometry based on a phase-height model.

Single-shot fringe projection profilometry (FPP) is widely used in the field of dynamic optical 3D reconstruction because of its high accuracy and efficiency. However, the traditional single-shot FPP methods are not satisfactory in reconstructing complex scenes with noise and discontinuous objects. Therefore, this paper proposes a Deformable Convolution-Based HINet with Attention Connection (DCAHINet), which is a dual-stage hybrid network with a deformation extraction stage and depth mapping stage. Specifically, the deformable convolution module and attention gate are introduced into DCAHINet respectively to enhance the ability of feature extraction and fusion. In addition, to solve the long-standing problem of the insufficient generalization ability of deep learning-based single-shot FPP methods on different hardware devices, DCAHINet outputs phase difference, which can be converted into 3D shapes by simple multiplication operations, rather than directly outputting 3D shapes. To the best of the author's knowledge, DCAHINet is the first network that can be applied to different hardware devices. Experiments on virtual and real datasets show that the proposed method is superior to other deep learning or traditional methods and can be used in practical application scenarios.