Phase-shifted Fringe Patterns Research Articles

Online nonlinearity elimination for fringe projection profilometry using slope intensity coding

The nonlinearity effect in the system of fringe projection profilometry can cause the non-sinusoidal deviation of the fringe patterns, inducing ripple-like phase errors and further affecting measurement accuracy. This paper presents an online nonlinearity elimination method based on slope intensity coding. Two sequences of sinusoidal phase-shifting fringe patterns with different frequencies, and one slope intensity pattern with one uniform intensity pattern are projected. The equations for the nonlinearity response are established using the defined mean and modulation parameters, the captured uniform intensity and two extracted background intensities. The nonlinearity response coefficients determined by solving the equations are used for pixel-wise nonlinearity correction on the captured images, which are employed for computing the wrapped phase, and further obtaining continuous phase by the multi-frequency phase unwrapping method. Experimental results demonstrate that the proposed method can eliminate the nonlinearity-induced phase error online by using fewer images and maintain the reliability of phase unwrapping in the measurement of isolated objects with complex surfaces.

Efficient and robust 3D reconstruction: Absolute phase extraction algorithm based on three gray images

The structured light three-dimensional (3D) measurement technology based on optical metrics has gained increasing applications, and there is a growing demand for capturing transient trajectory changes in dynamic scenes. Conventional temporal phase unwrapping (TPU) approaches often require multi-frequency fringe patterns for application. However, the inevitable conflict arises between the number of projected fringe patterns and the measurement speed. This paper proposes a highly accurate and robust 3D measurement method using only 3-step phase-shifting fringe patterns. To precisely reconstruct the 3D shape using the fewest fringe patterns, it is essential to extract as much information as possible from the available patterns. Therefore, we calculate the wrapped phase distribution using 3-step phase-shifting fringe patterns to ensure accuracy. Simultaneously, the trapezoidal phase-shifting method is employed to calculate the intensity ratio distribution. Through the proposed region-based indexing method, the intensity ratio map is transformed into an unwrapped phase map. Additionally, a novel self-correction method is introduced to correct the non-linear errors in the unwrapped phase map, which is then utilized for the calculation of the fringe order required for phase unwrapping. This high-speed measurement method utilizes only 3-step phase-shifting fringe patterns while retaining the advantages of TPU, meeting the requirements of high-speed and full-field measurements. Experimental results on both static and dynamic scenes validate the effectiveness and versatility of the proposed method, demonstrating its flexibility and simplicity in application. In our 3D measurement system, the method achieves 3D reconstruction of the dynamic object's trajectory at a measurement speed of 107 frames per second.

Variable-frequency phase-shifting algorithm with least-squares iteration for hybrid errors reduction under structured-light illumination

In this paper, a novel variable-frequency phase-shifting algorithm is proposed to reduce hybrid errors under structured-light illumination, which combines a least-squares iteration with regularization processing. Theoretical formulas and related calculation techniques for the variable-frequency fringe model are respectively derived and proposed considering phase-shifting errors, temporal intensity fluctuations and Gamma distortion. From the perspective of theoretical model, it is successfully verified and implemented to establish a minimal and sufficient system of equations using a least number of fringe patterns. To tackle the challenge of solving numerous unknown parameters in the derived model caused by these sources, a grouped step-by-step strategy is adopted during the least-squares iteration process. Error point determination and compensation tricks are further developed to reduce the phase jump problem caused by underdetermined solution. A Gamma prediction technique based on bi-frequency single-frame fringe patterns is also put forward to enhance the accuracy of phase retrieval results. Corresponding simulation and experimental results have demonstrated that the proposed algorithm with attractive potentiality can achieve a significant enhancement in accuracy using the least number of phase-shifting fringe patterns (taking bi-frequency two-step as an example in this study) compared to other existing competitive techniques.

Underwater 3D reconstruction based on double N-step orthogonal polarization state phase shift strategy

The nonlinear response of projection equipment and the uneven reflectance of HDR (high dynamic range) object surface pose challenges in underwater 3D reconstruction. As a result, the phase-shifting fringe pattern collected by the camera does not have good sinusoidality, leading to phase errors. Therefore, this paper proposes a double N-step orthogonal polarization state phase-shift strategy (DOPS). This technique uses the degree of linear polarization (DOLP) of polarized light to design a double N-step phase-shifting fringe pattern. The original phase shift fringes are encoded as horizontal polarization fringes, and the additional phase shift fringes are encoded as vertical polarization fringes. Then, the camera captures and extracts the phase information of the two groups of DOLP patterns, which are separately subjected to phase unwrapping. The resulting unwrapped phases are fused to reduce phase error. The experimental results indicate that this method reduces the measurement error by 57 % when dealing with 3D shape measurement of underwater HDR objects, compared to the polarization-encoded phase shift algorithm (PEPS). Additionally, this method is more accurate in measurement accuracy and improves measurement efficiency by 50 % compared to the double N-step phase shift method (DNPS).

A robust phase unwrapping method for phase shifting coding high-frequency color fringe projection profilometry

Phase unwrapping is one of the key problems in fringe projection profilometry (FPP). How to achieve unambiguous fringe step order to perform phase unwrapping with few patterns still face a major challenge. We propose a robust phase unwrapping method by using a pair of encoded color fringe patterns, one is a conventional three-step phase-shifting color fringe pattern with equal phase shifts, another is added an additional phase shift valve t(k) to the former, in which t(k) is related to the order of fringe periods. A single-valued solution function for t(k) is constructed and used to determine the unique fringe order. Besides, multiplexing this pair of fringes can help denoise the extracted wrapped phase to improve the accuracy. The simulation and experimental results proved that the proposed method can achieve unambiguous phase unwrapping for high-frequency fringe patterns. The proposed method is also applicable for isolated objects.

Electrically tunable lens assisted absolute phase unwrapping for large depth-of-field 3D microscopic structured-light imaging

This paper presents an absolute phase unwrapping method for large depth-of-field (DOF) microscopic structured-light 3D imaging. We calculate the wrapped phase maps and the fringe contrast maps from fringe images captured under various focus settings realized by an electrically tunable lens. From the fringe contrast maps, we extract each in-focus pixel and determine its approximate depth value from the focus settings. The approximate depth map is then used to create an artificial phase map. The geometric-constraint-based phase unwrapping method is adopted to unwrap the phase of all in-focus pixels using the artificial phase map. The unwrapped in-focus phase map is then used to reconstruct 3D coordinates for the entire DOF with the multi-focus pin-hole model. Experimental results demonstrate that our proposed method only requires three phase-shifted fringe patterns to achieve large DOF 3D imaging.

Effective camera calibration by using phase-shifting fringe patterns

The accuracy of conventional target-based camera calibration depends on the detection of feature points. To improve the calibration accuracy in the flat field, this paper proposes an innovative calibration method based on encoded phase-shifting fringe (PSF) patterns. By using arbitrary quadrilateral interpolation, the relationship between a point on the virtual phase plane and the corresponding four original phase points is determined, establishing a mapping between the simplified original phase field and the virtual phase field. This spatial relationship can be used for accurate camera calibration. The proposed research investigates the details of the PSF based camera calibration method and demonstrates its accuracy and effectiveness through experiments. The evaluation of this method includes the checkerboard edge length variance analysis and the linear distance analysis of the checkerboard fit, both of which serve as validation metrics. Camera distortion is described by the phase distribution, and the average phase error after correction is less than 0.001 pixels (px).

Harmonics elimination in phase-shifting fringe projection profilometry by use of a non-filtering algorithm in frequency domain.

In phase-shifting fringe projection profilometry, fringe harmonics caused by device nonlinearities as well as other factors may badly ruin the measurement results. Generally, the used phase-shifting algorithm enables restraint of effects of harmonics below a certain order depending on the number of phase shifts. When reducing the number of phase shifts for efficiency, high order harmonics will affect the phase-measuring results because of aliasing caused by insufficient sampling rate. To overcome this issue, this paper suggests a non-filtering technique operating in frequency domain, that enables improvement of measurement accuracy by eliminating effects of high order harmonics. With this technique, the phase-shifting algorithm is restated as a process of retrieving the fundamental complex fringes from the phase-shifted fringe patterns. Implementing a Fourier transform to this calculated complex fringe pattern, the actual fundamental signals and the aliased harmonics have their own lobes with separated peaks in the frequency domain. We reconstruct each order of the aliased harmonics by exploiting their relations with the fundamental signals and then estimate their magnitudes by using the spectral peaks. Instead of directly filtering the fringe spectrum, we subtract spectra of the harmonics from Fourier transform of the just calculated complex fringes, so that the Fourier spectrum of the fundamental fringes without harmonics is recovered through an iterative operation. Further, the phase map is measured accurately. Simulation and experimental results confirm that this proposed method can significantly suppress effects of fringe harmonics. Meanwhile, by taking advantage of non-filtering, it effectively preserves the edges and details of the measured surfaces from being blurred.

Accurate calibration for crosstalk coefficient based on orthogonal color phase-shifting pattern.

The crosstalk coefficient calibration of de-crosstalk in color fringe projection profilometry is an essential step for the high-accuracy measurement. In this paper, a novel approach for calibrating crosstalk matrix of color camera is proposed. The wrapped phase error model introduced by color crosstalk in orthogonal pattern is established. Compared with the existing calibration methods depending on calculating the modulation of the crosstalk channel, the crosstalk coefficients are obtained from phase error in our method. By projecting the designed color orthogonal phase-shifting fringe patterns onto a white plate, the phase-shifting fringe patterns in both horizontal and vertical directions can be separated from captured images. The coefficients between different channels are calibrated by fitting the error relationship between the wrapped phase containing crosstalk and the standard ones. Coefficient fitting simulations and experimental validations including shape measurement of a white plate and distance measurement of a step block are carried out to verify the effectiveness of the proposed method.

Saturation-Induced Phase Error Compensation Method Using Complementary Phase.

Intensity saturation can induce phase error and, thus, measurement error in fringe projection profilometry. To reduce saturation-induced phase errors, a compensation method is developed. The mathematical model of saturation-induced phase errors is analyzed for N-step phase-shifting profilometry, and the phase error is approximately N-folder of the frequency of the projected fringe. Additional N-step phase-shifting fringe patterns with initial phase-shift π/N are projected for generating a complementary phase map. The final phase map is obtained by averaging the original phase map extracted from the original fringe patterns and the complementary phase map, and then the phase error can be canceled out. Both simulations and experiments demonstrated that the proposed method can substantially reduce the saturation-induced phase error and realize accurate measurements for a highly dynamic range of scenarios.

Mask information-based gamma correction in fringe projection profilometry.

For fringe projection profilometry (FPP), the gamma effect of the camera and projector will cause non-sinusoidal distortion of the fringe patterns, leading to periodic phase errors and ultimately affecting the reconstruction accuracy. This paper presents a gamma correction method based on mask information. Since the gamma effect will introduce higher-order harmonics into the fringe patterns, on top of projecting two sequences of phase-shifting fringe patterns having different frequencies, a mask image is projected to provide enough information to determine the coefficients of higher-order fringe harmonics using the least-squares method. The true phase is then calculated using Gaussian Newton iteration to compensate for the phase error due to the gamma effect. It does not require projecting a large number of images, and only 2 × 3 phase shift patterns and 1 mask pattern minimum are required. Simulation and experimental results demonstrate that the method can effectively correct the errors caused by the gamma effect.

Accurate dynamic 3D deformation measurement based on the synchronous multiplexing of polarization and speckle metrology.

An accurate dynamic 3D deformation measurement method realized by the combination of phase-shifting speckle interferometry and speckle correlation is proposed. By converting the speckle field and the reference field into a circular polarized and linear polarized state, the three-step phase-shifting speckle interferograms and one specklegram were recorded directly and simultaneously within a single image by using a polarization camera. Then, the out-of-plane deformation was demodulated from the synchronous phase-shifting fringe patterns, and the in-plane deformation was measured by performing correlation calculations by using specklegrams with the effect of the reference field ignored. Thus, the full-field 3D deformation was obtained precisely. Experimental results demonstrated the accuracy and dynamic measurement ability of the proposed method, which is compact and feasible for actual dynamic scenes.

Generalized Fringe-to-Phase Framework for Single-Shot 3D Reconstruction Integrating Structured Light with Deep Learning.

Three-dimensional (3D) shape acquisition of objects from a single-shot image has been highly demanded by numerous applications in many fields, such as medical imaging, robotic navigation, virtual reality, and product in-line inspection. This paper presents a robust 3D shape reconstruction approach integrating a structured-light technique with a deep learning-based artificial neural network. The proposed approach employs a single-input dual-output network capable of transforming a single structured-light image into two intermediate outputs of multiple phase-shifted fringe patterns and a coarse phase map, through which the unwrapped true phase distributions containing the depth information of the imaging target can be accurately determined for subsequent 3D reconstruction process. A conventional fringe projection technique is employed to prepare the ground-truth training labels, and part of its classic algorithm is adopted to preserve the accuracy of the 3D reconstruction. Numerous experiments have been conducted to assess the proposed technique, and its robustness makes it a promising and much-needed tool for scientific research and engineering applications.

Adaptive horizontal scaling method for speckle-assisted fringe projection profilometry.

Phase-shifting method is widely used in fringe projection profilometry to obtain high-precision wrapped phase maps. The wrapped phase map needs to be converted to an absolute phase map to recover 3D information. The speckle pattern based phase unwrapping method requires only one additional auxiliary pattern, showing great potential for fast 3D measurements. In this paper, a speckle assisted four-steps phase-shifting method was proposed for 3D measurements. This method requires five structured light patterns to complete 3D measurements, including four-steps phase-shifting fringe patterns and a speckle pattern which is used to remove phase ambiguity. The main challenge of speckle based phase unwrapping method is to overcome the mismatch problem which often occurs in some very steep surfaces. In order to improve the speckle matching accuracy, an adaptive horizontal scaling method was proposed. A voting strategy based on phase-connected regions was proposed to reduce the computational overhead. The experiments demonstrate its superior performance, and an accuracy of 0.21 mm was achieved.

High-reflective surfaces shape measurement technology based on adaptive fringe projection

Fringe projection profilometry (FPP) is limited by the pixel depth of the camera and cannot correctly reconstruct complex objects with highly reflective surfaces. In this paper, we proposed an improved adaptive fringe technique to avoid image saturation. This method only needs to project a uniform gray level image and a set of horizontal and vertical phase-shifting fringe patterns to generate adaptive fringes. The gray level of saturated regions can be fitted by the uniform image and the coordinates of the areas are obtained by calculating the horizontal and vertical fringes. To avoid the process of recursion, we introduce the concept of the grayscale correction function. Furthermore, a modified mapping formula is adopted and refined, which enhances the mapping effect and reconstruction accuracy. Compared with the original adaptive fringe projection (AFP), the modified method further decreases the number of projected fringe patterns. We analyzed the error between the fitted value and the standard value to evaluate the performance of the method. Experimental results demonstrated that the proposed method maintains the accuracy of reconstruction while improving time efficiency compared to other high dynamic range (HDR) measurement methods. • The contribution of different fringe projection technology with highly reflective objects is investigated. • A novel adaptive fringe is proposed to measure the depth information of HDR objects. • An idea is proposed for the improvement of adaptive fringe projection.

Single-shot 3D shape acquisition using a learning-based structured-light technique.

Learning three-dimensional (3D) shape representation of an object from a single-shot image has been a prevailing topic in computer vision and deep learning over the past few years. Despite extensive adoption in dynamic applications, the measurement accuracy of the 3D shape acquisition from a single-shot image is still unsatisfactory due to a wide range of challenges. We present an accurate 3D shape acquisition method from a single-shot two-dimensional (2D) image using the integration of a structured-light technique and a deep learning approach. Instead of a direct 2D-to-3D transformation, a pattern-to-pattern network is trained to convert a single-color structured-light image to multiple dual-frequency phase-shifted fringe patterns for succeeding 3D shape reconstructions. Fringe projection profilometry, a prominent structured-light technique, is employed to produce high-quality ground-truth labels for training the network and to accomplish the 3D shape reconstruction after predicting the fringe patterns. A series of experiments has been conducted to demonstrate the practicality and potential of the proposed technique for scientific research and industrial applications.

Single frame color fringe projection profilometry based on empirical mode decomposition imbalance correction algorithm for fast three-dimensional reconstruction

Phase-shifting profilometry (PSP) is considered to be the most accurate technique for phase retrieval with fringe projection profilometry (FPP) systems. However, PSP requires that multiple phase-shifted fringe patterns be acquired, usually sequentially, which limits PSP to static or quasistatic imaging. We introduce color FPP (CFPP), which provides three-dimensional (3D) imaging using a single acquisition, for the application of real-time profiling. A single frame acquisition provides all three phase-shifted fringe patterns needed for the PSP phase retrieval algorithm. However, some problems must be overcome in CFPP, namely, color coupling and imbalance. Following up on the study, we propose a fixed coefficient decoupling method based on point-to-point correction and then introduce the empirical mode decomposition algorithm to decompose a color fringe pattern into three high-frequency components and three low-frequency components for imbalance correction, which improves measurement accuracy and realizes a single frame measurement for complex and discontinuous objects. After correction by the proposed method, the shape of the tested object can be recovered by one a single frame color fringe pattern, which meets the need for dynamic measurement, and the accuracy is not limited by the movement speed of object. The simulations and experimental results show that the proposed method can significantly reduce the influence of color coupling and imbalance in CFPP, and thus realize dynamic and high-precision 3D reconstruction.

Three-dimensional measurement method based on a three-step phase-shifting fringe and a binary fringe

Gray-code plus phase-shifting is currently a commonly used method for structured light three-dimensional (3D) measurement that is able to measure complex surfaces. However, the Gray-code fringe patterns tend to be complicated, making the measurement process time-consuming. To solve this problem and to obtain faster speed without sacrificing accuracy, a 3D measurement method based on three-step phase-shifting and a binary fringe is proposed; the method contains three phase-shifting fringe patterns and an additional binary fringe pattern. The period of the binary fringe is designed to be the same as the three-step phase-shifting fringe. Because of the specific pattern design strategy, the three-step phase-shifting algorithm is used to obtain the wrapped phase, and the connected region labeling theorem is used to calculate the fringe order. A theoretical analysis, simulation, and experiments validate the efficiency and robustness of the proposed method. It can achieve high-precision 3D measurement, which performs almost the same as the Gray-code plus phase-shifting method. Since only one additional binary fringe pattern is required, it has the potential to achieve higher measurement speed.

3D shape measurement based on the unequal-period combination of shifting Gray code and dual-frequency phase-shifting fringes

The combination of Gray code and phase-shifting fringes is widely used in the 3D structured light measurement. However, an error in the period order of the phase-shifting fringe obtained by the Gray code results in a period jump error approximately equal to an integer multiple of 2π. This error causes a gross measurement error, and even a measurement failure for a measured surface whose height changes drastically. In order to effectively reduce this error, a 3D measurement method, based on the unequal-period combination of shifting Gray code and dual-frequency phase-shifting fringes, is proposed by analyzing the generation mechanism of the period jump error. In the proposed method, on the basis of the unequal-period combination of shifting Gray code and single-frequency phase-shifting fringe, a strategy of additionally projecting a set of low-frequency phase-shifting fringe patterns is adopted to effectively correct the period jump error. A measurement model of this method is established, its applicable conditions are given, the experimental equipment is built, and verification experiments are carried out. The experimental results show that the proposed method can effectively reduce the period jump error, and can measure multiple isolated objects and objects with drastic changes in surface height such as plaster heads.

Virtual temporal phase-shifting phase extraction using generative adversarial networks.

In this paper, a virtual temporal phase-shifting method based on generative adversarial networks (GANs) is proposed to realize dynamic measurement, which is referred to as PSNet. The proposed PSNet can produce the virtual phase-shifting fringe patterns from the single-frame fringe pattern, and the standard n-step phase-shifting method is employed to obtain the wrapped phase from the virtual fringe patterns. The wrapped phase can further be unwrapped by the unwrapping algorithm. Simulation analysis shows that the PSNet can produce the virtual phase-shifting fringe patterns without noise, and thus, the denoising process is eliminated. The performance of the method is verified from the captured fringe in the interferometer, which demonstrates the practicability of the proposed method.