Temporal Phase Unwrapping Research Articles

Fringe projection profilometry based on deep learning phase demodulation combined with temporal phase unwrapping

Fringe projection profilometry based on deep learning phase demodulation combined with temporal phase unwrapping

Dual frequency composite pattern temporal phase unwrapping for 3D surface measurement

Phase shifting profilometry (PSP) is widely used in three-dimensional (3D) optical metrology applications such as mechanical engineering, industrial monitoring, computer vision, and biomedicine because of its high measurement accuracy. In PSP, multi-frequency temporal phase unwrapping (TPU) plays a dominant role due to its high-accuracy measurement of surfaces with discontinuities and isolated objects. However, it requires a large number of fringe patterns. To reduce the number of required patterns, a new dual-frequency composite-pattern TPU method is developed, which needs only three patterns. The new method combines two fringe patterns of different frequencies into three composite fringe patterns, reconstructs rough 3D points by demodulating the low-frequency fringe pattern using geometric constraints, and reprojects it to obtain a rough low-frequency absolute phase, which is then refined to correct edge errors. Finally, the high-frequency wrapped phase is unwrapped under the guidance of the refined low-frequency phase, thereby the accurate 3D points are reconstructed based on stereo-vision technology. Experimental results demonstrated that the proposed method can achieve 3D surface measurement with only three images, which greatly improves the measurement efficiency. The proposed method has great application potential for the rapid 3D measurement of discontinuous surfaces and isolated objects.

Two-plus-two fringe projection profilometry based on phase-shifted coding

In fringe projection profilometry based on temporal phase unwrapping, determining a fringe order map commonly requires a large number of fringes. To reduce the fringe number, this paper proposes a concise absolute phase retrieval algorithm just by projecting four fringes. The first two orthogonal fringes with relatively large frequency can collect reliable height information. The second two fringes are designed the same as the first two, but the only difference is that each 2π-phase of them is shifted by a unique amount, which can robustly label a large number of fringe orders. For decoding the fringes, we develop an average intensity one-time extraction algorithm, which allows for the rapid acquisition of the two pairs of alternating current components. From this, the wrapped phase containing height information and the stair-coded phase providing fringe orders can be directly extracted by arctangent operation in a point-to-point manner. Furthermore, we also develop a universal fringe order correction algorithm that can simultaneously correct the common errors and the misalignment between the wrapped phase and fringe orders. Experiment results demonstrate that this method achieves comparable accuracy and adaptability to the phase-coding method, while utilizing two fewer fringes.

Dynamic three-dimensional reconstruction with phase shift coding division multiplexing

Fringe projection profilometry with temporal phase unwrapping (FPP-TPU) has been gotten more and more attention in reconstructing the object with complex surfaces. However, a general contradiction in the existing FPP-TPUs is that more auxiliary coded patterns may limit sampling rate in dynamic three-dimensional (3D) measurement. In this paper, inheriting the single-frame auxiliary pattern of the phase-shifting temporal phase unwrapping (PS-TPU), an improved phase shift coding temporal phase unwrapping with higher robustness is proposed. And new projection stratagem with phase shift coding division multiplexing (PSC-DM) is also proposed to enhance the reconstructing rate in dynamic 3D scene. Firstly, six codewords instead of the traditional eight codewords are embedded into the single-frame auxiliary coding pattern, which guarantees the decoding robustness and provides a faster decoding process with single-step direction unifying. Then, the new projection sequence stratagem is designed with every two sets of 3-step phase-shifting patterns projection followed by the single-frame phase-shifting coding pattern projection. During phase unwrapping, every adjacent three frames of phase-shifting pattern sequentially resolving a wrapped phase and a highly robust 3D reconstruction can be achieved by using the closest phase-shifting coding pattern so as to achieve phase-shifting coding division multiplexing. It can obtain higher reconstructing rate. In addition, a phase shift segmentation is used to correct mismatched errors of fringe order. The experimental results demonstrate that the proposed method can increase the reconstructing rate from the rate of 30 fps in traditional PS-TPU method to 68.57 fps at the digital light projector (DLP) projection rate of 120 fps, which shows its potential prospective in dynamic 3D shape reconstruction.

Complementary phase interleaving-based fringe order recognition for temporal phase unwrapping

This paper introduces a complementary phase interleaving-based fringe order recognition method, designed to rectify phase errors encountered in temporal phase unwrapping. Addressing challenges arising from noise interference or misalignment between fringe order and the wrapped phase, this method employs an innovative integration of a smooth-wrapped phase and the interleaved complementary phases with precise π phase shifts. Furthermore, it maximizes measurement accuracy and prevents filtering-induced degradation by back-calculating the fringe order of noise-disturbed phases from the smooth-continuous phase. Its remarkable adaptability across various temporal phase unwrapping methods, without necessitating additional structural patterns, underscores its versatility in addressing a broad spectrum of phase error suppression challenges. Experimental validation substantiates the accuracy and adaptability of this method, demonstrating its efficacy in handling diverse types of phase errors within TPU methods. This advancement stands as a pivotal solution for achieving precise and comprehensive phase error suppression, particularly for multiple isolated objects.

Three-Shot Dual-Frequency Fringe Scheme Based on Spatial Computer-Generated Moiré Fringe

A highly robust dual-frequency hierarchical temporal phase unwrapping (DHTPU) based on the novel spatial computer-generated Moiré profilometry (SCGMP) is proposed. The method requires only three patterns: a high-frequency fringe to provide robust surface information, a multi-period low-frequency fringe to eliminate the 2π-phase ambiguities, and a flat pattern to remove the average intensity of the two fringes. In decoding, different from traditional Moiré profilometries that rely on spectrum filters, SCGMP only employs spatial-domain calculations to extract the wrapped phase, thereby preserving more detailed information. Furthermore, we fully explore SCGMP’s capability to significantly alleviate phase ambiguity and provide an algorithm to determine the maximum measurable height range for a fixed system, enabling the direct extraction of the continuous basic phase from the multi-period low-frequency fringe. Consequently, the proposed basic phase exhibits an enhanced signal-to-noise ratio, compared to the traditional basic phase derived from the single-period fringes, effectively releasing the high-frequency restriction in the traditional DHTPU. The experimental results verify that the proposed DHTPU method has considerable accuracy and great potential for high-speed measurements, due to there being only three shots required.

High performance holographic video compression using spatio-temporal phase unwrapping

In this work, we present a high-performance holographic data compression technique. This approach is based in the temporal correlation found in a holographic video generated using an optimized random phase. This temporal correlation makes possible high levels of compression by using a spatio-temporal (ST) representation, partial phase unwrapping, and a lossless compression algorithm like DEFLATE. In this approach, two holographic videos are created using the optimized random phase (ORAP) technique and the Gerchberg-Saxton (GS) algorithm. The phase holograms corresponding to each frame of the videos are arranged into a ST representation, enabling spatial and temporal phase unwrapping. We test phase unwrapping along the spatial and temporal dimensions separately, followed by volume reduction after applying the DEFLATE lossless compression algorithm to the unwrapped phase in each case. The holographic video is then reconstructed from the compressed temporal and spatial unwrapped phase. The results demonstrate the high efficiency of the proposed method, with ORAP holograms achieving a compression percentage of 92.670 % after applying DEFLATE to a ST representation with partial phase unwrapping along the temporal dimension.

Efficient and robust 3D reconstruction: Absolute phase extraction algorithm based on three gray images

The structured light three-dimensional (3D) measurement technology based on optical metrics has gained increasing applications, and there is a growing demand for capturing transient trajectory changes in dynamic scenes. Conventional temporal phase unwrapping (TPU) approaches often require multi-frequency fringe patterns for application. However, the inevitable conflict arises between the number of projected fringe patterns and the measurement speed. This paper proposes a highly accurate and robust 3D measurement method using only 3-step phase-shifting fringe patterns. To precisely reconstruct the 3D shape using the fewest fringe patterns, it is essential to extract as much information as possible from the available patterns. Therefore, we calculate the wrapped phase distribution using 3-step phase-shifting fringe patterns to ensure accuracy. Simultaneously, the trapezoidal phase-shifting method is employed to calculate the intensity ratio distribution. Through the proposed region-based indexing method, the intensity ratio map is transformed into an unwrapped phase map. Additionally, a novel self-correction method is introduced to correct the non-linear errors in the unwrapped phase map, which is then utilized for the calculation of the fringe order required for phase unwrapping. This high-speed measurement method utilizes only 3-step phase-shifting fringe patterns while retaining the advantages of TPU, meeting the requirements of high-speed and full-field measurements. Experimental results on both static and dynamic scenes validate the effectiveness and versatility of the proposed method, demonstrating its flexibility and simplicity in application. In our 3D measurement system, the method achieves 3D reconstruction of the dynamic object's trajectory at a measurement speed of 107 frames per second.

Dynamic phase-differencing profilometry with number-theoretical phase unwrapping and interleaved projection

High-speed 3D measurement is receiving increasing attention. However, simultaneously achieving high computational efficiency, algorithmic robustness, and reconstructing ratio is challenging. Therefore, a dynamic phase-differencing profilometry (DPDP) is proposed. By capturing the minimum three phase-shifting sinusoidal deformed patterns and establishing a brand-new model, the phase difference between the object on the reference plane and the reference plane is directly resolved to effectively improve computational efficiency. Although it is wrapped, by using only two auxiliary complementary gratings with a purposely designed lower frequency, a DPDP-based number-theoretical temporal phase unwrapping (NT-TPU) algorithm is also proposed to unwrap the wrapped phase difference rather than the phase itself with high robustness. Furthermore, compared to existing PSP-based NT-TPU, the proposed NT-TPU can normally work under more relaxed restrictions. In order to accomplish a high reconstructing ratio, a pentabasic interleaved projection (PIP) strategy based on time division multiplexing is proposed. It can improve the reconstructing ratio from one reconstruction per every five patterns to an equivalent of one reconstruction per every 1.67 patterns. Experimental results demonstrate that the proposed method achieves high computational efficiency, high algorithmic robustness, and high reconstructing ratio simultaneously and has prospective application in high-speed 3D measurement.

Dual-frequency phase unwrapping based on deep learning driven by simulation dataset

In fringe projection profilometry (FPP), hierarchical temporal phase unwrapping is used to reliably eliminate phase ambiguity in complex scenarios, in which triple-frequency hierarchical temporal phase unwrapping (THPU) has excellent accuracy but lower measurement speed, while dual-frequency hierarchical temporal phase unwrapping (DHPU) requires only two wrapped phases, but it is fragile to noise contamination. Some researchers have shown that deep learning techniques can be used to overcome this dilemma, but the high dataset building cost makes it difficult to apply them rapidly. In this paper, a novel simulation dataset generation method for FPP is proposed, which requires only a set of pre-collected fringe sequences on the reference plane along with some historical masks to generate a dataset, which is then used to train a neural network. The appropriately trained neural network is able to perform dual-frequency phase unwrapping in actual scenarios based on the principle of regression and segmentation respectively, and both of them show accuracy comparable to THPU, but segmentation is slightly better than regression. We verify its effectiveness through comparative experiments and demonstrate its robustness with different degrees of noise contamination. We believe that this paper can provide potential and beneficial ideas for simulation dataset driven deep learning techniques for optimizing various processes in the field of optical measurements.

Absolute phase retrieval based on spatial ternary phase coding with circular fringe projection

Fringe projection profilometry (FPP) based on temporal phase unwrapping (TPU) plays a vital role due to its high precision and fast speed in three-dimensional (3D) measurements. However, a fundamental contradiction arises between the necessity for numerous auxiliary patterns and the goal of enhancing measurement efficiency, particularly when ensuring a big codeword space for fringe order. Thus, a spatial ternary phase coding (STPC) method is proposed, which achieves a big codeword space by projecting just one auxiliary coding pattern while ensuring a robust decoding. By equally dividing each fringe period into several segments, ternary coding of fringe order is accomplished using an arccosine operation to code three distinct phase values within each segment. Meanwhile, circular grating is adopted due to its excellent performance in phase sensitivity, which offers a higher measuring accuracy in 3D reconstruction. Accordingly, a robust TPU is achieved by combining the introduced phase unwrapping scheme and the circle center-cosine theorem (CCCT) strategy. Furthermore, high-efficiency measurements are attained through projecting three phase-shifting circular gratings and one STPC grating, rendering it suitable for real-time measurements. Experimental results with various static objects and real-time scene demonstrate the feasibility and validity of the proposed method.

Real-time three-dimensional measurement for color object based on HSI multi-level Gray code demodulation temporal phase unwrapping

A real-time three-dimensional(3D) measuring method for color object based on hue-saturation-intensity (HSI) multi-level Gray code demodulation temporal phase unwrapping is proposed. In order to reduce the number of auxiliary fringe patterns to be more adopted to real-time measurement, the multi-level Gray code is used to assist temporal phase unwrapping. While measuring color object, the fringe order may be disorder by color-to-gray algorithm(C2G). In order to improve the characteristic of anti-color interference in real-time measurement, a multi-level Gray code demodulation method using HSI color space is proposed. Intensity in HSI color space instead of the gray in RGB space is used in multi-level Gray code decoding, at the same time, the color texture of the object surface can also be reconstructed without projecting additional auxiliary color texture patterns. Experimental results show that this method can effectively correct the fringe order confusion caused by color object chroma, and reduce errors in fringe order.

Orthogonal spatial binary coding method for high-speed 3D measurement.

Temporal phase unwrapping based on single auxiliary binary coded pattern has been proven to be effective for high-speed 3D measurement. However, in traditional spatial binary coding, it often leads to an imbalance between the number of periodic divisions and codewords. To meet this challenge, a large codewords orthogonal spatial binary coding method is proposed in this paper. By expanding spatial multiplexing from 1D to 2D orthogonal direction, it goes beyond the traditional 8 codewords to 27 codewords at three-level periodic division. In addition, a novel full-period connected domain segmentation technique based on local localization is proposed to avoid the time-consuming global iterative erosion and complex anomaly detection in traditional methods. For the decoding process, a purely spatial codewords recognition and a spatial-temporal hybrid codewords recognition methods are established to better suppress the percentage offset caused by static defocusing and dynamic motion, respectively. Obviating the need for intricate symbol recognition, the decoding process in our proposed method encompasses a straightforward analysis of statistical distribution. Building upon the development of special spatial binary coding, we have achieved a well-balance between low periodic division and large codewords for the first time. The experimental results verify the feasibility and validity of our proposed whole image processing method in both static and dynamic measurements.

Full-period fringe order correction method based on dual-threshold cellular automaton

In traditional temporal phase unwrapping (TPU) methods, the fringe order is obtained pixel-to-pixel by multi-frequencies or multi-wavelengths. The direct calculation and error non-accumulation features accelerate the prosperity of TPU. Nevertheless, the noise, defocusing, and non-uniform reflectivity may inevitably lead to some fringe order errors. These errors are powerless against the evolving TPUs such as Gray code and phase coding. In this paper, a full-period fringe order correction method based on dual-threshold cellular automaton is proposed for machine 3D vision. Initially, an advanced dual-threshold cellular automaton algorithm is employed to extract both the strong and weak edges of the wrapped phase individually. Superior to conventional single-threshold edge detection methods, this approach effectively mitigates the edge breakage problem caused by significant height variations. Subsequently, the N-filled map, N-residue mask, and N-start points concepts are introduced to sequentially segment independent connected domains based on the tightly graphical coupling. In addition, by performing edge compensation in the same period, accurate fringe order correction can be achieved even if excessive edge points are identified, as long as the broken edges are effectively connected. The experimental results exhibit exceptional correction performance of the proposed method for intricate isolated objects, objects with non-uniform reflectivity, and dynamic scenes.

Accurate fringe order recognition with adaptive morphological operations for 3D measurement

Temporal phase unwrapping (TPU) techniques are vulnerable to inaccurate recognition of fringe order due to various factors, including noise, motion blur and defocusing effects. The misalignment between the fringe order and wrapped phase can lead to abnormal jump errors and compromise measuring accuracy. To address this issue, an accurate fringe order recognition method with adaptive morphological operations for 3D measurement is proposed. Differs from the traditional half-period fringe order recognition method, a complete full-flow algorithm is constructed to accurately separate adhesive connected domains, which includes: segmentation and erosion of connected domains, localization of fully eroded connected regions, clustering of separated valid points, and adaptive selection of radius. In contrast to a simplistic morphological open arithmetic, the proposed method has the capability to perform clustering on all separated valid points, thereby achieving lossless fringe order recognition of the full field. And its row-correlation property gives it a larger correctable range than the previous row-independent approach. The experimental results demonstrate the generality, advancement, adaptivity, and precision of our method, built upon the traditional TPUs for obtaining fringe orders.

Unifying temporal phase unwrapping framework using deep learning.

Temporal phase unwrapping (TPU) is significant for recovering an unambiguous phase of discontinuous surfaces or spatially isolated objects in fringe projection profilometry. Generally, temporal phase unwrapping algorithms can be classified into three groups: the multi-frequency (hierarchical) approach, the multi-wavelength (heterodyne) approach, and the number-theoretic approach. For all of them, extra fringe patterns of different spatial frequencies are required for retrieving the absolute phase. Due to the influence of image noise, people have to use many auxiliary patterns for high-accuracy phase unwrapping. Consequently, image noise limits the efficiency and the measurement speed greatly. Further, these three groups of TPU algorithms have their own theories and are usually applied in different ways. In this work, for the first time to our knowledge, we show that a generalized framework using deep learning can be developed to perform the TPU task for different groups of TPU algorithms. Experimental results show that benefiting from the assistance of deep learning the proposed framework can mitigate the impact of noise effectively and enhance the phase unwrapping reliability significantly without increasing the number of auxiliary patterns for different TPU approaches. We believe that the proposed method demonstrates great potential for developing powerful and reliable phase retrieval techniques.

A novel 2+1 three-dimensional measurement for temporal phase unwrapping

A novel 2+1 three-dimensional (3D) measurement for temporal phase unwrapping is proposed. While two high-frequency complimentary sinusoidal fringes and one low-frequency sinusoidal fringe are projected onto the measured object respectively, the corresponding deformed patterns can be captured. Then the high-frequency wrapped phase can be extracted from the two high-frequency complimentary deformed patterns by high-accuracy computer-generated moiré profilometry (HCGMP). Due to its shared direct current (DC) component with the high-frequency deformed pattern, the low-frequency wrapped phase can still be extracted from the only one low-frequency deformed pattern by HCGMP. Because the extracted wrapped phase excludes the phase contribution of the reference plane, a sufficiently high low-frequency fringe projection can also make the low-frequency wrapped phase become its continuous phase without phase unwrapping. And then, the high-frequency wrapped phase is unwrapped, guided by the low-frequency unwrapped phase. Finally, the 3D profile of the measured object can be reconstructed successfully. Compared to the conventional 2+1 algorithm, the proposed algorithm can accomplish temporal phase unwrapping and improve the 3D measuring accuracy. It has a good application prospect in real-time 3D measurement. The experiment results show the feasibility and effectiveness of the proposed algorithm.

Real-time computer-generated frequency-carrier Moiré profilometry with three-frequency heterodyne temporal phase unwrapping

A real-time computer-generated frequency-carrier Moiré profilometry with three-frequency heterodyne temporal phase unwrapping is proposed. While three sinusoidal gratings shared the same direct current(DC) component and the same amplitude but with designed different frequencies are projected onto the measured object, three corresponding deformed patterns can be captured by camera. Then, the shared DC component is extracted from the highest-frequency pattern. And based on it, three AC components of the captured deformed patterns can also be effectively extracted by normalization. From this way, the number of the needed gratings can be reduced from six to three, and the disturbance of the object’s nonuniform reflectivity can be avoided efficiently. By using computer-generated frequency-carrier Moiré profilometry(CGFCMP), the carrier frequency designed as pure sinusoidal and cosinusoidal gratings is optimized in order to effectively extract the computer-generated frequency-carrier Moiré fringes, and finally-three wrapped phases at different frequencies can be obtained. Then the wrapped phase with the highest-frequency grating can be unwrapped by three-frequency heterodyne temporal phase unwrapping algorithm. The simulation and experimental results verify the feasibility and flexibility of the proposed method. It is suitable to measure the complex surface with abrupt slope and multiple isolated objects, and has prospective application in real-time three-dimensional measurement.

Time series phase unwrapping algorithm using LP-norm optimization compressive sensing

Time series phase unwrapping is an essential step in the time series interferometric synthetic aperture radar technique for deformation monitoring. However, traditional unwrapping algorithms are prone to unwrapping errors in the case of intense noise and steep deformation gradients, which directly affect the accuracy of the deformation information interpretation. To address this problem, a time series phase unwrapping algorithm using LP-norm optimization compressive sensing is proposed in this paper. This algorithm first transforms the minimizing L0-norm describing compressive sensing technique into a nonconvex problem with weaker constraints, that is, minimizing the LP-norm. Then, an optimized compressed sensing model was proposed by combining the constraint criterion of the phase triplet closure in the temporal domain and the solution strategy of the iterative reweighted least squares method. Improved temporal dimensional phase unwrapping and unwrapping error correction algorithms were developed based on this optimized model. Finally, the proposed algorithm was perfected by combining the above improved methods and the pseudo-three-dimensional phase unwrapping framework. Simulated and Sentinel-1A real datasets verify that the proposed algorithm has better noise robustness, unwrapping stability, and efficiency than traditional algorithms, particularly in the case of a steep deformation gradient and intense noise. The temporal coherence estimated by the proposed algorithm for the real dataset is generally greater than 0.99, which is far superior to traditional algorithms. Furthermore, the proposed algorithm facilitates high-precision interpretation of deformation information. The cross-validation results show that the deformation results obtained by traditional algorithms are underestimated by at least 73 mm/a and 68 mm at the maximum subsidence than the DSInSAR. However, the deformation result obtained by the proposed algorithm is highly consistent with that of the DSInSAR.

Fast fringe projection profilometry using 3 + 1 phase retrieval strategy and fringe order correction.

Fringe projection profilometry (FPP) has attracted wide attention in optical 3D measurement field for its high resolution, high accuracy, and simple system construction. However, the inherent requirements of FPP's measurement speed and accuracy are contradictory. We adopt a 3+1 phase retrieval strategy combining phase shift and the Hilbert transform algorithm to achieve high-speed, high-precision simultaneous measurements with only four binary fringe patterns. Furthermore, when using temporal phase unwrapping, fringe order errors (FOEs) are inevitable. Although these FOEs do not affect neighboring pixels, they are brought into the final point cloud coordinates by the absolute phase value. We propose a fringe order correction method to eliminate FOEs. The feasibility and effectiveness of this method are verified by the experimental results.