Composite Fringe Research Articles

Dual frequency composite pattern temporal phase unwrapping for 3D surface measurement

Phase shifting profilometry (PSP) is widely used in three-dimensional (3D) optical metrology applications such as mechanical engineering, industrial monitoring, computer vision, and biomedicine because of its high measurement accuracy. In PSP, multi-frequency temporal phase unwrapping (TPU) plays a dominant role due to its high-accuracy measurement of surfaces with discontinuities and isolated objects. However, it requires a large number of fringe patterns. To reduce the number of required patterns, a new dual-frequency composite-pattern TPU method is developed, which needs only three patterns. The new method combines two fringe patterns of different frequencies into three composite fringe patterns, reconstructs rough 3D points by demodulating the low-frequency fringe pattern using geometric constraints, and reprojects it to obtain a rough low-frequency absolute phase, which is then refined to correct edge errors. Finally, the high-frequency wrapped phase is unwrapped under the guidance of the refined low-frequency phase, thereby the accurate 3D points are reconstructed based on stereo-vision technology. Experimental results demonstrated that the proposed method can achieve 3D surface measurement with only three images, which greatly improves the measurement efficiency. The proposed method has great application potential for the rapid 3D measurement of discontinuous surfaces and isolated objects.

3D reconstruction with single-frame two-step phase-shift method based on orthogonal composite fringe pattern projection

Abstract In order to realize single-frame three-dimensional (3D) reconstruction, a single-frame two-step phase-shift method based on orthogonal composite pattern projection is proposed to solve the problem that the traditional N-step phase-shift profilometry needs multiple projections for 3D reconstruction. The orthogonal composite pattern uses only two carrier channels to reduce the spectrum overlapping influence on the demodulation accuracy of carrier and modulated fringes. A two-dimensional variational mode decomposition method is adopted to remove the background DC component of the sinusoidal fringe to overcome the mode overlap problem by controlling the size of the bandwidth. Thus, the two-step phase-shift method is applied to calculate the phases for 3D reconstruction. The experimental results show that, compared with the typical Fourier transform profilometry method, 3-step composite method and 2 + 1 composite method, the 3D reconstruction accuracy of the proposed method is improved by 49.1%,31.4% and 23.2% respectively according to mean absolute error, and by 73.0%, 58.4% and 56.8% respectively according to mean squared error as the evaluation index. Finally, the dynamic 3D reconstruction experiment demonstrates the good adaptability of dynamic 3D reconstruction.

Deep learning-based frequency-multiplexing composite-fringe projection profilometry technique for one-shot 3D shape measurement

The pursuit of precise and efficient 3D shape measurement has long been a focal point within the fringe projection profilometry (FPP) domain. However, achieving precise 3D reconstruction for isolated objects from a single fringe image remains a challenging task in the field. In this paper, a deep learning-based frequency-multiplexing (FM) composite-fringe projection profilometry (DFCP) is proposed, where an end-to-end absolute phase retrieval network (APR-Net) is trained to directly recover the absolute phase map from a FM composite fringe pattern. The obtained absolute phase map exhibits exceptional precision and is devoid of spectrum crosstalk or leakage disturbance typically encountered in traditional FM techniques. APR-Net is intricately crafted, incorporating a nested strategy along with the concept of centralized information interaction. A diverse dataset encompassing various scenarios and free from spectrum aliasing is assembled to guide the training process. A seven-map loss calculation scheme is employed to guide the training process, of which the efficacy is proved through ablation experiments. In the first qualitative experiment, DFCP demonstrates comparable phase accuracy to ground truths with 47 fewer projected images, outperforming other three methods with mean absolute phase errors of 0.0052 rad, 0.1761 rad, 0.0169 rad, and 0.0139 rad. The second qualitative experiment and the quantitative evaluation respectively prove DFCP’s capability in high dynamic range 3D measurement and in precise 3D measurement, with sphere diameter errors of 0.0780 mm and 0.0726 mm, and a spherical centroid distance error of 0.0555 mm.

A two-phase unwrapping three-dimensional measurement method based on phase shift coding

The periodic misalignment of the wrapping phase and fringe order based on the phase-shift + coding method is known to occur easily, leading to jump errors in absolute phase recovery. This paper proposes a two-phase unwrapping method based on shifting-phase coding. This approach encodes the fringe order information and the wrapping phase into the four-step phase shift mode. It generates four composite fringe patterns to directly resolve the wrapping phase and fringe order, thus improving the measurement speed by reducing the number of fringe projections. The interleaved wrapping phases are then constructed by a sequence that changes the initial wrapping phase. The two-phase unwrapping becomes the absolute phase, so the phase error is avoided and the measurement accuracy is improved. The proposed strategy’s effectiveness is proved by the comparison experiments of different methods. Similarly, the suggested method’s reliability is verified by the measurement experiments of several discontinuous objects.

An Anti-saturation phase retrieval algorithm based on optimal composite fringe patterns for 3D measurement with shiny surface

Three-dimensional (3D) measurement of shiny surfaces poses a challenge to the structure light method due to camera saturation caused by the reflective light. Addressing this problem, we propose an anti-saturation retrieval algorithm based on optimal composite fringe patterns. Based on the generalized expression of composite phase-shifting profilometry, we derive a complete phase error model, determine the best phase-shifting ratio theoretically, and finalize the optimal composite fringe pattern. Then, considering the phase retrieving problem occurred in the saturated regions, we present a novel anti-saturation retrieval algorithm by introducing a rejection coefficient, and accurately retrieve the phases of saturated region only with unsaturated values. Experimental results demonstrate that our method possesses a strong anti-saturation ability for shiny surface measurement with no need of adjusting camera exposure or projector brightness.

A novel phase unwrapping method for binocular structured light 3D reconstruction based on deep learning

Fringe projection has been prevalently adopted in the field of three-dimensional(3D) reconstruction. However, it is still a challenge to achieve high-precision and efficient measurement. To achieve real-time and efficient 3D shape measurement of the objects, we must extract the 3D coordinate data of the object from as few fringe patterns as possible. In the paper, we develop a new technique combining composite fringe coding and stair phase coding to solve the high-quality absolute phase and realize binocular 3D morphology reconstruction based on deep learning. This method only needs to project a composite fringe coding pattern to acquire the wrapped phase and a stair phase coding pattern to obtain the fringe order. Then we can recover the unwrapped phase maps of the captured fringe patterns. According to the calibration parameters of the binocular camera, the absolute phase is stereo matched to get the disparity map, and then converted into 3D spatial coordinates of the object. The influence of defocus and noise on the deviation between the wrapped phase and fringe order is solved by staggered phase unwrapping method. We demonstrated the validity and robustness of the novel method by experiments, which provides a promising research direction for 3D shape measurement based on binocular structured light.

Deep learning enabled single-shot absolute phase recovery in high-speed composite fringe pattern profilometry of separated objects

Deep learning enabled single-shot absolute phase recovery in high-speed composite fringe pattern profilometry of separated objects

Research on highly dynamic 3D measurement method based on RGB color fringe projection

Metal parts with highly dynamic areas often appear in industrial production measurements. However, if the traditional fringe projection technique is used to project fringe onto the surface of these metal parts, the light energy will be excessively concentrated and the image will be saturated, resulting thus in the loss of fringe information. To effectively address the high reflectivity problem of the object under test in fringe projection, background normalized Fourier transform contouring was combined with adaptive fringe projection in this work and a new method for performing highly dynamic 3D measurements was proposed. To reduce the number of the acquired images by the camera, a monochromatic fringe of different frequencies was put into the RGB channel to make color composite fringe, and then a color camera was used to acquire the deformed color composite fringe map. The images acquired by the color camera were then separated into three channels to obtain three deformed stripe maps. The crosstalk was also removed from these three images, and the 3D shape of the object was reconstructed by carrying out Fourier transform contouring with background normalization. From our experiments, it was demonstrated that the root mean square error of the proposed method can reach 0.191 mm, whereas, unlike the traditional methods, the developed method requires four images.

Modal analysis of beam-like structures using multipoint dynamic testing vision system based on composite fringe pattern

A multipoint dynamic testing vision-based system for beam-like structures based on composite fringe pattern (CFP) was proposed. The CFP was decorated on the surface of a beam as a sensor, whose image sequences were captured by a camera for dynamic identification. The CFP consists of two cosine fringe patterns. The fringes on both sides of the CFP are used to locate the measuring positions and the middle one is used to measure the spatial displacement of structures. The key advantage of the proposed method is that it can measure multipoint vibration information along the length direction of the beam at one measurement without point-by-point scanning. Experiments on two different beams were carried out, and the results show that the method can get multipoint vibration information and the modal shapes of the beam. Therefore, the vision and CFP-based measurement method is suitable for vibration monitoring and modal analysis of beam-like structures.

Superimposed structures, incremental strain and deformation path from field data to modelling: A case study from the Alpi Apuane metamorphic core complex (NW Tuscany, Italy).

In this study, superimposed folds and antitaxial strain fringes are investigated to unravel the deformation path of the core zones of the Alpi Apuane metamorphic complex (northwest Tuscany Italy). Detailed field mapping combined with microstructural analyses is used together with computer-modelling of a complex strain fringe system to disclose the main steps of the Alpi Apuane syn-metamorphic deformation history. A comparison between natural investigated fibres and those produced by paraffine wax experiments provides details of the relative core-object-fringe rotation allowing new insight on the growth kinematics of strain fringes. The results of our analyses show that our composite fringe system is characterized by a first set of fibres to be related with the main composite axial planar foliation of regional scale isoclinal non-cylindric folds. This first set of fibre fringes records a local non-coaxial strain with an absolute core-object rotation (relative to an external reference frame represented by the main foliation) of 140° and a calculated shear strain (γ) of about 4. The second set of fringes, developed after the tilting of the main foliation and the first fringe-set in a sub-vertical orientation, records a lower absolute rotation of the core-object (25°), a lower shear strain (γ) of about 1.4 and a coaxial strain associated with the vertical shortening. This part of the composite fringe system and the related structures may relate to the switch from the crustal scale contractional regime to the extensional one, which characterized the development of the inner northern Apennines orogen.

Rapid 3D measurement of colour objects based on three-channel sinusoidal fringe projection

A rapid 3D measurement method based on three-channel projection is proposed in this paper. Project a composite color fringe map and an all-white map to the object to be measured. Firstly, crosstalk correction is performed on the composite color fringe pattern collected by the camera then the reflectivity of fringe image is corrected by all-white image. Then, the background normalization principle is used to solve the spectrum aliasing problem when the Fourier transform obtains a truncated phase. Finally, the three-frequency heterodyne method is used for phase unwrapping to obtain the continuous phase and 3D shape. Compared with the traditional color fringe projection method, this method not only eliminates the spectrum aliasing problem but also solves the uneven fringe reflectivity problem when measuring color objects using an all-white map. This method only requires two images to achieve measurement of 3D color dynamic objects.

Three-dimensional translation vibration measurement system based on linear array sensor and composite fringe pattern

The vibration signal of a structure gives crucial information to understand its condition. A novel three-dimensional (3D) vibration measurement system is proposed, based on a linear array sensor (LAS) and a composite fringe pattern (CFP), for monitoring a rigid structure with translation displacement along each direction. The CFP is composed of three sinusoidal fringe sets. Two parallel sinusoidal fringe sets on both sides of the CFP and an inclined one in the middle. The displacement along the imaging optical axis could be obtained from the center distance variation of the parallel fringe sets in the recorded CFP. Meanwhile, the vertical displacement could be acquired from the variation of the center distance ratio of the fringe sets. The horizontal displacement could be obtained by locating the symmetric center of the parallel fringe intensity. A peak locating algorithm was created to realize sub-pixel displacement measurement without an increase in the amount of computation. Simulation results verify the feasibility, effectiveness and robustness of the proposed algorithms. The measurement performance of the proposed system was also verified by comparison experiments, whose results demonstrated that the proposed system is capable of recovering structural 3D displacements. Therefore, the proposed vibration monitoring system could be recommended for real engineering applications in the 3D translation vibration monitoring of rigid structures.

Composite fringe projection deep learning profilometry for single-shot absolute 3D shape measurement.

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.

Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement

Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects. For fringe projection profilometry (FPP), however, it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image. In this paper, we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies. The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods. Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.

Three-dimensional shape measurement based on speckle-embedded fringe patterns and wrapped phase-to-height lookup table

A three-dimensional (3D) shape measurement method based on speckle-embedded fringe patterns (composite fringe patterns) and wrapped phase-to-height lookup table (LUT) is proposed. As an auxiliary signal, speckle is embedded into the phase domain of the projected fringes, and the wrapped phase and deformed speckle modulated by the objects are demodulated by phase-shifting algorithm. With the assistance of the LUT established in advance, the candidate heights are directly obtained by the wrapped phase. Subsequently, the speckle correlation is used to eliminate the height ambiguity, and then the absolute spatial height of the measured objects can be reconstructed without projecting any additional pattern. In this paper, the cooperation of embedded speckle and LUT allows the speckle to do correlation operation in some certain smaller area of the candidate heights’ location, instead of matching within the whole measurement range, thus it can improve the computational efficiency in the determination of the unique accurate height. Theoretical analysis and experimental results have demonstrated that the proposed method can achieve high-accuracy and robust 3D shape measurement of dynamic and static complex scenes.

Joint Coding Strategy of the Phase Domain and Intensity Domain for Absolute Phase Retrieval

Codewords used to determine fringe orders are significant for absolute phase retrieval based on the phase-coding method. However, because of inevitable interference from image blurring, image noise, and system nonlinearity, it is challenging to significantly increase the number of codewords for high-accuracy phase retrieval. To address this challenge, this article proposes a joint coding strategy to perform the coding of high-density stripes. More specifically, codewords are simultaneously embedded into the phase domain and the intensity domain of composite fringe patterns, which greatly extends the encoding range of valid codewords. Moreover, to improve the anti-interference capability of the proposed coding strategy, a regional correction technique (RCT) is presented to remove jump phase errors induced by the inevitable interference. The results of experiments indicate that the proposed method is effective and accurate for both complex objects and a standard sphere.

Automatic optimal projected light intensity control for digital fringe projection technique

Automatically adapting the projected light intensity is crucial for high dynamic range 3D shape measurement, where minimum human intervention is usually desirable. This paper presents a method to automatically determine the optimal number of light intensities and corresponding intensity values for fringe pattern projection according to the surface reflectivity of the measured object. The raw fringe images captured at different light intensities are synthesized pixel-by-pixel into the composite fringe images, which can be further used for phase retrieval and conversion to 3D coordinates. Our experiments demonstrate that the proposed method can achieve high measurement accuracy and effectiveness when measuring an object with a large range of reflectivity variations.

Robust composite sine-trapezoidal phase-shifting algorithm for nonlinear intensity

Phase-shifting profilometry (PSP) is an efficient three-dimensional (3D) measurement technique. The accuracy of the PSP based 3D measurement system is generally affected by the nonlinear intensity response of the projector. In general, nonlinear errors can be reduced by projecting more fringe patterns, but this approach significantly increases the measuring time. To improve the measurement accuracy and speed, a new composite sine-trapezoidal phase-shifting profilometry technique that suppresses nonlinear error is proposed in this study. The main idea of the proposed method is projecting a set of sinusoidal fringe patterns and two sets of composite sine-trapezoidal fringe patterns with different intensity ranges in sequence. Among these patterns, the sinusoidal fringe is used to encode the pixel position, and the order of the wrapping phase is determined by the region code of the composite fringe. The flat intensity parts of the composite fringe are used to encode the nonlinear intensity response of the measurement system. Through system identification, the response parameters of the pixels can be obtained, and the gray value of the pixels can be corrected to gain a better phase distribution. Compared to the straight edges of the trapezoidal fringe, the sinusoidal edges of the composite fringe are more robust to the projector defocus, which improves the accuracy of the identification of the system intensity response model. Simulation results show that the proposed method can achieve a higher phase retrieval accuracy than the trapezoidal PSP and the sine PSP. The experiment results indicate that the proposed method has a better measurement accuracy than those of the standard 3-step and 4-step sine PSP methods.

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system.

In this paper, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

Vision-based system for simultaneous monitoring of shaft rotational speed and axial vibration using non-projection composite fringe pattern

Abstract Both the instantaneous rotational speed (IRS) and vibration of a rotating shaft are key information for condition monitoring and faults diagnosis of rotary machinery. Instead of using a rotational speed encoder and a displacement sensor to measure the IRS and the vibration of a rotating shaft, a non-projection fringe pattern and vision-based system was proposed to realize simultaneous measurement of IRS and axial vibration of a rotation shaft. A composite fringe pattern (CFP) was pasted around the surface of the shaft and worked as IRS and displacement sensors. Each line fringe intensity along the axial direction contained two fringe period densities (FPDs), in which each FPD ratio only corresponds to a rotational angle. A linear array image sensor was employed to record the fringe intensity, from which the time domain rotational angle could be obtained from the change of the FPD ratio and the axial displacement could be acquired by locating the maximum peaks of the cross correlation between the recorded fringe intensity and the reference one. Thus, the IRS could be obtained accurately and efficiently from the rotational angle by multiplying the sampling frame rate of the high-speed camera. Simulated and experimental results demonstrated that the proposed system could simultaneously extract the IRS and axial displacement with the advantage of non-contact and effectiveness. Therefore, the proposed system provides an alternative vision-based approach for the simultaneous measurement of IRS and axial vibration.