Fringe Projection Profilometry Method Research Articles

Experimental and numerical investigation of radial displacement of COPV using CT and FPP method

The radial displacement of hydrogen storage composite overwrapped pressure vessel (COPV) is a key factor related to the collapse failure and burst pressure. However, there is still lack of effective method to monitor the radial displacement during the hydrostatic test and service process of the vessels. This paper proposed a method to measure the radial displacement based on fringe projection profilometry (FPP). Hydrostatic test with a composite overwrapped pressure vessel verifies the proposed method. The variation of displacement changing with inner pressures observed in the experiments is in agreement with that obtained from finite element analysis. Also Computed Tomography (CT) scanning is conducted and a widely spread micro defects is found in the composite layer of the cylinder. 78.9% of the total observed defects are small (<1000 mm2). In regions where large-sized defects (>1000 mm2) are concentrated, the total area of defects reaches 16,200 mm2. According to the statistical analysis, the strong correlation is found between the defects area and radial displacement.

Dual-frequency phase unwrapping based on deep learning driven by simulation dataset

In fringe projection profilometry (FPP), hierarchical temporal phase unwrapping is used to reliably eliminate phase ambiguity in complex scenarios, in which triple-frequency hierarchical temporal phase unwrapping (THPU) has excellent accuracy but lower measurement speed, while dual-frequency hierarchical temporal phase unwrapping (DHPU) requires only two wrapped phases, but it is fragile to noise contamination. Some researchers have shown that deep learning techniques can be used to overcome this dilemma, but the high dataset building cost makes it difficult to apply them rapidly. In this paper, a novel simulation dataset generation method for FPP is proposed, which requires only a set of pre-collected fringe sequences on the reference plane along with some historical masks to generate a dataset, which is then used to train a neural network. The appropriately trained neural network is able to perform dual-frequency phase unwrapping in actual scenarios based on the principle of regression and segmentation respectively, and both of them show accuracy comparable to THPU, but segmentation is slightly better than regression. We verify its effectiveness through comparative experiments and demonstrate its robustness with different degrees of noise contamination. We believe that this paper can provide potential and beneficial ideas for simulation dataset driven deep learning techniques for optimizing various processes in the field of optical measurements.

Dual-stage hybrid network for single-shot fringe projection profilometry based on a phase-height model.

Single-shot fringe projection profilometry (FPP) is widely used in the field of dynamic optical 3D reconstruction because of its high accuracy and efficiency. However, the traditional single-shot FPP methods are not satisfactory in reconstructing complex scenes with noise and discontinuous objects. Therefore, this paper proposes a Deformable Convolution-Based HINet with Attention Connection (DCAHINet), which is a dual-stage hybrid network with a deformation extraction stage and depth mapping stage. Specifically, the deformable convolution module and attention gate are introduced into DCAHINet respectively to enhance the ability of feature extraction and fusion. In addition, to solve the long-standing problem of the insufficient generalization ability of deep learning-based single-shot FPP methods on different hardware devices, DCAHINet outputs phase difference, which can be converted into 3D shapes by simple multiplication operations, rather than directly outputting 3D shapes. To the best of the author's knowledge, DCAHINet is the first network that can be applied to different hardware devices. Experiments on virtual and real datasets show that the proposed method is superior to other deep learning or traditional methods and can be used in practical application scenarios.

A Y-shaped network based single-shot absolute phase recovery method for fringe projection profilometry

Fringe projection profilometry (FPP) is a widely used non-contact 3D measurement method. Though maturing in the last decade, it remains a significant challenge when facing the phase unwrapping of measured object surfaces in a single-shot measurement setting. With the rapid development of deep learning techniques, the adoption of a data-driven approach is gaining popularity in the field of optical metrology. This study proposes a new absolute phase recovery method based on the devised single-stage deep learning network. The aim is to ensure high-quality absolute phase recovery from a single-shot fringe projection measurement. Unlike most existing approaches, where the numerators and denominators of the wrapped phases and the fringe orders are predicted in various stages, the proposed method acquires the wrapped phases and the corresponding fringe orders within a single network, i.e. it can predict both wrapped phases and the corresponding fringe orders directly and simultaneously from the single fringe pattern projected in the single-shot mode based on a unified Y-shaped network. Experiments on benchmark datasets and models have demonstrated the effectiveness and efficiency of the technique, especially in terms of high-quality recovery of absolute phase information by using the lightweight single-stage network, and enabling the FPP-based phase 3D measurements in an online manner.

Machine learning enhanced high dynamic range fringe projection profilometry for in-situ layer-wise surface topography measurement during LPBF additive manufacturing

Fringe Projection Profilometry (FPP) is a cost-effective and non-destructive method, typically used for measuring finer features and reconstructing 3D topography of objects. However, to use the FPP method for measuring the dynamic topography of powder bed and printed layers during Laser Powder Bed Fusion (LPBF) based additive manufacturing (AM) process, unique challenges exist due to the varying material properties and ambient conditions in the build chamber. In this work, we aim to enhance the discernibility, accuracy, and resolution of FPP in the specific application scenario of measuring layer-wise surface topography during LPBF AM by integrating our recently developed LPBF-specific FPP sensing model that features localized sensor calibration and Fourier filter-aided unwrapping with an equipment-based High dynamic range (HDR) method and machine learning (ML) aided FPP data analysis. First, a projector based HDR method is applied to mitigate the shadowing and intensity saturation problems by projecting sinusoidal fringe patterns of varying intensities. Secondly, a ML framework is developed to improve the surface topography measurement accuracy (RMSE from 10.57 μm to 7.49 μm or even 4.35 μm for directly measurable points) and enhance resolution that is currently subjected to hardware limitations (from 38 μm to 5 μm laterally and from 10 μm to 1 μm vertically). Several different types of candidate neural networks (NNs) are trained and tested using the in-situ FPP measurement data and ex-situ standard optical microscopy characterization data. Multiple NN-based models are resulted and compared in terms of their ability to enhance the accuracy and resolution of FPP's end-result (height measurement). By selecting the best-performance NN enabled image super resolution model, the proposed ML integrated HDR FPP method is expected to measure the surface topography of printed layers during LPBF-AM more capably and efficiently, thus advancing the existing state-of-the-art methods towards the desired online inspection of LPBF print defects.

Calibration method for fringe projection profilometry based on rational function lens distortion model

The calibration of the fringe projection profilometry (FPP) system is susceptible to the camera lens distortion. This paper presents a calibration method based on the rational function lens distortion model. In this method, we integrate the rational function into the mathematical expression of the out-of-plane height to mitigate the impact of lens distortion. With arbitrarily arranged measurement components, we can flexibly measure the three-dimensional (3D) shape of objects with no expensive auxiliaries, high-precision setups, or complicated projector calibration. First, we project modulated sinusoidal fringe patterns with high frequency to the concentric circle grids calibration board to accurately estimate the unwrapped phase value, the pixel coordinate, and the out-of-plane height at each feature point. Then, the 3D coordinates can be recovered by determining the coefficients of the high order phase, coordinate, and height (PCH) mapping. Experiments on computer simulation and real data demonstrate that the modified calibration contributes to the flexibility and accuracy of the 3D shape measurement.

Single-shot 3D measurement of highly reflective objects with deep learning.

Three-dimensional (3D) measurement methods based on fringe projection profilometry (FPP) have been widely applied in industrial manufacturing. Most FPP methods adopt phase-shifting techniques and require multiple fringe images, thus having limited application in dynamic scenes. Moreover, industrial parts often have highly reflective areas leading to overexposure. In this work, a single-shot high dynamic range 3D measurement method combining FPP with deep learning is proposed. The proposed deep learning model includes two convolutional neural networks: exposure selection network (ExSNet) and fringe analysis network (FrANet). The ExSNet utilizes self-attention mechanism for enhancement of highly reflective areas leading to overexposure problem to achieve high dynamic range in single-shot 3D measurement. The FrANet consists of three modules to predict wrapped phase maps and absolute phase maps. A training strategy directly opting for best measurement accuracy is proposed. Experiments on a FPP system showed that the proposed method predicted accurate optimal exposure time under single-shot condition. A pair of moving standard spheres with overexposure was measured for quantitative evaluation. The proposed method reconstructed standard spheres over a large range of exposure level, where prediction errors for diameter were 73 µm (left) and 64 µm (right) and prediction error for center distance was 49 µm. Ablation study and comparison with other high dynamic range methods were also conducted.

Temporal phase-unwrapping in fringe projection profilometry: Increasing the accuracy with equidistant long time-steps sampling

Temporal phase-unwrapping methods for fringe projection profilometry, mainly represented by the multi-frequency and multi-wavelength classes, are fundamental to recovering the 3D information from the continuous phase, also allowing measure discontinuities and isolated objects. In this paper, it is proposed a temporal phase unwrapping method for fringe projection profilometry. This method is specially designed to unwrap temporal phase sequences with equidistant long-time steps. As will be shown, this kind of temporal sampling allows increasing the accuracy of measurements. Simulated and real experimental results realized on a flat plate demonstrate the superiority of this proposal compared with leading already reported methods. Also, some experiments with complex objects are presented to demonstrate its applicability.

Phase Unwrapping-Free Fringe Projection Profilometry for 3D Shape Measurement

Phase unwrapping is always the necessary procedure in fringe projection profilometry (FPP). This letter presents an efficient phase unwrapping-free FPP method. Both the wrapped phase maps of the measured object and the reference plane are extracted by phase-shifting algorithm. If the depth range of the measured object is within <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\pi $ </tex-math></inline-formula> in phase domain, the three-dimensional (3D) shape can be directly reconstructed by converting the wrapped phase difference into absolute phase difference without phase unwrapping process. Simulations and experiments are carried out, and their results demonstrate the effectiveness and robustness of the proposed method.

A Self-Adaptive Multiple Exposure Image Fusion Method for Highly Reflective Surface Measurements

Fringe projection profilometry (FPP) has been extensively applied in various fields for its superior fast speed, high accuracy and high data density. However, measuring objects with highly reflective surfaces or high dynamic range surfaces remains challenging when using FPP. A number of multiple exposure image fusion methods have been proposed and successfully improved measurement performance for these kinds of objects. Normally, these methods have a relatively fixed sequence of exposure settings determined by practical experiences or trial and error experiments, which may decrease the efficiency of the entire measurement process and may have less robustness with regard to various environmental lighting conditions and object reflective properties. In this paper, a novel self-adaptive multiple exposure image fusion method is proposed with two areas of improvement relating to adaptively optimizing the initial exposure and the exposure sequence. First, by introducing the theory of information entropy, combined with an analysis of the characterization of fringe image entropy, an adaptive initial exposure searching method is proposed. Then, an exposure sequence generation method based on dichotomy is further described. On the basis of these two improvements, a novel self-adaptive multiple exposure image fusion method for FPP as well as its detailed procedures are provided. Experimental results validate the performance of the proposed self-adaptivity multiple exposure image fusion method via the measurement of objects with differences in surface reflectivity under different ambient lighting conditions.

Active projection nonlinear γ correction method for fringe projection profilometry.

In fringe projection profilometry (FPP), the luminance nonlinearity generated by the superimposed γ effect of the projector and camera can lead to distortion of the intensity of the sinusoidal phase-shift fringe, resulting in a reduction of measurement precision and resolution. Traditional phase error compensation and γ-correction methods need to focus on the projector's optimal performance. However, commercial projectors often have huge apertures and are, therefore, unable to project a perfectly focused sinusoidal fringe image. This paper proposes an easy-to-implement active projection error correction method with high precision that is insensitive to projector defocus. After calibrating the projector to establish the nonlinear γ-response model of the optical measurement system, inverse γ compensation is performed. By generating and projecting a set of precorrected sinusoidal fringes, the camera can capture the high-quality sinusoidal fringe image and decrease the phase measurement error caused by the nonlinear γ effect of the FPP system. Computer simulations and experiments verify the effectiveness and feasibility of the proposed method for estimating and correcting the nonlinear γ distortion of the FPP system. The experimental results show that using the proposed active projection method to compensate for the error of the three-step phase-shift algorithm can achieve a high-precision measurement, and the influence of the system's nonlinear γ effect on the measurement accuracy is significantly suppressed.

Fringe projection profilometry method with high efficiency, precision, and convenience: theoretical analysis and development.

Fringe projector profilometry (FPP) is an important three-dimensional (3D) measurement technique, especially when high precision and speed are required. Thus, theoretical interrogation is critical to provide deep understanding and possible improvement of FPP. By dividing an FPP measurement process into four steps (system calibration, phase measurement, pixel correspondence, and 3D reconstruction), we give theoretical analysis on the entire process except for the extensively studied calibration step. Our study indeed reveals a series of important system properties, to the best of our knowledge, for the first time: (i) in phase measurement, the optimal and worst fringe angles are proven perpendicular and parallel to epipolar line, respectively, and can be considered as system parameters and can be directly made available during traditional calibration, highlighting the significance of the epipolar line; (ii) in correspondence, when two sets of fringes with different fringe orientations are projected, the highest correspondence precision can be achieved with arbitrary orientations as long as these two orientations are perpendicular to each other; (iii) in reconstruction, a higher reconstruction precision is given by the 4-equation methods, while we notice that the 3-equation methods are almost dominatingly used in literature. Based on these theoretical results, we propose a novel FPP measurement method which (i) only projects one set of fringes with optimal fringe angle to explicitly work together with the epipolar line for precise pixel correspondence; (ii) for the first time, the optimal fringe angle is determined directly from the calibration parameters, instead of being measured; (iii) uses 4 equations for precise 3D reconstruction but we can remove one equation which is equivalent to an epipolar line, making it the first algorithm that can use 3-equation solution to achieve 4-equation precision. Our method is efficient (only one set of fringe patterns is required in projection and the speed is doubled in reconstruction), precise (in both pixel correspondence and 3D reconstruction), and convenient (the computable optimal fringe angle and a closed-form 3-equation solution). We also believe that our work is insightful in revealing fundamental FPP properties, provides a more reasonable measurement for practice, and thus is beneficial to further FPP studies.

Phase retrieval for objects in rain based on a combination of variational image decomposition and variational mode decomposition in FPP.

As far as we know, there is no paper reported to retrieve the phase of an object in rain by the fringe projection profilometry (FPP) method. The fringe projection pattern taken in rain contains much rain noise, which makes it difficult to accurately retrieve the phase of the object. In this paper, we focus on the phase retrieval of the object in rain by the FPP method. We first decompose the original fringe projection pattern into a series of band-limited intrinsic mode functions by the two-dimensional variational mode decomposition (2D-VMD) method. Then we screen out fringe-associated modes adaptively based on mutual information and reconstruct the fringe projection pattern. Next, we decompose the reconstructed fringe projection pattern by the TGV-Hilbert-BM3D variational model to obtain the de-rained fringe component. Finally, we use the Fourier transform method, phase unwrapping method, and carrier-removal method to obtain the unwrapped phase. We test the proposed method on three fringe projection patterns taken in simulated rain weather, and we compare our proposed method with the phase-shifting method, windowed Fourier method, morphological operation-based bidimensional empirical mode decomposition method, 2D-VMD method, and the TGV-Hilbert-BM3D method. The experimental results demonstrate that, for the first time to our knowledge, our method can effectively retrieve the phase of an object in rain from a single fringe projection pattern.

A systematic study and framework of fringe projection profilometry with improved measurement performance for in-situ LPBF process monitoring

Fringe Projection Profilometry (FPP) is a cost-effective and non-invasive technology that has been shown to measure finer features. In this work, we developed an in-situ FPP method to measure the dynamic topography of powder bed and printed layer during Laser Powder Bed Fusion (LPBF) additive manufacturing (AM) process. A systematic study towards developing a comprehensive framework of LPBF-specific FPP is demonstrated to enhance and evaluate the performance of applying FPP for in-situ LPBF monitoring, including 1) a modified sensor model with localized correction; 2) improved phase unwrapping with FFT filtering 3) quantitative uncertainty analysis; and 4) experimental validation with ex-situ characterization. The developed LPBF-specific FPP system and methods are implemented on a commercial LPBF-AM machine, achieving better accuracy, more robustness, and increased field of view while maintaining sufficient measurement range and decent resolution, in contrast to literature methods. The established FPP framework will facilitate the development of closed-loop control strategies for advancing LPBF based AM.

Accurate 3D reconstruction via fringe-to-phase network

Learning 3D shape representations from structured-light images for 3D reconstructions has become popular in many fields. This paper presents a new approach integrating a fringe-to-phase network with a fringe projection profilometry (FPP) technique to achieve 3D reconstructions with superior accuracy and speed performance. The proposed fringe-to-phase network has a UNet-like architecture, capable of retrieving three wrapped phase maps directly from a color image comprising three fringe patterns with designated frequencies. Because the phase maps contain the 3D shape representations of the measurement target, they serve as an intermediary to transform the single-shot fringe-pattern image into the 3D shapes of the target. The datasets with ground-truth phase labels are generated by using a tri-frequency FPP method. Unlike the existing techniques, the proposed approach yields both high-accuracy and fast-speed 3D reconstructions. Experiments have been accomplished to validate the proposed technique, which provides a promising tool for numerous scientific research and industrial applications.

Quasi-Periodic Phase Coding for Long-Depth-Range 3-D Measurements of Moving Objects

Fringe projection profilometry (FPP) requires an auxiliary projection sequence or spatial landmark to remove phase ambiguity, thereby severely limiting the measurement speed or depth range for moving objects. In this article, we propose a novel quasi-periodic phase coding method for long-depth-range 3-D measurements of moving objects without temporal or spatial limitations. This article mainly focuses on the construction of the encoding schema for quasi-periodic fringe patterns and the execution of the phase decoding framework for long-depth-range 3-D measurements of moving surfaces. First, a set of stripes with similar periods is designed on the basis of a preset equivalent period, and the various combinations of these stripes constitute quasi-periodic fringe patterns. Second, we present a progressive phase recovery strategy for decoding quasi-periodic fringe images, in which the wrapped phase is converted first into a continuous phase distribution and then into the absolute phase. In this strategy, the absolute phase map is effectively screened via the piecewise linear transformation from the quasi-periodic phase to the equivalent phase. Compared with the existing FPP methods, the proposed method avoids the low-frame-rate constraint and the need for landmark configuration. Simulation and experimental results verify that the proposed method is applicable for dynamic 3-D measurements of moving objects over a large-depth range.

Two-dimensional complex wavelet with directional selectivity used in fringe projection profilometry.

The continuous wavelet transform (CWT) has been used in fringe projection profilometry (FPP) to recover a three-dimensional surface from a single-frame fringe. This Letter constructs a two-dimensional (2D) continuous complex wavelet employing a 2D real Mexican hat wavelet function, combined with the single-orthant analytical 2D Hilbert transform. The 2D complex wavelet has an asymmetric frequency envelope in the radial direction and the ability of direction selectivity, which results in better matching between the local fringe and daughter wavelets. We provide a complex wavelet construction model, present a theoretical analysis, and conduct experimental validation. The experiments demonstrate that the proposed method provides high phase accuracy in the single-frame FPP method.

Spatial phase-shifting profilometry by use of polarization for measuring 3D shapes of metal objects.

In this paper, we present a polarization spatial phase-shifting method for fringe projection profilometry. It enables us to measure the three-dimensional shape of a metal object in a fast way requiring only a single-shot implementation. With this method, a couple of projectors are equipped, in front of their lens, with linear polarization filters having orthogonal polarization directions, so that they can simultaneously cast two sinusoidal fringe patterns having different phase shifts onto the measured metal surfaces without mixture. To register the two projected patterns, we suggest a fringe alignment method based on the epipolar geometry between the projectors. By taking advantage of the property of metal surfaces in maintaining polarization state of incident light, the deformed fringe patterns on the measured surfaces are captured by using two coaxially-arranged polarization cameras. As a result, the fringe phases are calculated by using a two-step phase-shifting algorithm and further the 3D shapes of the measured surfaces are reconstructed. Experimental results demonstrate the proposed method to be valid and efficient in measuring metal objects.

High-accuracy projector calibration method for fringe projection profilometry considering perspective transformation.

Camera and projector are the key components of structured light three-dimensional (3-D) measurements, and Digital Light Processing (DLP) projector has been widely used for projecting digital structured light patterns for the measurement. The light projecting of projectors can be modeled as the inverse procedures of camera imaging, and its high-accuracy calibration is still a remaining challenge. Therefore, this paper proposes a novel projector calibration method to improve the calibration accuracy of DLP projector. By fixing the position of the camera and calibration board, this method essentially eliminates the perspective transformation error and effectively avoids the distortion of the extracted marker points. The proposed projector calibration procedures are given as follows: Firstly, the optical axis of the camera is adjusted parallel to the normal of the hollow ring calibration board, and a texture image is captured by the camera; Secondly, the horizontal and vertical fringe patterns with nine different positions and directions are projected onto the calibration board, and nine sets of projected images are taken; Finally, a one-to-one correspondence between the camera and the projector is established, and the projector is accurately calibrated using the phase equivalence. The experimental results show that the proposed projector calibration method is feasible and easy to operate, which can essentially eliminate the perspective transformation error and ensure the competitive accuracy.

High-Accuracy 3-D Sensor for Rivet Inspection Using Fringe Projection Profilometry with Texture Constraint.

Riveted workpieces are widely used in manufacturing; however, current inspection sensors are mainly limited in nondestructive testing and obtaining the high-accuracy dimension automatically is difficult. We developed a 3-D sensor for rivet inspection using fringe projection profilometry (FPP) with texture constraint. We used multi-intensity high dynamic range (HDR) FPP method to address the varying reflectance of the metal surface then utilized an additional constraint calculated from the fused HDR texture to compensate for the artifacts caused by phase mixture around the stepwise edge. By combining the 2-D contours and 3-D FPP data, rivets can be easily segmented, and the edge points can be further refined for diameter measurement. We tested the performance on a sample of riveted aluminum frame and evaluated the accuracy using standard objects. Experiments show that denser 3-D data of a riveted metal workpiece can be acquired with high accuracy. Compared with the traditional FPP method, the diameter measurement accuracy can be improved by 50%.