Captured Fringe Patterns Research Articles

Decomposition and compensation of fringe harmonic errors by use of their partial orthogonality in phase-shifting fringe projection profilometry

In phase-shifting fringe projection profilometry, fringe harmonics have been recognized as one of the main error-inducing factors. Generally, the response of a phase-shifting algorithm to fringe harmonics strongly depends on the used phase shifts and is usually unpredictable, especially when using nonuniform phase shifts. For this reason, it is difficult to eliminate the phase-measuring errors caused by fringe harmonics in a general case, for example, when the phase shifts are not uniform but arbitrarily valued. To overcome this problem, this paper analyzes the phase error function related to each fringe harmonic under the condition of using arbitrary phase shifts, reveals the partial orthogonality of these functions, and then derives an algorithm for decomposing the harmonics-induced phase errors. In the implementation, this algorithm calculates a coarse phase map first in the least squares sense from captured fringe patterns, and then estimates the coefficients of fringe harmonics from this calculated phase map by use of the partial orthogonality of the error functions. By using the estimated harmonic coefficients, the phase map is updated, thus having improved accuracies so that the harmonics-induced phase errors are compensated for through an iterative procedure. The simulation and experimental results demonstrate this method to be effective and efficient in measuring fringe phases.

Three-dimensional shape measurement technique for hot and shiny forging

For hot and shiny forging, rapid optical three-dimensional (3D) shape measurement can find the defects in time, improve manufacturing efficiency and reduce manufacturing cost. Optical 3D measurement of hot and shiny forging faces following three challenges. (a) Influence of blackbody radiation on the red band of visible light leads to saturation of captured fringe pattern. (b) Infrared band of blackbody radiation causes the camera to heat up. (c) Specular reflection of shiny forging leads to local saturation of captured fringe pattern. To address the above issues, we propose an effective optical 3D measurement scheme for hot and shiny forging in this paper. Firstly, the experiments demonstrate that the influence of blackbody radiation on visible light is concentrated in red light band and has no crosstalk in the blue light band, so we adopt blue light projection and blue light band-pass filter (BBF) to capture clear fringe pattern. Secondly, since the BBF cannot block the near-infrared component, we use an infrared light cutoff filter (ICF) to reduce the thermal effect of blackbody radiation. Finally, we analyze the Fourier transform (FT) spectrum of saturated fringe pattern caused by specular reflection, and propose an efficient Butterworth low-pass filtering (BLPF) method that does not require additional hardware, algorithm processing, and image assistance. The experimental results verify the effectiveness of the proposed scheme.

Nonlinear error reduction for phase-shifting profilometry considering periodicity and symmetry of a phase histogram.

Phase-shifting profilometry is extensively utilized for three-dimensional (3D) measurement. However, because of gamma nonlinearity, the image intensities of the captured fringe patterns are regrettably distorted. An effective nonlinear error reduction method without requiring parameter estimation is presented in this paper. Differing from the traditional whole-period phase histogram equalization (PHE) method, our method takes into account not only the periodicity but also the symmetry of the phase histogram. Taking a three-step phase-shifting algorithm as an example, the phase error frequency triples the fringe frequency; thus, we first propose a 1/3-period PHE method. Moreover, since the phase error distribution is sinusoidal with symmetry, we further propose a 1/6-period PHE method. Simulations and experiments both indicate that the 1/6-period PHE method, compared with the whole-period PHE and 1/3-period PHE methods, can further reduce the nonlinear error.

Quasi-pointwise two-step phase-shifting profilometry with the fringe parameters estimated statistically.

Fringe projection profilometry is a popularly used three-dimensional measurement technique in which phase-measuring algorithms based on two-step phase shifting are usually used because of their best tradeoff between measurement resolution and speed. Most two-step phase-shifting algorithms involve neighboring or other spatial operations, thus having degraded accuracies at edges and discontinuities of the measured object surface. Pointwise two-step algorithms enable overcoming this issue. With them, however, the offsets of the dynamic ranges of the projector and camera are usually improperly overlooked or inaccurately estimated, thus inducing errors in their measurement results. For solving this problem, this paper suggests a quasi-pointwise two-step phase-shifting algorithm for fringe projection profilometry. This algorithm models the captured fringe patterns practically by taking the offsets of the dynamic ranges of the projector and camera into account, and estimates the fringe parameters from the statistics of fringe intensities. As a result, we can calculate fringe phases in a pointwise way from two fringe patterns having a phase difference of π/2 radians. The simulation and experimental results demonstrate that the proposed method has a relatively low level of errors in measuring object surfaces having isolated regions and discontinuities.

Two-frame advanced iterative self-tuning algorithm for accurate phase retrieval

Two-shot random phase-shifting interferometry has been actively investigated and improved owing to more demanding requirements in investigating dynamic phenomena or sensing some transient events. Given the background intensity and phase step knowledge, a reconstruction algorithm can be used to recover a complex-valued signal from intensity-only measurements. Most algorithms assume the background intensity and modulation amplitude in a frame of interferogram are constant variables; however, in realistic systems, they vary laterally and axially across the field of view. The model can be used but often appears less precise. In this work, we propose an approach that leverages the iterative method to cope with spatially varying background intensity and modulation amplitude. Our approach, termed the two-frame advanced iterative self-tuning algorithm, uses the spatial mean algorithm paired with an iterative procedure to search the phase step from two captured fringe patterns. It represents a novel approach to reliable and practical two-step phase-shifting interferometry without pre-filtering due to the cancelation of simultaneously obtaining the phase shifts and measured phase through iterative operation in the spatial–temporal domain. Experimental results obtained using the proposed method indicate that it is a simple and robust solution for phase extraction from a two-frame unknown phase shift fringe pattern with spatially varying background intensity and modulation amplitude.

A novel phase unwrapping method for binocular structured light 3D reconstruction based on deep learning

Fringe projection has been prevalently adopted in the field of three-dimensional(3D) reconstruction. However, it is still a challenge to achieve high-precision and efficient measurement. To achieve real-time and efficient 3D shape measurement of the objects, we must extract the 3D coordinate data of the object from as few fringe patterns as possible. In the paper, we develop a new technique combining composite fringe coding and stair phase coding to solve the high-quality absolute phase and realize binocular 3D morphology reconstruction based on deep learning. This method only needs to project a composite fringe coding pattern to acquire the wrapped phase and a stair phase coding pattern to obtain the fringe order. Then we can recover the unwrapped phase maps of the captured fringe patterns. According to the calibration parameters of the binocular camera, the absolute phase is stereo matched to get the disparity map, and then converted into 3D spatial coordinates of the object. The influence of defocus and noise on the deviation between the wrapped phase and fringe order is solved by staggered phase unwrapping method. We demonstrated the validity and robustness of the novel method by experiments, which provides a promising research direction for 3D shape measurement based on binocular structured light.

Defect classification for specular surfaces based on deflectometry and multi-modal fusion network

Automated defect inspection for specular surfaces is still a challenge in the manufacturing industry because of their specular reflection property. Deflectometry provides surface information based on the captured fringe patterns through the reflection of the specular surfaces and has been widely applied in defect detection for specular surfaces. Conventional methods combined deflectometry with machine learning approaches, but the hand-crafted features need to be defined for each specific task. Combined with the deep neural network, the input images are obtained from deflectometry, and the network completes the identification of the defects. Nevertheless, conventional deep-learning-based defect inspection methods approached the problem as a binary classification, or only certain obvious defects can be correctly classified. In this study, we generated and released, for the first time, to the best of our knowledge, the benchmark dataset named SpecularDefect9 with various defects for specular surfaces, and the classification accuracy of some kinds of defects may be low with only one kind of input image. To classify all kinds of defects accurately, the proposed method applied the light intensity contrast map combined with the original captured fringe pattern as the input of the network, and a fusion network was introduced to extract features from multi-modal inputs. Experimental results based on the released benchmark dataset verified the effectiveness and robustness of the proposed multi-modal defect classification method.

Three-dimensional reconstruction of moving HDR object based on PSP

High quality 3D reconstruction of HDR object can be achieved by projecting extra fringe patterns with different intensity. However, it requires the object to be kept static. The object motion not only introduces phase shift among the fringe patterns, but also changes the position of the overexposure pixels. This paper proposes a new method to reconstruct the moving HDR object by utilizing the features of motion. For the specific point on the object, overexposure will not happen among all the fringe patterns as the overexposure position is changed with the motion. The unsaturation fringe patterns are selected from the captured fringe patterns and the correct phase information is retrieved based on unequal phase shift algorithm. The effectiveness of the proposed algorithm is verified by the experiments.

Polynomial color compensation model with neural network solver for color-coded fringe patterns in phase-measuring deflectometry

The total image display/projection and image acquisition time in a projector-based phase-measuring deflectometry (PMD) measurement is reduced when color-coded fringe patterns are used since only a single image of the fringe pattern is sufficient to extract phase information using the phase-stepping method. This benefit is, however, offset by the difficulty in resolving various sources of color intensity errors of the fringe patterns in the camera view, such as uneven illumination, color imbalance, color crosstalk, and deviation of color intensity response of the camera and the projector from the calibrated color targets. Thus, a polynomial color compensation model is proposed in this study to compensate for color intensity errors. The model coefficients were estimated using the relationship between the captured and targeted color intensities, which are sequences of monochromatic 8-bit red, green, and blue (RGB) images for projection. The model was applied to modify the color intensities of the generated fringe patterns before projection. The modified color fringe patterns were then used for PMD measurement on spherical concave and convex mirrors. Based on the experimental results, the root-mean-squared (RMS) color intensity errors in R, G, and B channels were reduced by 33.3%, 58.3%, and 36.4%, respectively, after conducting the polynomial color compensation. The RMS phase errors in the fringe patterns were reduced by 71.9%. The RMS height errors and the radius percentage errors of the reconstructed concave and convex heightmaps were 1.87 μm (0.12%) and 1.55 μm (0.17%), respectively. The performance of the proposed polynomial color compensation model in reducing color intensity errors of the captured fringe patterns for accurate PMD measurement was demonstrated successfully.

Pixel-wise structured light calibration method with a color calibration target.

We propose to use a calibration target with a narrow spectral color range for the background (e.g., from blue) and broader spectral color range for the feature points (e.g., blue + red circles), and fringe patterns matching the background color for accurate phase extraction. Since the captured fringe patterns are not affected by the high contrast of the calibration target, phase information can be accurately extracted without edging artifacts. Those feature points can be clearly "seen" by the camera if the ambient light matches the feature color or without the background color. We extract each calibration pose for three-dimensional coordinate determination for each pixel, and then establish pixel-wise relationship between each coordinate and phase. Comparing with our previously published method, this method significantly fundamentally simplifies and improves the algorithm by eliminating the computational framework estimate smooth phase near high-contrast feature edges. Experimental results demonstrated the success of our proposed calibration method.

Invalid point removal method based on error energy function in fringe projection profilometry

Invalid points, such as shadow and background, in the captured fringe patterns of fringe projection profilometry (FPP) are often inevitable due to the limited field of view measurements of three-dimensional (3D) imaging equipment. To ensure the quality of 3D reconstruction data, these invalid points must be identified and removed. FPP captures co-frequency-based fringe pattern sequences and approximately distributes this data along an ideal cosine curve. We propose an invalid points removal method based on an error energy function. By analyzing the relationship between the pixel values of a series of captured co-frequency patterns and an ideal cosine curve, we quantize the error energy by using a Gaussian weighted Euclidean distance. An improved Gaussian filtering method based on modulation intensity is used to significantly increase the error energy difference between the valid points of the target and the invalid points of the shadow/background area. Finally, the points whose error energy is greater than a certain error threshold are removed as invalid points. Experimental results indicate that this method can efficiently remove invalid points in fringe patterns and performs better than existing traditional methods.

Anti-aliasing phase reconstruction via a non-uniform phase-shifting technique.

The conventional phase-shifting techniques commonly suffer from frequency aliasing because of their number of phase shifts below the critical sampling rate. As a result, fringe harmonics induce ripple-like artifacts in their reconstructed phase maps. For solving this issue, this paper presents an anti-aliasing phase-measuring technique. Theoretical analysis shows that, with phase-shifting, the harmonics aliased with the fundamental frequency component of a fringe signal depend on the greatest common divisor (GCD) of the used phase shifts. This fact implies a possibility of removing such aliasing effects by selecting non-uniform phase shifts that together with 2π have no common divisors. However, even if we do so, it remains challenging to separate harmonics from the fundamental fringe signals, because the systems of equations available from the captured fringe patterns are generally under-determined, especially when the number of phase shifts is very few. To overcome this difficulty, we practically presume that all the points over the fringe patterns have an identical characteristic of harmonics. Under this constraint, using an alternate iterative least-squares fitting procedure allows us to estimate the fringe phases and the harmonic coefficients accurately. Simulation and experimental results demonstrate that this proposed method enables separating high order harmonics from as few as 4 fringe patterns having non-uniform phase shifts, thus significantly suppressing the ripple-like phase errors caused by the frequency aliasing.

High dynamic range fringe pattern acquisition based on deep neural network

Fringe projection profilometry has wide applications in three-dimensional measurement. When the fringes are projected onto the shiny surfaces with a large reflectivity, however, the saturation regions and dark regions are often generated on the surface due to the inconsistent reflection characteristics of the measured object. The saturation and dark regions make the modulation of captured fringe patterns degraded, and the accuracy of the three-dimensional measurement is significantly affected. To solve this problem, an improved network for high dynamic range fringe pattern is designed In this network, we introduce a low modulation region detection module based on U-net to obtain the saturation and dark regions in the fringe patterns. Then, the detailed fringe information in the low modulation regions can be predicted through a fringe enhancement module which takes advantage of the global distribution and detail intensity of the fringe patterns. In the experiment, a mass of simulated and actual low modulation fringe patterns are processed through the proposed network. The experimental results show that the designed network can effectively remove the noise of the captured fringe patterns and repair the low modulation regions captured from the shiny surfaces. Combined with phase calculation, the improved high dynamic range fringe patterns are used for three-dimensional data calculation. A standard metal gauge block with a height of 5 mm is measured and the root mean square error of the height is improved from 0.55 mm to 0.06 mm. This proves the proposed method can effectively improve the quality of fringe patterns projected on complex reflective surfaces.

One-shot deflectometry for high-speed inline inspection of specular quasi-plane surfaces

Deflectometry is a non-contact optical technique which uses a simple system setup to measure and inspect specular surfaces. Conventional deflectometry methods usually need very complicated system calibration and time-consuming phase-shifting methods, which are not practical for high-speed measurement. But if the measurement target is a quasi-plane surface where the local slope is directly modulated by phase, the complicated system calibration can be neglected. In this study, to reduce error, we investigated and analyzed error contribution when measuring quasi-plane surfaces. A simple and high speed one-shot phase retrieval method is proposed for quickly measuring surface phase. We used a single high carrier-frequency composite pattern for pattern projection, and derivation of the captured fringe pattern for phase retrieval. A phase-related slope is obtained for slope calculation and the 3D shape is reconstructed by integrating the surface slopes. We verified our proposed method by measuring representative quasi-plane surfaces and comparing results with conventional methods.

3-D Measurement Method for Multireflectivity Scenes Based on Nonlinear Fringe Projection Intensity Adjustment

The 3-D measurement methods based on fringe projection intensity adjustment (FPIA) have been widely adopted to measure the multireflectivity scenes. However, the signal-to-noise ratios (SNRs) of the captured fringe patterns in the FPIA-based methods are reduced because of a large range of reflectivity variations (LRRVs). In our research, a reflection model and a crosstalk model are built to analyze the disturbances induced by LRRVs on the SNRs in the captured fringe patterns, and then, a 3-D measurement method based on nonlinear FPIA (NLFPIA) is proposed to alleviate the LRRV-induced disturbances and improve the measurement accuracy. Moreover, to improve the efficiency of the proposed 3-D measurement method, we present a cyclic utilization of fringe patterns (CUFPs) technique to decrease the number of projection patterns. Experimental results show that the proposed NLFPIA-based 3-D measurement method reduces the root-mean-square error (RMSE) in the measurements of the two standard ceramic geometries by 52.9845% and 36.6393% compared with the FPIA-based method.

Large depth-of-field 3D measurement with a microscopic structured-light system

Fringe projection techniques have been widely used in three-dimensional (3D) microscopic measurement; however, the working volume is severely restricted due to the shallow depth-of-field (DOF) of the telecentric lenses used. In this paper, a large-DOF 3D measurement method for use with a microscopic structured-light system was proposed. By employing a precisely controlled moving platform, the height of the camera can be adjusted to capture several sets of fringe patterns with different amounts of defocusing. Then the proposed focusing map generation algorithm was employed to calculate the focusing degree of each pixel and generate a focusing map for each set of fringe patterns. Subsequently, local 3D shape data corresponding to in-focus pixels were merged with the guidance of the focusing maps, and global fine 3D measurement data without the influence of lens blur can be obtained. Specifically, to guide the compositing of the 3D shape data, a novel, simple focusing map generation algorithm was proposed to calculate the focusing degree of each set of captured fringe patterns. In contrast to existing DOF-extending 3D measurement techniques using multiple focal distances, the proposed algorithm does not need the object’s texture to calculate the gray-level gradient; hence, complex convolution or iteration is not required. The experimental results demonstrate that the proposed technique can increase the measurement DOF to approximately three times that of a conventional system.

Computer-generated moir\xe9 profilometry based on fringe-superposition

A computer-generated moiré profilometry based on algebraic addition instead of algebraic multiplication is proposed. Firstly, the two AC components of the captured fringe patterns on the reference plane with pi /2 phase difference are retrieved and saved in advance. While measuring, two sinusoidal gratings with pi phase difference are projected onto the measured object alternatively, and the corresponding deformed patterns are captured. Then the AC component of the captured deformed pattern can be separated exactly. When the positive and negative AC component of the captured deformed pattern are added to the two prestored AC components respectively, two moiré fringes only reflect sine and cosine of the object’s phase information can be successfully generated via a series of data processing procedures. Finally, the phase distribution of the measured object can be extracted by arctangent of the ratio of these two moiré fringes. Compared with computer-generated moiré profilometry based on algebraic multiplication, this proposed method can reduce the effect of high frequency noise and residual DC component on measurement and improve the measurement accuracy. While compared with pi phase shifting FTP, this method can measure more complex objects with better measurement capability. Experimental results verify the feasibility and validity of the proposed method.

Hue-Based Gray Coding Method for Three-Dimensional Surface Measurement of Cutlery With Specular Reflection

Many measurement methods have been proposed for use in automated production. Existing methods for measuring three-dimensional surface height data operate by projecting fringe patterns onto a target object and capturing images of them. However, these methods do not work well for cutlery, such as table forks and spoons, because specular reflection causes exposure change on the cutlery surface. To overcome this difficulty, this paper presents a measurement method for cutlery with specular reflection. The proposed method is based on a well-known measurement technique, the gray coding method, which is robust to exposure change because the captured fringe patterns are handled by discretizing their values. Furthermore, the gray coding method is improved by introducing hue-channel information into the captured fringe patterns. The hue-channel information is more robust to exposure change than brightness, which is used in existing methods. Experimental results for real cutlery show that the proposed hue-based gray coding method enables more robust measurement than existing methods.

条纹投射技术中的投影机非线性自校正方法研究进展

In fringe projection profilometry, the luminance nonlinearity of the projector has been recognized as one of the most crucial factors decreasing the measurement accuracy. It induces the ripple-like artifacts on the measured phase map. The self-adaptive correcting algorithms, i.e., self-correcting algorithms, allow us to suppress the effect of the projector nonlinearity without a prior calibration for the projector intensities or phase errors. This paper introduces the research progress in the self-correcting algorithms. Among them, the first algorithm is to determine a nonlinear curve representing the projector nonlinearity, directly from the captured fringe patterns, thus correcting the phase errors using this curve. The second one is to recognize and remove the nonlinearity-induced errors, directly from a calculated phase map. With the last one, error function coefficients are estimated from a couple of phase maps having different frequencies. Measurement results demonstrate these self-correcting algorithms to be effective in suppressing influences of the projector nonlinearity in the absence of any calibration information.

Three-dimensional shape measurement of aspheric mirrors with null phase measuring deflectometry

A null phase measuring deflectometry (PMD) is presented to measure the three-dimensional shape of aspheric mirrors with radial symmetry. Curved fringes displayed on a mobile screen are reflected by the surface under test, and a camera captures the fringe patterns. By analyzing the captured fringe patterns, the deviations along the normal of the mirror’s surface can be restored by integrating the slopes of these deviations, and then the surface under test can be reconstructed. Compared with non-null PMD, the qualitative information of surface in null PMD can be shown fast and easily for a surface with large deviations in the grinding stage. Computer simulation and experiment are presented to validate the presented method.