Digital Fringe Research Articles

Nonlinear correction method of digital fringe projection 3D measurement system based on precise precoding

Nonlinear correction method of digital fringe projection 3D measurement system based on precise precoding

Single frame digital phase-shift fringe projection profilometry based on symmetry transform

Single frame digital phase-shift fringe projection profilometry based on symmetry transform

Pixel-wise Phase Map Fusion Technique for High Dynamic Range 3-D Shape Measurement

Abstract Digital fringe projection technologies have been extensively applied in 3-D shape measurements. However, it is still limited to achieve accurate 3-D reconstruction of high reflective surfaces. To circumvent this problem, we propose a complete framework to realize pixel-by-pixel phase post-compensation of overexposed points, including the construction of hardware system and the guidelines of algorithm design. First, a virtual binocular vision system is established. Second, based on the fundamentals of projective and spatial transformation, three robust algorithms with low computational complexity are devised to generate a fused phase map that overcomes the information loss due to overexposure. Furthermore, experiments are conducted to evaluate the performance of the proposed method. Qualitative experiment results show that the presented technique achieve a similar surface coverage compared with the other two mainstream methods, but is superior to them in terms of measurement efficiency (13 images vs. 144 images, 13 images vs. 32 images). Quantitative experimentation further verifies the feasibility of our method in precise 3-D measurement, with sphere diameter errors of 0.0367mm and 0.0361mm, spherical centroid distance error of 0.0269mm, and plane distance error of 0.0186mm.

Image-based non-isotropic point light source calibration using digital fringe projection.

Accurate light source calibration is critical in numerous applications including physics-based computer vision and graphics. We propose an image-based method for calibrating non-isotropic point light sources, addressing challenges posed by their non-radially symmetric radiant intensity distribution and the non-Lambertian properties of the calibration target. We deduce an image formation model, and capture the intensity of the calibration target at multiple poses, coupled with accurate 3D geometry acquisition using the digital fringe projection technique. Finally, we design an iterative computational framework to optimize all the light source parameters simultaneously. The experiment demonstrated the high accuracy of the calibrated light source model and showcased its application by estimating the relative reflectance of diverse surfaces.

The optimal algorithm for eliminating nonlinear error in phase measurement profilometry based on global statistical phase feature function

In a digital fringe projection structured light system, the nonlinear phase error is generated by the gamma effect of both the projector, camera, and other electronic devices. One of the existing nonlinear correction methods is active correction by projecting ideal fringes as far as possible, and the other is passive compensation after capturing aberrant fringes. The former has higher accuracy but needs to capture a large number of fringe patterns, while the latter does not need many fringe patterns, but is not only greatly affected by random noise and out-of-focus effects, but also has poor accuracy. In this paper, an optimal algorithm for eliminating nonlinear error based on global statistical phase feature function (GSPF) is proposed. The phase distribution can be estimated from the difference between the global cumulative distribution function (CDF) and the normalized (CDF). For an ideal fringe pattern without nonlinear error and a fringe pattern with nonlinear error, the region wrapped by the x-axis normalized CDF is much smaller than the region wrapped by the x-axis global CDF, and the larger the nonlinear error is, the larger the difference between the two is. Therefore, the GSPF can be used for nonlinear error correction. Then the optimal nonlinear error correction is performed based on the minimum difference between the compensated phase entropy and the ideal phase entropy. The method does not require too many steps of phase-shifting, and only three fringe patterns are needed to realize accurate and robust correction. Experimental results show that the method is fast, highly accurate and robust. Using this technique, high accuracy measurements can be achieved with the traditional three-step phase-shifting algorithm.

Motion-Induced Error Reduction for Motorized Digital Fringe Projection System

Motion-Induced Error Reduction for Motorized Digital Fringe Projection System

Recovery of the response function of a flexible membrane to a volume oscillation using double digital fringe projection technique and frequency filtering

This study presents a method to recover the response function of a flexible latex membrane undergoing a dynamic event by volume oscillation using Double Digital Fringe Projection to measure the profile of the surface in a sequence of frames. By applying a frequency filter to the interference pattern, a moiré pattern was obtained. Optical phase retrieval from individual frames was performed using the Isotropic Quadrature Transform Algorithm. This fringe projection technique enables full-field measurements, including edges where other techniques could not perform due to shadows generated by the tilt angle. The procedure was devised as a non-contact system to retrieve the deformation function of a flexible surface, enhancing a valuable aspect of the methodology. The results demonstrate successful recovery of the membrane response function, highlighting the relevance of the method for accurate deformation measurements using a straightforward system. These outcomes provide significant value for the development of precise measurement techniques for flexible structures in engineering applications, such as soft robotics, biomedical devices, and material science.

Human Respiration Rate Measurement with High-Speed Digital Fringe Projection Technique.

This paper proposes a non-contact continuous respiration monitoring method based on Fringe Projection Profilometry (FPP). This method aims to overcome the limitations of traditional intrusive techniques by providing continuous monitoring without interfering with normal breathing. The FPP sensor captures three-dimensional (3D) respiratory motion from the chest wall and abdomen, and the analysis algorithms extract respiratory parameters. The system achieved a high Signal-to-Noise Ratio (SNR) of 37 dB with an ideal sinusoidal respiration signal. Experimental results demonstrated that a mean correlation of 0.95 and a mean Root-Mean-Square Error (RMSE) of 0.11 breaths per minute (bpm) were achieved when comparing to a reference signal obtained from a spirometer.

Uncertainty of digital fringe projection measurement caused by structural parameters

Digital fringe projection (DFP) has been widely applied in three-dimensional (3D) precision measurement owing to its advantages of non-contact, high-precision, and large-field measurement. The DFP measurement system undergoes a calibration process to define the parameters that reconstruct the surface points from the camera images. As the DFP system is nonlinear, evaluating the measurement uncertainty caused by the calibration deviation of the structural parameters is a great challenge. In this study, we performed an uncertainty analysis based on the binocular vision model of a DFP system. A virtual DFP system based on the structural parameters of the actual system was built to remove the influence of other uncertainty factors. The adaptive Monte Carlo method (AMCM) was used to evaluate the uncertainty caused by the deviation of the parameters during calibration. The model and method for calculating the uncertainty of the DFP system proposed in this study are helpful in better understanding the propagation of the uncertainty of structural parameters, determining the critical factors affecting the measurement uncertainty, and providing a theoretical basis for subsequent error compensation and accuracy improvement for DFP measurement systems.

Deep learning-based correction of defocused fringe patterns for high-speed 3D measurement

Digital fringe projection profilometry often faces a trade-off between measurement accuracy and efficiency. Defocus technology is commonly employed to address this challenge and improve the efficiency of high-speed three-dimensional (3D) measurement. This technology uses 1-bit binary fringe patterns instead of traditional 8-bit sinusoidal patterns, but accurately measuring 3D shapes with both high speed and accuracy remains a challenge due to defocus errors. These errors are introduced by the manual adjustment of lens focal length and reduce both fringe pattern quality and measurement accuracy. To overcome this limitation, we propose a multi-stage generative adversarial network with a self-attention mechanism to correct inaccurate fringe patterns and transform them into more ideal sinusoidal fringe patterns. Our generation network comprises a multi-stage feature extraction network with a self-attention mechanism and an encoder-decoder network. A multi-stage network integrating residual and transformer modules is constructed to mine global feature information. The self-attention mechanism accurately detects key areas for correction, and the encoder-decoder network generates rectified sinusoidal fringe patterns by combining the feature information with the attention area. We use a discriminative network to evaluate whether the output of the generative network is good enough to be true. In our experiments, we considered different fringe widths and measured objects of various types and colors. The results show that our proposed method improves the quality of defocus fringe patterns and the accuracy of subsequent 3D reconstruction compared to existing direct defocus methods.

Automatic exposure control method for 3D shape measurement of specular surface based on digital fringe projection

Digital fringe projection (DFP) is widely applied in three-dimensional (3D) shape measurements. However, its performance is severely disturbed while measuring complex surfaces with an extensive range of reflectivity. In this paper, to enhance the underexposed regions and reduce the saturated regions, an automatic and effective method is proposed for the multi-exposure determination of the DFP system. During the whole process, only one unsaturated image must be captured preliminarily, which could establish an adaptive strategy based on the intensity value distribution function. Raw absolute phase maps captured with different exposure times are synthesized at the pixel level for phase retrieval and 3D reconstruction. To evaluate the effectiveness of our method, a high dynamic range method and a global optimal exposure method are introduced for comparison. Verification experiments demonstrate that our method performs better for the 3D reconstruction results of complex surfaces with a large reflectivity range.

Rapid and flexible calibration of DFPP using a dual-sight fusion target.

The parameter calibration of a digital fringe projection profilometry (DFPP) system is a fundamental step and directly related to 3D measurement accuracy. However, existing solutions based on geometric calibration (GC) suffer from the weakness of limited operability and practicality. In this Letter, a novel, to the best of our knowledge, dual-sight fusion target is designed for flexible calibration. The novelty of this target is the ability to directly characterize control rays for ideal pixels of the projector, and to transform the rays into the camera coordinate system, which replaces the traditional phase-shifting algorithm and avoids the error from the nonlinear response of the system. Attributed to the excellent position resolution of a position-sensitive detector within the target, the geometric relationship between the projector and camera can be easily established by projecting only one diamond pattern. Experimental results demonstrated that the proposed method using only 20 captured images is capable of achieving comparable calibration accuracy to the traditional GC method (20 images versus 1080 images, 0.052 pixels versus 0.047 pixels), which is suitable for rapidly and accurately calibrating the DFPP system in the 3D shape measurement field.

Digital Fringe Projection-Based Clamping Force Estimation Algorithm for Railway Fasteners

The inspection of railway fasteners to assess their clamping force can be used to evaluate the looseness of the fasteners and improve railway safety. Although there are various methods for inspecting railway fasteners, there is still a need for non-contact, fast inspection without installing additional devices on fasteners. In this study, a system that uses digital fringe projection technology to measure the 3D topography of the fastener was developed. This system inspects the looseness through a series of algorithms, including point cloud denoising, coarse registration based on fast point feature histograms (FPFH) features, fine registration based on the iterative closest point (ICP) algorithm, specific region selection, kernel density estimation, and ridge regression. Unlike the previous inspection technology, which can only measure the geometric parameters of fasteners to characterize the tightness, this system can directly estimate the tightening torque and the bolt clamping force. Experiments on WJ-8 fasteners showed a root mean square error of 9.272 N·m and 1.94 kN for the tightening torque and clamping force, demonstrating that the system is sufficiently precise to replace manual measurement and can substantially improve inspection efficiency while evaluating railway fastener looseness.

Improving the localisation of features for the calibration of cameras using EfficientNets.

Camera-based methods for optical coordinate metrology, such as digital fringe projection, rely on accurate calibration of the cameras in the system. Camera calibration is the process of determining the intrinsic and distortion parameters which define the camera model and relies on the localisation of targets (in this case, circular dots) within a set of calibration images. Localising these features with sub-pixel accuracy is key to providing high quality calibration results which in turn allows for high quality measurement results. A popular solution to the localisation of calibration features is provided in the OpenCV library. In this paper, we adopt a hybrid machine learning approach where an initial localisation is given by OpenCV which is then refined through a convolutional neural network based on the EfficientNet architecture. Our proposed localisation method is then compared with the OpenCV locations without refinement, and to an alternative refinement method based on traditional image processing. We show that under ideal imaging conditions, both refinement methods provide a reduction in the mean residual reprojection error of approximately 50%. However, in adverse imaging conditions, with high noise levels and specular reflection, we show that the traditional refinement degrades the results given by pure OpenCV, increasing the mean residual magnitude by 34%, which corresponds to 0.2 pixels. In contrast, the EfficientNet refinement is shown to be robust to the unideal conditions and is still able to reduce the mean residual magnitude by 50% compared to OpenCV. The EfficientNet feature localisation refinement, therefore, enables a greater range of viable imaging positions across the measurement volume. leading to more robust camera parameter estimations.

Gamma estimation method based on phase error coefficients vector in digital fringe projection system

For phase-measuring profilometry systems, gamma effect is important to the accuracy while its gamma factor cannot be measured goodly when there are a few images. To tackle the problem, this paper proposes a novel idea: calculate the phase errors coefficients vector (PECV) of a series of heuristic gamma values, and measure the distances from the PECV of the current measurement to those, respectively; then, the gamma corresponding to minimal distance is the measured gamma factor. The idea does not measure the gamma factor straightforwardly and novelly measures the phase error distances. The technique is fast and robust, and only needs at least six images. Experimental results demonstrate the feasibility and performance.

Optimized dithering approach for 3D shape measurement using a phase-based chaos genetic algorithm.

A binary defocusing method has a significant impact on improving 3D shape measurement quality for digital fringe projector (DFP) techniques. In this paper, an optimization framework using the dithering method is presented. This framework utilizes genetic algorithm and chaos maps to optimize the bidirectional error-diffusion coefficients. It can effectively avoid quantization errors of binary patterns in a specific direction and obtain the fringe patterns with better symmetry and higher quality. In the process of the optimization, chaos initialization algorithms are used to generate a series of bidirectional error-diffusion coefficients as initial individuals. Additionally, mutation factors generated by chaotic maps, compared with the mutation rate, determine whether the individual position will mutate. Both simulations and experiments demonstrate that the proposed algorithm can improve the quality of phase and reconstruction at different defocus levels.

An Area Modulation Binary Defocusing-Based Digital Fringe Projection 3-D Shape Measurement Method for Print Circuit Board Inspection

An Area Modulation Binary Defocusing-Based Digital Fringe Projection 3-D Shape Measurement Method for Print Circuit Board Inspection

Adaptive Phase Error Suppression Concerning 3D surface Deformation Measurement on Color Digital Fringe Projection Profilometry

3D surface measurement based on phase-shifting profilometry (PSP) has been actively developed in recent years. Three color channels of RGB that are modulated to generate a one-shot PSP method is a concept of color digital fringe pattern profilometry (CDFPP). The CDFPP is a promising technique for the 3D imaging profile of dynamic surface deformation if several phase errors in the one-shot PSP method can be suppressed. This work proposes a processing scheme for phase error suppression schemes (PESS) based on retrieving the modulated sinusoidal fringe and color fringe normalization in PSP using RGB color channel. The processing of PESS consists of tunable bandpass filtering (BPF) followed by fringe normalization. The initial BPF function is defined based on a smoothing spline data set of frequency and power spectrum from the baseline color fringe image. The predefine BPF function could be tunable during the imaging process by considering each frame's condition and RGB channel spectrum mapping. The corrected fringe images are then normalized from the color imbalance, and the phase shift is calculated using the conventional three-step PSP. For evaluation, PESS is performed to reconstruct simulator membrane deformation from four different static profiles and tested to observe the 3D surface of continuous membrane deformation. The PESS could suppress the phase errors of less than 30% less absolute errors than the conventional method and successfully reconstruct the 3D surface for low-frequency continuous membrane deformation with minimizing phase errors.

Warpage characterization of a large size fan-out panel subjected to inhomogeneous heating by a digital fringe projection system

Warpage is an essential issue of fan-out wafer/panel level packaging. Increasing size of wafers/panels poses new challenges to the warpage characterization system. In this work, a warpage measurement system based on digital fringe projection method is reported. It can cover 600 mm × 600 mm area with 4 × 4 camera array. Benchmark tests were carried out to evaluate the measurement error, which was found to be 0.053 %. The warpage evolution of a 515 mm × 510 mm panel with inhomogeneous heating was then studied by heating from room temperature to 90 °C. It was found that the warpage evolution of a large size panel can be very complex. Based on finite element modeling results, this can be attributed to temperature gradient, and several other factors, including mismatch of coefficient of thermal expansion, gravity, initial stress state and original layout design of the panel (such as size and count of chips, RDL design and chip position).

Theoretical analysis and experimental investigation of the Floyd-Steinberg-based fringe binary method with offset compensation for accurate 3D measurement.

Digital fringe projection (DFP) with defocused binary fringe patterns has the ability to overcome the projector nonlinearity and achieve a high-speed 3D measurement. The Floyd-Steinberg (FS) dithering technique is one of the most commonly adopted binary fringe coding methods due to its relatively high measurement accuracy. Nevertheless, we found that the FS binary fringe would cause a fixed error in the recovered phase, which is proven to be invariable for various defocusing levels and various phase-shift steps according to the analysis of the phase error based on noise model of phase-shifting profilometry. It means that FS binary fringe would have a certain offset in space, compared with standard sinusoidal fringe, which is verified to be essentially constant for different fringe pitches through simulation and experiment. This offset would distort the 3D geometry of the tested target for monocular systems relying on triangulation, which needs to be compensated to improve 3D measurement accuracy. Experiments are presented to demonstrate the enhanced 3D result after compensation.