Projected Fringe Pattern Research Articles

3D shape measurement of shiny surfaces based on optimized combination of fringe patterns of different intensity

Image saturation is a challenging problem for three-dimensional shape measurement of objects with shiny surfaces using fringe projection profilometry technology. This paper presents a new method based on projection of multiple fringe patterns, providing an effective solution to the problem. First, a set of phase shifted sinusoidal fringe patterns with different intensity levels are projected to the object of interest, and the reflections are captured by the camera. With the captured images a set of masks are created, which are able to pick up the unsaturated data from the captured images. The unsaturated data are then combined based on maximal Signal-to-noise Ratio (SNR) scenario, and the combined images are used to retrieve the phase information. Comparing to existing methods, the proposed technique is advantageous by improved measurement accuracy due to the improved SNR. The performance of the proposed method is verified by the experiments.

A three-dimensional measurement system calibration method based on red/blue orthogonal fringe projection

Structured light measurement system that consists of a camera and a projector has been widely used in three-dimensional (3-D) shape measurement. The premise of measurement is to calibrate the structured light system, and the calibration accuracy directly affects the accuracy of 3-D measurement system. The traditional calibration board-based methods suffer from either complex operations or uncompetitive accuracy. To overcome these limitations, this paper proposes a novel calibration method by introducing a liquid crystal display (LCD) with holographic projection film as a phase target to simplify the operations and improve the speed and accuracy of system calibration. The proposed calibration procedures are given as follows: Firstly, the LCD is turned on and kept still to display the red/blue orthogonal fringe pattern, and it is then turned off to reflect the projected fringe pattern with the holographic projection film, the brightness and frequency of the two patterns are adjusted and captured; Secondly, the accurate absolute phase maps are obtained using optimized phase calculation and overall error compensation method; Finally, the above two steps are repeated at several different positions and orientations in the measured field of view. The one-to-one correspondence between the camera and the projector pixel coordinate system and the phase target world coordinate system is established by using the phase equality, and the intrinsic and extrinsic parameters of the system are obtained to reconstruct 3-D shape of the objects. The experiment results demonstrate that the proposed method is feasible and the equipment is simple, which can effectively reduce the overall error, ensure the synchronous calibration of the camera and the projector, avoid the problem of time-consuming, and eventually improve the accuracy of the system calibration.

Automatic optimal projected light intensity control for digital fringe projection technique

Automatically adapting the projected light intensity is crucial for high dynamic range 3D shape measurement, where minimum human intervention is usually desirable. This paper presents a method to automatically determine the optimal number of light intensities and corresponding intensity values for fringe pattern projection according to the surface reflectivity of the measured object. The raw fringe images captured at different light intensities are synthesized pixel-by-pixel into the composite fringe images, which can be further used for phase retrieval and conversion to 3D coordinates. Our experiments demonstrate that the proposed method can achieve high measurement accuracy and effectiveness when measuring an object with a large range of reflectivity variations.

3D shape measurement with an uncalibrated camera using the feature quantity type whole-space tabulation method

A prototype of a 3D shape measurement device with two cameras was developed, and the prototype was used to confirm the effectiveness of the 3D shape measurement method that does not require camera parameters. For 3D shape measurement using a fringe projection method, generally the pixel coordinate in the image and phase information of the projected fringe pattern are used; however, 3D coordinates can be obtained from only three fringe phase values. Recently, authors proposed this method as a feature quantity type whole-space tabulation method. There were no camera parameters required because pixel coordinates were not used, and thus a camera calibration-free 3D shape measurement can be realized. Moreover, the experimental evaluation was performed using the prototype having two cameras. Although these cameras were located at different positions and had lenses with different focal lengths, their measured shapes of an object were almost identical. An experiment of 3D shape measurement using an uncalibrated camera was performed. The effectiveness of the proposed method was quantitatively validated from experimental result obtained using the developed prototype.

Method of Contactless Measurement of Large Object Deformations Based on Structured Illumination

The possibility of determining deformations of large-sized objects by structured illumination is considered. The method is based on projection of sinusoidal fringe patterns on the surface of the object under study. Using the step-by-step phase shift allows one to distinguish information about the phase distribution reflecting the surface relief. Comparison of phase distributions before and after the action on the object makes it possible to establish the deformation field. A new method of finding phase differences is presented; it considerably reduces demands to phase ambiguity elimination algorithms. An experimental facility for the determination of ultimate sensitivity of the method when investigating deformations of a clamped cantilever beam is developed. It is shown that the sensitivity of the method is less than 10 $$\mu$$ m at the free end of the beam.

End-to-end deep learning-based fringe projection framework for 3D profiling of objects

Fringe projection techniques are widely used for precise three-dimensional depth profiling of objects. Existing signal processing-based fringe projection techniques measure the phase deformation of the projected fringe patterns with a sequence of operations, such as fringe denoising, fringe analysis for wrapped phase extraction, followed by phase unwrapping. However, the error induced in any of the stages leads to erroneous depth estimation. Furthermore, any aliasing in frequency domain fringe analysis and ill-posed nature of phase unwrapping limit the overall accuracy of the Fringe Projection Profilometry (FPP). To this end, unlike the traditional approaches, we propose a paradigm shift by introducing a novel end-to-end deep learning-based framework for FPP that does not need any frequency domain filtering and phase unwrapping. The proposed framework directly reconstructs the object’s depth profile from the deformed fringe itself through a multi-resolution similarity assessment convolutional neural network. We compare the performance of the proposed framework with two widely used conventional approaches. The evaluations are performed for various challenging and important scenarios such as low fringe-frequency, depth profiles with high dynamic range and noisy fringes. The results demonstrate that the proposed framework achieves promising results in all these scenarios.

Hue-Based Gray Coding Method for Three-Dimensional Surface Measurement of Cutlery With Specular Reflection

Many measurement methods have been proposed for use in automated production. Existing methods for measuring three-dimensional surface height data operate by projecting fringe patterns onto a target object and capturing images of them. However, these methods do not work well for cutlery, such as table forks and spoons, because specular reflection causes exposure change on the cutlery surface. To overcome this difficulty, this paper presents a measurement method for cutlery with specular reflection. The proposed method is based on a well-known measurement technique, the gray coding method, which is robust to exposure change because the captured fringe patterns are handled by discretizing their values. Furthermore, the gray coding method is improved by introducing hue-channel information into the captured fringe patterns. The hue-channel information is more robust to exposure change than brightness, which is used in existing methods. Experimental results for real cutlery show that the proposed hue-based gray coding method enables more robust measurement than existing methods.

High-frequency color-encoded fringe-projection profilometry based on geometry constraint for large depth range.

In multi-view fringe projection profilometry (FPP), a limitation of geometry-constraint based approaches is the reduced measurement depth range often used to reduce the number of candidate points and increase the corresponding point selection reliability, when high-frequency fringe patterns are used. To extend the depth range, a new method of high-frequency fringe projection profilometry was developed by color encoding the projected fringe patterns to allow reliable candidate point selection even when six candidate points are in the measurement volume. The wrapped phase is directly retrieved using the intensity component of the hue-saturation-intensity (HSI) color space and complementary-hue is introduced to identify color codes for correct corresponding point selection. Mathematical analyses of the effect of color crosstalk on phase calculation and color code identification show that the phase calculation is independent of color crosstalk and that color crosstalk has little effect on color code identification. Experiments demonstrated that the new method can achieve high accuracy in 3D measurement over a large depth range and for isolated objects, using only two high-frequency color-encoded fringe patterns.

Real-time high-speed three-dimensional surface imaging using band-limited illumination profilometry with a CoaXPress interface.

High-speed three-dimensional (3D) surface imaging by structured-light profilometry is currently driven by numerous applications. However, the limited speeds in fringe pattern projection, image acquisition, and data transmission have strained the existing methods from reaching kilohertz-level acquisition, processing, and display of 3D information during the occurrence of dynamic events (i.e., in real time). To overcome these limitations, we have developed band-limited illumination profilometry (BLIP) with a CoaXPress interface (CI), which enables real-time high-speed 3D surface imaging. We have demonstrated the system's performance by imaging various static and fast-moving 3D objects in real time. We have also applied this system in fluid mechanics by imaging dynamics of a flag, which allowed observation of the wave propagation, gravity-induced phase mismatch, and asymmetric flapping motion. We expect CI-BLIP to find diverse scientific and industrial applications.

Improving 3D reconstruction accuracy in wavelet transform profilometry by reducing shadow effects

Wavelet transform profilometry is a three-dimensional (3D) reconstruction method based on the structured light technique of fringe pattern projection, widely used because it is a non-invasive, high-performance 3D reconstruction method. The presence of shadows created by the object in the image capture process is an obstacle in obtaining accurate 3D reconstructions, as they add noise to the phase data, leading to artefacts in object reconstruction, even when using robust phase-unwrapping algorithms. Since shadows present diverse intensities and shapes, detecting and eliminating their effects are challenging tasks. This work presents a novel method to detect shadow regions and reduce their effects in 3D reconstruction. The proposed method uses coloured fringe patterns to detect the shadows and mathematical morphology to condition the outlines of the shadow regions. The shadow outline information is used to interpolate the background-plane fringe pattern onto the captured scene, where the shadows are detected. The mean squared error (MSE) of the reconstructed objects is reduced to 25% of the MSE without shadow removal, on an average, when using the Bioucas phase-unwrapping method. When using the Ghiglia phase-unwrapping method, the MSE reduction is to 8.3%, on an average, of the MSE in the shadow case.

Determination and adjustment of optimal defocus level for fringe projection systems.

The sinusoidal fringe pattern is widely used in fringe projection profilometry. Too much or too little defocusing will affect the quality of sinusoidal fringe patterns and consequently jeopardize the accuracy of measurement results. This paper proposes a method to quantify and ascertain the defocus level by simulations and experiments. By simulating the defocus pattern with a Gaussian low-pass filter, the optimum defocus level of the fringe pattern is determined so that the projected fringe pattern is closer to the sinusoidal function. Then, a method is proposed to adjust the projector to make the projected pattern in the optimal defocus degree. Experiments show the feasibility and the validity of the proposed method, and the accuracy is improved up to 9.9%, compared with the focus-projected pattern.

Theoretical proof of parameter optimization for sinusoidal fringe projection profilometry

In sinusoidal fringe projection profilometry (FPP), many factors are known to improve measurement accuracy. However, the parameters of the projected fringe pattern, such as modulation and pitch, are the most basic factors. Some empirical principles have been described by researchers, but there is still a lack of theoretical support. In this study, we attempt to address this problem by providing a theoretical proof. We first build an error model for the FPP system. The model reveals how the pitch, modulation, and noise of the fringe pattern affect the measurement error. Combined with optical projection analysis, an optimal projection pitch is delivered that yields the best performance. Both simulations and experiments verify the proposed theoretical proof and optimization method, which will facilitate the construction of FPP systems and help users avoid empiricism and time-consuming experiments.

Fringe optimization for structured illumination super-resolution microscope with digital micromirror device

Structured illumination microscopy (SIM) is a promising super-resolution technique for imaging subcellular structures and dynamics due to its compatibility with most commonly used fluorescent labeling methods. Structured illumination can be obtained by either laser interference or projection of fringe patterns. Here, we proposed a fringe projector composed of a compact multi-wavelength LEDs module and a digital micromirror device (DMD) which can be directly attached to most commercial inverted fluorescent microscopes and update it into a SIM system. The effects of the period and duty cycle of fringe patterns on the modulation depth of the structured light field were studied. With the optimized fringe pattern, [Formula: see text] resolution improvement could be obtained with high-end oil objectives. Multicolor imaging and dynamics of subcellular organelles in live cells were also demonstrated. Our method provides a low-cost solution for SIM setup to expand its wide range of applications to most research labs in the field of life science and medicine.

Robust Single-shot Fringe Projection Profilometry Based on Morphological Component Analysis.

In a fringe projection profilometry (FPP) process, the captured fringe images can be modeled as the superimposition of the projected fringe patterns on the texture of the objects. Extracting the fringe patterns from the captured fringe images is an essential procedure in FPP; but traditional single-shot FPP methods often fail to perform if the objects have a highly textured surface. In this paper, a new single-shot FPP algorithm which allows the object texture and fringe pattern to be estimated simultaneously is proposed. The heart of the proposed algorithm is an enhanced morphological component analysis (MCA) tailored for FPP problems. Conventional MCA methods which use a uniform threshold in an iterative optimization process are inefficient to separate fringe-like patterns from image texture. We extend the conventional MCA by taking advantage of the low-rank structure of the fringe's sparse representation to enable an adaptive thresholding process. It ends up with a robust single-shot FPP algorithm that can extract the fringe pattern even if the object has a highly textured surface. The proposed approach has a side benefit that the object texture can be simultaneously obtained in the fringe pattern estimation process, which is useful in many FPP applications. Experimental results have demonstrated the improved performance of the proposed algorithm over the conventional single-shot FPP approaches.

Fringe Pattern Based Plane-to-Plane Visual Servoing for Robotic Spray Path Planning

Robotic spray path planning is still a challenging task for large-scale structures because deformation is difficult to consider in conventional path planning, which can lead to the nonuniformity of paint film. To overcome this problem, a visual servoing approach based on fringe pattern projection is proposed to optimize the robotic spray path in this paper. In this method, the spray gun orientation is optimized to be perpendicular to the part surface at a desired distance during the spray procedure. Due to the textureless property of the surface of large-scale structures, the fringe pattern is shot onto the surface; thus, the deformed pattern directly reflects the position and orientation of the robot end-effector. Simulation experiments and real scenarios are used to demonstrate the validity and robustness of the proposed method. The results show that the proposed method would make automatic spray painting more feasible for large-scale structures.

Adaptive pixel-to-pixel projection intensity adjustment for measuring a shiny surface using orthogonal color fringe pattern projection

Three-dimensional (3D) shape measurement based on fringe pattern projection techniques has been commonly used in various fields. One of the remaining challenges in fringe pattern projection is that camera sensor saturation may occur if there is a large range of reflectivity variation across the surface that causes measurement errors. To overcome this problem, a novel fringe pattern projection method is proposed to avoid image saturation and maintain high-intensity modulation for measuring shiny surfaces by adaptively adjusting the pixel-to-pixel projection intensity according to the surface reflectivity. First, three sets of orthogonal color fringe patterns and a sequence of uniform gray-level patterns with different gray levels are projected onto a measured surface by a projector. The patterns are deformed with respect to the object surface and captured by a camera from a different viewpoint. Subsequently, the optimal projection intensity at each pixel is determined by fusing different gray levels and transforming the camera pixel coordinate system into the projector pixel coordinate system. Finally, the adapted fringe patterns are created and used for 3D shape measurement. Experimental results on a flat checkerboard and shiny objects demonstrate that the proposed method can measure shiny surfaces with high accuracy.

Nonlinear gamma correction via normed bicoherence minimization in optical fringe projection metrology

We propose a method to compensate for the projector intensity nonlinearity induced by gamma effect in three-dimensional (3-D) fringe projection metrology by extending high-order spectra analysis and bispectral norm minimization to digital sinusoidal fringe pattern analysis. The bispectrum estimate allows extraction of vital signal information features such as spectral component correlation relationships in fringe pattern images. Our approach exploits the fact that gamma introduces high-order harmonic correlations in the affected fringe pattern image. Estimation and compensation of projector nonlinearity is realized by detecting and minimizing the normed bispectral coherence of these correlations. The proposed technique does not require calibration information and technical knowledge or specification of fringe projection unit. This is promising for developing a modular and calibration-invariant model for intensity nonlinear gamma compensation in digital fringe pattern projection profilometry. Experimental and numerical simulation results demonstrate this method to be efficient and effective in improving the phase measuring accuracies with phase-shifting fringe pattern projection profilometry.

3-D optical profilometry at micron scale with multi-frequency fringe projection using modified fibre optic Lloyd’s mirror technique

A new fibre optic Lloyd’s mirror method is developed for extracting 3-D height distribution of various objects at the micron scale with a resolution of 4 µm. The fibre optic assembly is elegantly integrated to an optical microscope and a CCD camera. It is demonstrated that the proposed technique is quite suitable and practical to produce an interference pattern with an adjustable frequency. By increasing the distance between the fibre and the mirror with a micrometre stage in the Lloyd’s mirror assembly, the separation between the two bright fringes is lowered down to the micron scale without using any additional elements as part of the optical projection unit. A fibre optic cable, whose polymer jacket is partially stripped, and a microfluidic channel are used as test objects to extract their surface topographies. Point by point sensitivity of the method is found to be around 8 µm, changing a couple of microns depending on the fringe frequency and the measured height. A straightforward calibration procedure for the phase to height conversion is also introduced by making use of the vertical moving stage of the optical microscope. The phase analysis of the acquired image is carried out by One Dimensional Continuous Wavelet Transform for which the chosen wavelet is the Morlet wavelet and the carrier removal of the projected fringe patterns is achieved by reference subtraction. Furthermore, flexible multi-frequency property of the proposed method allows measuring discontinuous heights where there are phase ambiguities like 2π by lowering the fringe frequency and eliminating the phase ambiguity.

Three-dimensional shape measurement of high reflective objects based on adaptive fringe-pattern projection

Three-dimensional shape measurement of high reflective objects based on adaptive fringe-pattern projection

Adaptive projection intensity adjustment for avoiding saturation in three-dimensional shape measurement

Phase-based fringe projection methods have been commonly used for three-dimensional (3D) measurements. However, image saturation results in incorrect intensities in captured fringe pattern images, leading to phase and measurement errors. Existing solutions are complex. This paper proposes an adaptive projection intensity adjustment method to avoid image saturation and maintain good fringe modulation in measuring objects with a high range of surface reflectivities. The adapted fringe patterns are created using only one prior step of fringe-pattern projection and image capture. First, a set of phase-shifted fringe patterns with maximum projection intensity value of 255 and a uniform gray level pattern are projected onto the surface of an object. The patterns are reflected from and deformed by the object surface and captured by a digital camera. The best projection intensities corresponding to each saturated-pixel clusters are determined by fitting a polynomial function to transform captured intensities to projected intensities. Subsequently, the adapted fringe patterns are constructed using the best projection intensities at projector pixel coordinate. Finally, the adapted fringe patterns are projected for phase recovery and 3D shape calculation. The experimental results demonstrate that the proposed method achieves high measurement accuracy even for objects with a high range of surface reflectivities.