Sinusoidal Fringe Research Articles

Binocular composite grayscale fringe projection profilometry based on deep learning for single-shot 3D measurements

The efficiency and precision of 3D shape reconstruction has been a focus in the fringe projection profilometry (FPP). However, achieving high-quality 3D measurement for isolated or overlapping objects from single fringe image is still a challenging task in the field. In this paper, a binocular composite grayscale fringe projection profilometry (BCGFPP) based on deep learning is proposed, in which a two-stage one-to-three network (TONet) is trained to predict the images required for phase unwrapping. The obtained absolute phase map exhibits high precision and eliminates deviation error and periodic ambiguity typically encountered in traditional sinusoidal composite fringe coding scheme. Haar transform principle is employed to form a Haar-like composite fringe pattern (HCFP), which consists of three different frequencies, serving as the input. TONet architecture is designed to predict images required by the tri-frequency four-step phase-shifting method (TFPM). Further, the absolute phase is calculated and the disparity map is obtained by matching the absolute phase of the left and right cameras. Finally, the 3D shape of the object can be restored by the system calibration parameters. Experimental results demonstrate the approach can greatly reduce the number of fringes required and achieve the accuracy of the absolute phase close to the training set.

Prior-guided restoration of intense local specular highlight in fringe projection profilometry images

This paper presents a novel prior-guided restoration method, to our knowledge, aimed at removing and recovering intense local specular highlight in fringe projection profilometry (FPP) images of specular objects. Local reflections, caused by the direct reflection of the projector on smooth surfaces, often saturate pixel intensities, posing a significant obstacle to 3D shape reconstruction. The proposed method combines sinusoidal fringe projection principles with improved fitting techniques. By analyzing fringe patterns in non-highlight regions, the constant and amplitude parameters of the fringes are determined by non-highlight regions. For the critical initial phase parameter, the continuity of highlight regions and the fixed relative geometry between the projector and object are leveraged, which enables an iterative calculation strategy that progressively estimates fringe intensity within specular regions. The results show a seamless integration of the restored fringe data with the original non-highlight information, ensuring global consistency and continuity. 3D measurement experiments demonstrate effective restoration of morphological distortions and filling of point cloud holes.

Robust phase-coding: a solution to suppress fringe order errors.

Three-dimensional (3D) shape measurements based on code-based fringe projection profilometry have been extensively used for scientific research and industrial applications. However, the fringe order errors always influence the measurement result. Although numerous methods have been proposed to eliminate fringe order errors, they may compromise computational cost, measurement speed, measurement range and the failure to eliminate all types of errors. To address this issue, a comprehensive investigation has been conducted into the formation mechanisms of fringe order errors. This has enabled a deeper understanding of the underlying causes of fringe order errors and the development of a set of guidelines for the design of fringe patterns. Based on these guidelines, this paper proposes a robust measurement technique based on phase-coding for enhanced measurement performance. Unlike traditional methods, shifting the value of the codeword and pre-staggering fringes prevents the occurrence of different types of fringe order errors. The measurement range is subsequently extended by coding fringe order into the sinusoidal fringes. Experimental results successfully demonstrate that the proposed method suppresses fringe order errors and achieves high-quality, efficient 3D shape measurements in complex scenarios.

Ultra high-speed 3D shape measurement technology for specular surfaces based on μPMD

Phase measuring deflectometry (PMD) has been extensively applied to measure specular surfaces due to its non-contact, high-precision, full-field measurement capabilities. Liquid crystal display (LCD) screen is the most common structured light source in PMD. However, the response time of liquid crystal molecules limits its frame rate to around 100 frames per second (fps). Therefore, it is quite difficult for traditional PMD to measure rapidly moving surfaces. This paper proposes a 3D dynamic sensing technique, microsecond-PMD (µPMD) based on the high-frame-rate sinusoidal fringe display (HSFD). In the proposed method, the switching time for each fringe pattern display is at a sub-microsecond level, enabling high-speed fringe acquisition with kHz-level area array detection or 100kHz-level line array scanning. The HSFD method uses a specially designed LED array and two-step optical expansion. The high-speed switching characteristic of LED sources is utilized to allow a superfast display rate. Moreover, the superior sinusoidal property can be achieved by the combination of the specially designed discrete sinusoidal LED array, the light-diffracting effect of orthogonal gratings, and the filtering effect of the light diffuser. The mechanism and analytic model of fringe generation are thoroughly analyzed and discussed in this work. Furthermore, the swarm optimization algorithm and corresponding weighted fringe quality evaluation function are presented to obtain the optimal fringes. To the best of our knowledge, the proposed µPMD, for the first time, achieved a superfast fringe acquisition rate of 4000fps with sub-micrometer precision in three-dimensional (3D) reconstruction for specular surfaces. We envision this proposal to be broadly implemented for real-time monitoring in manufacturing.

High precision full-field vibration measurement by LDV-induced stroboscopic fringe projection

A method for full-field vibration measurement that combines Fourier Transform Profilometry (FTP) and Laser Doppler Vibrometry (LDV) is proposed and validated experimentally on various structures. This technique projects sinusoidal fringe patterns onto the test sample using a flashing LED at an oblique angle, while a CCD camera captures the fringe images vertically. Vibration data is derived from 3-D phase maps generated by applying FTP to these images and is further calibrated using a motorized translation stage. To improve the precision of FTP and extend the measurable frequency range beyond that of normal-rate cameras, initial single-point vibration data from the test sample is obtained using LDV. This prior information enables noise decoupling and down-sampling techniques to be employed. The experimental results confirm that this method can precisely reconstruct the high-order operational deflection shapes of vibration on structures with different materials, achieving a precise measurement for vibration amplitude of around 500 nm (1/2000 of fringe spacing) on a 10 cm x 10 cm plastic plate and a 2 cm x 14 cm aluminum beam with approximately 1 mm fringe spacing and 26° projection angle.

3D reconstruction with single-frame two-step phase-shift method based on orthogonal composite fringe pattern projection

Abstract In order to realize single-frame three-dimensional (3D) reconstruction, a single-frame two-step phase-shift method based on orthogonal composite pattern projection is proposed to solve the problem that the traditional N-step phase-shift profilometry needs multiple projections for 3D reconstruction. The orthogonal composite pattern uses only two carrier channels to reduce the spectrum overlapping influence on the demodulation accuracy of carrier and modulated fringes. A two-dimensional variational mode decomposition method is adopted to remove the background DC component of the sinusoidal fringe to overcome the mode overlap problem by controlling the size of the bandwidth. Thus, the two-step phase-shift method is applied to calculate the phases for 3D reconstruction. The experimental results show that, compared with the typical Fourier transform profilometry method, 3-step composite method and 2 + 1 composite method, the 3D reconstruction accuracy of the proposed method is improved by 49.1%,31.4% and 23.2% respectively according to mean absolute error, and by 73.0%, 58.4% and 56.8% respectively according to mean squared error as the evaluation index. Finally, the dynamic 3D reconstruction experiment demonstrates the good adaptability of dynamic 3D reconstruction.

Large-depth-range 3D measurement based on optimized multi-focal projection system.

High-speed and large-depth-range 3D measurement technology is of great importance in a variety of fields. Conventional binary defocusing methods can achieve high measurement speed, but their depth range is limited because the defocusing effect is insufficient near the focal plane in the ordinary projection system. To address this problem, we propose an optimized multi-focal projection system by introducing a cylindrical lens with optimal parameter configuration. Compared to traditional methods, the proposed method could overcome the constraint of defocusing region to obtain high-quality sinusoidal fringe patterns over large depth range without sacrificing the measurement speed. In this paper, the principle of multi-focal projection system and the related scheme of parameter configuration are presented in order to illustrate the role of the cylindrical lens and system parameters on modulating the distribution of defocusing kernel and removing the limitations of defocusing region. Experimental results show that the proposed multi-focal projection system with optimal parameter configuration increased the depth range from 150 mm to 725 mm over the conventional single-focal system, achieving greater depth range and better measurement results.

Online nonlinearity elimination for fringe projection profilometry using slope intensity coding

The nonlinearity effect in the system of fringe projection profilometry can cause the non-sinusoidal deviation of the fringe patterns, inducing ripple-like phase errors and further affecting measurement accuracy. This paper presents an online nonlinearity elimination method based on slope intensity coding. Two sequences of sinusoidal phase-shifting fringe patterns with different frequencies, and one slope intensity pattern with one uniform intensity pattern are projected. The equations for the nonlinearity response are established using the defined mean and modulation parameters, the captured uniform intensity and two extracted background intensities. The nonlinearity response coefficients determined by solving the equations are used for pixel-wise nonlinearity correction on the captured images, which are employed for computing the wrapped phase, and further obtaining continuous phase by the multi-frequency phase unwrapping method. Experimental results demonstrate that the proposed method can eliminate the nonlinearity-induced phase error online by using fewer images and maintain the reliability of phase unwrapping in the measurement of isolated objects with complex surfaces.

An accurate measurement of high-reflective objects by using 3D structured light

Using three-dimensional structured light to measure high-reflective objects accurately is a big challenge. Some overexposed and blurred areas in the structured light fringe patterns lead to the loss of semantic and texture information on the object surface, further leading to 3D model reconstruction errors. To solve this problem, a lightweight novel network, Deformable Convolutional and Multi-scale Convolutional Network (DcMcNet), is proposed for repairing highly reflectively distorted regions in sinusoidal fringe patterns and the Gray code binary fringe patterns. DcMcNet uses deformable convolution and multi-scale convolution to realize the effective utilization of multiscale global features in repairing highly reflective distorted regions and can adapt well to the different shapes of these regions. In addition, virtual software is used to acquire large-scale and high-fidelity datasets and a diverse loss function is constructed to better optimize DcMcNet. Depth reconstruction experiments of sinusoidal fringe patterns and point cloud reconstruction experiments of Gray code binary fringe patterns show that our method can repair highly reflective distortion regions with high accuracy, and for aeronautical blades, the MAE of the depth map is reduced from 0.1608 to 0.0675, and the point cloud coverage is improved from 76.64% to 98.95%.

High-frame rate, large-depth-range structured light projector based on the step-designed LED chips array.

Among numerous mature optical 3D measurement techniques, phase-shift profilometry (PSP) has been widely used because of its high precision and insensitivity to ambient light, and high-speed PSP has become a research hotspot in recent years. Current mainstream high-frame rate PSP projection techniques employ binary defocusing projection schemes, which limit the available measurement depth. We propose a high-frame rate, large-depth-range sinusoidal fringe projection technique based on step-designed LED chips array. In principle, on the one hand, the LED chips array still produces a binary pattern, so high-frame rate switching can be achieved, on the other hand, whether focusing or defocusing can generate sinusoidal fringes, avoiding the limit of defocusing projection on the depth range of measurement. A PSP projector is designed and manufactured, and 3D reconstruction of static human face mask and dynamic rotating fan is carried out at 1 kHz frame rate. In another experiment, the PSP projector projected the fringes at a 100kHz frame rate and detected the fringes with a single point photodetector, and the output waveform showed that the projection technique had the potential to be much higher than the 100 kHz frame rate. These results show that the PSP projection technology has the advantages of high-frame rate and large-depth-range, and is very useful for three-dimensional measurement of moving targets.

Three-dimensional measurement based on equal spacing binary fringe coding

Abstract Binary fringe projection technology can effectively avoid the measurement error caused by nonlinearity in structured light three-dimensional measurement system. In this paper, a binary fringe projection coding based on equal spacing is proposed firstly, the image sequence projected by binary fringes with equal spacing is corresponding to the sinusoidal intensity values in the same period one by one, and then the sinusoidal fringes are generated by weighted superposition. The experimental results show that, compared with the traditional four-step and 12-step phase shifting methods, the mean square error of the synthesized sinusoidal pattern is reduced by 36.74% and 18.24% respectively, and the mean square error of the distance between the obtained spherical point cloud of the standard sphere and the center of the fitting standard sphere is reduced by 89.36% and 77.27% respectively.

High-Efficiency Dynamic Three-Dimensional Topography Measurement Using the Phase Shift Generation Method

In the fringe projection profilometry (FPP), the traditional phase-shifting (TPS) algorithm and the Fourier transform (FT) algorithm are beset with a conundrum where measurement efficiency and conflicts with measurement accuracy, thereby limiting their application in dynamic three-dimensional (3D) measurements. Here, we propose a phase shift generation (PSG) method, which acquires the sinusoidal fringes by sparse sampling and reconstructs the complete phase-shifting sequence by generating the missing fringes with superimposed coupling of adjacent fringes. According to our proposed PSG method in which the sinusoidal fringe sequence size is about half of the TPS method, meaning that the PSG method will be timesaving in the phase-shifting sequence sampling process. Moreover, because of the utilization of multiframe fringes, our PSG method allows for a more accurate measurement than the FT method. Both simulation and experimental results demonstrate that our proposed PSG method can well balance the measurement accuracy and efficiency with a lower sampling rate, bearing a great potential to be applied in both scientific and industrial areas.

Adaptive phase retrieval algorithm for local highlight area based on a piecewise sine function.

Phase measuring profilometry (PMP) has been widely used in industries for three-dimensional (3D) shape measurement. However, phase information is often lost due to image saturation results from high-reflection object surfaces, leading to subsequent 3D reconstruction errors. To address the problem, we propose an adaptive phase retrieval algorithm that can accurately fit the sinusoidal fringes damaged by high reflection in the saturated regions to retrieve the lost phase information. Under the proposal, saturated regions are first identified through a minimum error thresholding technique to narrow down regions of interest and so that computation costs are reduced. Then, images with differing exposures are fused to locate peak-valley coordinates of the fitting sinusoidal fringes. And the corresponding values of peak-valley pixels are obtained based on a least squares method. Finally, an adaptive piecewise sine function is constructed to recover the sinusoidal fringe pattern by fitting the pattern intensity distribution. And the existing PMP technology is used to obtain phase information from the retrieved sinusoidal fringes. To apply the developed method, only one (or two) image with different exposure times is needed. Compared with existing methods for measuring reflective objects, the proposed method has the advantages of short operation time, reduced system complexity, and low demand on hardware equipment. The effectiveness of the proposed method is verified through two experiments. The developed methodology provides industry an alternative way to measure high-reflection objects in a wide range of applications.

Robust structured light 3D measurement method based on polarization-encoded projection patterns.

Fringe projection profilometry (FPP) is widely used in 3D vision measurement because of its high robustness and measurement accuracy. In the case of HDR objects, due to the problem of surface reflectivity, the obtained image will be overexposed. This will cause the sinusoidality of the fringes projected on the surface of the object in the acquired image to be interfered, resulting in a phase error in the calculated wrapped phase. Therefore, a polarization-encoded sinusoidal structured light is proposed to enhance the sinusoidality of the fringe. The phase information contained in the polarized sinusoidal structured light fringe is only related to the polarization state, not to the light intensity. A polarization coding assisted structured light measurement strategy (PASM) is proposed. This method uses polarization coding assisted polarization phase-shifting fringes for phase unwrapping. The angle of the linear polarizer is set to zero in this method, and it does not require rotating the polarizer. It only needs a single exposure to improve the fringe quality and obtain a more stable unwrapping phase. The experimental results show that the obtained polarization fringes have better sinusoidality, and the phase unwrapping can be more accurate. The reconstructed 3D point cloud also does not appear missing and has better accuracy. It is a reliable method for vision measurement of HDR objects.

A non-iterative frame-reduced structured illumination microscopy using checkerboard modulation

Structured illumination microscopy (SIM) is a tremendously potential super-resolution microscopy technique requiring nine raw images to enhance two-fold lateral resolution. For SIM, it is a challenging dilemma to achieve super-resolution imaging with a reduced number of images and fast data processing speed at the same time. Here, we demonstrate a new SIM reconstruction framework that uses only four raw images without requiring extensive iterative computation. The four-frame method is achieved by applying checkerboard pattern illumination modulation to supersede the sinusoidal fringe illumination. It is worth noting that the proposed scheme prominently shortens image acquisition time, combined with a remarkably higher image reconstruction rate compared to current frame-reduced methods. Meanwhile, the reconstruction process is non-iterative and unlimited by the field of view size. Consequently, our work is expected to bring about breakthroughs in facilitating the improvement of real-time live cell observation.

Error-diffusion-kernel parameters for binary pattern in 1-bit fringe projection profilometry.

In fringe projection profilometry, 1-bit processing of 8-bit raster patterns is a common method to suppress nonlinear errors in commercial projectors and realize high-speed projection in industrial projectors. In the process of generating 1-bit fringes from sinusoidal fringes, the generation of high-order harmonics is inevitable; choosing to introduce fewer high-order harmonics of the algorithm is conducive to defocus to obtain a better sinusoidal pattern. This paper proposes a method to expand the error-diffusion kernel of the conventional Floyd-Steinberg diffusion dithering algorithm from 2×3 to 3×5, which can reduce the grayscale change of surrounding pixels and generate 1-bit fringes with fewer high-order harmonics. Meanwhile, this paper optimizes the parameters of the 3×5 error-diffusion kernel and proposes the optimal parameters for this kind of diffusion kernel. The simulation results show that the fringes generated by the proposed 3×5 error-diffusion-kernel algorithms are closer to sinusoidal fringes after Gaussian low-pass filtering. The experimental results show that the accuracy of the 3×5 diffusion kernel algorithms is higher.

Adaptive phase measuring profilometry for robustly detecting saturated pixels

In this paper, we propose an adaptive method for phase measuring profilometry to more accurately reconstruct 3D point clouds of high-reflective surfaces. First, we project high-frequency sinusoidal fringe patterns to robustly detect saturated pixels and then obtain their corresponding optimal projection intensity. Second, based on the coordinate mapping relationship between a camera-projector pair, we generate a set of fringe patterns with adaptive intensity. Finally, we project the adaptive fringes patterns to achieve more accurate 3D reconstruction. The experimental results show that our method can more accurately detect saturated pixels, and thus improve the quality of 3D reconstruction.

Wide field 3D optical profilometry using a diffraction Lloyd's mirror interferometer.

Interference fringe projection is used as a non-contact optical profilometry method for accurate 3D measurements. In interferometric fringe projection schemes, the maximum measurable size of the test object is limited by the optics of the interferometer. In this work, we report the application of a diffraction Lloyd's mirror interferometer (DLMI) as a wide-field sinusoidal fringe projection system for 3D shape measurement. The DLMI works on diffracted light and therefore generates interference fringes over a large area. This enables measurement of large objects using DLMI as compared to a conventional Lloyd's mirror interferometer. The performance of the proposed system is evaluated in terms of its stability and reproducibility of the results through measurement of the standard deviation in the phase values.

Deep learning-based correction of defocused fringe patterns for high-speed 3D measurement

Digital fringe projection profilometry often faces a trade-off between measurement accuracy and efficiency. Defocus technology is commonly employed to address this challenge and improve the efficiency of high-speed three-dimensional (3D) measurement. This technology uses 1-bit binary fringe patterns instead of traditional 8-bit sinusoidal patterns, but accurately measuring 3D shapes with both high speed and accuracy remains a challenge due to defocus errors. These errors are introduced by the manual adjustment of lens focal length and reduce both fringe pattern quality and measurement accuracy. To overcome this limitation, we propose a multi-stage generative adversarial network with a self-attention mechanism to correct inaccurate fringe patterns and transform them into more ideal sinusoidal fringe patterns. Our generation network comprises a multi-stage feature extraction network with a self-attention mechanism and an encoder-decoder network. A multi-stage network integrating residual and transformer modules is constructed to mine global feature information. The self-attention mechanism accurately detects key areas for correction, and the encoder-decoder network generates rectified sinusoidal fringe patterns by combining the feature information with the attention area. We use a discriminative network to evaluate whether the output of the generative network is good enough to be true. In our experiments, we considered different fringe widths and measured objects of various types and colors. The results show that our proposed method improves the quality of defocus fringe patterns and the accuracy of subsequent 3D reconstruction compared to existing direct defocus methods.

Adaptive structured illumination optical-sectioning microscopy based on the prior knowledge of sample structure

Fast and accurate online detection is of great significance to improve mass production efficiency and reduce the costs of semiconductor chips and other micro-nano devices. Compared with other 3D imaging detection techniques, structured illumination optical-sectioning microscopy has faster 3D imaging because it uses wide-field sinusoidal fringe structured illumination. However, commercial structured illumination optical-sectioning microscopes are designed for general needs, and the orientations of the fringe patterns they project are not optimized for the structures of the sample. To obtain an isotropic result, they need to project multiple sinusoidal fringe patterns with different orientations and perform multiple optical-sectioning imaging, and then average the optical-sectioning results, resulting in low imaging efficiency. This paper proposes an adaptive structured illumination optical-sectioning microscopy. It exploits prior knowledge of the sample structure to design the sinusoidal fringe orientation for each local structure and generates an adaptive structured illumination. In this way, we can achieve the best imaging effect with only one optical-sectioning imaging, thereby improving the imaging efficiency, which is beneficial to the improvement of the mass production efficiency of products. The method proposed in this paper will provide useful guidance on how to correctly and efficiently use structured illumination optical-sectioning microscopy to detect defects of micro-nano devices such as semiconductor chips with known designed structures.