Binary Fringe Research Articles

An accurate measurement of high-reflective objects by using 3D structured light

Using three-dimensional structured light to measure high-reflective objects accurately is a big challenge. Some overexposed and blurred areas in the structured light fringe patterns lead to the loss of semantic and texture information on the object surface, further leading to 3D model reconstruction errors. To solve this problem, a lightweight novel network, Deformable Convolutional and Multi-scale Convolutional Network (DcMcNet), is proposed for repairing highly reflectively distorted regions in sinusoidal fringe patterns and the Gray code binary fringe patterns. DcMcNet uses deformable convolution and multi-scale convolution to realize the effective utilization of multiscale global features in repairing highly reflective distorted regions and can adapt well to the different shapes of these regions. In addition, virtual software is used to acquire large-scale and high-fidelity datasets and a diverse loss function is constructed to better optimize DcMcNet. Depth reconstruction experiments of sinusoidal fringe patterns and point cloud reconstruction experiments of Gray code binary fringe patterns show that our method can repair highly reflective distortion regions with high accuracy, and for aeronautical blades, the MAE of the depth map is reduced from 0.1608 to 0.0675, and the point cloud coverage is improved from 76.64% to 98.95%.

Three-dimensional measurement based on equal spacing binary fringe coding

Abstract Binary fringe projection technology can effectively avoid the measurement error caused by nonlinearity in structured light three-dimensional measurement system. In this paper, a binary fringe projection coding based on equal spacing is proposed firstly, the image sequence projected by binary fringes with equal spacing is corresponding to the sinusoidal intensity values in the same period one by one, and then the sinusoidal fringes are generated by weighted superposition. The experimental results show that, compared with the traditional four-step and 12-step phase shifting methods, the mean square error of the synthesized sinusoidal pattern is reduced by 36.74% and 18.24% respectively, and the mean square error of the distance between the obtained spherical point cloud of the standard sphere and the center of the fitting standard sphere is reduced by 89.36% and 77.27% respectively.

Phase retrieval from single-shot square wave fringe based on image denoising using deep learning.

Fringe-structured light measurement technology has garnered significant attention in recent years. To enhance measurement speed while maintaining a certain level of accuracy using binary fringe, this paper proposes a phase retrieval method with single-frame binary square wave fringe. The proposed method utilizes image denoising through deep learning to extract the phase, enabling the use of a trained image denoiser as a low-pass filter, which adaptively replaces the manual selection of the appropriate band-pass filter. The results demonstrate that this method achieves higher reconstruction accuracy than the traditional single-frame algorithm while preserving more object details.

Deep learning-based correction of defocused fringe patterns for high-speed 3D measurement

Digital fringe projection profilometry often faces a trade-off between measurement accuracy and efficiency. Defocus technology is commonly employed to address this challenge and improve the efficiency of high-speed three-dimensional (3D) measurement. This technology uses 1-bit binary fringe patterns instead of traditional 8-bit sinusoidal patterns, but accurately measuring 3D shapes with both high speed and accuracy remains a challenge due to defocus errors. These errors are introduced by the manual adjustment of lens focal length and reduce both fringe pattern quality and measurement accuracy. To overcome this limitation, we propose a multi-stage generative adversarial network with a self-attention mechanism to correct inaccurate fringe patterns and transform them into more ideal sinusoidal fringe patterns. Our generation network comprises a multi-stage feature extraction network with a self-attention mechanism and an encoder-decoder network. A multi-stage network integrating residual and transformer modules is constructed to mine global feature information. The self-attention mechanism accurately detects key areas for correction, and the encoder-decoder network generates rectified sinusoidal fringe patterns by combining the feature information with the attention area. We use a discriminative network to evaluate whether the output of the generative network is good enough to be true. In our experiments, we considered different fringe widths and measured objects of various types and colors. The results show that our proposed method improves the quality of defocus fringe patterns and the accuracy of subsequent 3D reconstruction compared to existing direct defocus methods.

Robust binary fringe pattern optimization for optical three-dimensional shape measurement

Current studies show that binary fringe patterns can be optimized in the phase or intensity domain. The problem with the former is the poor sensitivity to defocusing levels and the problem with the latter is that phase errors can not be restricted efficiently. In this paper, a binary fringe pattern optimization technology, combining intensity similarity and phase similarity, is proposed to improve fringe pattern structure similarity and defocusing robustness. Moreover, a binary patch optimization framework is also proposed to solve the time-consuming problem. Simulation and experiment results show that the proposed optimization can produce binary fringe patterns with high phase quality and defocusing robustness.

Performance of Background Oriented Schlieren with different background patterns and image processing techniques

Synthetic tests of the accuracy, spatial resolution and robustness with respect to large gradient of the apparent displacement are performed for Background Oriented Schlieren (BOS) measurements using various background patterns and image processing techniques, including multi-pass cross-correlation, iterative Horn–Schunck and Lucas–Kanade optical flow algorithms and Fourier Transform Profilometry (FTP). It is shown that FTP with periodic sinusoidal or binary fringe pattern outperforms all the other methods. With optimal pattern period λpattern=6−15pix it is able to resolve perturbations with wavelength down to λpattern2 and to obtain reliable results for images with displacement gradient up to 0.9 pix/pix. Better performance of FTP, compared to multi-pass cross-correlation, is confirmed for real experimental measurements of water temperature in natural convection near a heated vertical plate. FTP advantages and limitations, including possible nonlocal error, associated with phase unwrapping, are discussed.

Single-shot dense depth sensing with a tricolor RGB fringe pattern

A single-shot fringe pattern composed of red (R), green (G), and blue (B) is proposed for dynamic scene depth acquisition. A tricolor binary fringe is adopted and filtered in the frequency domain to obtain sinusoidal fringes. A Gabor filter is employed to estimate the wrapped phase. To overcome the phase ambiguity, we use the color-encoded De Bruijn sequence to denote the period of the phase information and assign the sequence to two channels of the pattern, while the third channel is equal to the inverse of the composition of the two channels. Benefiting from this coding strategy, two disparity maps with different phase shifts can be acquired in only one shot, and a disparity consistency check is used to eliminate errors. Quantitative and qualitative experiments are conducted to verify the performance of the proposed method and prove that it can achieve a high accuracy and dense depth map.

Dynamic computer-generated moiré profilometry based on high-density binary coding.

A dynamic computer-generated moiré profilometry based on high-density binary coding is proposed. For making full use of the maximum refresh rate and the maximum resolution of the digital light projector (DLP), the binary coded fringe is used to replace the conventional 256-gray-scale sinusoidal fringe, which can increase the refresh rate from the traditional 120Hz to more than 1kHz and meet the needs of dynamic measurement from the source. To realize the minimum equivalent wavelength and obtain the purest calculated moiré fringe, a minimum four-pixel period high-density binary fringe that satisfies the sampling theorem is designed for the DLP. The measuring accuracy of computer-generated moiré profilometry is effectively improved due to its minimum equivalent wavelength. The experimental results show the feasibility and practicability of the proposed method. It not only possesses higher measuring accuracy, but also possesses a proper potential application in dynamic three-dimensional measurement.

Fast fringe projection profilometry using 3 + 1 phase retrieval strategy and fringe order correction.

Fringe projection profilometry (FPP) has attracted wide attention in optical 3D measurement field for its high resolution, high accuracy, and simple system construction. However, the inherent requirements of FPP's measurement speed and accuracy are contradictory. We adopt a 3+1 phase retrieval strategy combining phase shift and the Hilbert transform algorithm to achieve high-speed, high-precision simultaneous measurements with only four binary fringe patterns. Furthermore, when using temporal phase unwrapping, fringe order errors (FOEs) are inevitable. Although these FOEs do not affect neighboring pixels, they are brought into the final point cloud coordinates by the absolute phase value. We propose a fringe order correction method to eliminate FOEs. The feasibility and effectiveness of this method are verified by the experimental results.

High-quality binary fringe generation by multi-scale optimization on intensity and phase.

The optimization-based binary fringe generation method can greatly improve the quality of projected fringes for structured light measurement. However, the existing phase-based and intensity-based methods are either sensitive to the projector defocus levels or cannot reduce the phase error efficiently, and these two methods only optimize the phase or intensity at a single defocus scale. A new method is proposed to optimize the intensity and phase of binary fringe on multiple defocus scales, to the best of our knowledge, which can effectively reduce the phase error and ensure its robustness. The algorithm is accelerated by block optimization. We also proposed a strategy based on objective index evaluation to simplify the process of selecting the best binary patch. Simulation and experiment results show that the binary fringe generated by the proposed method is of high-phasing quality while keeping robust to different defocus levels of the projector.

Three-dimensional face measurement based on stripe edge detection using infrared light

Subject to the poor imaging quality of the infrared camera, the projector defocusing, and the subsurface scattering, the classical phase-shifting method is constrained for reconstructing three-dimensional (3D) face geometry. To address this issue, a face measurement method based on stripe edge detection using infrared light is proposed by designing the global codeword correction. Binary fringe instead of sinusoidal fringe is employed to resist the above disturbance. Significantly, an assistant fringe is developed according to the parity of the gray order, which then is utilized to generate the correction orders. Subsequently, the correction orders are employed to guide the decoding by the discriminator of the rising and falling edges in binary patterns. In this way, the 3D face without jump errors can be reconstructed using an infrared structured light system. Experiments verify the effectiveness of our method not only on the human face but also on the standard sphere and models.

High-efficiency and robust binary fringe optimization for superfast 3D shape measurement.

By utilizing 1-bit binary fringe patterns instead of conventional 8-bit sinusoidal patterns, binary defocusing techniques have been successfully applied for high-speed 3D shape measurement. However, simultaneously achieving high accuracy and high speed remains challenging. To overcome this limitation, we propose a high-efficiency and robust binary fringe optimization method for superfast 3D shape measurement, which consists of 1D optimization and 2D modulation. Specifically, for 1D optimization, the three-level OPWM technique is introduced for high-order harmonics elimination, and an optimization framework is presented for generating the 'best' three-level OPWM pattern especially for large fringe periods. For 2D modulation, a single-pattern three-level OPWM strategy is proposed by utilizing all the dimensions for intensity modulation to decrease the required projection patterns. Thus, the proposed method essentially belongs to the 2D modulation technique, yet iterative optimization is carried out along one dimension, which drastically improves the computational efficiency while ensuring high accuracy. With only one set of optimized patterns, both simulations and experiments demonstrate that high-quality phase maps can be consistently generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) and different amounts of defocusing, and it can achieve superfast and high-accuracy 3D shape measurement.

Theoretical analysis and experimental investigation of the Floyd-Steinberg-based fringe binary method with offset compensation for accurate 3D measurement.

Digital fringe projection (DFP) with defocused binary fringe patterns has the ability to overcome the projector nonlinearity and achieve a high-speed 3D measurement. The Floyd-Steinberg (FS) dithering technique is one of the most commonly adopted binary fringe coding methods due to its relatively high measurement accuracy. Nevertheless, we found that the FS binary fringe would cause a fixed error in the recovered phase, which is proven to be invariable for various defocusing levels and various phase-shift steps according to the analysis of the phase error based on noise model of phase-shifting profilometry. It means that FS binary fringe would have a certain offset in space, compared with standard sinusoidal fringe, which is verified to be essentially constant for different fringe pitches through simulation and experiment. This offset would distort the 3D geometry of the tested target for monocular systems relying on triangulation, which needs to be compensated to improve 3D measurement accuracy. Experiments are presented to demonstrate the enhanced 3D result after compensation.

Robust binary fringe generation method with defocus adaptability

Current binary defocus technology mainly focuses on fringe generation suitable for specific frequencies without considering fringe adaptability for defocus-degree variation, which decreases the measuring accuracy for this scenario. To achieve a high-quality measurement, we propose a robust binary fringe generation method to minimize the phase error caused by changes in defocus and the random error in measurement. In the method, we establish a complete phase error model and construct a novel objective function, to the best of our knowledge, to optimize the binarization threshold of each pixel. Through derivation of the threshold gradient calculation formula, we quickly obtain optimal binary fringes that can adapt to different fringe pitches and various degrees of defocus. The experimental results verify that the proposed method can generate robust binary fringes adaptive to different fringe pitches and defocus degree variation, and thus achieve high-quality 3D measurements.

Correcting projector lens distortion in real time with a scale-offset model for structured light illumination.

In fringe projection profilometry, inevitable distortion of optical lenses decreases phase accuracy and decreases the quality of 3D point clouds. For camera lens distortion, existing compensation methods include real time look-up tables derived from the related parameters of camera calibration. However, for projector lens distortion, so far, post-undistortion methods iteratively correcting lens distortion are relatively time-consuming while, despite avoiding iteration, pre-distortion methods are not suitable for binary fringe patterns. In this paper, we aim to achieve real-time phase correction for the projector by means of a scale-offset model that characterizes projector distortion by four correction parameters within a small-enough area, and thus we can speed up the post-undistortion by looking up tables. Experiments show that the proposed method can suppress the distortion error by a factor of 20 ×, i.e., the error of root mean square is less than 45 µm/0.7‰, while also proposed improving the computation speed by a factor of 50× over traditional iterative post-undistortion.

Three-dimensional measurement method based on a three-step phase-shifting fringe and a binary fringe

Gray-code plus phase-shifting is currently a commonly used method for structured light three-dimensional (3D) measurement that is able to measure complex surfaces. However, the Gray-code fringe patterns tend to be complicated, making the measurement process time-consuming. To solve this problem and to obtain faster speed without sacrificing accuracy, a 3D measurement method based on three-step phase-shifting and a binary fringe is proposed; the method contains three phase-shifting fringe patterns and an additional binary fringe pattern. The period of the binary fringe is designed to be the same as the three-step phase-shifting fringe. Because of the specific pattern design strategy, the three-step phase-shifting algorithm is used to obtain the wrapped phase, and the connected region labeling theorem is used to calculate the fringe order. A theoretical analysis, simulation, and experiments validate the efficiency and robustness of the proposed method. It can achieve high-precision 3D measurement, which performs almost the same as the Gray-code plus phase-shifting method. Since only one additional binary fringe pattern is required, it has the potential to achieve higher measurement speed.

High speed fringe projection for dynamic shape measurement using binary phase mask. Part 1: theory and simulation

Projection rates of up to 30,000 greyscale fringe patterns per second have been achieved recently by defocusing binary fringe patterns from a digital micromirror device (DMD) based projector. Part 1 of this two-part paper describes the design of a binary phase mask, based on a virtual scatter plate, for the purpose of enhancing the performance of a binary fringe projector. The phase mask's anisotropic point spread function (PSF) produces a well-defined blur of the fringes parallel to the fringe direction, thereby minimising degradation of fringe contrast. The shape of the PSF is also shown, by means of a polychromatic Fourier optics model, to be insensitive to projection distance over a range of ±10% of the standoff distance. Two new binary fringe design methods are proposed, including extensions to optimize the system performance in the case of a mismatch between camera and projector framing rates. Expressions for the phase noise are derived as a function of the phase mask design parameters, which demonstrate that fringe quality comparable to traditional 8-bit greyscale fringes is achievable at projection rates over two orders of magnitude higher.

High-quality 3D shape measurement with binary half truncated sinusoidal fringe pattern

Digital fringe projection (DFP) with defocused binary fringe patterns could overcome the projector nonlinearity and achieve a high-speed 3D measurement. Most of the binary defocusing techniques take sinusoidal fringe as the target of binarization, hoping the binary fringes can become “quasi-sinusoidal” after defocusing. However, considering that the phase-shifting algorithm (PSA) has an inherent resistance to certain order harmonics components, it is not the best choice to make the defocused binary fringe become more sinusoidal. This paper proposed a new kind of fringe called half truncated sinusoidal (ht-sinusoidal) fringe, which has the same phase result as the sinusoidal fringe with 4-step PSA theoretically. And this kind of fringe is easier to be binarized for the reason that half of it has 0 intensity values already. Therefore it can achieve significant phase quality improvements compared with the binary sinusoidal fringe. We mainly verified that the ht-sinusoidal fringe will help substantially improve the measurement accuracy of different error diffusion methods through both simulations and experiments.

High speed fringe projection for dynamic shape measurement using binary phase mask. Part 2: manufacture and test

Dynamic measurement of object shape is now achievable at projection rates of up to 30,000 binary fringe patterns per second from a digital micromirror device (DMD) based projector. This two-part paper describes the design, manufacture and test of a binary phase mask in the projection system that produces high-quality greyscale fringes from the binary patterns, with much reduced phase noise compared to the projector defocus approach that has been used until now. Part 2 describes the experimental verification and characterization of the phase masks, manufactured using photolithography and dry etching from the design methods described in Part 1. The height distribution of two masks with point spread function (PSF) dimensions of 1 × 256 and 4 × 64 pixels were assessed using coherence scanning interferometry and contact surface profilometry. Phase noise was determined using standard temporal unwrapping fringe sequences and found to be 20–50% higher than a traditional 8-bit greyscale approach. The projector lens distortion has been observed, and the corresponding compensation method shown to significantly improve the quality of the projected fringes. Increasing the number of projector frames per camera frame reduced phase noise, to less than one-half the 8-bit values for a 10-slice sequence. Fringe image acquisition rates for the 1-slice and 10-slice sequences are up to 255 × and 25 × faster than for the 8-bit greyscale sequence, respectively. Finally, examples are presented of fringe patterns and wrapped phase maps from a high-speed system used to measure moving objects, with projection and acquisition rates of up to 9515 frames s−1.

Binary Fringe Defocused Projection Technology: A Review

Binary Fringe Defocused Projection Technology: A Review