Binary Defocusing Research Articles

Fast Three-Dimensional Profilometry with Large Depth of Field.

By applying a high projection rate, the binary defocusing technique can dramatically increase 3D imaging speed. However, existing methods are sensitive to the varied defocusing degree, and have limited depth of field (DoF). To this end, a time-domain Gaussian fitting method is proposed in this paper. The concept of a time-domain Gaussian curve is firstly put forward, and the procedure of determining projector coordinates with a time-domain Gaussian curve is illustrated in detail. The neural network technique is applied to rapidly compute peak positions of time-domain Gaussian curves. Relying on the computing power of the neural network, the proposed method can reduce the computing time greatly. The binary defocusing technique can be combined with the neural network, and fast 3D profilometry with a large depth of field is achieved. Moreover, because the time-domain Gaussian curve is extracted from individual image pixel, it will not deform according to a complex surface, so the proposed method is also suitable for measuring a complex surface. It is demonstrated by the experiment results that our proposed method can extends the system DoF by five times, and both the data acquisition time and computing time can be reduced to less than 35 ms.

A phase error compensation method for three-dimensional shape measurement using improved iteration algorithm

The binary defocusing has been extensively studied in the three-dimensional measurement. But if the projector is slightly defocused, the binary fringes after defocusing still contain high-order harmonics compared with the ideal sinusoidal fringes, so the phase error caused by the nonlinear response is not negligible. In this paper, two models are proposed to calculate and compensate phase error, which include a double-precision iterative compensation model (DPICM) and a dual-domain iterative compensation model (DDICM). These two models obtain accurate phase errors by fusing the phases at different precision and domains. DPICM is composed of the double precision method and the improved iterative algorithm, and DDICM consists of the dual-domain method and the improved iterative algorithm. This improved iterative algorithm is used to compensate phase error, which can improve phase accuracy. DPICM and DDICM reduce the RMS error by 25.5% and 13.5% respectively.

A Novel Defocus-Degree-Based Phase Unwrapping and Fusion Algorithm for High-Speed and Large-Depth-Range 3D Measurement

Binary defocusing technique is typically used in high-speed 3-D measurement. As the defocus degree varies with the measuring depth range, current defocusing technique has limitations while measuring large-depth-range (LDR) scene. Addressing this difficulty, this article proposes a novel defocus-degree-based phase unwrapping and phase fusion algorithm for LDR 3-D measurement. Firstly, we establish a mathematical model to present the relationship between defocus degree and phase, and present the defocus-degree-based phase unwrapping algorithm to directly obtain its accurate fringe order. Then, we derive a phase error model to estimate its phase error, and develop a phase fusion algorithm to obtain the lowest phase error by fusing two frequency phases based on the estimated error. With implementation of the proposed phase unwrapping and fusion algorithm, experiments are performed and the results demonstrate that our method can achieve a high-speed, high-precision and low-outlier 3-D measurement for LDR scene.

Quantitative Evaluation of Binary Defocusing in Fringe Projection Profilometry Based on Digital Correlation

Quantitative Evaluation of Binary Defocusing in Fringe Projection Profilometry Based on Digital Correlation

Determination of optimal binary defocusing based on digital correlation for fringe projection profilometry

In fringe projection profilometry (FPP), non-sinusoidal projected fringes due to the gamma effect of a projector will cause phase measurement errors. To solve this problem, a binary defocusing technique has been introduced in recent years, while it is difficult to determine an appropriate amount of defocus quantitatively with high sensitivity and high accuracy. In this paper, a novel approach based on digital correlation is proposed to determine an optimal amount of defocus, aiming to get sinusoidal fringes from a binary-structured fringe pattern with projector defocusing. Firstly, a window containing multiple fringes is selected from the captured binary defocusing fringe pattern projected onto a calibration plate or a flat part of the surface of the object being measured. Then calculate the correlation coefficient between the selected window and its second-order differential in the direction perpendicular to the fringe. Since the second-order differential of a sinusoidal wave remains still sinusoidal, the maximum correlation coefficient should correspond to the optimal sinusoidality of the binary defocusing fringe. Therefore, by finding the maximum correlation coefficient while defocusing the projector, the optimal binary defocusing level can be determined accordingly. One of the characteristics of the proposed method is that only one fringe window is required to judge the defocus level in practical measurement, which may speed up 3D measurement. In addition, the proposed method has no requirements for fringe pitch and is applicable to fringes of any spatial frequency. Both numerical simulations and experimental results have been demonstrated to exhibit the validity and high accuracy of the proposed method.

Passive binary defocusing for large depth 3D measurement based on deep learning.

Phase-shifting profilometry (PSP) based on the binary defocusing technique has been widely used due to its high-speed capability. However, the required adjustment in projector defocus by traditional method is inaccurate, inflexible, and associated with fringe pitch. Instead of manual defocusing adjustment, a passive defocus of the binary patterns based on deep learning is proposed in this paper. Learning the corresponding binary patterns with a specifically designed convolutional neural network, high-quality three-step sinusoidal patterns can be generated. Experimental results demonstrate that the proposed method could reduce phase error by 80%-90% for different fringe pitches without projector defocus and outperform the traditional method by providing more accurate and robust results within a large measuring depth.

Binary defocusing technique based on complementary decoding with unconstrained dual projectors

Binary defocusing technique can effectively break the limitation of hardware speed, which has been widely used in the real-time three-dimensional (3D) reconstruction. In addition, fusion technique can reduce captured images count for a 3D scene, which helps to improve real-time performance. Unfortunately, it is difficult for binary defocusing technique and fusion technique working simultaneously. To this end, our research established a novel system framework consisting of dual projectors and a camera, where the position and posture of the dual projectors are not strictly required. And, the dual projectors can adjust defocusing level independently. Based on this, this paper proposed a complementary decoding method with unconstrained dual projectors. The core idea is that low-resolution information is employed for high-resolution phase unwrapping. For this purpose, we developed the low-resolution depth extraction strategy based on periodic space-time coding patterns and the method from the low-resolution order to high-resolution order of fringe. Finally, experimental results demonstrated the performance of our proposed method, and the proposed method only requires three images for a 3D scene, as well as has strong robustness, expansibility, and implementation.

Evaluating binary defocusing quantitatively in real-time for fringe projection profilometry

In fringe projection profilometry (FPP), non-sinusoidal fringes due to the gamma effect of a projector will cause measurement errors. To solve this problem, a binary defocusing technique has been introduced in recent years, yet the appropriate defocusing is hard to evaluate quantitatively. An innovative approach to quantify binary defocusing in real-time is proposed. No matter how the projector and camera are arranged in FPP system, the fringe period in a captured defocusing fringe image is strictly varying; therefore, unlike previous methods, the proposed method uses a limited window for defocusing evaluation. This not only improves the accuracy of defocusing evaluation, but also greatly improves the evaluation speed, thus in turn enabling real-time evaluation. In addition, numerical differentiation is introduced on the window using a modified five-interval-point algorithm to effectively improve the sensitivity to improper (inadequate or excessive) defocusing. The difference between the numerical differentiation and its fundamental harmonic extracted by Levenberg–Marquardt iterative is taken as an evaluation value of binary defocusing. The smaller the difference, the more appropriate is the defocusing. By minimizing this difference, the most appropriate defocusing can be obtained quantitatively. Both numerical simulations and experiments validate the high sensitivity and high speed of the proposed method.

Phase error analysis and compensation for motion in high-speed phase measurement profilometry

High-speed three-dimensional (3D) measurement is increasingly important in many fields. Phase measurement profilometry (PMP) based on the binary defocusing technique has been applied to the high-speed 3D measurement scene for its higher measurement resolution and precision, and breaking the speed limitations of projector. However, because the PMP needs three phase-shifting (3-PS) patterns, motion error is inevitable to measuring dynamic objects. In this research, we construct a complete high-speed 3-PS PMP system, and re-derive two clearer motion error models than those in Weise’s research [Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2007), pp. 110.1109/CVPR.2007.383291]. Then, we theoretically analyze the effects of the truncation error on the model accuracy, especially when the motion error is higher. To this end, a polynominal-based motion error model by fitting coefficient matrix of pre-simulation is proposed. Meanwhile, its corresponding error compensation method based on local domain estimation of the Nelder-Mead algorithm is developed. Finally, both simulations and quantitative and qualitative experiments verify the accuracy and effectiveness of the proposed method, as well as demonstrate the proposed method has improvements compared with the Weise’s research.

Reconstruction Method for Binary Defocusing Technique Based on Complementary Decoding With Dual Projectors

Real-time three-dimensional reconstruction is increasingly important in many fields. There are mainly two ways to improve real-time performance: increasing hardware rate and reducing captured frames count. For increasing hardware rate, the binary defocusing technique can effectively break the limitation of speed. For reducing captured frames count, the fusion technique is typically introduced. However, the simultaneous work of binary defocusing technique and fusion technique is hard to achieve. To this end, we establish a novel system framework with a camera and dual projectors. The dual projectors can control defocusing level independently, i.e. enhance the flexibility of projector. Based on this, our paper exhibits two decoding phase methods: the strict-alignment method and the loose-alignment method. The strict-alignment method requires the projection chips to be strictly aligned, and a virtual example demonstrated effectiveness with three captured frames. In order to further improve practicality, the loose-alignment method, which relaxes the restriction, is proposed. Finally, experiments verify the performance of our proposed method.

High-quality 3D shape measurement by kernel-optimized high sinusoidal similarity dither patterns

The error-diffusion dithering technique is widely used in three-dimensional (3D) measurement. The sinusoid similarity of dithering fringe patterns greatly affects measurement accuracy, while the traditional dithering methods lack this quality. In this paper, we present a kernel-optimized dithering technique based on the Floyd-Steinberg dithering method and establish an optimization strategy considering both intensity and phase aspects to generate high-quality sinusoidal fringes for 3D measurement by utilizing the binary defocusing technique. Both simulations and experiments demonstrate that the proposed method could improve the performance of 3D measurement effectively under different defocusing amounts, especially in narrow strips.

Multilevel symmetric pattern design and optimization for high-speed and high-accuracy 3D shape measurement

The binary defocusing technique has enabled speed breakthroughs for 3D shape measurement, yet simultaneously achieving high accuracy and high speed remains difficult. To overcome this limitation, we propose to utilize multilevel symmetric pattern for high-speed and high-accuracy 3D shape measurement. Compared to conventional binary patterns, multilevel pattern could bring more flexibility for eliminating undesired high-frequency harmonics, thus has the potential to greatly enhance the phase quality and measurement accuracy. In this paper, the symmetric pattern design principle and related optimization procedure are presented in order to find the “best” multilevel fringe patterns. Both simulation and experiments verify that with respect to conventional methods, the proposed method could consistently generate better fringe patterns for a wide range of fringe periods. Furthermore, we developed an absolute 3D shape measurement system with the speed of 667 Hz, verifying that the proposed method is applicable for high-speed, high-accuracy applications.

A comparison of n-ary simple code and n-ary gray code phase unwrapping in high-speed fringe projection profilometry

Binary defocusing technique has been widely used in fringe projection profilometry for high-speed measurement. This technique projects designed binary patterns with a defocused projector to generate fringes for wrapped phase calculation, and n-ary simple codes (nSC) or n-ary gray codes (nGC) for phase unwrapping. The unwrapped phase reflecting the object surface profile is calculated by a phase-shifting algorithm with nSC/nGC phase unwrapping. However, it remains unclear which phase unwrapping method, nSC or nGC, is more appropriate for high-accuracy and high-speed measurement. This paper comprehensively compares nSC and nGC with the consideration of various factors, including the strategies for error removal, the base n, the step heights and invalid regions on the measured objects, defocusing level and noise level of the system. From simulations and experiments, we conclude that, (i) in general, nSC with a proposed identification and binary classification (IBC) method has a similar result as nGC; (ii) when accuracy is critical, binary simple code (nSC with n = 2) and binary gray code (nGC with n = 2) methods are recommended; (iii) when speed is critical, quaternary simple code (nSC with n = 4) method with continuity/geometry constraints is recommended.

Depth range enhancement of binary defocusing technique based on multi-frequency phase merging.

Binary defocusing technique has demonstrated various merits for high-speed and high-accuracy three-dimensional measurement. However, the existence of excessive defocusing zone (EDZ) limits the depth range of binary defocusing system. To overcome this problem, this paper proposes a multi-frequency phase merging (MFPM) approach, which makes it possible to measure the object surface in large depth range (LDR). The method is based on our finding that for different fringe frequencies, the associated EDZs of binary defocusing system are different and not totally overlapped. Thus by merging the phase maps of multiple binary fringes, we could effectively enhance the measurement depth range. Meanwhile, a strategy to determine the optimal combination of fringe frequencies is also proposed by analyzing the phase error distribution under different defocusing degrees. Both simulations and experiments verify the effectiveness and robustness of the proposed method.

High-speed three-dimensional shape measurement based on cyclic complementary Gray-code light.

The binary defocusing technique has been widely used in high-speed three-dimensional (3D) shape measurement because it breaks the bottlenecks in high-speed fringe projection and the projector's nonlinear response. However, it is challenging for this method to realize a two- or multi-frequency phase-shifting algorithm because it is difficult to simultaneously generate high-quality sinusoidal fringe patterns with different periods under the same defocusing degree. To bypass this challenge, we proposed a high-speed 3D shape measurement technique for dynamic scenes based on cyclic complementary Gray-code (CCGC) patterns. In this proposed method, the projected phase-shifting sinusoidal fringes kept one same frequency, which is beneficial to ensure the optimum defocusing degree for binary dithering technique. The wrapped phase can be calculated by phase-shifting algorithm and unwrapped with the aid of complementary Gray-code (CGC) patterns in a simple and robust way. Then, the cyclic coding strategy further extends the unambiguous phase measurement range and improves the measurement accuracy compared with the traditional Gray-coding strategy under the condition of the same number of projected patterns. High-quality 3D results of three complex dynamic scenes-including a cooling fan and a standard ceramic ball with a free-falling table tennis, collapsing building blocks, and impact of the Newton's cradle-were demonstrated at a frame rate of 357 fps. This verified the proposed method's feasibility and validity.

Binarized dual phase-shifting method for high-quality 3D shape measurement.

It has been demonstrated that using 1-bit binary patterns is better than using 8-bit sinusoidal patterns for high-speed applications. However, the phase quality generated from binary patterns is lower, especially when the projector is nearly focused. A dual phase-shifting method can effectively reduce the dominant periodical phase errors caused by high-order harmonics. Yet, such method requires an additional set of phase-shifting patterns, which thus slows down the measurement speed. To overcome this problem, this paper proposes a binarized dual phase-shifting method (BDPS) to generate a high-quality phase map using just three phase-shifted binary patterns. The basic idea is to merge the two sets of squared binary phase-shifting patterns into one set of patterns with three gray levels, which is further binarized by employing 2D area modulation. Given this, the BDPS method can realize error reduction for the binary defocusing technique while maintaining its speed advantage by neither adding the total number of patterns nor the number of bits. The effectiveness of the proposed method is verified by both a quantitative evaluation by measuring a white flat board and an additional quantitative evaluation by measuring a complex sculpture surface.

Out-of-Focus Projector Calibration Method with Distortion Correction on the Projection Plane in the Structured Light Three-Dimensional Measurement System

The three-dimensional measurement system with a binary defocusing technique is widely applied in diverse fields. The measurement accuracy is mainly determined by out-of-focus projector calibration accuracy. In this paper, a high-precision out-of-focus projector calibration method that is based on distortion correction on the projection plane and nonlinear optimization algorithm is proposed. To this end, the paper experimentally presents the principle that the projector has noticeable distortions outside its focus plane. In terms of this principle, the proposed method uses a high-order radial and tangential lens distortion representation on the projection plane to correct the calibration residuals caused by projection distortion. The final accuracy parameters of out-of-focus projector were obtained using a nonlinear optimization algorithm with good initial values, which were provided by coarsely calibrating the parameters of the out-of-focus projector on the focal and projection planes. Finally, the experimental results demonstrated that the proposed method can accuracy calibrate an out-of-focus projector, regardless of the amount of defocusing.

Optimal defocus selection based on normed Fourier transform for digital fringe pattern profilometry.

Owing to gamma-effect robustness and high-speed imaging capabilities, projector defocusing of binary-coded fringe patterns is by far the most widely used and effective technique in generating sinusoidal fringe patterns for three-dimensional optical topography measurement with digital fringe projection techniques. However, this technique is not trouble-free. It is borne with uncertainty and challenges mainly because it remains somewhat difficult to quantify and ascertain the level of defocus required for desired fidelity in sinuousness of the projected fringe pattern. Too much or too little defocusing will affect the sinuosity accuracy of fringe patterns and consequently jeopardize the quality of the measurement results. In this paper, by combining intrinsic phase spectral sensitivities and normed Fourier transform, a method to quantify the amount of defocus and subsequently select the optimal degree of sinuosity for generating digital sinusoidal fringe patterns with projector defocusing for fringe pattern optical three-dimensional profilometry is proposed. Numerical simulations plus experiments give evidence of the feasibility and validity of the proposed method in enabling an improved digital binary defocusing technique for optical phase-shift profilometry using the digital fringe projection technique.

High-quality octa-level fringe pattern generation for improving the noise characteristics of measured depth maps

Since the introduction of the binary defocusing technique, various algorithms have been proposed to optimize binary fringe patterns for reducing the phase root mean square (rms) error. Our recent study showed that octa-level fringe patterns can further reduce the phase rms error at no extra cost and patch-based fringe patterns can cause harmonic distortion to the measured depth map. This paper presents a novel method to produce patch-based octa-level fringe patterns of ideal noise characteristics by (1) formulating the optimization problem in a better way, (2) starting the optimization process with a better initial estimate and (3) adopting a necessity-oriented strategy to refine the fringe patterns during the optimization process.

Calibration method for fringe projection profilometry with a binary defocusing technique

A novel and simple calibration method for fringe projection profilometry with a binary defocusing technique is proposed and verified by experiments. This paper theoretically presents the principle that the one-to-one mapping servers as spatially invariant to environment in the phase domain when the projector is defocused. In terms of this principle, only two reference planes are introduced in the volume of projector defocusing (the measured volume) to specify one constraint, i.e., the emitted light beams, which are determined by the sub-pixel homologous pairs from the measured object to the reference planes in the phase domain. The nonlinear camera model is utilized to define the other constraint, i.e., the reflected light beams. Then, combining these two constraints will calibrate the 3D information of the measured object. It is pointed out that the proposed method does not require the calibration of the projector. Experiments demonstrate the performance of our proposed approach: the measurement accuracy can reach about 0.0916 mm with the binary defocusing technique.