High-speed Three-dimensional Shape Measurement Research Articles

Semi-Global Matching Assisted Absolute Phase Unwrapping.

Measuring speed is a critical factor to reduce motion artifacts for dynamic scene capture. Phase-shifting methods have the advantage of providing high-accuracy and dense 3D point clouds, but the phase unwrapping process affects the measurement speed. This paper presents an absolute phase unwrapping method capable of using only three speckle-embedded phase-shifted patterns for high-speed three-dimensional (3D) shape measurement on a single-camera, single-projector structured light system. The proposed method obtains the wrapped phase of the object from the speckle-embedded three-step phase-shifted patterns. Next, it utilizes the Semi-Global Matching (SGM) algorithm to establish the coarse correspondence between the image of the object with the embedded speckle pattern and the pre-obtained image of a flat surface with the same embedded speckle pattern. Then, a computational framework uses the coarse correspondence information to determine the fringe order pixel by pixel. The experimental results demonstrated that the proposed method can achieve high-speed and high-quality 3D measurements of complex scenes.

Digital image correlation assisted absolute phase unwrapping.

This paper presents an absolute phase unwrapping method for high-speed three-dimensional (3D) shape measurement. This method uses three phase-shifted patterns and one binary random pattern on a single-camera, single-projector structured light system. We calculate the wrapped phase from phase-shifted images and determine the coarse correspondence through the digital image correlation (DIC) between the captured binary random pattern of the object and the pre-captured binary random pattern of a flat surface. We then developed a computational framework to determine fringe order number pixel by pixel using the coarse correspondence information. Since only one additional pattern is used, the proposed method can be used for high-speed 3D shape measurement. Experimental results successfully demonstrated that the proposed method can achieve high-speed and high-quality measurement of complex scenes.

Time-overlapping structured-light projection: high performance on 3D shape measurement for complex dynamic scenes.

High-speed three-dimensional (3D) shape measurement has been continuously researched due to the demand for analyzing dynamic behavior in transient scenes. In this work, a time-overlapping structured-light 3D shape measuring technique is proposed to realize high-speed and high-performance measurement on complex dynamic scenes. Time-overlapping structured-light projection is presented to maximumly reduce the information redundancy in temporal sequences and improve the measuring efficiency; generalized tripartite phase unwrapping (Tri-PU) is used to ensure the measuring robustness; fringe period extension is achieved by improving overlapping rate to further double the encoding fringe periods for higher measuring accuracy. Based on the proposed measuring technique, one new pixel-to-pixel and unambiguous 3D reconstruction result can be updated with three newly required patterns at a reconstruction rate of 3174 fps. Three transient scenes including collapsing wood blocks struck by a flying arrow, free-falling foam snowflakes and flying water balloon towards metal grids were measured to verify the high performance of the proposed method in various complex dynamic scenes.

High-speed three-dimensional shape measurement based on tripartite complementary Gray-coded light.

In phase-shifting profilometry based on the Gray code, the jump error is inevitably generated and is further amplified in dynamic scenes. To tackle this problem, we propose the robust tripartite complementary Gray code method (TCG). Without projecting additional patterns, TCG uses different combinations of Gray code to calculate three complementary orders able to avoid jump error in the unwrapping process. TCG is efficient and robust, as it fully utilizes the redundant information of the Gray code. Experimental results demonstrate that TCG can realize high-efficiency and high-speed three-dimensional shape measurement at a rate of 500 fps.

High-speed three-dimensional shape measurement with inner shifting-phase fringe projection profilometry

Fringe projection profilometry (FPP) has been extensively studied in the field of three-dimensional (3D) measurement. Although FPP always uses high-frequency fringes to ensure high measurement accuracy, too many patterns are projected to unwrap the phase, which affects the speed of 3D reconstruction. We propose a high-speed 3D shape measurement method using only three high-frequency inner shifting-phase patterns (70 periods), which satisfies both high precision and high measuring speed requirements. Besides, our proposed method obtains the wrapped phase and the fringe order simultaneously without any other information and constraints. The proposed method has successfully reconstructed moving objects with high speed at the camera’s full frame rate (1700 frames per second).

A DIC-assisted fringe projection profilometry for high-speed 3D shape, displacement and deformation measurement of textured surfaces

High-speed three-dimensional (3D) shape measurement techniques based on fringe projection profilometry (FPP) have undergone huge advances over the past two decades. However, accurate 3D displacement mapping and deformation analysis of dynamic scenes using FPP remains an unsolved problem. Because fringe patterns are projected rather than attached on the tested surfaces, the full-field point-to-point correspondence cannot be accurately established between any two 3D shape results. To deal with this challenge, a DIC-assisted FPP for high-speed 3D shape, displacement and deformation measurement of textured surfaces is proposed. Firstly, a high-speed 3D shape measurement system is adopted using our recently proposed robust and efficient Gray-coded coding strategy, which can accurately reconstruct full-field shape of discontinuous surfaces with rich texture information. Then, the modulation-based method is proposed to retrieve high-quality texture map from three phase-shifting fringe patterns, which can eliminate the adverse influence of the nonuniform and time-varying ambient light. By matching the retrieved surface texture images at difference states using DIC, accurate point tracking between the measured 3D shape data can be fulfilled, leading to precise 3D displacement and deformation measurement. Experiments have verified that the proposed method can achieve 3D shape, displacement and deformation measurement of dynamic scenes at a frame rate of 542 fps without any extra expense of hardware or calibration for the FPP system. The presented method is reliable and promising for further 3D displacement mapping and deformation analysis of dynamic scenes using FPP.

Improve Temporal Fourier Transform Profilometry for Complex Dynamic Three-Dimensional Shape Measurement.

The high-speed three-dimensional (3-D) shape measurement technique has become more and more popular recently, because of the strong demand for dynamic scene measurement. The single-shot nature of Fourier Transform Profilometry (FTP) makes it highly suitable for the 3-D shape measurement of dynamic scenes. However, due to the band-pass filter, FTP method has limitations for measuring objects with sharp edges, abrupt change or non-uniform reflectivity. In this paper, an improved Temporal Fourier Transform Profilometry (TFTP) algorithm combined with the 3-D phase unwrapping algorithm based on a reference plane is presented, and the measurement of one deformed fringe pattern producing a new 3-D shape of an isolated abrupt objects has been achieved. Improved TFTP method avoids band-pass filter in spatial domain and unwraps 3-D phase distribution along the temporal axis based on the reference plane. The high-frequency information of the measured object can be well preserved, and each pixel is processed separately. Experiments verify that our method can be well applied to a dynamic 3-D shape measurement with isolated, sharp edges or abrupt change. A high-speed and low-cost structured light pattern sequence projection has also been presented, it is capable of projection frequencies in the kHz level. Using the proposed 3-D shape measurement algorithm with the self-made mechanical projector, we demonstrated dynamic 3-D reconstruction with a rate of 297 Hz, which is mainly limited by the speed of the camera.

High-speed three-dimensional absolute shape measurement with three projected binary patterns

Reducing the number of structured patterns for three-dimensional (3-D) reconstruction is of great importance for high-speed 3-D shape measurement. We present a method that reconstructs absolute 3-D shape using three projected binary patterns: one direct current (DC), one low-frequency, and one high-frequency fringe pattern. The procedures are (1) take the difference between the sinusoidal fringe patterns and the DC pattern; (2) apply Hilbert transform to the difference images to generate two phase maps; (3) employ the geometric-constraint-based phase unwrapping method to unwrap the low-frequency phase map; (4) unwrap the high-frequency phase map using the unwrapped low-frequency phase map; and (5) reconstruct 3-D shape. We developed a prototype system that can capture two-dimensional images at 6000 Hz, achieving 2000 Hz 3-D shape measurement speed.

High-speed three-dimensional shape measurement based on shifting Gray-code light.

The measuring technique combining a phase-shifting algorithm and Gray-code light has been widely used in three-dimensional (3D) shape measurement for static scenes because of its high robustness and anti-noise ability. However, in the high-speed measurement, phase unwrapping errors occur easily on the boundaries of adjacent Gray-code words because of the defocus of the projector, the motion of the objects and the non-uniform reflectivity of the surface. To overcome this challenge, a high-speed 3D shape measurement method based on shifting Gray-code light has been proposed in this paper. Firstly, the average intensity of three captured phase-shifting fringe images are used as a pixel-wise threshold to binarize the Gray codes and to eliminate most phase unwrapping errors caused by the non-uniform reflectivity, ambient light variations, and the defocus of projector. Then, the shifting Gray-code (SGC) coding strategy is proposed to avoid the remaining errors of phase unwrapping on the edge of the code words. In this strategy, no additional patterns are projected, and two sets of decoding words with staggered boundaries are built in the temporal sequences for one wrapped phase. Finally, the simple, efficient, and robust phase unwrapping can be achieved in the high-speed dynamic measurement. This proposed method has been applied to reconstruct 3D shape of randomly collapsing objects in a large depth range, and the experimental results demonstrate that it can reliably obtain high-quality shape and texture information at 310 frames per second.

High-speed three-dimensional shape measurement based on cyclic complementary Gray-code light.

The binary defocusing technique has been widely used in high-speed three-dimensional (3D) shape measurement because it breaks the bottlenecks in high-speed fringe projection and the projector's nonlinear response. However, it is challenging for this method to realize a two- or multi-frequency phase-shifting algorithm because it is difficult to simultaneously generate high-quality sinusoidal fringe patterns with different periods under the same defocusing degree. To bypass this challenge, we proposed a high-speed 3D shape measurement technique for dynamic scenes based on cyclic complementary Gray-code (CCGC) patterns. In this proposed method, the projected phase-shifting sinusoidal fringes kept one same frequency, which is beneficial to ensure the optimum defocusing degree for binary dithering technique. The wrapped phase can be calculated by phase-shifting algorithm and unwrapped with the aid of complementary Gray-code (CGC) patterns in a simple and robust way. Then, the cyclic coding strategy further extends the unambiguous phase measurement range and improves the measurement accuracy compared with the traditional Gray-coding strategy under the condition of the same number of projected patterns. High-quality 3D results of three complex dynamic scenes-including a cooling fan and a standard ceramic ball with a free-falling table tennis, collapsing building blocks, and impact of the Newton's cradle-were demonstrated at a frame rate of 357 fps. This verified the proposed method's feasibility and validity.

Single-shot 3D shape measurement of discontinuous objects based on a coaxial fringe projection system.

Fringe projection profilometry has been widely used in high-speed three-dimensional (3D) shape measurement. To improve the speed without loss of accuracy, we present a novel single-shot 3D shape measuring system that utilizes a coaxial fringe projection system and a 2CCD camera. The coaxial fringe projection system, comprising a visible light (red, green, and blue) projector and an infrared (IR) light projector, can simultaneously project red, green, blue, and IR fringe patterns. The 2CCD camera, as the name suggests, has two CCD chips that can acquire visible and IR fringe patterns at the same time. Combining the two-step phase-shifting algorithm, Fourier transform profilometry, and the optimum three-frequency selection method, 3D shape measurement of complex surfaces such as large slopes or discontinuous objects can be obtained from single-shot acquisition. A virtual fringe projection measurement system has been established to generate pre-deformed fringe patterns to correct positional deviations of the coaxial fringe projection system. This method has been applied to simulations and experiments on static and dynamic objects with promising results.

High-speed three-dimensional shape measurement using geometry-constraint-based number-theoretical phase unwrapping

In this paper, we propose a high-speed three-dimensional (3-D) shape measurement technique for dynamic scenes using geometry-constraint-based number-theoretical phase unwrapping. As a classical algorithm for temporal phase unwrapping (TPU), the number-theoretical approach is suitable for the binary defocusing fringe projection system since it can retrieve an absolute phase without using low-frequency fringe patterns. However, the conventional number-theoretical TPU approach cannot provide sufficient stability to unwrap a high-frequency phase since it requires the two fringe frequencies to be coprime within the global range of the projector coordinate. In contrast, using low-frequency fringe patterns tends to make phase unwrapping more reliable, but at the expense of the measurement precision. By introducing depth constraint into the traditional number-theoretical TPU, we only need to eliminate the phase ambiguity of each pixel within a small period range defined by the depth range, which means that our method just requires the two fringe frequencies to be coprime within the local period range instead of the conventional global range. Due to the reduction of fringe order candidates and the unambiguous phase range, the reliability of phase unwrapping can be significantly improved compared with the traditional number-theoretical TPU approach even when high-frequency fringe patterns are used. The proposed method has been successfully implemented on a high-frame-rate fringe projection system, achieving high-precision, robust, and absolute 3-D shape measurement at 3333 frames per second.

Calibration of an array projector used for high-speed three-dimensional shape measurements using a single camera.

Geometric calibration of digital light processing projectors in single-camera, fringe-projecting 3D measurement systems have been studied assuming the projector is inverse pinhole modeled. Conversely, a high-speed multi-aperture array projector (MAAP) projecting aperiodic fringes is not dependent on a digital mirror device and cannot be pinhole modeled. With MAAP projection, a stereo camera setup is required. This paper presents a model-less method to calibrate a MAAP by direct measurement of its illumination field and re-enables 3D measurements with a single camera even with surface discontinuities present. Experimental proof of principle and preliminary measurement performance are shown.

Full-field phase error analysis and compensation for nonsinusoidal waveforms in phase shifting profilometry with projector defocusing

Phase shifting profilometry (PSP) using binary fringe patterns with projector defocusing is widely used for high-speed three-dimensional (3D) shape measurement. In this technique, the projector projects binary fringe patterns instead of the sinusoidal fringe patterns to perform 3D shape measurement. However, inappropriate defocusing degree of the projector causes the captured fringe patterns to be nonsinusoidal waveforms or remaining binary structure, which results in phase error and therefore measurement error. In this paper, a novel method was proposed not only to measure the phase error directly but also to extract the full-field phase error distribution in a large depth range of projector defocusing. Then a mathematical phase error function in terms of the depth z was established. Finally, with help of the detected phase error distribution, the phase error function was calibrated and used to compensate the phase error at arbitrary depth ranges within the calibration volume. The experimental results demonstrated that the proposed method can detect full-field phase error distribution successfully, and improve the phase accuracy significantly.

High-speed three-dimensional shape measurement using phase measurement profilometry without phase unwrapping

In the fast three-dimensional (3-D) shape measurement, it is an important factor to use the least number of the fringe pattern to get 3-D shape measurement of arbitrary objects. Although one-shot technologies can use one-frame deformed fringe pattern to get the wrapped phase, they are easily affected by the surface property and suffer from poor spatial resolution. In addition to the process, phase unwrapping may affect the quality of absolute phase. This paper proposes a fast measurement method based on a phase measurement profilometry and stereo vision system. This method can reconstruct 3-D surface without phase unwrapping. Using original image matching constraint, a rough parallax is used in the phase matching. To resist the false matching, subpixel parallax optimization is used to reduce the matching errors. To detect the edge point of wrapped phase, the average phase is calculated. Experimental results verify the feasibility of the proposed method and it can measure complex objects without phase unwrapping.

BRDF-dependent accuracy of array-projection-based 3D sensors.

In order to perform high-speed three-dimensional (3D) shape measurements with structured light systems, high-speed projectors are required. One possibility is an array projector, which allows pattern projection at several tens of kilohertz by switching on and off the LEDs of various slide projectors. The different projection centers require a separate analysis, as the intensity received by the cameras depends on the projection direction and the object's bidirectional reflectance distribution function (BRDF). In this contribution, we investigate the BRDF-dependent errors of array-projection-based 3D sensors and propose an error compensation process.

Pixel-by-pixel absolute three-dimensional shape measurement with modified Fourier transform profilometry

The single-pattern Fourier transform profilometry (FTP) and double-pattern modified FTP methods have great value in high-speed three-dimensional shape measurement, yet it is difficult to retrieve absolute phase pixel by pixel. This paper presents a method that can recover absolute phase pixel by pixel for the modified FTP method. The proposed method uses two images with different frequencies, and the recovered low-frequency phase is used to temporally unwrap the high-frequency phase pixel by pixel. This paper also presents the computational framework to reduce noise impact for robust phase unwrapping. Experiments demonstrate the success of the proposed absolute phase recovery method using only two fringe patterns.

Phase unwrapping for fringe projection three-dimensional measurement with projector defocusing

For high-speed three-dimensional shape measurement, techniques with projector defocusing have become increasingly important. When the projector is defocused, structured patterns used for phase unwrapping are blurred. These blurred patterns need to be normalized into binary distributions in actual measurement. We propose a method to normalize these blurred structured patterns. The retrieved absolute phase is correct even when the measured objects contain discontinuous parts or contrasting colors.

High-speed three-dimensional shape measurement using GOBO projection

A projector which uses a rotating slide structure to project aperiodic sinusoidal fringe patterns at high frame rates and with high radiant flux is introduced. It is used in an optical three-dimensional (3D) sensor based on coded-light projection, thus allowing the analysis of fast processes. Measurements of an inflating airbag, a rope skipper, and a soccer ball kick at a 3D frame rate of more than 1300 independent point clouds per second are presented.

Experimental comparison of aperiodic sinusoidal fringes and phase-shifted sinusoidal fringes for high-speed three-dimensional shape measurement

Stereo vision-based triangulation systems are commonly used to reconstruct the three-dimensional surface shape of objects. In order to increase accuracy, they typically contain a projector which projects a sequence of patterns onto the object’s surface. However, in spite of a considerable demand to measure dynamic processes, such systems are often limited to static objects. In order to enhance their speed, apart from fast cameras and projection systems, it is useful to use pattern sequences as short as possible. In this contribution, we experimentally compare two different illumination approaches, aperiodic sinusoidal fringes and phase-shifted sinusoidal fringes, with regard to the achievable point cloud completeness and accuracy depending on the pattern parameters and sequence length.