Structured Light System Research Articles

SGE: Structured Light System Based on Gray Code with an Event Camera

SGE: Structured Light System Based on Gray Code with an Event Camera

LAVOLUTION: Tunable structured light for bridge displacement measurement

Assessing bridge conditions is critical, but field measurements often face challenges like sensor installation and accuracy requirements. This paper introduces a novel approach using structured light (SL) for non-target-based measurements, eliminating the need for physical markers. A tunable SL system, featuring four laser pointers aligned with a commercial camera, was developed to estimate the relative angle between the camera and the target structure, and to accurately calculate the scale factor. The proposed method involves three main processes: (1) laser alignment, (2) angle estimation, and (3) real-time displacement measurement. By precisely aligning the SL system and minimizing discrepancies between projected SL points and an ideal numerical model, the method achieves accurate results. Validation through numerical simulations, lab-scale tests, and full-scale bridge tests demonstrated a maximum error of 2.69% in scale and 2.53% in displacement, confirming the method’s effectiveness and applicability.

Robust structured light with efficient redundant codes

Structured light (SL) systems acquire high-fidelity 3D geometry with active illumination projection. Conventional systems exhibit challenges when working in environments with strong ambient illumination. This paper studies a general-purposed solution to improve the robustness of SL by projecting a redundant number of patterns. Despite sacrificing the signal-noise-ratio at each frame, projected signals become more distinguishable from errors. Thus, the geometry can be recovered easily. We systematically analyze the redundant SL code design rules to achieve high accuracy with minimum redundancy. Based on the more reliable correspondence cost volume and the natural image prior, we integrate spatial context-aware disparity estimators into our system to further boost performance. We also demonstrate the application of such techniques in iterative error detection and refinement. We demonstrate significant performance improvements of efficient redundant code SL systems in both simulations and challenging real-world scenes.

Single-shot calibration method based on Fourier transform profilometry.

This Letter introduces a novel, to the best of our knowledge, calibration method for structured light systems that simplifies the calibration process and reduces time consumption. We combine vertical and horizontal fringe patterns into a single composite pattern and retrieve the bidirectional phase based on Fourier transform profilometry (FTP). The entire calibration process becomes faster and more simplified by capturing only a single-shot pattern. Experimental results demonstrate that the proposed method achieves high accuracy and robustness, comparable to the conventional method that requires capturing numerous fringe patterns for the typical phase-shifting algorithm.

Fringe projection profilometry (FPP) based point clouds fusion for the binocular and monocular structured light systems

Fringe projection profilometry (FPP) based point clouds fusion for the binocular and monocular structured light systems

Handheld structured light system for panoramic 3D measurement in mesoscale

Abstract The measurement of complete 3D topography in mesoscale plays a vital role in high-precision reverse engineering, oral medical modeling, circuit detection, etc. Traditional structured light systems are limited to measuring 3D shapes from a single perspective. How to achieve high-quality mesoscopic panoramic 3D measurement remains challenging, especially in complex measured scenarios such as dynamic measurement, scattering medium, and high reflectance. To overcome these problems, we develop a handheld mesoscopic panoramic 3D measurement system for such complex scenes together with the fast point-cloud-registration and accurate 3D-reconstruction, where a motion discrimination mechanism is designed to ensure that the captured fringe is in a quasi-stationary case by avoiding the motion errors caused during fringe scanning; a deep neural network is utilized to suppressing the fringe-degradation caused by scattering mediums resulting to significantly improves the quality of the 3D point cloud; a strategy based on phase averaging is additionally proposed to simultaneously correct the saturation-induced errors and gamma nonlinear errors. Finally, the proposed system with a multi-threaded data processing framework is further developed to verify the proposed method and the corresponding experiments verify its feasibility.

A calibration method of line-structured light system for measuring outer circle dimension during machining

The line-structured light system is widely utilized for various 3D profile measurements due to its convenience and high efficiency. However, the existing calibration methods primarily focus on establishing the relationship between object points and image points, which imposes stringent requirements on the calibration target. To address this issue, a practical calibration method based on interpolation function is proposed in order to measure a specific 3D profile - namely, the diameter of a rotating workpiece during machining. Firstly, the light stripe projected onto an ordinary machined workpiece by a light plane is captured using a Charge-Coupled Device (CCD) camera and subsequently fitted to a quadratic elliptic curve after undergoing image processing. Subsequently, the cubic spline interpolation function is selected to directly establish a mapping relationship between geometric characteristics of the elliptic curve and the diameter measurement. Several experiments are conducted to evaluate the proposed methods. The experimental results demonstrate that compared with two comparison methods, our approach reduces the calibration error of linear structured light systems by 69 % and 51 %, respectively; furthermore, it enables restoration of smooth and detailed 3D models through parallel movement of the system. Moreover, our entire calibration process proves practicable in simplifying experimental procedures while remaining suitable for industrial inspection applications.

Realization of Impression Evidence with Reverse Engineering and Additive Manufacturing

Significant advances in reverse engineering and additive manufacturing have the potential to provide a faster, accurate, and cost-effective process chain for preserving, analyzing, and presenting forensic impression evidence in both 3D digital and physical forms. The objective of the present research was to evaluate the capabilities and limitations of five 3D scanning technologies, including laser scanning (LS), structured-light (SL) scanning, smartphone (SP) photogrammetry, Microsoft Kinect v2 RGB-D camera, and iPhone’s LiDAR (iLiDAR) Sensor, for 3D reconstruction of 3D impression evidence. Furthermore, methodologies for 3D reconstruction of latent impression and visible 2D impression based on a single 2D photo were proposed. Additionally, the FDM additive manufacturing process was employed to build impression evidence models created by each procedure. The results showed that the SL scanning system generated the highest reconstruction accuracy. Consequently, the SL system was employed as a benchmark to assess the reconstruction quality of other systems. In comparison to the SL data, LS showed the smallest absolute geometrical deviations (0.37 mm), followed by SP photogrammetry (0.78 mm). In contrast, the iLiDAR exhibited the largest absolute deviations (2.481 mm), followed by Kinect v2 (2.382 mm). Additionally, 3D printed impression replicas demonstrated superior detail compared to Plaster of Paris (POP) casts. The feasibility of reconstructing 2D impressions into 3D models is progressively increasing. Finally, this article explores potential future research directions in this field.

VDI/VDE 2634-2 and ISO 10360-13 performance evaluation tests, and systematic errors in structured light systems

Structured light systems (SLSs) are commonly used for precision dimensional measurements. These are active systems in that the projected structured light pattern is used to establish correspondence between the projector and the camera(s) to provide more accurate 3D reconstruction as opposed to passive camera-based systems that rely on the user or features in images to identify corresponding points between the cameras. Errors in the calibration of the model parameters of an SLS can lead to systematic errors in dimensional measurements. In this paper, we explore the topic of performance testing of an SLS, specifically, the sensitivity of the length tests described in the VDI/VDE 2634-2 guideline and the ISO 10360-13 standard to the model parameters (such as intrinsic parameters of the camera and the projector, and camera-projector geometry parameters) of an SLS. The results are based on a hybrid approach of simulations performed using data obtained experimentally from an SLS mounted on a Cartesian coordinate measuring machine (CMM) which is used primarily to position the target at different points within the measurement volume. The results show that the length tests described in the ISO 10360-13 standard provide better sensitivity to the model parameters than the VDI/VDE 2634-2 guideline. However, even the ISO 10360-13 length tests do not detect all parameters with high sensitivity. A model-based approach in identifying length tests (i.e., the position and orientation of reference lengths) such as described in this paper provides better sensitivity to model parameters and therefore is more likely to detect systematic errors in SLSs.

Pixelwise calibration method for a telecentric structured light system.

This paper introduces a pixelwise calibration method designed for a structured light system utilizing a camera attached with a telecentric lens. In the calibration process, a white flat surface and a flat surface with circle dots serve as the calibration targets. After deriving the properties of the pinhole projector through a conventional camera calibration method using circle dots and determining the camera's attributes via 3D feature points estimation through iterative optimizations, the white surface calibration target was positioned at various poses and reconstructed with initial camera and projector calibration data. Each 3D reconstruction was fitted with a virtual ideal plane that was further used to create the pixelwise phase-to-coordinate mapping. To optimize the calibration accuracy, various angled poses of the calibration target are employed to refine the initial results. Experimental findings show that the proposed approach offers high calibration accuracy for a structured light system using a telecentric lens.

A Coupled Calibration Method for Dual Cameras-Projector System with Sub-Pixel Accuracy Feature Extraction.

Binocular structured light systems are widely used in 3D measurements. In the condition of complex and local highly reflective scenes, to obtain more 3D information, binocular systems are usually divided into two pairs of devices, each having a Single Camera and a Projector (SCP). In this case, the binocular system can be seen as Dual Cameras-Projector (DCP) system. In the DCP calibration, the Left-SCP and Right-SCP need to be calibrated separately, which leads to inconsistent parameters for the same projector, thus reducing the measurement accuracy. To solve this problem and improve manoeuvrability, a coupled calibration method using an orthogonal phase target is proposed. The 3D coordinates on a phase target are uniquely determined by the binocular camera in DCP, rather than being calculated separately in each SCP. This ensures the consistency of the projector parameters. The coordinates of the projector image plane are calculated through the unwrapped phase, while the parameters are calibrated by the plane calibration method. In order to extract sub-pixel accuracy feature points, a method based on polynomial fitting using an orthogonal phase target is exploited. The experimental results show that the reprojection error of our method is less than 0.033 pixels, which improves the calibration accuracy.

Fringe Projection Profilometry for Three-Dimensional Measurement of Aerospace Blades

The aero-engine serves as the “heart” of an aircraft and is a primary factor determining the aircraft’s performance. Among the crucial components in the core of aero-engines, aero-engine compressor blades stand out as extremely important. They are not only numerous but also characterized by a multitude of parameters, making them the most complex parts in an aero-engine. This paper aims to address the trade-off between accuracy and efficiency in the existing measurement methods for asymmetric blades. Non-contact measurements were conducted using a structured light system composed of a stereo camera and a DLC projector. The point cloud data of the blades are processed using methods such as the PCA (Principal Component Analysis) algorithm, binary search, and least squares fitting. This paper established a fringe-projection profilometry light sensor system for the multi-view measurement of the blades. High-precision rotary tables are utilized to rotate and extract complete spatial point cloud data of aviation blades. Finally, measurements and comparative experiments on the blade body are conducted. The obtained blade point cloud data undergo sorting and denoising processes, resulting in improved measurement accuracy. The measurement error of the blade chord length is 0.001%, the measurement error of blade maximum thickness is 0.895%, compared to CMM (Coordinate Measuring Machine), where the measurement error of chord is 0.06%.

Novel calibration technique for hybrid structured-light three-dimensional measurement system

Hybrid structured-light measurement technique is used for form measurement of structured composite surfaces. A hybrid structured-light measurement system contains a phase measuring deflectometry (PMD) subsystem and a fringe projection profilometry (FPP) subsystem. Each subsystem measures specular surfaces and rough surfaces based on structured-light reflection and projection principle, respectively. Calibration’s accuracy extremely effects data stitching precision between the subsystems. A novel calibration technique is explored for the hybrid structured-light system to complete reliable measurement accuracy. Calibration algorithms are developed based on designed calibration targets. Information of the calibration procedure are discussed and presented. Effectiveness of the proposed calibration technique has been conducted and verified through experiments by measuring structured composite samples. Experimental results demonstrate that the proposed technique can significantly improve data fusion accuracy of a hybrid structured-light measurement system.

Fast 3D reconstruction via event-based structured light with spatio-temporal coding.

Event-based structured light (SL) systems leverage bio-inspired event cameras, which are renowned for their low latency and high dynamics, to drive progress in high-speed structured light systems. However, existing event-based structured light methods concentrate on the independent construction of either time-domain or space-domain features for stereo matching, ignoring the spatio-temporal consistency towards depth. In this work, we build an event-based SL system that consists of a laser point projector and an event camera, and we devise a spatial-temporal coding strategy that realizes depth encoding in dual domains through a single shot. To exploit the spatio-temporal synergy, we further present STEM, a novel Spatio-Temporal Enhanced Matching approach for 3D reconstruction. STEM is comprised of two parts, the spatio-temporal enhancing (STE) algorithm and the spatio-temporal matching (STM) algorithm. Specifically, STE integrates the dual-domain information to increase the saliency of the temporal coding, providing a more robust basis for matching. STM is a stereo matching algorithm explicitly tailored to the unique characteristics of event data modality, which computes the disparity via a meticulously designed hybrid cost function. Experimental results demonstrate the superior performance of our proposed method, achieving a reconstruction rate of 16 fps and a low root mean square error of 0.56 mm at a distance of 0.72 m.

Suppressing blooming interference in structured light systems by adaptively reducing camera oversaturated energy.

Structured light systems often suffer interference of the fringes by blooming when scanning metal objects. Unfortunately, this problem cannot be reliably solved using conventional methods such as the high dynamic range (HDR) method or adaptive projection technique. Therefore, this study proposes a method to adaptively suppress the oversaturated areas that cause blooming as the exposure time increases and then fuse the multi-exposure time decoding results using a decoding inheritance method. The experimental results demonstrate that the proposed method provides a more effective suppression of blooming interference than existing methods.

Phase retrieval for dual-projection pattern based on orthogonal frequency encoding to suppress superposing effects.

When grating patterns are simultaneously projected by a dual-projection structured-light system, interference-like blur and brightness overexposure in the superposed area often cause miscalculation of the phase of the grating pattern. In this study, we proposed a novel method, to the best of our knowledge, that utilizes orthogonal grating encoding to retrieve the phases of superposed grating patterns. Specifically, we determined the frequency of the dual-projection pattern based on the condition that enabled the separation of superposed orthogonal signals in wireless communication. Additionally, the maximum intensity of the projected pattern was determined using the intensity-saturation relationship. By performing a discrete Fourier transform on a series of superposed grating patterns, we obtained the wrapped phase of the corresponding projected grating patterns in the space-time dimension. Finally, we reconstructed the measured object by fusing the point clouds obtained from the dual-projection structured-light system. The experimental results demonstrated that the encoded orthogonal grating patterns could eliminate interference-like blurring and brightness overexposure during superposition and obtain high-precision phase maps and 3D reconstruction results, which provides the possibility for the simultaneous reconstruction of multiprojection structured light.

Phase-domain modulated hybrid phase-shifting structured light based efficient 3D measurement

Phase-shifting profilometry (PSP) measurement technology remains a research hotspot in 3D imaging. However, existing PSP methods require projecting additional patterns, which limits measurement space, or involves compressing fringe amplitudes to eliminate phase ambiguity. These practices result in reduced measurement efficiency and robustness. To overcome these challenges, we present a novel phase-domain modulated hybrid phase-shifting (PDM-HPS) structured light 3D measurement approach to solve the phase error of existing spatial-domain modulated phase-shifting (SDM-PS) methods without imposing additional pattern or space constraints. The method integrates phase-shifting speckle and phase-shifting fringe methodologies to create a hybrid phase-shifting coding pattern. In the decoding phase, the offset speckle phase is detected and corrected by the low-precision phase extracted in the frequency domain. The separation of binary speckle and high-precision wrapped phase is realized. Ultimately, an efficient binary speckle-aided stereo phase matching process is employed to achieve unambiguous phase unwrapping. We test the PDM-HPS method in multiple complex real-world scenarios using binocular structured light systems, and compare its accuracy and efficiency with three existing advanced methods. The qualitative and quantitative results show that the proposed method can achieve the measurement accuracy similar to that of the multi-frequency (MF) method by using only 1/3 projection patterns.

A Polarized Structured Light Method for the 3D Measurement of High-Reflective Surfaces

The reflection phenomenon exhibited by highly reflective surfaces considerably affects the quality of captured images, thereby rendering the task of structured light (SL) 3D reconstruction. In this paper, a polarized SL method is proposed to address the reconstruction issues on high-reflectance surfaces. The SL system we build in this paper involves a four-channel polarizing camera and a digital light processing (DLP) projector equipped with a polarizer in the lens. The built system enables the simultaneous acquisition of four groups of fringe images, each with different brightness differences. Then, a binary time-multiplexing SL method is adopted to obtain four distinct point clouds. Additionally, a fusion algorithm is proposed to merge the four point clouds into a single, precise, and complete point cloud. Several experiments have been conducted to demonstrate that the proposed method is capable of achieving excellent reconstruction outcomes on highly reflective surfaces.

Pixel-wise rational model for a structured light system.

This Letter presents a novel structured light system model that effectively considers local lens distortion by pixel-wise rational functions. We leverage the stereo method for initial calibration and then estimate the rational model for each pixel. Our proposed model can achieve high measurement accuracy within and outside the calibration volume, demonstrating its robustness and accuracy.

Flexible structured light system calibration method with all digital features.

We propose an innovative method for single-camera and single-projector structured light system calibration in that it eliminates the need for calibration targets with physical features. Instead, a digital display such as a liquid crystal display (LCD) screen is used to present a digital feature pattern for camera intrinsic calibration, while a flat surface such as a mirror is used for projector intrinsic and extrinsic calibration. To carry out this calibration, a secondary camera is required to facilitate the entire process. Because no specially made calibration targets with real physical features are required for the entire calibration process, our method offers greater flexibility and simplicity in achieving accurate calibration for structured light systems. Experimental results have demonstrated the success of this proposed method.