Structured Light Method Research Articles

Vision sensing and surface fitting for real-time detection of tight butt joints

Weld joint detecting technology is significant and indispensable to intelligent welding manufacturing. And vision sensing is considered as a promising weld joint detecting technology, with the advantages of rich information, non-contact measuring, high accuracy and anti-electromagnetic interference. Many structural components used in the aerospace field are manufactured through welding with tight butt joints. However, the conventional structured light method is not suitable for the detection of tight butt joints, since they rely on the light stripes’ geometric distortion caused by the grooves. This study presents a real-time visual method for the tight butt joints’ detection. Uniform surface light is employed to highlight tight butt joint and acquire 2D information, and structured light with a double-line pattern is projected onto the seam to acquire 3D point cloud data by optical trigonometry. Surface fitting with moving Least Squares (MLS) method is conducted in an iterative manner to combine 2D and 3D information and calculate the seam’s positions and poses. Experimental results indicate that the proposed method is capable of detecting stably and accurately tight butt joints with a gap of less than 0.2 mm. The technique has the potential to be applied in tracking tight butt joints of complex structures in the aerospace industry for an automatic welding process.

Rethinking phase height model of fringe projection profilometry: A geometry transformation viewpoint

The phase height model is developed in the phase measuring profilometry, while the geometry transformation model is proposed in the structured light method and the active stereovision. However the inner connection or coherence of them has not been clearly discussed. In this paper, the gap between them is bridged. The phase height model is reasoned from the geometry transformation model, which can be denoted as an explicit function of the geometry parameters. The reasoned phase height model is consistent with two traditional phase height models, whether the system is coplanar or non-coplanar optical axes. On the other hand, this generic phase height model explains the connection of two traditional phase height models. The simulation and experiments affirm that the reasoned model is valid and the proposed calibration method is feasible.

Camera calibration method based on phase encoding for out-of-focus condition

The state-of-art camera calibration method requires the user to provide accurate pixel coordinates of calibration plate feature points. For some cameras with special sensing range, general calibration objects' (such as calibration plates with a centimeter-long dimension) using range is outside their clear sensing range. Using these cameras to take a picture for general calibration objects, you can only get out-of-focused blurred images that can not accurately extract feature points' pixel coordinates. This paper analyzes the influence on the phase of the structured light based on sine grating (abbreviated as sinusoidal structured light) when optical system is in defocus state. Based on the fact that the state of focus is independent of the phase of sinusoidal structured light, a method of phase-shifted sinusoidal structured light encoding by phase shift is proposed to encode the feature points on the calibration object and this method realizes the calibration of the camera under out-of-focus condition. The experimental results show that the maximal deviation of focal length from the real value is 0.47% and the maximal pixel reprojection error is 0.17 pixels. This paper provides a solution to camera calibration with a special sensing range.

Invalid-point removal based on epipolar constraint in the structured-light method

In structured-light measurement, there unavoidably exist many invalid points caused by shadows, image noise and ambient light. According to the property of the epipolar constraint, because the retrieved phase of the invalid point is inaccurate, the corresponding projector image coordinate (PIC) will not satisfy the epipolar constraint. Based on this fact, a new invalid-point removal method based on the epipolar constraint is proposed in this paper. First, the fundamental matrix of the measurement system is calculated, which will be used for calculating the epipolar line. Then, according to the retrieved phase map of the captured fringes, the PICs of each pixel are retrieved. Subsequently, the epipolar line in the projector image plane of each pixel is obtained using the fundamental matrix. The distance between the corresponding PIC and the epipolar line of a pixel is defined as the invalidation criterion, which quantifies the satisfaction degree of the epipolar constraint. Finally, all pixels with a distance larger than a certain threshold are removed as invalid points. Experiments verified that the method is easy to implement and demonstrates better performance than state-of-the-art measurement systems.

Single shot laser speckle based 3D acquisition system for medical applications

The state of the art techniques used by medical practitioners to extract the three-dimensional (3D) geometry of different body parts requires a series of images/frames such as laser line profiling or structured light scanning. Movement of the patients during scanning process often leads to inaccurate measurements due to sequential image acquisition. Single shot structured techniques are robust to motion but the prevalent challenges in single shot structured light methods are the low density and algorithm complexity. In this research, a single shot 3D measurement system is presented that extracts the 3D point cloud of human skin by projecting a laser speckle pattern using a single pair of images captured by two synchronized cameras. In contrast to conventional laser speckle 3D measurement systems that realize stereo correspondence by digital correlation of projected speckle patterns, the proposed system employs KLT tracking method to locate the corresponding points. The 3D point cloud contains no outliers and sufficient quality of 3D reconstruction is achieved. The 3D shape acquisition of human body parts validates the potential application of the proposed system in the medical industry.

A Weld Position Recognition Method Based on Directional and Structured Light Information Fusion in Multi-Layer/Multi-Pass Welding.

Multi-layer/multi-pass welding (MLMPW) technology is widely used in the energy industry to join thick components. During automatic welding using robots or other actuators, it is very important to recognize the actual weld pass position using visual methods, which can then be used not only to perform reasonable path planning for actuators, but also to correct any deviations between the welding torch and the weld pass position in real time. However, due to the small geometrical differences between adjacent weld passes, existing weld position recognition technologies such as structured light methods are not suitable for weld position detection in MLMPW. This paper proposes a novel method for weld position detection, which fuses various kinds of information in MLMPW. First, a synchronous acquisition method is developed to obtain various kinds of visual information when directional light and structured light sources are on, respectively. Then, interferences are eliminated by fusing adjacent images. Finally, the information from directional and structured light images is fused to obtain the 3D positions of the weld passes. Experiment results show that each process can be done in 30 ms and the deviation is less than 0.6 mm. The proposed method can be used for automatic path planning and seam tracking in the robotic MLMPW process as well as electron beam freeform fabrication process.

Three-Dimensional Hand Reconstruction by Single-Shot Structured Light Line Pattern

3-D hand reconstruction is required in many applications, e.g., hand gesture controls, hand modeling, biomechanical analysis, humanoid robot hand design and analysis, hand surgery, and so on. In most situations, it is impossible or difficult for one to keep his hand static. Thus, real-time single-shot 3-D hand reconstruction is usually required in most applications. As a result, the popular phase shift-based structured light methods could not meet the requirement, because they need at least three images to reconstruct the hand. In this paper, we propose a single-shot 3-D hand reconstruction approach based on the structured light line pattern. The proposed approach calculates a line coordinate shift map from the distorted line pattern in a single image and then reconstructs the 3-D hand with the line coordinate shift map and the calibration parameters. To calculate the line coordinate shift map accurately, we propose a specific line clustering method based on the features of the hand. We also propose a slope-based compensation method to correct the distorted line coordinate shift map. Experimental results verified the effectiveness of the proposed line clustering method based on hand features, the proposed coordinate shift map correction method, and the proposed 3-D hand reconstruction approach.

Error of image saturation in the structured-light method.

In the phase-measuring structured-light method, image saturation will induce large phase errors. Usually, by selecting proper system parameters (such as the phase-shift number, exposure time, projection intensity, etc.), the phase error can be reduced. However, due to lack of a complete theory of phase error, there is no rational principle or basis for the selection of the optimal system parameters. For this reason, the phase error due to image saturation is analyzed completely, and the effects of the two main factors, including the phase-shift number and saturation degree, on the phase error are studied in depth. In addition, the selection of optimal system parameters is discussed, including the proper range and the selection principle of the system parameters. The error analysis and the conclusion are verified by simulation and experiment results, and the conclusion can be used for optimal parameter selection in practice.

Capture of hair geometry using white structured light

Structured lighting (SL) scanning technology has been successfully applied to reconstruct personalized avatars, providing high-quality geometry for the face and body. However, previous white SL methods have typically been unable to capture and reconstruct hair geometry, preventing a complete, realistic avatar surface from being reconstructed. In this paper, we propose a novel hair capture system, producing a full-head manifold with complete hair, based on four white SL scanners made by ourselves. The key technical contribution is a robust strip-edge-based coding algorithm, using a projection pattern of 18 stripes; it allows geometric acquisition to an accuracy of 1 mm. Experiments are given to show that our system produces high-quality hair geometry, comparable to that captured by more expensive state-of-the-art multiview stereo systems. Our system is however easier to configure, and is more suited to real-world setups.

Single-Shot Dense Depth Sensing with Color Sequence Coded Fringe Pattern.

A single-shot structured light method is widely used to acquire dense and accurate depth maps for dynamic scenes. In this paper, we propose a color sequence coded fringe depth sensing method. To overcome the phase unwrapping problem encountered in phase-based methods, the color-coded sequence information is embedded into the phase information. We adopt the color-encoded De Bruijn sequence to denote the period of the phase information and assign the sequence into two channels of the pattern, while the third channel is used to code the phase information. Benefiting from this coding strategy, the phase information distributed in multiple channels can improve the quality of the phase intensity by channel overlay, which results in precise phase estimation. Meanwhile, the wrapped phase period assists the sequence decoding to obtain a precise period order. To evaluate the performance of the proposed method, an experimental platform is established. Quantitative and qualitative experiments demonstrate that the proposed method generates a higher precision depth, as compared to a Kinect and larger resolution ToF (Time of Flight) camera.

Calibration method for a large-scale structured light measurement system.

The structured light method is an effective non-contact measurement approach. The calibration greatly affects the measurement precision of structured light systems. To construct a large-scale structured light system with high accuracy, a large-scale and precise calibration gauge is always required, which leads to an increased cost. To this end, in this paper, a calibration method with a planar mirror is proposed to reduce the calibration gauge size and cost. An out-of-focus camera calibration method is also proposed to overcome the defocusing problem caused by the shortened distance during the calibration procedure. The experimental results verify the accuracy of the proposed calibration method.

Three-dimensional surface reconstruction via a robust binary shape-coded structured light method

A binary shape-coded structured light method for single-shot three-dimensional reconstruction is presented. The projected structured pattern is composed with eight geometrical shapes with a coding window size of 2×2. The pattern element is designed as rhombic with embedded geometrical shapes. The pattern feature point is defined as the intersection of two adjacent rhombic shapes, and a multitemplate-based feature detector is presented for its robust detection and precise localization. Based on the extracted grid-points, a topological structure is constructed to separate the pattern elements from the obtained image. In the decoding stage, a training dataset is first established from training samples that are collected from a variety of target surfaces. Then, the deep neural network technique is applied for the classification of pattern elements. Finally, an error correction algorithm is introduced based on the epipolar and neighboring constraints to refine the decoding results. The experimental results show that the proposed method not only owns high measurement precision but also has strong robustness to surface color and texture.

Application of optical scanning system to determine the machining allowances

The paper contains a description of replacing a process of marking-out iron casts with 3D scanning operation and the profits it implies. It presents some of the available scanning methods which is a part of surface metrology. The paper includes a description of scanning operation using structured light method and all information necessary to perform such operation, from the equipment construction up to creation of a report for the CNC machinist.

Review and Comparison of High-Dynamic Range Three-Dimensional Shape Measurement Techniques

In the last decade, a significant number of techniques for three-dimensional (3D) shape measurement have been proposed. There are a large number of measurement demands for metallic workpieces with shiny surfaces in industrial applications; however, such shiny surfaces cannot be directly measured using the conventional structured light method. Therefore, various techniques have been investigated to solve this problem over the last few years. Some reviews summarize the different 3D imaging techniques; however, no comprehensive review exists that provides an insight into high-dynamic range (HDR) 3D shape measurement techniques used for shiny surfaces. We present a survey of recent HDR techniques for the digitization of shiny surfaces and classify and discuss the advantages and drawbacks of different techniques with respect to each other.

3D inspection technology combining passive stereo matching and active structured light for steel plate surface sample

Three-dimensional (3D) inspection technology is a new important research hotspot in the field of steel surface defect inspection. However, current 3D information acquisition methods (i.e., the passive stereo vision method and active structured light method) still suffer from several issues. In order to solve these issues, a three-dimensional inspection system is designed in this paper. In this system, the passive stereo vision method and active structured light method are combined to obtain the surface sample height information of high temperature steel plate. Furthermore, the Hexagonal grid Census (Hg_Census) transform is proposed to improve the robustness of census transform stereo matching. Moreover, the passive stereo disparity map is adopted as a constraint condition to realise phase matching using wrapped phase. In addition, the local phase matching and sub-pixel disparity refinement are proposed to obtain high measuring accuracy. Three high temperature steel plate surface samples are picked to verify the effectiveness of the proposed method. The actual experimental results present the average error is less than 1 mm.

3D inspection technology combining passive stereo matching and active structured light for steel plate surface sample

Three-dimensional (3D) inspection technology is a new important research hotspot in the field of steel surface defect inspection. However, current 3D information acquisition methods (i.e., the passive stereo vision method and active structured light method) still suffer from several issues. In order to solve these issues, a three-dimensional inspection system is designed in this paper. In this system, the passive stereo vision method and active structured light method are combined to obtain the surface sample height information of high temperature steel plate. Furthermore, the Hexagonal grid Census (Hg_Census) transform is proposed to improve the robustness of census transform stereo matching. Moreover, the passive stereo disparity map is adopted as a constraint condition to realise phase matching using wrapped phase. In addition, the local phase matching and sub-pixel disparity refinement are proposed to obtain high measuring accuracy. Three high temperature steel plate surface samples are picked to verify the effectiveness of the proposed method. The actual experimental results present the average error is less than 1 mm.

High-speed surface reconstruction of a flying bird using structured light.

Birds fly effectively and maneuver nimbly by dynamically changing the shape of their wings during each wingbeat. These shape changes have yet to be quantified automatically at high temporal and spatial resolution. Therefore, we developed a custom 3D surface reconstruction method, which uses a high-speed camera to identify spatially encoded binary striped patterns that are projected on a flying bird. This non-invasive structured-light method allows automated 3D reconstruction of each stand-alone frame and can be extended to multiple views. We demonstrate this new technique by automatically reconstructing the dorsal surface of a parrotlet wing at 3200framess-1 during flapping flight. From this shape we analyze key parameters such as wing twist and angle of attack distribution. While our binary 'single-shot' algorithm is demonstrated by quantifying dynamic shape changes of a flying bird, it is generally applicable to moving animals, plants and deforming objects.

Design and verification of diffractive optical elements for speckle generation of 3-D range sensors

The optical projection using speckles is one of the structured light methods that have been applied to three-dimensional (3-D) range sensors. This paper investigates the design and fabrication of diffractive optical elements (DOEs) for generating the light field with uniformly distributed speckles. Based on the principles of computer generated holograms, the iterative Fourier transform algorithm was adopted for the DOE design. It was used to calculate the phase map for diffracting the incident laser beam into a goal pattern with distributed speckles. Four patterns were designed in the study. Their phase maps were first examined by a spatial light modulator and then fabricated on glass substrates by microfabrication processes. Finally, the diffraction characteristics of the fabricated devices were verified. The experimental results show that the proposed methods are applicable to the DOE design of 3-D range sensors. Furthermore, any expected diffraction area and speckle density could be possibly achieved according to the relations presented in the paper.

Advanced ultrasonic testing of complex shaped composite structures

Due to the wide application of composite materials it is necessary to develop unconventional quality control techniques. One of the methods that can be used for this purpose is ultrasonic tomography. In this article an application of a robotic ultrasonic system is considered. Precise positioning of the robotic scanner and path generating are defined as ones of the most important aspects. This study proposes a non-contact calibration method of a robotic ultrasonic system. Path of the scanner requires a 3D model of controlled objects which are created in accordance with the proposed algorithm. The suggested techniques are based on implementation of structured light method.

Wearable Structured Light System in Non-Rigid Configuration

Traditionally, structured light methods have been studied in rigid configurations. In these configurations the position and orientation between the light emitter and the camera are fixed and known beforehand. In this paper we break with this rigidness and present a new structured light system in non-rigid configuration. This system is composed by a wearable standard perspective camera and a simple laser emitter. Our non-rigid configuration permits free motion of the light emitter with respect to the camera. The point-based pattern emitted by the laser permits us to easily establish correspondences between the image from the camera and a virtual one generated from the light emitter. Using these correspondences, our method computes rotation and translation up to scale of the planes of the scene where the point pattern is projected and reconstructs them. This constitutes a very useful tool for navigation applications in indoor environments, which are mainly composed of planar surfaces.