Structured Light Vision Research Articles

Omni-Directional Vision System for Mobile Robot Using Structured Lights

Vision system can obtain rich environmental information, and vision is one of the most important sensing methods for mobile robot navigation and positioning. An omni-directional vision system using laser illumination is presented. The system can obtain the distance information of the obstacles around the mobile robot by only one image. The principle of the vision system is expounded; the mapping relationship between the image plane and the laser plane is given based on the analysis of the system model. The system benefits from the advantages of the omni-directional vision and the structured light vision, such as large ranged environmental information, high precision and robustness, etc. Experimental results of measurement and map construction verify the vision system.

Pipe Defect Detection and Reconstruction Based on 3D Points Acquired by the Circular Structured Light Vision

The inspection of pipe inside is very important so as to ensure the safety of pipe transmission. The pipe defect detection method based on inside 3D points acquired by circle structured light vision system was discussed in this paper. Based on the characteristics of 3D points inside a pipe acquired by circle structured light, a method by which pipe inner surface convex defect and pit defect were identified by the normal vector angle between adjacent points change on the same circle was proposed. A distance constraint between each pair of corresponding adjacent two points was proposed in order to recognize holes in the pipe inside. Based on Delaunay triangulation principle, the triangulation method for pipe inner surface circle structured light vision 3D points with hole and without hole was discussed and 3D reconstruction of the pipe was then realized. On the VC++ 6.0 software platform, pipe inner surface defect simulation data, hole simulation data and real 3D data were used to test the correctness of pipe defect identification method.

A Sphere-Based Calibration Method for Line Structured Light Vision Sensor

A major difficulty in calibrating line structured light vision sensor is how to obtain enough calibration feature points, due to the fact that known world points on the calibration target do not rightly fall onto the light stripe plane. This paper presents a calibration method using a target, consisting of one sphere and a reference board, to obtain feature points on the light stripe, which is the intersection of the structured-light plane and the reference board. By moving the sphere into several positions on the fixed reference board, we can get a plane in parallel with the reference one. From the acquired right-circular cone and known radius of the sphere, the function of the reference board can be deduced. Moving the target randomly into different positions, enough points lying on the structured-light plane can be obtained. Experiment results show that the accuracy of the proposed calibration method can reach 0.102 mm within the view field of 200 mm * 200 mm; meanwhile, robustness and easy operation can be indicated.

A novel laser vision sensor for omnidirectional 3D measurement

The combination of omnidirectional imaging and structured-light vision technique provides an efficient solution for omnidirectional 3D measurement. However, the existing omnidirectional structured-light vision sensors are constructed on the basis of curved mirrors, such as parabolic mirrors and conic mirrors, which inevitably generate two obvious problems: first, the difficulty of camera calibration is undoubtedly increased because the camera mathematical model becomes nonlinear and the traditional camera model is no longer applicable in those cases and second, the measurement accuracies of those sensors are far worse than those of traditional structured-light sensors since the captured images are severely warp-deformed. To solve these problems, we developed a novel omnidirectional laser vision sensor by adopting plane mirrors instead of curved mirrors so that the simple traditional camera model can still be used, and more important, the measurement accuracy can be maintained at the same level with the traditional structured-light vision. With respect to the sensor designing, the omnidirectional camera is constructed by a traditional perspective camera and a pyramid reflector, which is made up of four identical plane mirrors of isosceles triangular shape. Moreover, four laser projectors are used to project structured-light to the surrounding scene by the way of splicing. The sensor is calibrated by a 2D planar target and a quasi-3D target. The experimental results show that the relative calibration and measurement errors of the sensor are less than 0.08% and 0.16% respectively. It is proved that complete omnidirectional 3D measurement with high accuracy is feasible with the proposed sensor system.

Feature points matching for face reconstruction based on the window unique property of pseudo-random coded image

Structured light vision systems have been successfully used for accurate measurement of the 3D surfaces of an object, in which a pseudo-random coded structured light image pattern is projected onto the target object through a projector, and the coded information image produced by its surface is captured by a CCD camera in order to recover its 3D surfaces. In this kind of computer vision technology, 3D face reconstruction is a hot research topic. This paper presents a method for feature points matching used in 3D reconstruction. In this method, the feature points can be identified exclusively taking advantage of the window unique property of a pseudo-random array. Thus, the matching problem can be solved by finding the correspondence between 2D coordinates of feature points in the pixel image and those in the code of the projected template. Then, the 3D reconstruction can be carried out with only a single image with the benefit of easy operation and simple calculation. An experiment for 3D face reconstruction with simulated data is given. The performances show that this method has high matching precision for object matching of feature points.

Calibration Method for Multiple Line Structured Light Vision Sensors

Calibration Method for Multiple Line Structured Light Vision Sensors

Error Modeling and Compensation Approach for Three-dimensional Surface Scanning Robot

For the three-dimensional surface scanning robotic system,the system measuring accuracy mainly depends on the following aspects: Robot positioning accuracy,line structured light vision sensor measuring accuracy,hand to eye calibration accuracy.The three aspects mentioned above can be considered as part ones.Whatever how much error will these three aspects be,the final error will be reflected in the obtained measuring data.However,the error model of the system is difficult to be established via mathematical models.As to enhance the system measuring accuracy,the error model is constructed via particle swarm optimization-radial basis function neural network(PSO-RBFNN).The input of PSO-RBFNN is chosen as the image coordinates of laser stripes,while the output of PSO-RBFNN is chosen as the errors between measured points and closest points after iterative closest point technique.The centers and widths of RBFNN are optimized by particle swarm optimization(PSO) to find more suitable parameters and thus reducing the number of neurons in the hidden layer.Experimental results demonstrate that the system measuring accuracy is improved after the error compensation scheme via constructed error network model.

线结构光视觉测量系统运动轴线的简易标定方法

线结构光视觉测量系统运动轴线的简易标定方法

Simple and fast rail wear measurement method based on structured light

In this paper, a fast and accurate rail wear measurement method based on simple equipments is presented. The inner rail profile is measured by a line structured light vision sensor. Using the centers of the big and small circle from the rail waist profile as control points, the measured rail profile is registered to the reference profile. The rail wear, including the vertical and horizontal rail wear, is computed by comparing the registered measured profile with the reference profile. The method has three key contributions: (1) the rail waist light stripe center points in the images are located fast and accurately by first tracking the region containing the rail waist light stripe using the Kalman filter and then computing the sub-pixel precision image coordinates by Hessian matrix at pixels. (2) The rail waist profile is segmented automatically into arcs of big and small circles by thresholding the normal angle curve of the measured rail waist profile. The centers of the two circles are used as control points for registering the measured rail profile to the reference profile. (3) The fast location of rail wear points in the images is realized by projecting the rail wear constraint points to the image, which simplifies the problem of computing rail wear from 2d image processing to 1d searching along the line segment connecting two rail wear constraint points. Experiments show that the proposed method can achieve 500 fps measurement frequency. At a train speed of 350 km/h, the interval between two consecutive measurements is about 190 mm. The system is tested on a real running train, and the measurement results are compared with those rail wear measured manually by special gage. The RMS errors of vertical and horizontal rail wears are 0.34 and 0.30 mm, respectively.

Calibration Method for Structured Light Vision Sensor Based on Planar Target

The calibration of structured light vision sensor is the key technique in structured light 3D vision measurement. In this paper, a novel method for structured light vision sensor calibration is presented. In our method, a simple 2D planar target is used, and the corresponding world coordinate system is set for the target at different positions as well as the transformation relationships between world coordinate system camera coordinate system and image coordinate system. The intersecting line equations at different positions are unified under the camera coordinate system after processing. Then we can use least square method to fit the structured light equation. The experimental results show that the proposed method is an efficient method with high precision and simple procedure.

A Vision-Based Self-Tuning Fuzzy Controller for Fillet Weld Seam Tracking

In this paper, an intelligent seam-tracking system is developed for fillet weld workpiece. Structured light vision is employed to detect the deviation of the welding torch. Two stripe lines are formed on the surface of the fillet weld workpiece by the structured laser light. Their intercepts are selected as the image feature, which can result in an approximately uncoupled relationship between the image feature and the movement of adjustment mechanism. A two-step feature-extraction method is designed to robustly detect the two lines in image. In order to get a favorable tracking precision in the welding process, a new self-tuning fuzzy controller is designed. Its input and output scaling factors are tuned according to its active working region. Moreover, a supervisory level is introduced into the control system to limit the output pulses in order to guarantee steady and safe tracking. Finally, experiments are well conducted to verify the performance of the proposed system and methods.

Optimization Design of the Secondary Calibration of Line Structured Light Vision System

As the traditional calibration methods are difficult to ensure the calibration accuracy and system stability, a new optimization design of the secondary calibration based on Levenberg- Marquard (LM) algorithm is proposed. It is the optimization and correction of the first calibration results. The method introduces the optimization idea of the LM algorithm. The required objective function can be obtained by capturing the image of standard block after the first system calibration. The problem is solved satisfactorily with the optimization theory of LM algorithm. Experimental results show the optimized system accuracy is less than 0.01mm, RMS error is less than 0.02mm. The proposed algorithm has significant effect on accuracy improvement, and the system is more stable.

A two-step method for spatial circle orientation with a structured light vision sensor and error analysis

A novel two-step method for spatial circle orientation with a structured light vision sensor is proposed for a 3D flexible visual inspection system guided by an industrial robot. Firstly the z coordinate of a spatial circle center is estimated, secondly the x and y coordinates are estimated with the center orientation relative to the camera optic center, and then its radius is computed. Simultaneously, the x, y and z coordinate orientation errors are analyzed in detail. It shows that the method is feasible and valid, and the orientation accuracy for the spatial circle exceeds 0.15 mm by experiment. It eliminates the bottleneck of the traditional orientation method with a stereovision sensor, and greatly expands the application of the structured light visual inspection system.

Novel calibration method for a multi-sensor visual measurement system based on structured light

We focus on two key problems in the calibration of multi-sensor visual measurement systems based on structured light: the calibration of the structured light vision sensor, and the global calibration of multiple vision sensors. In the calibration of the vision sensor, the light-plane equation is computed by combining the Plucker matrices of light stripes obtained at different target positions. Since the light-plane equation is optimized by using all the light-stripe center points, the robustness and accuracy of calibration are considerably improved. For the global calibration of multiple vision sensors, the relative positions of the two vision sensors with non-overlapping fields of view are calibrated by means of two planar targets (fixed together), using the constraint that the relative positions of the two targets are invariable. The mutual transformations between the two targets need not be known. Using one of the vision sensors as the base vision sensor, the global calibration of multiple vision sensors is achieved by calibrating each of the other vision sensors with the base vision sensor. The proposed method has already been successfully applied in practic.

A novel method for round steel measurement with a multi-line structured light vision sensor

We present a novel method for on-line measurement of round steel parameters with a multi-line structured light vision sensor. In this paper, the mathematical model of the multi-line structured light vision sensor is established, and the measurement principle and algorithms are proposed. Firstly, the 3D coordinates of cross section ellipses formed by the structured light planes and the round steel surface are extracted, and the spatial ellipse centers are fitted. Secondly, the virtual axis of round steel is constructed based on the fitted spatial centers, and a virtual plane perpendicular to the round steel virtual axis through an arbitrary point on the axis is established; then cross section ellipses are projected to the virtual plane vertically and we can get the corresponding projective circles which denote the actual circumference of the measured round steel. Finally, the diameter, roundness, straightness and other parameters of round steel can be estimated. It shows that the proposed method is valid, flexible and with high precision by experiment, and the measured relative error of the round steel diameter is less than 0.8% and repeatability exceeds 0.15 mm.

Design of Robot Welding Seam Tracking System with Structured Light Vision

Robot welding is an important developing direction of welding automation and intelligentization, and automatic seam tracking technology is one of principal research domains. Nowadays, seam tracking system with structured light vision becomes a hot research. Structured light vision seam tracking products abroad are generally very expensive and can only be applied on special occasions. In China, the research of structured light vision seam tracking system is still just on the stage of experiments. A robot real-time seam tracking system with line structured light vision is designed. The hardware system is set up, a filtering method for line structure seam image is improved, and compared with common filtering, it has better effect and characteristic of real time. Two methods, fast template matching and fast Hough transform, to recognize the image coordinates of seam center are improved. Two new image recognition methods, structure element matching and corner detecting, are proposed. The comparison of seam image recognition shows that fast template matching and corner detecting are more precise and stable than the other two methods, and corner detecting is the best in real time. A simultaneous calibration for camera parameters and robot hand-eye is also proposed, and calculation shows that the calibration is effective and feasible. The robot seam tracking tests for linear and folded lap-joint are performed, which are based on the above four image recognition methods, and the results indicate that four image recognition methods are all applicable to real-time seam tracking, and the whole system sufficed for the requirements of real-time seam tracking. Automatic seam tracking with line structured light vision is feasible and has good versatility.

Calibrating Structured-light Vision Sensor with One-dimensional Target

In order to calibrate the three-dimensional measurement model parameters of structured-light vision in situ,a novel approach to easily calibrating a structured-light vision sensor with free-moving one-dimensional target(at least 3 feature-points aligned on a feature-line and one of them chosen as the origin) is proposed.In the camera coordinate frame,the vanishing point of the feature-line can be obtained on the basis of 1D projective transformation,with at least three collinear feature-points on the feature-line,so the direction vector of the feature-line can be determined by joining the projective center and the vanishing point.The camera coordinates of the origin can be computed in accordance with the direction of the feature-line and the known distance between the origin and the other feature-point.The camera coordinates of a control point on the light-stripe can be calculated by the feature-line equation and its distance to the origin which is calculated with its projective coordinates based on 1D projective transformation.The light plane is fitted with at least three nonlinear control points in the camera coordinate frame.The experimental results prove that the proposed approach can achieve the same calibration accuracy as the planar-target-calibration approach under general laboratory conditions.Compared to planar target,1D target with high accuracy and large dimensions is manufactured more easily and maintained more simply.Moreover,the proposed technique is also important in practice especially when calibrating structured-light vision sensor with large-scale measuring range.

Measuring method for micro-diameter based on structured-light vision technology

Based on structured-light vision measurement technology, we study a measuring method for microdiameter. The measurement principle and mathematical model are described. A novel grayscale barycenter extraction algorithm along the radial direction is proposed, which can precisely gather the image coordinates of the ellipse-shaped light-stripe centers. The accuracy of the measurement result shows marked improvement by using the algorithm. The method executes circle fitting to the measured three-dimensional (3D) data using linear least square method, which can acquire the diameter, surface profile, and other information of the object effectively. On the scene, a line-structured light vision system using the presented method is applied to measure the curvature radius of metal blades. Experimental results show that the measurement precision of the system is higher than 2 \mu m.

Accurate and Robust Surface Measurement Using Optimal Structured Light Tracking Method

Structured light vision systems are based on the fact that the pixel location of light in an image has a unique association with the object depth. However, their applications are mainly limited to the properties of the object surface and the lighting conditions. This paper presents a robust vision system for accurate acquisition of 3-D surface data based on optimal structured light. To achieve depth measurement for a dynamic scene, the data acquisition must be performed with only a single image. Our special arrangement makes the image of the light stripe remaining sharp while the background becomes blurred. Moreover, a dynamic programming approach is proposed to track the optimal path while the laser beam is invisible or divergent under extreme condition. The principle and necessary mathematics for implementing the algorithm are presented. The robustness of the system against uncalibrated errors is demonstrated.

A novel 1D target-based calibration method with unknown orientation for structured light vision sensor

The problem associated with calibrating a structured light vision sensor is that it is difficult to obtain the coordinates of the world calibration points falling on the light stripe plane. In this paper, we present a novel method to address this problem by randomly moving a 1D (one-dimension) target within the sensor's view field. At each position where the target is set, the world coordinates with one calibration point on the light stripe plane that can be obtained based on at least three preset known points on the 1D target by a proposed two-stage technique. Thus, as long as the 1D target is at least set at three different positions, not less than three such calibration points can be obtained to perform the structured light vision sensor calibration. The simulation and real experiments conducted reveal that the proposed approach has an accuracy of up to 0.065 mm. The advantages of the proposed method are: (1) 1D target is easily machined with high accuracy, which reduces the cost of the calibration equipment; (2) the method simplifies the calibration operation and can be convenient in on-site calibration; (3) the method is suitable for use in confined spaces.