Structured Light Sensor Research Articles

On Exploiting Haptic Cues for Self-Supervised Learning of Depth-Based Robot Navigation Affordances

This article presents a method for online learning of robot navigation affordances from spatiotemporally correlated haptic and depth cues. The method allows the robot to incrementally learn which objects present in the environment are actually traversable. This is a critical requirement for any wheeled robot performing in natural environments, in which the inability to discern vegetation from non-traversable obstacles frequently hampers terrain progression. A wheeled robot prototype was developed in order to experimentally validate the proposed method. The robot prototype obtains haptic and depth sensory feedback from a pan-tilt telescopic antenna and from a structured light sensor, respectively. With the presented method, the robot learns a mapping between objects' descriptors, given the range data provided by the sensor, and objects' stiffness, as estimated from the interaction between the antenna and the object. Learning confidence estimation is considered in order to progressively reduce the number of required physical interactions with acquainted objects. To raise the number of meaningful interactions per object under time pressure, the several segments of the object under analysis are prioritised according to a set of morphological criteria. Field trials show the ability of the robot to progressively learn which elements of the environment are traversable.

Consistent depth video segmentation using adaptive surface models.

We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using adaptive surface models. Starting from an initial configuration, the algorithm converges to a stable segmentation through a new iterative split-and-merge procedure, which includes an adaptive mechanism for the creation and removal of segments. This allows the segmentation to adjust to changing input data along the movie, leading to stable, temporally coherent, and traceable segments. We tested the method on a large variety of data acquired with different range imaging devices, including a structured-light sensor and a time-of-flight camera, and successfully segmented the videos into surface segments. We further demonstrated the feasibility of the approach using quantitative evaluations based on ground-truth data.

超大尺度线结构光传感器现场标定技术

超大尺度线结构光传感器现场标定技术

基于共面靶标的线结构光传感器标定新方法

基于共面靶标的线结构光传感器标定新方法

Structured-light sensor using two laser stripes for 3D reconstruction without vibrations.

3D reconstruction based on laser light projection is a well-known method that generally provides accurate results. However, when this method is used for inspection in uncontrolled environments, it is greatly affected by vibrations. This paper presents a structured-light sensor based on two laser stripes that provides a 3D reconstruction without vibrations. Using more than one laser stripe provides redundant information than is used to compensate for the vibrations. This work also proposes an accurate calibration process for the sensor based on standard calibration plates. A series of experiments are performed to evaluate the proposed method using a mechanical device that simulates vibrations. Results show excellent performance, with very good accuracy.

Robust surface tracking in range image sequences

A novel robust method for surface tracking in range-image sequences is presented which combines a clustering method based on surface models with a particle-filter-based 2-D affine-motion estimator. Segmented regions obtained at previous time steps are used to create seed areas by comparing measured depth values with those obtained from surface-model fitting. The seed areas are further refined using a motion-probability region estimated by the particle-filter-based tracker through prediction of future states. This helps resolving ambiguities that arise when surfaces belonging to different objects are in physical contact with each other, for example during hand-object manipulations. Region growing allows recovering the complete segment area. The obtained segmented regions are then used to improve the predictions of the tracker for the next frame. The algorithm runs in quasi real-time and uses on-line learning, eliminating the need to have a priori knowledge about the surface being tracked. We apply the method to in-house depth videos acquired with both time-of-flight and structured-light sensors, demonstrating object tracking in real-world scenarios, and we compare the results with those of an ICP-based tracker.

Research on Calibration Method of Linear Structured Light Vision Measurement System

This template explains and demonstrates how to design a measurement system based on the size of the linear structured light vision, the system could works at realized the high precision and fast measurement of the size of mechanical parts, and accurate calibration of the system. First of all, this paper set up the experimental platform based on linear structured light vision measurement. Secondly, this paper established a system of measurement model, and puts forward a new method of calibration of structured light sensor and set up the mathematical model of sensor calibration. This calibration method only need to use some gage blocks of high precision as the target, the target position need not have a strict requirements, and the solving process will be more convenient, much easier to field use and maintenance. Finally, measuring accuracy on the system by gage blocks with high precision is verified, the experiment shows that measurement accuracy within 0.050 mmin the depth of 0-80 - mm range. This system can satisfy the demands of precision testing of most industrial parts .with its simple calibration process and high precision, it is suitable for the structured light vision calibration.

Simultaneous calibration of the intrinsic and extrinsic parameters of structured-light sensors

A novel approach for simultaneously calibrating the intrinsic and extrinsic parameters of structured-light sensors is presented in this paper. A planar target etched with grid lines is adopted to generate calibration points. By intersecting the laser plane with the grid lines on the target some co-linear points are formed, and their 3-D coordinates are computed according to the rule of cross-ratio invariance. For obtaining non-colinear points in the laser plane, the sensor is located at several different positions by the moving of coordinate measuring machine (CMM). However, the co-linear points corresponding to different CMM positions locate at different coordinate frame. In order to establish conjugate pairs to calibrate the sensor, all of the generated points in the laser plane are transformed into the CMM machine coordinate frame. Then using the 3D points in this coordinate frame a 2D coordinate frame is established with determined direction vectors, i.e., the extrinsic parameters are calibrated. By transforming the 3D points in the CMM machine coordinate frame into the 2D coordinate frame, conjugate pairs for calibrating the intrinsic parameters are established. Experimental studies show that the planar target is simple, and it can be manufactured with high accuracy. The calibrating process is facilitating, and the calibration result possesses high accuracy.

A novel laser vision sensor for weld line detection on wall-climbing robot

In this paper, we present a cross-structure light (CSL) sensor, that consists a structured light projector and a camera, for weld line detection. The structured light projector projects cross laser beams on the weldment to form cross stripes, which are captured in images by a CCD camera for measurement. We use feature points, a planar target and a homograph matrix to calibrate the sensor. We also propose an effective approach to extract laser stripes in images for weld line detection. Experiments show that the CSL sensor can capture 3D information of the weldment with very low measurement error, and the weld line detection approach is effective in wall-climbing robotic platform navigation. ● We design a cross structured light sensor to detect and measure the weld lines. ● We model and calibrate the cross structured light sensor. ● A new approach is presented to extract the laser stripe. ● The sensor system is applied on a wall climbing robot platform.

A New Method of Self-calibration of Hand-Eye Systems Based on Active Vision

This paper presents a new approach to self-calibration of the robotics hand-eye relations based on active vision systems. At the same time, the calibration of the line structured light sensor can be accomplished. According to the special designed motion of the camera, the whole calibration process can be divided into two steps, including two groups of four linearly independent translations and one pure rotation. This method requires no reference object and only needs two characteristic points in the scene. Besides, it can accomplish the self-calibration of the camera intrinsic parameters and the rotational matrix as a whole, which reduce the computation errors and cost. After verification by our experiments, the method is proved to be convenient and rapid with the precision required by the industrial fields with much fewer steps of calibration.

3D Wide FOV Scanning Measurement System Based on Multiline Structured-Light Sensors

Structured-light three-dimensional (3D) vision measurement is currently one of the most common approaches to obtain 3D surface data. However, the existing structured-light scanning measurement systems are primarily constructed on the basis of single sensor, which inevitably generates three obvious problems: limited measurement range, blind measurement area, and low scanning efficiency. To solve these problems, we developed a novel 3D wide FOV scanning measurement system which adopted two multiline structured-light sensors. Each sensor is composed of a digital CCD camera and three line-structured-light projectors. During the measurement process, the measured object is scanned by the two sensors from two different angles at a certain speed. Consequently, the measurement range is expanded and the blind measurement area is reduced. More importantly, since six light stripes are simultaneously projected on the object surface, the scanning efficiency is greatly improved. The Multiline Structured-light Sensors Scanning Measurement System (MSSS) is calibrated on site by a 2D pattern. The experimental results show that the RMS errors of the system for calibration and measurement are less than 0.092 mm and 0.168 mm, respectively, which proves that the MSSS is applicable for obtaining 3D object surface with high efficiency and accuracy.

Vision Control System of Pipe Welding Robot

The author of this article designs a non-track automatic pipe welding robot, which mainly studies the image processing system of visual welding tracking. With the requirement of various interference noise and tracking accuracy in the welding process, this study adopts structure light CCD sensor checking system and image acquisition card processing images of computer software, in which sample filtering, edge checking, contour tracking, laser centerlines selection and checking of its characteristics. This processing method has the advantages of good effect and speedy processing that is able to meet the timely requirement of tracking system.

Frequency-based underwater terrain segmentation

A method for segmenting three-dimensional data of underwater unstructured terrains is presented. The three-dimensional point clouds are converted to two-dimensional elevation maps and analyzed for segmentation in the frequency domain. The lower frequency components represent the slower varying undulations of the underlying ground. The cut-off frequency, below which the frequency components form the ground surface, is determined automatically using peak detection. The user can also specify a maximum admissible size of objects to drive the automatic detection of the cut-off frequency. The points above the estimated ground surface are clustered via standard proximity clustering to form object segments. The precision of the segmentation is compared against ground truth hand labelled data acquired by a stereo camera pair and a structured light sensor. It is also evaluated for registration error when the extracted segments are used for sub-map alignment. The proposed approach is compared to three point cloud based and two image based segmentation algorithms. The results show that the approach is applicable to a range of different terrains and is able to generate features useful for navigation.

Towards a Stable Robotic Object Manipulation Through 2D-3D Features Tracking

In this paper, a new object tracking system is proposed to improve the object manipulation capabilities of service robots. The goal is to continuously track the state of the visualized environment in order to send visual information in real time to the path planning and decision modules of the robot; that is, to adapt the movement of the robotic system according to the state variations appearing in the imaged scene. The tracking approach is based on a probabilistic collaborative tracking framework developed around a 2D patch-based tracking system and a 2D-3D point features tracker. The real-time visual information is composed of RGB-D data streams acquired from state-of-the-art structured light sensors. For performance evaluation, the accuracy of the developed tracker is compared to a traditional marker-based tracking system which delivers 3D information with respect to the position of the marker.

Three-Dimensional Scanning System with Double Structured-Light Sensors

Three-dimensional (3D) vision scanning measurement is widely used in industry for its ability to obtain the 3D surface data of the object. Aiming at overcoming the shortcomings of 3D scanning measurement system with single structured-light sensor, such as limited measurement range and blind measurement area, a scanning system based on double structured-light sensors (DSS) is established. The object is scanned from two different directions, and the 3D surface coordinates are unified to the measurement coordinate system to obtain the 3D surface of the measured object. In this paper, the mathematical model of the DSS scanning system is established. Meanwhile, an on-site calibration approach based on planar target is proposed to complete the system calibration task. Finally, experimental results of practical data are given to show the feasibility and validity of the proposed system.

A novel laser vision sensor for omnidirectional 3D measurement

The combination of omnidirectional imaging and structured-light vision technique provides an efficient solution for omnidirectional 3D measurement. However, the existing omnidirectional structured-light vision sensors are constructed on the basis of curved mirrors, such as parabolic mirrors and conic mirrors, which inevitably generate two obvious problems: first, the difficulty of camera calibration is undoubtedly increased because the camera mathematical model becomes nonlinear and the traditional camera model is no longer applicable in those cases and second, the measurement accuracies of those sensors are far worse than those of traditional structured-light sensors since the captured images are severely warp-deformed. To solve these problems, we developed a novel omnidirectional laser vision sensor by adopting plane mirrors instead of curved mirrors so that the simple traditional camera model can still be used, and more important, the measurement accuracy can be maintained at the same level with the traditional structured-light vision. With respect to the sensor designing, the omnidirectional camera is constructed by a traditional perspective camera and a pyramid reflector, which is made up of four identical plane mirrors of isosceles triangular shape. Moreover, four laser projectors are used to project structured-light to the surrounding scene by the way of splicing. The sensor is calibrated by a 2D planar target and a quasi-3D target. The experimental results show that the relative calibration and measurement errors of the sensor are less than 0.08% and 0.16% respectively. It is proved that complete omnidirectional 3D measurement with high accuracy is feasible with the proposed sensor system.

Terrain surface classification with a control mode update rule using a 2D laser stripe-based structured light sensor

It is necessary for autonomous ground vehicles operating on outdoor terrains to identify and adapt to different terrains in order to improve their mobility and safety. This work presents a classification scheme to identify outdoor terrains and an update rule to reduce the possibility of implementing control modes based on classification inaccuracies. A laser stripe-based structured light sensor, which has a laser and infrared camera component, is used to sense terrains directly in front of the vehicle ( < 1 m ). Use of this infrared vision system allows sensing at night, without external lighting, unlike many previous vision-based approaches that rely on stand-alone cameras. Also, unlike many previous results, the classification algorithm presented here does not rely on measures of color, which are subject to illumination and weather conditions. Instead, the method presented here relies on spatial relationships which are captured in two quantities: spatial frequency from range data and texture from camera data. The presented terrain classification scheme uses a probabilistic neural network classifier to exploit the spatial differences in four terrains: asphalt, grass, gravel and sand. This approach yields empirical results that report a greater than 97% classification accuracy when both spatial frequency and texture features are used. Color robustness and lighting robustness is then shown through additional experiments. Furthermore, instead of implementing control modes directly from the identified terrains, it is shown that the use of current and past terrain detections allows for the rejection of misclassifications with minimal effect on the rate at which a new control mode can be implemented.

Accurate extrinsic calibration method of a line structured-light sensor based on a standard ball

In order to implement 3D measurement in different measurement systems with line structured-light sensor, the extrinsic matrix for transforming the position between the structured-light sensor and the measurement platform must be calibrated. An extrinsic calibration method, achieved by measuring the fixed point (the centre of a standard ball) of the measuring system with the sensor in different positions, was proposed. In order to calculate the coordinate of the ball centre in the laser plane frame more accurately, the images containing fewer pixels were filtered and the noise around the slice was removed. Experimental study was carried out on a four-axis coordinate measuring machine (CMM) with a ball supplied by CIMCORE and showed that the method is simple; it can also be used in a non-orthogonal measurement system.

Calibration of Line Structured Light Sensor for Robotic Inspection System

With the development of the green manufacture, a robotic inspection system is presented to meet the requirement of remanufacturing engineering. It solves the problem of acquiring the information of the three dimensions (3D) shape feature in a timely and effective manner. As to set up this system efficiently, a calibration approach of the main part of the system, the line structured light sensor, based on a six degree of freedom (6-DOF) robot is proposed. This approach takes the advantage of the flexibility of the robot for simplifying the traditional sensor calibration approach and generates the calibration control points with concentric circle to refine the precision of locating its center. The actual experiment demonstrates that this approach is suitable for field calibration with conveniences and good accuracy.

이동 로봇의 강인 위치 추정을 위한 단안 비젼 센서와 레이저 구조광 센서의 베이시안 센서융합

This paper describes a procedure of the map-based localization for mobile robots by using a sensor fusion technique in structured environments. A combination of various sensors with different characteristics and limited sensibility has advantages in view of complementariness and cooperation to obtain better information on the environment. In this paper, for robust self-localization of a mobile robot with a monocular camera and a laser structured light sensor, environment information acquired from two sensors is combined and fused by a Bayesian sensor fusion technique based on the probabilistic reliability function of each sensor predefined through experiments. For the self-localization using the monocular vision, the robot utilizes image features consisting of vertical edge lines from input camera images, and they are used as natural landmark points in self-localization process. However, in case of using the laser structured light sensor, it utilizes geometrical features composed of corners and planes as natural landmark shapes during this process, which are extracted from range data at a constant height from the navigation floor. Although only each feature group of them is sometimes useful to localize mobile robots, all features from the two sensors are simultaneously used and fused in term of information for reliable localization under various environment conditions. To verify the advantage of using multi-sensor fusion, a series of experiments are performed, and experimental results are discussed in detail.