Laser Plane Research Articles

3D surface reconstruction of transparent objects using laser scanning with a four-layers refinement process.

Acquiring the 3D geometry of objects has been an active research topic, wherein the reconstruction of transparent objects poses a great challenge. In this paper, we present a fully automatic approach for reconstructing the exterior surface of a complex transparent scene. Through scanning a line laser by a galvo-mirror, images of the scene are captured from two viewing directions. Due to the light transmission inside the transparent object, the captured feature points and the calibrated laser plane can produce large number of 3D point candidates with large incorrect points through direct triangulation. Various situations of laser transmission inside the transparent object are analyzed and the reconstructed 3D laser point candidates are classified into two types: first-reflection points and non-first-reflection points. The first-reflection points means the first reflected laser points on the front surface of measured objects. Then, a novel four-layers refinement process is proposed to extract the first-reflection points step by step from the 3D point candidates through optical geometric constraints, including (1) Layer-1 : fake points removed by single camera, (2) Layer-2 : ambiguity points removed by the dual-camera joint constraint, (3) Layer-3 : retrieve the missing first-reflection exterior surface points by fusion and (4) Layer-4 : severe ambiguity points removed by contour-continuity. Besides, a novel calibration model about this imaging system is proposed for 3D point candidates reconstruction through triangulation. Compared with traditional laser scanning method, we pulled in the viewing angle information of the second camera and a novel four-layers refinement process is adopted for reconstruction of transparent objects. Various experiments on real objects demonstrate that proposed method can successfully extract the first-reflection points from the candidates and recover the complex shapes of transparent and semitransparent objects.

Robot line structured light vision measurement system: light strip center extraction and system calibration

We investigate the light strip center extraction and system calibration method of a robot line structured light vision measurement system to realize fast on-line measurement of complex objects. First, the model of a robot vision measurement system is established to analyze the key technologies involved for improving the measurement accuracy of the system. Second, on the basis of the traditional gray gravity method, a gray gravity method based on the normal direction is proposed to extract the center of the laser stripe. Considering the influence of camera distortion, the accuracy of extracting the center stripe can be further improved. Third, a two-dimensional checkerboard is applied to synchronously calibrate the camera parameters, laser plane parameters, and hand-eye relationship of the robot, thus improving the efficiency and accuracy of system calibration. Finally, three-dimensional topography and the geometric dimensions of a complex part are measured by the constructed system with high accuracy. The experimental results show that the average measurement error is <0.2 mm and the measurement accuracy is <1.3 % , indicating that the system can be effectively used in actual industrial measurement.

Multisensor data processing in dimensional metrology for collaborative measurement of a laser plane sensor combined to a touch probe

Dimensional inspection, which consists in verifying the geometric conformity of parts in terms of ISO specifications, is an essential step in the product life cycle. In recent years, new optical measurement technologies, such as laser sensors or structured light sensors, have become increasingly important. However, the quality of the data limits their expansive use for metrology applications and touch probes with CMM (Coordinate Measurement Machine) remain the reference in terms of measurement quality. On the other hand, the cooperative use of an optical system and a contact sensor presents a good solution which takes advantage of the characteristics of both systems while maintaining a quality level satisfying the requirements of metrology. However, measurement with a laser sensor and a touch probe mounted on a CMM results in two heterogeneous point clouds that must be processed before the final evaluation of specifications. This paper presents a methodology that addresses the processing of measurement data in this scenario. The proposed approach consists of four phases. In the first phase, the measurement data resulting from the laser sensor is processed to remove outliers, reduce noise and partition the point cloud. In the second phase, the point cloud resulting from both sensors are unified in a unique coordinate system. In the third phase, a fusion method is proposed to fuse the point clouds. The specifications are finally evaluated based on the fused point clouds in the fourth phase. A case study is conducted to illustrate the proposed approach with the measurement data of a test part defined by its CAD model and specifications. The results are compared with a reference inspection report obtained by a touch probe measurement system.

STRICT GEOMETRIC CALIBRATION OF AN UNDERWATER LASER TRIANGULATION SYSTEM

Abstract. This paper will describe a novel approach for the calibration of an underwater laser triangulation system. Underwater triangulation systems, consisting of a line laser and a camera can be used to determine the geometry of submerged objects or the topography of a water body bottom. Placing camera and laser line projector inside a waterproof housing leads to refraction effects at the air-glass-water interfaces, both of the laser light-sheet and image rays. This implies a deformed laser plane in the water and a curved line on the object surface. The proposed approach strictly models the geometry between camera, laser and housing. First experiments show, that the calibration method can be applied for water depth measurements with accuracies of 0.2–0.3 mm at depths in the order of 100 mm.

Motion of classical charged particles with magnetic moment in external plane-wave electromagnetic fields

We study the motion of a charged particle with magnetic moment in external electromagnetic fields utilizing covariant unification of Gilbertian and Amperian descriptions of particle magnetic dipole moment. Considering the case of a current loop, our approach is verified by comparing classical dynamics with the classical limit of relativistic quantum dynamics. We obtain motion of a charged particle in the presence of an external linearly polarized EM (laser) plane wave field incorporating the effect of spin dynamics. For specific laser-particle initial configurations, we determine that the Stern-Gerlach force can have a cumulative effect on the trajectory of charged particles.

A universal method for the calibration of swing-scanning line structured light measurement system

As a non-contact, low cost and efficient method for three dimensional (3D) measurements, the swing-scanning line structured light measurement system has a more compact structure than the linear scanning system. In measuring process, the laser line projector is driven by a step motor with the laser plane intersecting the object sequentially. To obtain the intersection profiles, the corresponding laser plane equations need to be real-time calculated. Moreover, the laser plane can be an arbitrary relative position with the swing axis due to the mount error of laser projector. These would make the calibration model and the solution rather difficult. To address these problems, a universal calibration method is proposed which bridges the swing angle and the coefficients of laser plane with a constant mapping matrix directly. This matrix is determined by use of several (three at least) specific laser planes. The whole calibration process can be achieved by one freely moved checkerboard and is very convenient to use. Several experiments are carried out to validate the proposed method including the measurement of the planar surface, the steps and the sculptured surface.

Development of 3D environmental laser scanner using pinhole projection

Three-dimensional (3D) information of capturing and reconstructing an object existing in its environment is a big challenge. In this work, we discuss the 3D laser scanning techniques, which can obtain a high density of data points by an accurate and fast method. This work considers the previous developments in this area to propose a developed cost-effective system based on pinhole projection concept and commercial hardware components taking into account the current achieved accuracy. A laser line auto-scanning system was designed to perform close-range 3D reconstructions for home/office objects with high accuracy and resolution. The system changes the laser plane direction with a microcontroller to perform automatic scanning and obtain continuous laser strips for objects’ 3D reconstruction. The system parameters were calibrated with Matlab’s built-in camera calibration toolbox to find camera focal length and optical center constraints. The pinhole projection equation was defined to optimize the prototype rotating axis equation. The developed 3D environmental laser scanner with pinhole projection proved the system’s effectiveness on close-range stationary objects with high resolution and accuracy with a measurement error in the range (0.05–0.25) mm. The 3D point cloud processing of the Matlab computer vision toolbox has been employed to show the 3D object reconstruction and to perform the camera calibration, which improves efficiency and highly simplifies the calibration method. The calibration error is the main error source in the measurements, and the errors of the actual measurement are found to be influenced by several environmental parameters. The presented platform can be equipped with a system of lower power consumption, and compact smaller size

21.2: 3D Laser AI‐vision in Smart Welding System

In terms of modern industrial welding, the teaching programming mode currently is still the major method applied with the welding robots, whereas as a matter of fact the operation of this method is comparatively time‐consuming, easily affected by human uncertainty and thus, requires experienced workers. In an effort to develop the autonomous and adaptive abilities of the welding robots, enhancing their flexibility as well as accomplishing more efficient industrial welding, this paper proposes an intelligent robot welding system based on laser vision and machine learning, applying a self‐designed laser scanning sensor. The crucial procedures of system calibration are illustrated step by step, involving camera calibration, laser plane calibration, hand‐eye calibration and etc. AI tracing method is implemented via computer vision and machine learning to substitute for teaching programming method to actively lead the laser scanning sensor of the welding robots to trace a welding seam for gathering the spatial coordinates of seam points. The method is tested on the complex intersection welding joint, demonstrating its reliability and superiority. Commonly, during the industrial welding process, some environmental factors, e.g. overheating, can cause some deformation of the welding seam. In order to ensure that welding torch is controlled to still follow and perform welding on the correct trajectory, smart welding seam tracking is proposed to tackle this welding issue. In addition, seam pattern mismatching detector based on Learnable Template is proposed to be used with Kernelized Correlation Filter (KCF) for accurately tracking the positive seam pattern as well as detecting the signal of mismatching to make KCF stop tracking the negative seam pattern. The corresponding experimental results have demonstrated the effectiveness, dependability and efficiency of the intelligent robot welding system.

A Multi Camera and Multi Laser Calibration Method for 3D Reconstruction of Revolution Parts.

This paper describes a method for calibrating multi camera and multi laser 3D triangulation systems, particularly for those using Scheimpflug adapters. Under this configuration, the focus plane of the camera is located at the laser plane, making it difficult to use traditional calibration methods, such as chessboard pattern-based strategies. Our method uses a conical calibration object whose intersections with the laser planes generate stepped line patterns that can be used to calculate the camera-laser homographies. The calibration object has been designed to calibrate scanners for revolving surfaces, but it can be easily extended to linear setups. The experiments carried out show that the proposed system has a precision of 0.1 mm.

A Camera Intrinsic Matrix-Free Calibration Method for Laser Triangulation Sensor.

Laser triangulation sensors (LTS) are widely used to acquire depth information in industrial applications. However, the parameters of the components, e.g., the camera, of the off-the-shelf LTS are typically unknown. This makes it difficult to recalibrate the degenerated LTS devices during regular maintenance operations. In this paper, a novel one-dimensional target-based camera intrinsic matrix-free LTS calibration method is proposed. In contrast to conventional methods that calibrate the LTS based on the precise camera intrinsic matrix, we formulate the LTS calibration as an optimization problem taking all parameters of the LTS into account, simultaneously. In this way, many pairs of the camera intrinsic matrix and the equation of the laser plane can be solved and different pairs of parameters are equivalent for displacement measurement. A closed-form solution of the position of the one-dimensional target is proposed to make the parameters of the LTS optimizable. The results of simulations and experiments show that the proposed method can calibrate the LTS without knowing the camera intrinsic matrix. In addition, the proposed approach significantly improves the displacement measurement precision of the LTS after calibration. In conclusion, the proposed method proved that the precise camera intrinsic matrix is not the necessary condition for LTS displacement measurement.

An automatic calibration algorithm for laser vision sensor in robotic autonomous welding system

Visual sensor plays an important part in intelligentized welding systems, and the calibration of the vision sensor is the indispensable part of visual systems. Aiming at the problem of the tedious calibration process, this paper describes an automatic calibration algorithm. First, the robot motion equation and the motion range constraint equation are proposed to ensure that the collected images of calibration grid and laser line meet the calibration requirements. Based on these two equations, the automatic collection procedure can be realized. Second, the simplified visual servoing method and the Extended Kalman filter were used to adjust the images and rectify system parameters, respectively, which will improve the stability of the calibration motion. Third, to reduce the impact of the complex welding environments, a robustness feature extraction algorithm based on local threshold is studied. And then, the laser plane and hand-eye matrix are fitted with optimization algorithms to ensure calibration accuracy. Finally, the simulation experiments prove the feasibility and stability of the proposed algorithm. And the actual calibration tests suggest that the algorithm can significantly improve calibration efficiency. Moreover, the experimental results of welding guidance and seam tracking confirm that the calibration precision has met the welding requirement.

Design and Analysis of High-Accuracy Telecentric Surface Reconstruction System Based on Line Laser

Non-contact measurement technology based on triangulation with cameras is extensively applied to the development of computer vision. However, the accuracy of the technology is generally not satisfactory enough. The application of telecentric lenses can significantly improve the accuracy, but the view of telecentric lenses is limited due to their structure. To address these challenges, a telecentric surface reconstruction system is designed for surface detection, which consists of a single camera with a telecentric lens, line laser generator and one-dimensional displacement platform. The designed system can reconstruct the surface with high accuracy. The measured region is expanded with the used of the displacement platform. To achieve high-accuracy surface reconstruction, we propose a method based on a checkerboard to calibrate the designed system, including line laser plane and motor direction of the displacement platform. Based on the calibrated system, the object under the line laser is measured, and the results of lines are assembled to make the final surface reconstruction. The results show that the designed system can reconstruct a region of 20×40 mm2, up to the accuracy of micron order.

Data-Driven Laser Plane Optimization for High-Precision Numerical Calibration of Line Structured Light Sensors

Line structured light sensor has been applied in various three dimensional (3D) measurement scenes with the advantages of non-contact, low cost and high speed. Its accuracy, which is directly determined by the calibration method, needs to be further improved to fulfill the measuring tasks of precision parts. Here, we proposed a numerical method that can eliminate the errors of the model based methods through two strategies. One is the establishment of the numerical mapping relationship between stripe pixel coordinates and their world coordinates through piecewise cubic interpolation. Corner points of a checkerboard target are used to obtain sufficient interpolating nodes. This target can be manually aligned with the laser plane and the alignment error would be eliminated via the point projection. The other is the data-driven laser plane optimization. The data set of reference interval distance is computed based on the invariance of cross ratio. The optimization model is to minimize the root mean squared error of measured interval distances by adjusting the laser plane coefficients. After the optimization, a higher numerical mapping relationship can be achieved. It bypasses the camera and the distortion models and reaches a calibration error of only 0.005mm. The comparison studies and the measurement of the steps further validate the proposed method.

A Sparse Branch and Bound Optimization of Noisy Weighted DAG Modification Under Constraints: A Method for Monocular Data Association to Multiple Laser Planes

The line segments of multiple (and nearly parallel) planar laser sources appearing in monocular video are objectively associated to their respective laser sources. Such association is a precursor to 3D surface digitization where each laser plane develops a 3D surface. In this work, the detected laser line segments are modeled as a weighted directed acyclic graph (wDAG), in which the graph nodes represent the line segments and their maximum path hopes from any source node represents the laser source label. However, due to noise and occlusions, the structure of the wDAG will normally vary from that expected in the ideal cases. Furthermore, it is essential to robustness to incorporate the association information of the previous frames in the new frame associations. The proposed work objectively modifies the current frame’s wDAG under these constraints so the resulting wDAG becomes in an appropriate structure for the labeling of the detected line segments. Branch and bound (BnB) optimization has been a good choice for global optimization of combinatorial problems. Rather than approaching the optimization as a pure (or full) combinatorial problem, the proposed prioritizations of the branch modification actions are nearly optimal. The high priority actions will very likely lead to an optimal modification. This aspect is exploited by the proposed sparse variant of BnB (SBnB) optimization. It significantly reduces the number of the branches to be considered, in addition to those contributed by the bounding part of the BnB optimization. The proposed approach was tested on numerous monocular videos and plausible association results were efficiently achieved.

Extrinsic Visual–Inertial Calibration for Motion Distortion Correction of Underwater 3D Scans

Underwater 3D laser scanners are an essential type of sensor used by unmanned underwater vehicles (UUVs) for operations such as navigation, inspection, and object recognition and manipulation. Scanners that acquire 3D data by sweeping a laser plane across the scene can provide very high lateral resolution. However, their data may suffer from rolling shutter effect if the change of pose of the robot with respect to the scanned target during the sweep is not negligible. In order to compensate for motion-related distortions without the need for point cloud postprocessing, the 6-DoF pose at which the scanner acquires each line needs to be accurately known. In the underwater domain, autonomous vehicles are often equipped with a high-end inertial navigation system (INS) that provides reliable navigation data. Nonetheless, the relative pose of the 3D scanner with respect to the inertial reference frame of the robot is not usually known a priori. Therefore, this paper uses an ego-motion-based calibration algorithm to calibrate the extrinsic parameters of the visual-inertial sensor pair. Simulations are performed to quantify how miscalibration affects motion-related distortion. The method is also evaluated experimentally in laboratory conditions.

A Vision Measurement Method for the Gear Shaft Radial Runout With Line Structured Light

The gear shaft needs to measure its radial runout for straightening before finishing, while it is difficult to measure the gear shaft radial runout without rotating the gear shaft by the existing measurement technology, which results the measurement inefficient. Present work puts forward a method of measuring a gear shaft radial runout with line structured light vision. This method can measure the radial runout value and direction online for straightening the bending of the gear shaft. The center position of the gear shaft cross section is obtained by the ellipse geometric fitting on the laser plane to eliminate the influence of projection distortion. The measurement method is tested by a specialized experiment, and the experimental results show that measurements of the method meet the requirements of straightening the gear shaft. The method can realize online measurement of the gear shaft radial runout just by a single image without rotating the gear shaft, so it features fast measurement compared with the existing measurement technology.

Optical Dual Laser Based Sensor Denoising for OnlineMetal Sheet Flatness Measurement Using Hermite Interpolation.

Flatness sensors are required for quality control of metal sheets obtained from steel coils by roller leveling and cutting systems. This article presents an innovative system for real-time robust surface estimation of flattened metal sheets composed of two line lasers and a conventional 2D camera. Laser plane triangulation is used for surface height retrieval along virtual surface fibers. The dual laser allows instantaneous robust and quick estimation of the fiber height derivatives. Hermite cubic interpolation along the fibers allows real-time surface estimation and high frequency noise removal. Noise sources are the vibrations induced in the sheet by its movements during the process and some mechanical events, such as cutting into separate pieces. The system is validated on synthetic surfaces that simulate the most critical noise sources and on real data obtained from the installation of the sensor in an actual steel mill. In the comparison with conventional filtering methods, we achieve at least a 41% of improvement in the accuracy of the surface reconstruction.

A Calibration Method for Mobile Omnidirectional Vision Based on Structured Light

Mobile omnidirectional structured light vision is increasingly used in scene perception and robot navigation. A wide range of information is obtained by means of the vision system by only one image and laser image features are detected and extracted easily and quickly. In this paper a novel calibration method for mobile omnidirectional camera based on structured light is presented. Firstly, a set of parallel laser planes is emitted on the walls of corridor as auxiliary targets by structured light and intersects with wall orthogonally. Secondly, the constraint relationship is analyzed between the vanishing points in fisheye images and intrinsic parameters of imaging model. Finally, effects of the laser stripes' interval and the angle between the wall which contains laser stripes and ground on calibration results are evaluated. Compared to Scaramuzza method, the calibration method shows its superiority in terms of both feasibility and efficiency. The method with the characteristic of self-calibration since the planar target is replaced by actively projected laser stripes. The result illustrates that our method has the advantages of simple and feasible operation, but result is effective and accurate. The calibration parameters are independent of the laser stripes' interval and the angle between the wall and ground. Therefore, the method of the mobile omnidirectional structured light vision presented in this paper can be applied to many areas.

Vision-based reconstruction of laser projection with invariant composed of points and circle on 2D reference

A vision-based reconstruction method is conducted by the point-circle invariant and the planar laser. The planar laser is coplanar to the two-dimensional (2D) reference. The combination of a circle on the 2D reference, a point on the 2D reference and a random point on the laser stripe is considered as the invariant, which is impervious to the projection from the laser plane to the image. Therefore, the reconstruction model is achieved by the invariant, which is generated from the projections of the above geometric features. The experiments are performed to verify the performance and reconstruction error of the method. The minimum error is 0.473 mm for the camera-reference distance of 600 mm, the scaleplate-projector distance of 100 mm and the reference length of 30 mm. Besides, the maximum error is 4.960 mm, for the camera-reference distance of 900 mm, the scaleplate-projector distance of 400 mm and the reference length of 120 mm. The reconstruction error means of 0.891 mm, 1.365 mm, 2.578 mm and 3.767 mm are observed with respect to the test conditions of 600–900 mm, which proves the applicability of the reconstruction method with the point-circle invariant.

Omnidirectional Vision System With Laser Illumination in a Flexible Configuration and Its Calibration by One Single Snapshot

An omnidirectional vision system consisting of a fisheye camera and structured light projector has its own distinctive advantages toward other vision systems. A vision system provides a wide range of information by one single capture. Moreover, the laser feature is easy to detect and extract from the given image for further analysis. In order to obtain the reliable results, the vision system must be calibrated. There have been improvements toward calibration techniques for vision systems with laser illumination. However, these techniques are not suitable for an omnidirectional vision system in a particular configuration. Motivated by the abovementioned problem, this article presents a novel omnidirectional vision system with laser illumination in a flexible configuration and a method for its extrinsic calibration. Flexible configuration means that both camera and laser plane are no longer parallel to the floor. The calibrated vision system is evaluated experimentally by using simulated and real data to demonstrate that the proposed calibration method is more robust and provides a higher measurement accuracy.