Laser Line Projector Research Articles

Development of a vision system integrated with industrial robots for online weld seam tracking

This paper develops a laser-structured light vision system for the industrial welding manipulator. The designed structured light vision system consisting of a line laser projector and an industrial camera is a decent 3D object data acquisition platform. By applying the principle of triangulation, the depth value of the object can be obtained via the laser plane in the captured image. According to hand-eye calibration and the principle of triangulation, the 3D position of the objects in the robot coordinate system is calculated. Based on the discriminated convolution tracker and deep reinforcement learning that increases the accuracy and processing time of the tracker, allowing simultaneous scanning for seam data acquisition and welding along the weld path, a real-time seam tracking algorithm is developed to handle the square-groove butt joint with small gaps (i.e., <1 mm). Experiments are undertaken using a Yaskawa industrial welding robot integrating the designed laser-structured light vision system. Experiment results show the significant performance of the developed system in welding square-groove butt joints with curved profiles and small gaps, i.e., the welding absolute mean error is 0.31 mm, whereas in the case of using convolution tracker only, the welding absolute mean error 1.7 mm.

Calibration method of a three-dimensional scanner based on a line laser projector and a camera with 1-axis rotating mechanism

Calibration method of a three-dimensional scanner based on a line laser projector and a camera with 1-axis rotating mechanism

Fabrication of Graphene Optical Devices for Endoscope Calibration

Endoscopic devices used in the clinical examination of pathologies such as glottic tumors can record real-time images of the vocal cords during vibration. Quantitative analysis of the glottic surface requires the metric system to obtain the absolute dimensions of the larynx in a series of acquired images. However, so far, there is no effectively automated and stable calibration method for clinical use. Therefore, most metrics used by professionals are invasive and subjective. Thus, this study reports the microfabrication of a diffractive optical element (DOE) that could potentially quantify cancer cells from the glottic surface of patients. It has been observed that the image of the glottis seen through an endoscope is shaped by its geometric properties and by the shading of color and brightness. Therefore, tumor location and size can be quantified by overlaying the DOE metric with light spots on images that include the glottic region. DOE was developed from processes used in the fabrication of integrated circuits and MEMS, already quite mature. To obtain high-quality images from optical reconstruction of the glottis, the use of graphene monolayers grown by chemical vapor deposition (CVD) on copper (Cu) foil was investigated. This was transferred to a glass substrate by wet method (Fishing method) and the etching steps of the Cu substrate were carried out. In this method, polymethylmethacrylate (PMMA) was used as the supporting polymer through the spinner. Two lithographs were made under the glass substrate with different approaches (using a Karl Suss MJB3 mask aligner system with a maximum resolution of 1 µm). The first, which preceded the thin-film transfer, aimed to delineate the graphene transfer area using an aluminum (Al) deposition technique by sputtering. The second lithography, linked to oxygen Ashing plasma corrosion, made it possible to study the modulation of a microrelief containing parallel and serpentine patterns, of amplitude and phase, spaced from 1 to 10 μm. An optical system containing an EOD and a Fourier lens was attached to the endoscope end to project a laser line parallel to the vocal cords. Image processing methods automatically identify pixels in the image data that belong to the projected laser line. Thus, the detection of the diffractive pattern was based on segmentation of endoscopic images centered on Sobel filtering and on the method with active boundary regions of the watershed. These use successively robust temporal and spatial characteristics of the laser lines projected onto the endoscopic image sequence. In short, the combination of laser projection device and image processing allows calibration of endoscopic images and can provide quantitative metric data for tumors on the surface of the vocal cords. Therefore, the proposed calibration method can be considered a powerful tool to aid in the prognosis of laryngeal pathologies. Figure 1

Micro-Scale Surface Recognition via Microscope System Based on Hu Moments Pattern and Micro Laser Line Projection

The surface engineering of metals develops high technology to detect microscale convex, concave and flat surface patterns. It is because the manufacturing industry requires technologies to recognize microscale surface features. Thus, it is necessary to develop microscopic vision technology to recognize microscale concave, convex and flat surfaces. This study addresses microscale concave, convex and flat surface recognition via Hu moments’ patterns based on micro-laser line contouring. In this recognition, a Hu moments’ pattern is generated from a Bezier model to characterize the surface recovered through microscopic scanning. The Bezier model is accomplished by employing a genetic algorithm and surface coordinates. Thus, the flat, convex and concave surfaces are recognized based on the Hu moments’ pattern of each one. The microscope system projects a 40 μm laser line on the object and a camera acquires the object’s contour reflection to retrieve topographic coordinates. The proposed technique enhances the microscale convex, concave, flat, and surface recognition, which is performed via optical microscope systems. The contribution of microscopic shape recognition based on the Hu moments’ pattern and microscopic laser line is elucidated by a discussion based on the microscopic shape recognition performed through the optical microscopic image processing.

SINGLE IMAGE LASER TRIANGULATION SYSTEM FOR 360° EXTERNAL INSPECTION OF CYLINDRICAL SHAPED OBJECTS USING INVERSE TRIANGULATION ALGORITHM

Abstract. This paper proposes a new Laser Triangulation System (LTS) that allows an 360° external inspection for each acquired image using two planar mirrors, one laser line projector, and one camera. The new measurement algorithm is based on the inverse triangulation in which the analysis is made from 3D object space to image space, the opposite of traditional LTS algorithms, with laser peak detection in favourable directions even in an image with multiple reflections of the object. Measurement system principle, calibration and metrological evaluation are described based on simulated experimental data and 3D reconstructions examples. Uncertainty for a 20 mm plain plug gauge measurement was ±0.07 mm for 95.45% confidence.

Polarization Image Laser Line Extraction Methods for Reflective Metal Surfaces

In this work, we propose a novel pipeline method for laser line extraction from images with a polarization image sensor. The proposed method is specifically developed for strong laser beam reflections from metal surfaces. For the preprocessing stage, we propose a demosaicing algorithm for color polarizer filter array (CPFA) sensors. This can be implemented by using either one-quarter or full resolution of the sensor. In addition, we propose two methods for optimizing the information available in a 12-channel color polarization image. The first method is based on the minimum linearly polarized irradiance, and the second method is based on the linear polarization intensity. These preprocessing and optimization methods are combined with laser line extraction methods. The laser line extraction is done with either the polarized finite impulse response (FIR) center of gravity (COG), where the laser line coordinates are computed from the filtered laser intensity distribution, or with the polarized FIR-Peak, where the laser line coordinates are calculated from the first derivative of the filtered laser signal. The performance of the proposed algorithms is studied experimentally using a laser line scanner assembly, made of a polarization camera, and a laser line projector operating in the blue wavelength range.

Multi-Objective Optimization via GA Based on Micro Laser Line Scanning Data for Micro-Scale Surface Modeling

Industry 4.0 represents high-level methodologies to make intelligent, autonomous, and self-adaptable manufacturing systems. Additionally, the surface modeling technology has become a great tool in industry 4.0 for representing the surface point cloud. Thus, the micro-scale machining technology requires efficient models to represent micro-scale flat and free-form surfaces. Therefore, it is fundamental to perform surface modeling through artificial intelligence for representing small surfaces. This study addressed multi-objective optimization via genetic algorithms and micro laser line projection to accomplish surface models for representing micro-scale flat and free-form surfaces, where an optical microscope system retrieves micro-scale topography via micro laser line coordinates and the multi-objective optimization constructs the flat and free-form surface models through genetic algorithms and micro-scale topography. The multi-objective optimization determines the surface model parameters through exploration and exploitation, and the solution space is deduced via surface data. The surface model generated through the multi-objective optimization fit accurately to the micro-scale target surface. Thus, the proposed technique enhanced the fitting of micro-scale flat and free-form surface models, which were deduced via gray-level images of an optical microscope. This enhancement was validated by a discussion between the multi-objective optimization via genetic algorithms and the micro-scale surface modeling via optical microscope imaging systems.

Micro-Scale Spherical and Cylindrical Surface Modeling via Metaheuristic Algorithms and Micro Laser Line Projection

With the increasing micro-scale manufacturing industry, the micro-scale spherical and cylindrical surface modeling has become an important factor in the manufacturing process. Thus, the micro-scale manufacturing processes require efficient micro-scale spherical and cylindrical models to achieve accurate assembly. Therefore, it is necessary to implement models to represent micro-scale spherical and cylindrical surfaces. This study addresses metaheuristic algorithms based on micro laser line projection to perform micro-scale spherical and cylindrical surface modeling. In this technique, the micro-scale surface is recovered by an optical microscope system, which computes the surface coordinates via micro laser line projection. From the surface coordinates, a genetic algorithm determines the parameters of the mathematical models to represent the spherical and cylindrical surfaces. The genetic algorithm performs exploration and exploitation in the search space to optimize the models’ mathematical parameters. The search space is constructed via surface data to provide the optimal parameters, which determine the spherical and cylindrical surface models. The proposed technique improves the fitting accuracy of the micro-scale spherical and cylindrical surface modeling performed via optical microscope systems. This contribution is elucidated by a discussion about the model fitting between the genetic algorithms based on micro laser line projection and the optical microscope systems.

Accuracy Analysis of an Oblique Underwater Laser Lightsheet Triangulation System

Laser lightsheet triangulation is a well-established optical measurement method, which is frequently used in industrial applications. With some adaptions, the technique can also be used underwater. Placing a green laser line projector and a camera at a fixed base inside a watertight housing enables flexible and accurate underwater measurements at low cost. To achieve near-orthogonal intersections on the object surface, camera and laser need to be placed oblique to the housing interface. Refraction influences have to be considered strictly in geometric modelling, with the light propagating through the air–glass and glass–water interfaces. The measurement and calibration methods presented in the paper utilize a concept of splitting the lightsheet into multiple sub-beams, which are traced, refracted, and intersected with image observations to receive 3D measurements. In a calibration step, the relative orientation between camera, laser and interfaces is determined. In a theoretical accuracy analysis, the error influences caused by the calibrated parameters is estimated with a standard deviation of 0.2 to 0.4 mm in a depth range of up to 15 cm. A prototype triangulation sensor is applied for practical measurements of two test objects with known geometry. The predicted accuracy is validated in stationary single scan measurements. The sensor is furthermore moved along the test objects, using a six-degrees-of-freedom method to determine its exterior orientation. The combination of multiple single scans results in dense point clouds with 0.3 mm standard deviation compared to a reference.

STRICT GEOMETRIC CALIBRATION OF AN UNDERWATER LASER TRIANGULATION SYSTEM

Abstract. This paper will describe a novel approach for the calibration of an underwater laser triangulation system. Underwater triangulation systems, consisting of a line laser and a camera can be used to determine the geometry of submerged objects or the topography of a water body bottom. Placing camera and laser line projector inside a waterproof housing leads to refraction effects at the air-glass-water interfaces, both of the laser light-sheet and image rays. This implies a deformed laser plane in the water and a curved line on the object surface. The proposed approach strictly models the geometry between camera, laser and housing. First experiments show, that the calibration method can be applied for water depth measurements with accuracies of 0.2–0.3 mm at depths in the order of 100 mm.

A universal method for the calibration of swing-scanning line structured light measurement system

As a non-contact, low cost and efficient method for three dimensional (3D) measurements, the swing-scanning line structured light measurement system has a more compact structure than the linear scanning system. In measuring process, the laser line projector is driven by a step motor with the laser plane intersecting the object sequentially. To obtain the intersection profiles, the corresponding laser plane equations need to be real-time calculated. Moreover, the laser plane can be an arbitrary relative position with the swing axis due to the mount error of laser projector. These would make the calibration model and the solution rather difficult. To address these problems, a universal calibration method is proposed which bridges the swing angle and the coefficients of laser plane with a constant mapping matrix directly. This matrix is determined by use of several (three at least) specific laser planes. The whole calibration process can be achieved by one freely moved checkerboard and is very convenient to use. Several experiments are carried out to validate the proposed method including the measurement of the planar surface, the steps and the sculptured surface.

Automatic pavement rutting measurement by fusing a high speed-shot camera and a linear laser

This study proposes an inexpensive system using a linear laser and a high speed-shot sports camera to compute the rutting depth in static and dynamic modes with acceptable accuracy. Static and dynamic data collection is performed in a laboratory environment and multiple Bus Rapid Transit (BRT) lanes, respectively. The camera captures the projected laser line on the pavement surface, and this image is processed by applying image quality enhancement filters. A clustering method is used to detect the borders of ruts. Moreover, the effect of light level and the distance from the defect surface are measured. The results illustrate that the system has its best performance in 0.0001 lx light level with a relative error of 4%. Based on the paired sample t-test outputs, the optimal distance between the sensors and the ground should be 1 m. By validating the outputs of the system with the standard measurement method, the results show that this system is capable of computing the rut depth between 0.05 and 5.5 cm with a relative error of 9.31%, which confirms its capability for accurate data collection. Moreover, the replicability tests confirm the capability of using this system as an accurate data collection tool.

Underwater High-Precision 3D Reconstruction System Based on Rotating Scanning.

This paper presents an underwater high-precision line laser three-dimensional (3D) scanning (LLS) system with rotary scanning mode, which is composed of a low illumination underwater camera and a green line laser projector. The underwater 3D data acquisition can be realized in the range of field of view of 50° (vertical) × 360° (horizontal). We compensate the refraction of the 3D reconstruction system to reduce the angle error caused by the refraction of light on different media surfaces and reduce the impact of refraction on the image quality. In order to verify the reconstruction effect of the 3D reconstruction system and the effectiveness of the refraction compensation algorithm, we conducted error experiments on a standard sphere. The results show that the system’s underwater reconstruction error is less than 0.6 mm within the working distance of 140 mm~2500 mm, which meets the design requirements. It can provide reference for the development of low-cost underwater 3D laser scanning system.

Extrinsic Camera Calibration with Line-Laser Projection.

Knowledge of precise camera poses is vital for multi-camera setups. Camera intrinsics can be obtained for each camera separately in lab conditions. For fixed multi-camera setups, the extrinsic calibration can only be done in situ. Usually, some markers are used, like checkerboards, requiring some level of overlap between cameras. In this work, we propose a method for cases with little or no overlap. Laser lines are projected on a plane (e.g., floor or wall) using a laser line projector. The pose of the plane and cameras is then optimized using bundle adjustment to match the lines seen by the cameras. To find the extrinsic calibration, only a partial overlap between the laser lines and the field of view of the cameras is needed. Real-world experiments were conducted both with and without overlapping fields of view, resulting in rotation errors below 0.5°. We show that the accuracy is comparable to other state-of-the-art methods while offering a more practical procedure. The method can also be used in large-scale applications and can be fully automated.

Line Structured Light 3D Sensing With Synchronous Color Mapping

A line structured light measurement system is proposed which can obtain both the 3D geometrical and color information of objects. The profile sensor of the system is consisted of a black-and-white camera and an infrared laser line projector. A high precision calibration method is presented for the range sensor by use of a planar target. Then, the point coordinates of the measurement profile can be computed according to center coordinates of laser stripe and the calibrated sensor parameters. The wavelength of the laser projector is 850nm which is out of the spectrum response range of the color camera. Thus, the projected laser line would not affect the color acquisition. The transformation matrix between the two camera coordinate systems is determined and then optimized based on the invariance of cross ratio. By use of the transformation matrix, each point on the profile can be projected onto the image plane of the color camera, and the color information can be obtained synchronously. The measurement results show that the mean radius error of a ceramic ball can reach 0.009mm, and the mean mapping error of profile points on color image plane is 0.16pixel. The 3D measurement results of the textured objects further validate the proposed system.

A Smartphone-Based Laser Measuring System for Gap and Flush Assessment in Car Body

This article presents a portable smartphone-based laser measurement system (SLM) for measuring gaps and flushes on a car body. The system is designed to replace conventional gauges that are used by human operators. The developed device consists of a smartphone and an off-the-shelf laser-line projector held together by a three-dimensional-printed structure. It captures images of a laser line scattered on target surfaces, identifies the extreme points on the laser profiles, and obtains gap and flush measurements with minimal uncertainty. Since the smartphone is used as the operational unit, the measurement data can be stored and allows the capability for data transfer with other storage locations. Experiments were performed on the specimens and real cars to validate the application of the developed system. The measurement uncertainty on a real car was reported as ±0.201 mm for gap and ±0.154 mm for flush.

Real-Time Stripe Width Computation Using Back Propagation Neural Network for Adaptive Control of Line Structured Light Sensors.

A line structured light sensor (LSLS) is generally constituted of a laser line projector and a camera. With the advantages of simple construction, non-contact, and high measuring speed, it is of great perspective in 3D measurement. For traditional LSLSs, the camera exposure time is usually fixed while the surface properties can be varied for different measurement tasks. This would lead to under/over exposure of the stripe images or even failure of the measurement. To avoid these undesired situations, an adaptive control method was proposed to modulate the average stripe width (ASW) within a favorite range. The ASW is first computed based on the back propagation neural network (BPNN), which can reach a high accuracy result and reduce the runtime dramatically. Then, the approximate linear relationship between the ASW and the exposure time was demonstrated via a series of experiments. Thus, a linear iteration procedure was proposed to compute the optimal camera exposure time. When the optimized exposure time is real-time adjusted, stripe images with the favorite ASW can be obtained during the whole scanning process. The smoothness of the stripe center lines and the surface integrity can be improved. A small proportion of the invalid stripe images further proves the effectiveness of the control method.

Microscope vision system based on micro laser line scanning for characterizing microscale topography.

A microscope vision system to characterize a microscale surface via micro laser line projection is presented. The characterization is performed by means of surface descriptors, which include the root mean square, kurtosis, skewness, homogeneity, entropy, contrast, and correlation. These descriptors are computed from surface irregularities, which are retrieved by means of the micro laser line projection. The characterization is carried out by an optical microscope system on which a CCD camera and a 36 µm laser line are attached. Thus, the microscope vision system projects the micro laser line on the surface, and the CCD camera captures the line reflection, which provides the surface contour. The contour dimension is computed via Bezier networks by means of the micro laser line coordinates. Thus, the surface descriptors are computed by means of the surface contour to accomplish the characterization. The proposed characterization improves the accuracy of the optical microscope imaging systems, which characterize the microscale surface by means of the gray-level intensity. Thus, the capability of the characterization via micro laser line projection is established by means of the descriptors' accuracy. This contribution is corroborated by characterizing metal and paper surfaces.

Orientation detection of fruits by means of convolutional neural networks and laser line projection for the automation of fruit packing systems

Sorting and quality inspection of fruits is still a time-consuming task because of the low level of automation. Natural products exhibit a greater variance compared to manufactured products regarding quality and dimensions, which complicates their automated processing. In our paper, we propose a method to determine the orientation of fruits by applying a specially trained Convolutional Neural Network to 2.5-dimensional image data generated by a combination of a color CMOS camera and a red line laser. We achieve an accuracy of the orientation prediction of approximately 5 degrees, which presents very promising results for the automation of fruit packing systems.

SCOUT3D – AN UNDERWATER LASER SCANNING SYSTEM FOR MOBILE MAPPING

Abstract. This paper presents an underwater laser scanning system and GNSS based trajectory estimation system for scanning from a surface vessle in shallow water. The system has an above-the-water and an underwater component. Above-the-water two low-cost multiband GNSS receivers with an antenna baseline of one meter are used for RTK positioning with heading. The full 6-DOF is estimated by fusing the satellite navigation data with a MEMS-based INS. The 3D data is captured in water using a structured light scanner consisting of a low-light underwater camera and a green cross line laser projector. We describe the development of the system and employed hardware components. We show results of scanning a large test object in a water tank acquired by from a tripod with a motorized yaw axis. Additionally, we demonstrate first results of mobile mapping from a floating platform. We evaluate the performance of the system by measuring the 6-DOF trajectory with an external optical tracking system. Additionally, we assess the quality of the created point cloud using reference objects placed in the scene.