Multiline Laser Research Articles

Multi-line laser 3D reconstruction method based on spatial quadric surface and geometric estimation

The traditional multi-line laser 3D reconstruction is based on binocular epipolar constraints and the equation of the laser optical plane. Firstly, matching points are identified based on the epipolar constraints. Then, the correct matching points are selected based on the optical plane equation of the multi-line laser. Finally, 3D reconstruction is performed using the selected matching points. In actual production processes involving multi-line lasers, noticeable distortion occurs due to the projection of Diffractive Optical Elements (DOE) at a large angle. This results in curved laser light planes and curved reflections on the measured objects. Under such circumstances, efficiently screening matching points using the traditional method based on the laser plane equation becomes challenging. Additionally, inherent noise in multi-line laser systems introduces errors in extracted laser center coordinates, making it impossible to directly obtain high-precision 3D data. To address these issues, this paper proposes a method that utilizes spatial quadric surfaces and geometric estimation for completing multi-line laser 3D reconstructions. By analyzing the distortion principle of DOE and the positional relationship after optical diffraction, the multi-line laser manifests as a quadric surface on the optical output plane. Consequently, by calibrating the equations of the quadric surface and applying binocular polar constraints, suitable matching points for the multi-line laser can be chosen. Once an accurate matching point is identified, a minimum geometric distance estimation can be established based on the distance constraint between the point and its corresponding epipolar line. This distance represents the separation between the left and right camera’s laser center points and their respective epipolar lines. Utilizing this estimate of the minimum geometric distance, a refined calculation can be performed to better satisfy epipolar constraints and obtain new matching points. Ultimately, utilizing these new matching points enables the completion of 3D reconstruction for multi-line lasers. In comparison with methods relying on spatial plane and epipolar constraints, our algorithm exhibits a superior degree of matchability and accuracy.According to multiple groups of test data, the average error before optimization is 0.3126 mm, and can be increased to 0.0336 mm after optimization.

Measurement and optimization method for aero-engine rotors based on binocular multi-line laser sensing and virtual assembly

During the assembly of aero-engine rotors, eccentricity and tilt errors along the radial and axial direction result in uneven loading along the mating surface, which significantly impacts the operational lifespan of the rotor. This paper proposes a three-dimensional contour measurement system based on binocular multi-line laser sensing and conducts an aircraft engine rotor stacking virtual assembly technology system, to response the practical requirements of aircraft engine rotor unit production assembly engineering. We combine multi-line structured light active projection device with the binocular stereo vision model to construct the contour measurement system, which enhances the surface features of the projected object by adhering to the common constraint of structured line and stereo vision. Furthermore, the error propagation model for multi-stage rotor assembly to investigate the mechanism of error impact is established aiming to facilitate quantitative research on virtual assembly mechanisms. By manipulating the rotor phases and evaluating the coaxiality under various conditions, effective prediction of phase variation can be achieved, the multi-objective benchmark virtual assembly method based on the error propagation model of multi-stage rotor coaxiality is presented. The experimental results indicate that the proposed measurement system achieves the maximum disparity between the spatial data points procured by the proposed measurement system and the predetermined value of 2.1 μm, and the maximum disparity between the target plane’s average movement distance and the preset value of 3.7 μm. The precision provided by this system effectively meets the objective of assembly phase optimization for multi-stage rotors. In the empirical experiment conducted on the three-stage rotors, both systems yielded identical optimized phases: 0° for the first-stage rotor, 270° for the second-stage rotor, and 120° for the third-stage rotor, thereby validating the efficacy of the proposed measurement and optimization method for aero-engine multi-stage rotors with virtual assembly.

Multi-line laser scanning reconstruction with binocularly speckle matching and trained deep neural networks

A multi-line laser scanning system for 3D topography measurement is proposed. This method not only has the advantages of high precision of laser scanning technology, but also has high reconstruction efficiency. In this paper, speckle reconstruction technique, multi-line laser technique and Biocular reconstruction technique are used to construct a 3D reconstruction system, and test equipment is built, and the problems existing in the system establishment process are actually studied. In order to solve the problem of mismatching in binocular multi-line laser matching, a method to sort out the correspondence of multiple laser lines in binocular images based on speckle matching results is proposed. In order to optimize the multi-line laser matching effect, a speckle matching network based on deep learning is proposed, which integrates the grayscale images of the left and right cameras as supplementary information, and takes the speckle image and grayscale image as the input of the network model to obtain more accurate and edge-complete matching results. Finally, the matching results of the multi-line laser and the camera calibration parameters were used to reconstruct the object point cloud. Experimental results show that the proposed speckle matching method can make binocular multiline laser point cloud reconstruction more robust and stable than the traditional method, and through the accuracy analysis of the system, it is proved that the average measurement accuracy of the proposed method can reach 0.05 mm.

Accurate extraction method of multi-laser stripes for stereo-vision based handheld scanners in complex circumstances

A technique for extracting multiline lasers is proposed for handheld scanner. First, a robust matching method for multi-lasers is presented in this work to distinguish every laser stripe in the left and right images by using back-projection algorithm. Then, an improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering method is promoted to detect noise in the laser images caused by complex scenarios. This approach is focused on the features of the laser stripes in the images, such as normal vectors, stripe width and gray values. These parameters are used as the criteria to find the noise in the images. After the calibration results of stereo-vision and laser-matching results are combined, the 3D information of the noise parts in the laser stripes are recovered accurately. Finally, a subpixel interpolation method based on Shepard interpolation is used to achieve the precise subpixel centerline extraction of laser stripes. And the experiments further valide the effectiveness of the proposed method.

Multi-line structured light stripes clustering based on a custom iterative window

ObjectiveThis paper proposes a multi-line structured light stripe clustering method based on a custom iterative window to address the issue of stripes confusion caused by stripe breakage and noise points during the multi-line laser stripe matching process in binocular stereo vision. MethodsFirstly, the paper segments the measured object from the carrier plane. Secondly, the extracted centerlines are clustered using the custom iterative window. Finally, the stripes are numbered one by one after clustering the left and right images using barycentric coordinates of the stripes, based on the property of sequential consistency constraint, to complete the stereo matching of the stripes with the same number. ResultsThe experimental results show that the method can effectively cluster the centerline stripes on the measured object and solve the problem of stripes mis-corresponding when stereo matching of multi-line structured light stripes.

Depth extraction of 3D defects on curved surfaces with multi-line lasers

Abstract The adoption of computer vision technology has significantly impacted surface defect inspection by providing a non-contact, cost-effective solution that has been widely accepted. Among the various techniques available, 3D defect inspection using multi-line lasers is notable for its simplicity, high detection speed, and extensive coverage. The accuracy of this method is significantly constrained by the precision of laser stripe extraction. In industrial environments, achieving accurate extraction is hindered by the intricate surface geometries of objects and the challenge of maintaining uniform brightness in multi-line laser stripes. To address these challenges, we propose a novel approach to extract the depth of 3D defects on surfaces using multi-line lasers. Our method combines guided filtering and the Frankle-McCann Retinex (FMR) algorithms to improve the quality of captured images. We have refined the laser stripe extraction process and proposed an advanced adaptive threshold segmentation technique that utilizes the OTSU method to determine threshold coefficients, followed by secondary segmentation based on a neighborhood search. The extracted laser strips are then processed using the quadratic weighted gray gravity method. Additionally, we proposed an innovative region-growth segmentation method based on neighborhood search that effectively segments individual laser strips. We also design a strategy for determining 3D defect depths in situations where precise camera calibration is challenging. The efficacy of our proposed method was rigorously tested on a hot-rolled seamless steel tube with a diameter of 145 mm. The resulting 3D defect depth exhibited an error of less than 0.5 mm, meeting the stringent standards required for practical applications.

A comprehensive laser image dataset for real-time measurement of wheelset geometric parameters

Railway transportation has experienced significant growth worldwide, offering numerous benefits to society. Most railway accidents are caused by wheelset faults so it’s significant to monitor wheelset conditions. Therefore, we need to collect wheelset images, repaint them, extract laser stripe centerlines, construct 3D contour, and measure their geometric parameters to judge the wheelset’s conditions. Deep learning can fulfill the tasks satisfyingly because it’s adaptable, robust, and generalize compared with traditional methods. To train the deep learning models effectively, we need rich and high-quality wheelset datasets. So far, there are no applicable public train wheelset datasets available, which greatly hinders the research on train wheelsets. Thus we construct a publicly available Wheelset Laser Image Dataset (WLI-Set). WLI-Set consists of four sub-datasets, Original, Inpainting, Segmentation, and Centerline. The dataset contains abundant annotated multiline laser stripe images that can facilitate the research on train wheelsets effectively.

Spatial quadric calibration method for multi-line laser based on diffractive optical element

The traditional multi-line laser calibration method relies primarily on calibrating the spatial plane equations of multiple laser lines and reconstructing the multi-line laser in three dimensions through the plane equation and camera parameters. In traditional methods, the multi-line laser is projected as a straight line by default, but in reality, the laser line projected by the multi-line laser is not a plane due to the Diffractive Optical Element (DOE). Instead, it is a surface with a certain curvature in space. During the production process, the multi-line laser experiences significant distortion because of the DOE projecting at a large angle. Consequently, if traditional methods are used, accurate calibration of the multi-line laser cannot be achieved. To address this issue, this paper introduces a new calibration algorithm for multi-line laser based on a spatial quadric equation. By calibrating the spatial quadric equation of the multi-line laser, the spatial characteristics of the laser can be accurately captured. The algorithm involves projecting a crossed 7-line laser onto a calibration plate and changing the position of the calibration plate while capturing images. This allows for obtaining the spatial three-dimensional coordinates of each laser line in the camera coordinate system. Next, spatial quadric equations are fitted for each laser line based on the three-dimensional coordinates. Finally, 3D reconstruction is performed using the quadratic surface parameters of the multi-line laser and camera parameters. The results from 3D reconstruction demonstrate that, compared to traditional laser calibration methods based on planes, the proposed algorithm achieves higher calibration accuracy. It also improves the number of successful binocular multi-line laser matches and overall accuracy.

Broadband sum-frequency conversion of multiline Q-switched CO laser emission under its double-pass through AR-coated ZnGeP2 crystal

Broadband sum-frequency conversion of multiline Q-switched CO laser emission under its double-pass through AR-coated ZnGeP2 crystal

High accuracy calibration method for multi-line structured light three-dimensional scanning measurement system based on grating diffraction.

Multi-line structured light three-dimensional (3D) scanning measurement system enables to obtain the richer 3D profile data of the object simultaneously during one frame, ensuring high accuracy while structured light is deformed for the modulation by the object. Nevertheless, current calibration methods cannot fully take advantage of its high precision. In this paper, a fast and high-accuracy 3D measurement system based on multi-line lasers with a spatially precise structure via integrating a diffraction grating was proposed. This helps achieve precise calibration results of the light planes by introducing spatial constraint relations of the diffractive light, thus improving measurement accuracy. The operating principle and the workflow of the proposed system were described in detail. The measurement accuracy of the developed prototype was verified through contrastive experiments. At a working distance of 400 mm, the results show that the root mean square error (RMSE) of the proposed system is 0.083 mm, which is improved by 37.6% compared to the traditional calibration method of light planes for the ranging system. The system utilizing a grating that facilitates the integration of the device has great application value.

Development of a novel multi-line laser triangulation scanning system based on the rotary diffraction grating

The laser triangulation scanning system (LTSS) is widely employed in three-dimensional (3D) measurement fields owing to its high measurement accuracy. However, due to the limitations of the exploited motion, the traditional LTSS fails to be satisfactory for certain precise and efficient industrial applications. In this paper, a new multi-line laser triangulation scanning system (MLTSS) via integrating a rotary diffraction grating was proposed to address this problem. The image sequence of the divergent laser lines with variable angles on the object can be captured as the grating rotates. This enables to obtain the multiple profiles simultaneously during one frame. A novel method to distinguish laser lines in a single frame which enhances the measurement range was described. The measurement principle and workflow of the proposed system are explained in detail. The calibration algorithm of light planes of the proposed system utilizing the equivalent grating equation is simplified compared to the traditional algorithm. The developed setup and the measured object remain stationary throughout the whole scanning, which relieves the mechanical error caused by the motion. Moreover, the utilized grating facilitates to miniaturize the device. The experiments demonstrate the validity of the proposed system.

Spatially modulated erbium YAG laser as a treatment for diabetic ulcer.

Diabetic ulcers are a significant healthcare challenge, capable of diminishing quality of life, lengthening hospitalisation stay, and incurring substantial costs for patients and healthcare systems. Erbium-doped yttrium-aluminum-garnet (Er-YAG) laser has been evolving as a prospective intervention for addressing wounds of various aetiologies. Despite this, the literature remains limited in appraising the effectiveness of laser therapy specifically in diabetic wounds. This study investigates the impact of employing a spatially modulated Er-YAG laser as a therapeutic approach for treating diabetic ulcers. In a single-arm study conducted from November 2017 to April 2023, patients with hard-to-heal ulcers were treated in a two-step approach of wound debridement using Er-YAG laser for ablation and biostimulation through deep tissue resonance using RecoSMA (Multiline laser system, LINLINE MS, Latvia) laser technology. Ulcers received weekly laser treatment, together with routine care until healing occurred and were then followed up to observe any recurrence. The primary outcome measure was wound closure; the secondary outcome measures were time to closure, and the number of laser treatments required. Data related to sociodemographic details, size and number of diabetic ulcers, and number of sessions related to laser treatment were collected using a predesigned, pretested questionnaire before initiating the treatment. A total of 59 patients attending the clinic during the study period with diabetic ulcers were included in the study. The mean wound surface area at baseline was 25.7cm2 (median: 12cm2). The average number of sessions of laser treatment required was 4.41, ranging from 1-11. The size of the ulcer reduced with each session of laser treatment. The diabetic ulcers healed completely at the end of the treatment, indicating the effectiveness of the Er-YAG/RecoSMA two-step approach. Spatially modulated erbium YAG laser is effective as a therapeutic approach for treating diabetic ulcers.

URS: A Light-Weight Segmentation Model for Train Wheelset Monitoring

To detect the wheelset deformation and wear, an intuitive method is to first collect wheelset multi-line laser stripe images by monitors, and then extract the centerlines to construct 3D contours that are transferred to the cloud data center by 6G communication. The images, however, contain flairs and fractures due to the influence of environmental interference and reflected light on the smooth surfaces. The image defects affect the accurate extraction of stripe centerlines. To segment the defects and inpaint them, we propose a new lightweight U-shaped segmentation model URS. A target-shaped receptive field is designed to efficiently extract the details of the local, the ring-shaped, and the cross-shaped context around the local, which facilitates segmenting various defects. A scale-select sub-module is designed to adjust the weights of features from different receptive fields. To train the model, a multi-line laser image defect segmentation dataset MLIDSD is constructed. Experiments demonstrate that the performance of our model surpasses twelve SOTA models explicitly and can achieve a balance between the accuracy and the lightweight requirement.

Broadband sum-frequency conversion of multiline Q-switched CO laser emission under its double-pass through uncoated ZnGeP2 crystal

Broadband sum-frequency conversion of multiline Q-switched CO laser emission in a single uncoated ZnGeP2 crystal was experimentally studied by application of single-pass and double-pass optical schemes. The laser emitted sub-microsecond pulses of 4 kW peak power at 100 Hz pulse repetition rate on ∼60 spectral lines within ∼5–6 μm wavelength interval. The sum-frequency emission spectrum consisted of at least ∼100 spectral lines covering the wavelength interval of ∼2.5–2.9 μm. The double-pass scheme provided about 30% higher efficiency as compared to a traditional single-pass one, while the spectral bandwidth of frequency converted radiation was the same. Moreover, a feasibility of slight spectral bandwidth extension by a variation of a reverse beam direction was demonstrated for the double-pass scheme.

Research on SLAM and Path Planning Method of Inspection Robot in Complex Scenarios

Factory safety inspections are crucial for maintaining a secure production environment. Currently, inspections are predominantly performed manually on a regular basis, leading to low efficiency and a high workload. Utilizing inspection robots can significantly improve the reliability and efficiency of these tasks. The development of robot localization and path planning technologies ensures that factory inspection robots can autonomously complete their missions in complex environments. In response to the application requirements of factory inspections, this paper investigates mapping, localization, and path planning methods for robots. Considering the limitations of cameras and laser sensors due to their inherent characteristics, as well as their varying applicability in different environments, this paper proposes SLAM application systems based on multi-line laser radar and visual perception for diverse scenarios. To address the issue of low efficiency in inspection tasks, a hybrid path planning algorithm that combines the A-star algorithm and time elastic band method is introduced. This approach effectively resolves the problem of path planning becoming trapped in local optima in complex environments, subsequently enhancing the inspection efficiency of robots. Experimental results demonstrate that the designed SLAM and path planning methods can satisfy the inspection requirements of robots in complex scenarios, exhibiting excellent reliability and stability.

P‐2.7: Slam Fusion Location Mapping Based on Lidar

Aiming at the low accuracy of GNSS/INS integrated navigation caused by the outdoor road environment without GNSS signal or poor GNSS signal, this thesis proposes to combine slam technology with on-board laser scanning system, which can still maintain good robustness in the weak GNSS area. By coupling slam and IMU through the factor graph optimization algorithm, it can improve the mapping results affected by the high speed in the on-board laser scanning system. this thesis uses the system calibration to improve the lio-sam algorithm.Based on the factor graph optimization algorithm, the multi line laser is added to the vehicle lidar scanning system to solve the long-distance and high-precision positioning and attitude determination in the case of weak GNSS signal. The feasibility and effectiveness are verified by the specific application in the field.

Efficient 3D scanning measurement system based on asymmetric trinocular vision and a multi-line laser.

The laser scanning measurement system has a pivotal role in precision measurement thanks to the non-contacting and low-cost advantages, but traditional methods and systems are inadequate in terms of accuracy, efficiency, and adaptability. In this study, an efficient 3D scanning measurement system based on asymmetric trinocular vision and a multi-line laser is developed to improve the measurement performance. The system design, working principle, and 3D reconstruction method are explored, as well as the innovation of the developed system. Furthermore, an efficient multi-line laser fringes indexing method is presented based on K-means ++ clustering and hierarchical processing to improve processing speed with guaranteed accuracy, which is the key point of the 3D reconstruction method. Various experiments are conducted to verify the capability of the developed system, and the results show that the developed system fulfills measurement needs in adaptability, accuracy, effectiveness, and robustness. The developed system achieves better results than commercial probes for complex measurement conditions, and measurement precision can be achieved to within 18µm.

Void-Defect Location Control of Laser-Crystallized Silicon Thin Films with Hole-Pattern

High-performance low-temperature polycrystalline silicon (LTPS) thin-film transistors (TFTs) have been developed for larger applications than flat panel displays (FPDs) such as three-dimensional integrated circuits (3D-ICs) and glass sheet computers. The crystallinity of poly-Si thin films has been the key factor determining TFTs’ performance. In this work, a void-defect location has been controlled by patterning amorphous silicon (a-Si) thin films with rectangular and square holes before crystallized by multiline continuous-wave laser beam to avoid the effect of void-defects on the TFTs’ performance. Instead of randomly appearing in the poly-Si thin films, void-defects were only observed at the backsides of the patterned holes. Interestingly, large crystal grains without void-defects were laterally crystallized at Si strips between holes. By observing the crystallinities of poly-Si thin film around the patterned holes, both the mechanism of the void formation and crystal growth based on temperature gradient was clarified.

A novel hierarchical iterative hypothesis strategy for intrinsic parameters calibration of laser structured-light weld vision sensor

In automatic welding, multi-line laser structured-light vision sensors provide more adequate information to compared with traditional single-line structured light and are gradually being widely designed. However, when calibrating the sensor, a global calibration approach should be adopted in order that the laser intrinsic parameters follow the inherent constraints. In this paper, a Lagrange solution is shown and its unsuitability for general cases is qualitatively analyzed. We then propose a novel hierarchical iterative hypothesis (HIH) strategy to deal with issues with the following conditions. First, the configuration and pose (including rotation and translation) of a 3D model need to be solved. Second, the model can be obtained by scaling a known model in several dimensions. Third, part of sampling points of the model can be acquired. The proposed hypothesis is introduced based on the coordinate transformation theory. Hierarchical iteration, including global and local iteration, is elaborated mathematically, which utilizes singular value decomposition (SVD) and gradient descent technology. The process of applying hierarchical iterative hypothesis strategy HIH is illustrated, and a reconfirmation for transformation is introduced. The steps in the intrinsic parameters calibration for weld vision sensor are described. A calibration experiment is witnessed that the ambiguity on the intersection issue is reduced to compared with ordinary least square method for structured light calibration. Several quality measures were introduced that allow us to judge the performance of our method. Two experiments for verifying calibration results reveal that the proposed strategy earns a great calibration performance that can meet weld seam tracking needs.

Multi-line laser structured light fast visual positioning system with assist of TOF and CAD

As a reliable and high-precision three-dimensional (3D) vision method, line laser structured light vision has been widely used in the engineering field. However, to obtain overall 3D shape of target to be measured, scanning mechanisms need to be added. Multi-line laser structured light vision can obtain multi-line array 3D shape of workpiece without scanning; however, the encoding of multi-line lasers is a difficult task. This paper proposes a composite vision system, which is composed of time-of-flight(TOF) camera, color camera and vertical cross multi-line laser structured light. Color camera and multi-line laser form a multi-line array 3D vision system, while the coded signal of multi-line laser stripes is provided by TOF camera and corresponding coding algorithm is also proposed in this paper. Based on this system, automatic positioning of welding seam is realized by registration of CAD data and multi-line laser 3D data. The registration is divided into two steps, which are coarse registration and precise registration. Coarse registration is performed by using CAD point cloud data and original point cloud data of multi-line laser; while precise registration is performed with CAD welding seam point cloud data and welding seam point cloud data of multi-line laser. This paper also analyzes accuracy of vision system based on constructing structured light vision model. The measurement accuracy of vision system in x, y and z directions is 0.25, 0.06 and 0.29 mm respectively. This composite vision system can realize precise positioning of target within a single exposure time, which provides possibility for positioning under movement.