Laser Stripe Research Articles

Identifying the Laser Stripes via Ray Model for Multiline Structured Light Stereo Vision Sensors

Identifying the Laser Stripes via Ray Model for Multiline Structured Light Stereo Vision Sensors

Research on a Feature Point Detection Algorithm for Weld Images Based on Deep Learning

Laser vision seam tracking enhances robotic welding by enabling external information acquisition, thus improving the overall intelligence of the welding process. However, camera images captured during welding often suffer from distortion due to strong noises, including arcs, splashes, and smoke, which adversely affect the accuracy and robustness of feature point detection. To mitigate these issues, we propose a feature point extraction algorithm tailored for weld images, utilizing an improved Deeplabv3+ semantic segmentation network combined with EfficientDet. By replacing Deeplabv3+’s backbone with MobileNetV2, we enhance prediction efficiency. The DenseASPP structure and attention mechanism are implemented to focus on laser stripe edge extraction, resulting in cleaner laser stripe images and minimizing noise interference. Subsequently, EfficientDet extracts feature point positions from these cleaned images. Experimental results demonstrate that, across four typical weld types, the average feature point extraction error is maintained below 1 pixel, with over 99% of errors falling below 3 pixels, indicating both high detection accuracy and reliability.

A Net Shape Profile Extraction Approach for Exploring the Forming Appearance of Inclined Thick-Walled Structures by Wire Arc Additive Manufacturing

Wire arc additive manufacturing (WAAM) offers a viable solution for fabricating large-scale metallic parts, which contain various forms of inclined thick-walled structure. Due to the variety of heat dissipation conditions at different positions, the inclined thick-walled structure is a major challenge in fabrication that may produce collapses and defects. However, there is a lack of effective sensing method for acquiring the forming appearance of individual beads in the structure. This paper proposes a novel approach for extracting individual bead profiles during the WAAM process. The approach utilizes a structured-laser sensor to capture the morphology of the surface before and after deposition, thereby enabling an accurate acquisition of the bead profile by integrating the laser stripes. Utilizing the proposed approach, the research investigated the forming mechanism of beads in inclined thick-walled components that were fabricated by various deposition parameters. The width of the overlapping area at the overhanging feature decreased as the layer number increased, while the height of the same area increased. The height of the overlapping area in each layer increased with an increase in deposition current and decreased when the deposition speed was increased. These phenomena suggest that the heat input is a major factor that influences the formation of the overhanging feature. Both the deposition current and deposition velocity influence heat input, and thereby have an effect in enhancing the geometrical accuracy of an overhanging feature. The experimental results indicate that the proposed approach facilitates morphology change investigation, providing a sufficient reference for optimizing deposition parameters.

Anti-noise weld seam tracking algorithm based on spatio-temporal memory mechanism

In laser vision-based robotic welding, existing seam tracking methods are seriously affected by noise and lack effective utilization of spatio-temporal information during welding process. To address this, this paper proposes an anti-noise weld seam tracking algorithm based on spatio-temporal memory mechanism. First, a memory mechanism with welding history memory information is designed, which can better adapt to seam appearance changes and noise interference during welding. Subsequently, a laser stripe tracker based on spatio-temporal memory is constructed, it enables the current frame to adaptively search historical valid information to eliminate effects of noise and appearance changes. The experimental results show that the proposed method has an average response speed of 43 FPS and an average tracking error of 1.35 pixels for various weld seams. The actual tracking error is less than 0.6 mm. The introduction of memory mechanism improves the weld tracking accuracy by 52.1 % compared to a single tracking template.

An enhanced centerline extraction algorithm for complex stripes in linear laser scanning measurement

Centerline extraction is a basic and critical step for linear laser scanning measurement. However, in practical measurement, the inconsistent reflectivity of the measured surface and the external noise interference can result in complex variation of the stripe pattern, which will influence the centerline extraction accuracy of the stripe pattern and hence the measurement accuracy of the laser scanner. To solve this problem, an enhanced centerline extraction algorithm for complex laser stripes is proposed. In the preprocessing of the stripe pattern, a region separation strategy is employed to mitigate the effects of complex variations, and multi-region self-adaptive convolution is adopted to enhance the stripe pattern quality. For high-precision extraction of the stripe centerline, different convolution kernels are applied to compute the Hessian matrix for the stripe patterns in different regions to determine the normal direction of the laser line. The second order Taylor expansion is performed along this direction to work with the denoising algorithm to determine the subpixel positions of the center points on the line, and then the whole centerline of the stripe pattern is obtained by piecewise cubic Hermite interpolation methods. Experimental results show that the effectiveness of the proposed algorithm in addressing centerline extraction for complex laser stripe patterns due to stray light interference, reflective stripes, and laser stripe linewidth changes in laser scanning measurement.

Laser stripe measurement of near-wall solids distribution in channel flow with intense transport of bimodal sediment

Laser stripe measurement of near-wall solids distribution in channel flow with intense transport of bimodal sediment

Method of 3D reconstruction of underwater concrete by laser line scanning

High-precision 3D (three-dimensional) reconstruction of underwater concrete is helpful for the detection and analysis of underwater concrete diseases. A prevalent method involves mounting optical cameras in waterproof housings to enable the 3D reconstruction of underwater concrete. Achieving millimetric accuracy in underwater 3D reconstruction poses a significant challenge in current research. The refraction of light on surfaces of different media and its reflection underwater complicate the process. In this paper, a 3D reconstruction method of underwater concrete based on laser line scanning is proposed. Firstly, the underwater imaging model is established by the ray tracing method, and the refraction conversion matrix is derived. Then, based on this matrix, the conversion relationship between the measured 3D size and the real 3D size of the underwater object is obtained. Finally, image restoration technology is introduced in the process of laser stripe extraction to eliminate the influence of reflected light on the extraction of laser stripe center, to improve the reconstruction accuracy. We used the independently developed laser line 3D scanning equipment to carry out experimental analysis. The results showed that the error rate caused by direct reconstruction of underwater objects was 9.75 %. Additionally, the error rate obtained from 3D reconstruction of underwater objects using the conversion model was 2.89 %. This study provides technical support for the 3D detection of underwater concrete.

Accurate extraction method of multi-laser stripes for stereo-vision based handheld scanners in complex circumstances

A technique for extracting multiline lasers is proposed for handheld scanner. First, a robust matching method for multi-lasers is presented in this work to distinguish every laser stripe in the left and right images by using back-projection algorithm. Then, an improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering method is promoted to detect noise in the laser images caused by complex scenarios. This approach is focused on the features of the laser stripes in the images, such as normal vectors, stripe width and gray values. These parameters are used as the criteria to find the noise in the images. After the calibration results of stereo-vision and laser-matching results are combined, the 3D information of the noise parts in the laser stripes are recovered accurately. Finally, a subpixel interpolation method based on Shepard interpolation is used to achieve the precise subpixel centerline extraction of laser stripes. And the experiments further valide the effectiveness of the proposed method.

Multi-line structured light stripes clustering based on a custom iterative window

ObjectiveThis paper proposes a multi-line structured light stripe clustering method based on a custom iterative window to address the issue of stripes confusion caused by stripe breakage and noise points during the multi-line laser stripe matching process in binocular stereo vision. MethodsFirstly, the paper segments the measured object from the carrier plane. Secondly, the extracted centerlines are clustered using the custom iterative window. Finally, the stripes are numbered one by one after clustering the left and right images using barycentric coordinates of the stripes, based on the property of sequential consistency constraint, to complete the stereo matching of the stripes with the same number. ResultsThe experimental results show that the method can effectively cluster the centerline stripes on the measured object and solve the problem of stripes mis-corresponding when stereo matching of multi-line structured light stripes.

New insights into enhancing the mechanical properties of 6061-T6 aluminum alloy MIG welded joints by constructing bionic heterostructure

The trade-off between strength and toughness is a long-term challenge for engineering metal structural materials. However, natural materials overcome the inverse relationship between strength and toughness through their unique structures. Inspired by plant leaves and dragonfly wings, a ‘soft-hard’ alternating bionic heterostructure is prepared on 6061-T6 Al alloy MIG welded joints by laser surface remelting (LSR) technology. After LSR, the grain size of the laser treated zone is significantly refined, dislocation density is increased, and the formation of fine and stable GP zone and β′′ phase is promoted, enhancing microhardness and tensile properties of the LSRed sample. By adjusting the laser stripe spacing and distribution angle, stress transfer and redistribution can be optimized, effectively improving the tensile properties of the welded joint can be. Compared to as-welded sample, the tensile strength, yield strength and elongation of LSRed sample can be increased by 30.6 %, 13.6 % and 102 %, respectively. In addition, the bionic heterostructure composed of grain size gradient, dislocation gradient and precipitated phases gradient can effectively develop heterogeneous deformation induced (HDI) stress strengthening and HDI strain hardening, enabling synergistic strengthening of strength and toughness.

3D measurement of precise part in complex circumstance using line structured light and improved U-Net

Abstract Line structured light scanning is extensively utilized for the 3D measurement of precise metal parts, but the curvature surfaces and specific materials of these parts generate specular reflection, making it challenging to accurately extract the center of the laser stripe in complex circumstances. Therefore, the primary challenges are the noise separation in the laser stripe image and the extraction of the laser stripe center under complex circumstances. To solve the above problems, an improved U-Net semantic segmentation algorithm is proposed by adding an attention mechanism and modifying skip connections to the classical U-Net network structure for accurate laser stripe segmentation. Secondly, the dual smoothing method of mean smoothing and Savitzky–Golay smoothing is combined with the Hessian matrix algorithm to complete the subpixel extraction of the center point of the laser stripe. Finally, taking the blade and shaft part as the measurement objects, the experimental results demonstrate that the method can obtain more complete, smoother, and denser results than the traditional method under highly reflective surfaces, vital interference spots, and strong ambient light. The proposed method is used for the 3D measurement of the shaft parts, and the diameter measurement maximum error is 0.029 mm, verifying the feasibility of the proposed method.

Depth extraction of 3D defects on curved surfaces with multi-line lasers

Abstract The adoption of computer vision technology has significantly impacted surface defect inspection by providing a non-contact, cost-effective solution that has been widely accepted. Among the various techniques available, 3D defect inspection using multi-line lasers is notable for its simplicity, high detection speed, and extensive coverage. The accuracy of this method is significantly constrained by the precision of laser stripe extraction. In industrial environments, achieving accurate extraction is hindered by the intricate surface geometries of objects and the challenge of maintaining uniform brightness in multi-line laser stripes. To address these challenges, we propose a novel approach to extract the depth of 3D defects on surfaces using multi-line lasers. Our method combines guided filtering and the Frankle-McCann Retinex (FMR) algorithms to improve the quality of captured images. We have refined the laser stripe extraction process and proposed an advanced adaptive threshold segmentation technique that utilizes the OTSU method to determine threshold coefficients, followed by secondary segmentation based on a neighborhood search. The extracted laser strips are then processed using the quadratic weighted gray gravity method. Additionally, we proposed an innovative region-growth segmentation method based on neighborhood search that effectively segments individual laser strips. We also design a strategy for determining 3D defect depths in situations where precise camera calibration is challenging. The efficacy of our proposed method was rigorously tested on a hot-rolled seamless steel tube with a diameter of 145 mm. The resulting 3D defect depth exhibited an error of less than 0.5 mm, meeting the stringent standards required for practical applications.

Advanced Parallel Laser Line-Camera System for Automatic and Precise Crack Width Measurement

Accurate quantification of cracks is crucial in evaluating building conditions. However, current non-contact measurement technologies encounter challenges in measuring crack widths, particularly in hard-to-reach areas. This paper presents a novel portable parallel laser line-camera system that is specifically designed for automatic and precise measurement of crack widths from different angles and over long distances. The system is composed of a digital camera, parallel line laser diodes, and rigid positioning arms, forming a triangulation-based configuration. A newly developed image processing algorithm is at the core of this innovation, allowing for the extraction of accurate pixel scale information from laser strips, thus overcoming the challenge of pixel distortion in non-perpendicular photography. The U-Net algorithm is utilized to efficiently identify crack regions, followed by the Canny algorithm for meticulous extraction of crack features. By utilizing the pixel scale data obtained from the laser strips, the actual width of cracks can be accurately determined. The system's efficacy and precision have been validated through laboratory and field tests. This automatic, precise, and portable solution significantly advances the field of on-site crack measurement, offering substantial potential for practical applications in building condition assessments.

A novel reflective interference mitigation model for laser stripe extraction

In practical applications of line-structured light measurement technology, the transmitted laser may be reflected several times due to the reflective properties of the measured object’s surface, which directly affects the accuracy of three-dimensional (3-D) measurements. To obtain precise laser stripes under such reflective conditions to improve measurement accuracy, this paper proposes a semantic segmentation method called GuideNet. Firstly, a dual-branch encoder is introduced to simultaneously extract high-resolution spatial features and high-level semantic features from the input image. Secondly, a Guidance module is designed to fuse spatial features and semantic features and enhance the model’s ability to focus on relevant regions. Finally, the multi-level feature fusion decoder is performed to obtain the final results. Experiments on RIDataset and 3-D measurement experiments demonstrate that the proposed method improves the anti-reflection ability of line-structured light sensors.

Extraction of laser stripe centerlines from translucent optical components using a multi-scale attention deep neural network

The precise extraction of laser stripe centerlines is critical for line-laser 3D scanning systems. However, conventional methods relying on threshold segmentation and morphological operations face significant challenges when confronted with pervasive optical phenomena, including specular reflection, scattering, and bleeding, which are commonly observed in translucent optical components. These methods typically require complex preprocessing procedures and often yield poor precision in centerline extraction. In this paper, we introduce a novel learning-based approach, complemented by a meticulously curated dataset, explicitly designed to address these challenges. Our proposed method leverages a multi-scale attention U-Net-like architecture, initially tasked with the segmentation of laser stripes from the complex background environment. Subsequently, it employs the Steger algorithm for the precise extraction of laser stripe centerlines. The experimental results, obtained by comprehensively evaluating real-world captured images, clearly demonstrate the effectiveness of our deep neural network combined with the Steger algorithm. This combined approach exhibits exceptional accuracy even when challenged by the interferences from specular reflection, scattering, and bleeding artifacts. Specifically, our method achieves a mean intersection over union (mIoU) of 84.71% for the laser stripe detection task, accompanied by a mean square error (MSE) of 10.371 pixels. Also, the average execution time for the centerline extraction task is notably efficient at 0.125 s.

Active visual continuous seam tracking based on adaptive feature detection and particle filter tracking

Welding seam tracking based on online programming is the future trend of intelligent production. However, most of the existing image processing methods have certain limitations in the adaptability, accuracy, and robustness of weld feature point detection. The online welding method of gas metal arc welding (GMAW) based on active vision sensing is studied in this paper. The Steger sub-pixel detection method is used to guarantee the accuracy of feature point extraction, and a self-adaptive search window and self-adaptive slope extraction are proposed on this basis. The self-adaptive window is generated according to the linear information of the weld area, and the scale factor and range threshold constraint are added to realize the real-time detection of the weld feature information. Screening the center pixel of the laser stripe in the self-adaptive window of the current frame by the initial slope or the self-adaptive slope of the previous frame, the linear information of the weld area is obtained. The self-adaptive slope of the current frame is fitted by the random sampling consistency method, and the pixel margin is retained to adapt to the linear detection of different continuous welds. When arc light and other serious interference make it difficult to obtain weld information, a particle filter is used to make the best prediction of the weld position. Finally, the welding robot platform based on laser vision sensing was built to test various continuous welds of the butt weld, fillet weld, and lap weld. Experimental results show that the detection speed is 27 ms, and the accuracy of detection and tracking can respectively reach 0.03 mm and 0.78 mm, which meets the requirements of weld detection and tracking.

A comprehensive laser image dataset for real-time measurement of wheelset geometric parameters

Railway transportation has experienced significant growth worldwide, offering numerous benefits to society. Most railway accidents are caused by wheelset faults so it’s significant to monitor wheelset conditions. Therefore, we need to collect wheelset images, repaint them, extract laser stripe centerlines, construct 3D contour, and measure their geometric parameters to judge the wheelset’s conditions. Deep learning can fulfill the tasks satisfyingly because it’s adaptable, robust, and generalize compared with traditional methods. To train the deep learning models effectively, we need rich and high-quality wheelset datasets. So far, there are no applicable public train wheelset datasets available, which greatly hinders the research on train wheelsets. Thus we construct a publicly available Wheelset Laser Image Dataset (WLI-Set). WLI-Set consists of four sub-datasets, Original, Inpainting, Segmentation, and Centerline. The dataset contains abundant annotated multiline laser stripe images that can facilitate the research on train wheelsets effectively.

Crack mitigation strategies for a high-strength Al alloy Al92Ti2Fe2Co2Ni2 fabricated by additive manufacturing

Laser powder bed fusion is capable of fabricating aluminum (Al) alloy parts with great geometrical flexibility and rapid prototyping for various industries. However, high strength Al alloys generally suffer from solidification cracks due to the steep thermal gradient associated with the laser fusion process. Here, we report several strategies to mitigate the hot crack susceptibility of a high strength Al92Ti2Fe2Co2Ni2 alloy. Routine processing parameter optimization based on varying laser power and scanning speed has to trade off porosity for producing crack-free parts, making it not suitable for load-bearing structural applications. Furthermore, secondary printing parameters, including laser strip length, laser defocus, scanning strategies, etc., improved printability but were insufficient to eliminate all the cracks. Crack morphology and residual stress measurements indicate that the cracks are generated in the solid state driven by large tensile residual stress, instead of solidification cracking or liquation cracking. Thus, an attempt was made to alleviate the residual stress in a controlled manner. By properly introducing a compliant, sacrificial, scaffold support structure to regulate crack propagation, near fully dense, crack-free parts could be successfully printed. The results were further verified by micro computed tomography, showing that cracking could be arrested in the support before propagating through the parts. This method can be readily applied to other alloy systems without modifying the chemistry.

Extraction of structured light stripes center from Non-uniform scattering surfaces fused with deep learning

The non-uniform strongly scattering surface enormously influences the measurement accuracy of high-precision structured light Three-dimensional (3D) surface topography. Accurately extracting the central profile of non-uniform highly scattering surfaces is the basis for 3D measurement using point cloud data. In this paper, based on the analysis of the properties of structured light stripes on non-uniform scattering surfaces, a high-precision subpixel laser stripes center robust detection method is proposed. This method first used the Stripe-Segmentor network to accurately segment the Region of Interest (ROI) of the linear structured light stripes, then used the gradient descent method to determine the normal direction of the center of the ROI area light stripes, and finally used the gray-scale centroid method to realize the sub-pixel in the center of the linear structured light stripes precision extraction. Through validation, it is obtained that the laser stripes center extraction algorithm proposed in this study has good accuracy and real-time performance, and has certain engineering value.

Improved YOLO-based laser stripe region extraction method for welding robots

Improved YOLO-based laser stripe region extraction method for welding robots