Laser Vision Sensor Research Articles

Structural Analysis of Thermal Diffusion and Non-Uniform Temperature Distribution along the Sidewall Thickness of STS316L during Gas Tungsten Arc Butt Welding

This study investigated how welding affects the thermal deformation of square cells produced for casks, which are dry storage containers for spent nuclear fuel. We aimed to minimize structural deformation by utilizing STS316L as the material for the square cells. We explored a method of subdividing the square cells and joining them through butt welding. Keeping the upper plate thickness constant, GTA butt welding was conducted while varying the column’s wall thickness, followed by measurement with a laser vision sensor. The heat conduction and thermal strain were then calculated using a finite element analysis (FEM). Both experimental and analytical results confirmed that there was significant thermal deformation in the cases of thick-walled columns due to variations in heat conduction distribution, with the resulting deformation patterns depending on thickness.

A multi-sensor based online monitoring system for laser hot-wire surface cladding process

A multi-sensor online monitoring system, including a high-speed camera, a spectrometer, and an infrared camera, was developed to enhance the laser hot wire cladding process. This innovative process feeds a preheated wire, using a hot-wire power supply, into a shallow molten pool formed on the substrate surface by a rectangular diode laser beam. This study assesses the capability of the monitoring system to detect variations in cladding process conditions that significantly influence the geometry and quality of the clad layer. Specifically, the study involved four sets of laser hot-wire cladding experiments, altering substrate surface conditions, substrate thicknesses, laser powers, and scanning speeds. The real-time data from the three sensors were synchronized and analyzed in depth, focusing on the morphology and microhardness of the clad layers. A laser vision sensor was employed to examine the surface profile and roughness of the clads. The findings indicated that the crucial role of surface preparation in influencing the dynamics of the molten pool, subsequently affecting the geometry, dilution depth, and microhardness of the clads. The sensors demonstrated increased sensitivity to changes in surface conditions compared to variations in other conditions like substrate thickness, laser power, and scanning speed. A significant correlation was found between the dilution depth, clad height, surface roughness, and microhardness of the clad layer. These correlations are in response to adjustments in laser power, scanning speed, and surface preparation and can be reliably and rapidly detected by the multi-sensor system.

Research on welding seam tracking algorithm for automatic welding process of X-shaped tip of concrete piles using laser distance sensor

AbstractThe manufacturing of the X-shaped tip of prestressed centrifugal concrete piles is nowadays done half automatically by combining the manual worker and the automatic welding robot. To make this welding process full automatically, the welding seam tracking algorithm is considered. There are many types of sensors that can be used to detect the welding seam such as vision sensor, laser vision sensor, arc sensor, or touch sensor. Each type of sensor has its advantages and disadvantages. In this paper, an algorithm for welding seam tracking using laser distance sensor is proposed. Firstly, the fundamental mathematics theory of the algorithm is presented. Next, the positioning table system supports the procedure is designed and manufactured. The object of this research is the fillet joint because of the characteristics of the X-shaped tip of the concrete piles. This paper proposes a new method to determine the welding trajectory of the tip using laser distance sensor. After that, the experimental results are received to verify the proposed idea. Finally, the improved proposal of the algorithm is considered to increase the accuracy of the suggested algorithm.

A guidance system for robotic welding based on an improved YOLOv5 algorithm with a RealSense depth camera

Vision-based automatic welding guidance technology plays an essential role in robotic welding. A laser vision sensor (LVS) relies on manual intervention to guide the robot when near the workpiece, which reduces the autonomy of the welding robot and productivity. To solve this problem, a robot welding guidance system based on an improved YOLOv5 algorithm with a RealSense Depth Camera was proposed. A coordinate attention (CA) module was embedded in the original YOLOv5 algorithm to improve the accuracy of weld groove detection. The center of the predicted frame of the weld groove in the pixel plane was combined with the depth information acquired by a RealSense depth camera to calculate the actual position of the weld groove. Subsequently, the robot was guided to approach and move over the workpiece. Then, the LVS was used to guide the welding torch installed at the end of the robot to move along the centerline of the weld groove and complete welding tasks. The feasibility of the proposed method was verified by experiments. The maximum error was 2.9 mm in guiding experiments conducted with a distance of 300 mm between the depth camera and the workpiece. The percentage error was within 2% in guidance experiments conducted with distances from 0.3 to 2 m. The system combines the advantages of the depth camera for accurate positioning within a large field and the LVS for high accuracy. Once the position of the weld groove of the workpiece to be welded has been determined, the LVS combined with the robot can easily track the weld groove and realize the welding operation without manual intervention.

Prediction Model for Tensile Shear Strength of Gas Metal Arc Weld using a Laser Vision Sensor

This study proposes a method to predict tensile shear strength of the overlap welded joint of aluminum alloy (Al5083-O, Al6061-T5) plates applied to the cowl-cross part of a vehicle. The profile of a weld bead was measured using a laser vision sensor, and a technology that can predict tensile shear strength of a welded joint was developed and evaluated. Welded joints were fabricated by using AC pulse GMAW to overlap the configuration of the aluminum alloy plates. The data required for training the prediction model were obtained by measuring the profiles of the welded joints using a laser vision sensor and conducting a tensile test. A CNN-based regression model was thus developed to predict tensile shear strength of welded joints. The model uses a weld?bead profile and material information as input and estimates tensile shear strength of a welded joint as output. The average prediction error of the proposed model was calculated to be approximately 3%.

Adaptive parameter optimization approach for robotic grinding of weld seam based on laser vision sensor

Adaptive parameter optimization approach for robotic grinding of weld seam based on laser vision sensor

Enhancing Robotic-Based Propeller Blade Sharpening Efficiency with a Laser-Vision Sensor and a Force Compliance Mechanism.

The edge sharpness of a propeller blade plays a vital role in improving energy transmission efficiency and reducing the power required to propel the vehicle. However, producing finely sharpened edges through casting is challenging due to the risk of breakage. Additionally, the blade profile of the wax model can deform during drying, making it difficult to achieve the required edge thickness. To automate the sharpening process, we propose an intelligent system consisting of a six-DoF industrial robot and a laser-vision sensor. The system improves machining accuracy through an iterative grinding compensation strategy that eliminates material residuals based on profile data from the vision sensor. An indigenously designed compliance mechanism is employed to enhance the performance of robotic grinding which is actively controlled by an electronic proportional pressure regulator to adjust the contact force and position between the workpiece and abrasive belt. The system's reliability and functionality are validated using three different workpiece models of four-blade propellers, achieving accurate and efficient machining within the required thickness tolerances. The proposed system provides a promising solution for finely sharpened propeller blade edges, addressing challenges associated with the earlier robotic-based grinding studies.

A novel hand-eye calibration method for industrial robot and line laser vision sensor

PurposeThe purpose of this study is solving the hand–eye calibration issue for line structured light vision sensor. Only after hand–eye calibration the sensor measurement data can be applied to robot system.Design/methodology/approachIn this paper, the hand–eye calibration methods are studied, respectively, for eye-in-hand and eye-to-hand. Firstly, the coordinates of the target point in robot system are obtained by tool centre point (TCP), then the robot is controlled to make the sensor measure the target point in multiple poses and the measurement data and pose data are obtained; finally, the sum of squared calibration errors is minimized by the least square method. Furthermore, the missing vector in the process of solving the transformation matrix is obtained by vector operation, and the complete matrix is obtained.FindingsOn this basis, the sensor measurement data can be easily and accurately converted to the robot coordinate system by matrix operation.Originality/valueThis method has no special requirement for robot pose control, and its calibration process is fast and efficient, with high precision and has practical popularized value.

Prediction of Weld Tensile-Shear Strength using ANN Based on the Weld Shape in Aluminum Alloy GMAW

Weld shape and size generally determine the quality of gas metal arc welding. Auto parts manufacturers prescribe the size and shape of the weld because they can indicate the mechanical properties of the weld. It is impossible to evaluate the quality of all welds through destruction inspection. Therefore, research on welding quality inspection using laser vision sensors as a non-destructive inspection method is underway. Although the external profile of the weld can be measured using a laser vision sensor, studies to predict the weld strength are insufficient. In this study, an artificial neural network (ANN) model was developed to predict the welding strength of the lap-fillet weld of an aluminum alloy. Input date for weld size was obtained in two ways. In the first method, a bead profile was acquired using a laser vision sensor, whereas the size of the weld was obtained through the acquired bead profile. In the second method, the size of the weld was obtained directly from cross-section analysis. The output data on the strength of the weld was obtained through a tensile shear test. Two models for predicting the tensile shear strength based on ANN were developed. By predicting the tensile strength of both models, the average error rate was within 10%, but the prediction accuracy using the laser vision sensor was better than that of the cross-sectional method.

Square groove detection based on Förstner with Canny edge operator using laser vision sensor

Square groove detection based on Förstner with Canny edge operator using laser vision sensor

Research on 3D curved weld seam trajectory position and orientation detection method

The "teaching-reproduction" mode and off-line programming mode are widely used in traditional welding robots. For curved welding workpieces, the operators need to visually inspect the position and orientation of the weld seam during processing to select the teaching points or the starting point for programming. The welding accuracy depends on the operators’ experience. Re-teaching or re-programming is required after adjusting the welding position or orientation, resulting in poor welding accuracy and efficiency. To enhance the adaptability of the robot to various welding conditions, we propose a 3D curved weld seam trajectory position and orientation detection method. A local welding workpiece point cloud is obtained by laser vision sensor scanning. The point cloud is smoothed by using the moving least squares (MLS) method. The surface variation of each point is calculated, and the surface variation threshold is set to extract the weld seam feature points. The weld seam orientation is calculated by solving the basic vectors of the spatial curve according to the parametric equation of the weld seam trajectory. The rotation angles of the robot are calculated by the transformation matrix between the weld seam local coordinate system and the robot base coordinate system. According to the experiments, the proposed method can adapt to various orientations of the curved welding workpiece and improve the robot's adaptability to orientations, therefore, the accuracy and efficiency of welding will be enhanced.

A novel visual guidance framework for robotic welding based on binocular cooperation

• An initial welding position guidance framework based on uncalibrated visual servoing and binocular cooperation is studied. The proposed method combines the advantages of passive and active vision sensors, and improves the flexibility of the vision system. • The global camera first controls the robot to approach the workpiece through the proposed EKF based uncalibrated visual servoing. Then, the derived cooperative motion equation will lead the robot towards the IWP. • 2 The features extraction algorithms for realizing UVS are studied. The method to estimate the desired position of welding torch is discussed. And a compression strategy for Yolo-v4 is designed to realize real-time welding torch detection, which can be an optional solution for target detection tasks on computational-resource-limited devices. • The simulation experiments demonstrate the feasibility and stability of the proposed guidance framework. The comparison experiments indicate that the framework can increase the flexibility of the vision system while maintaining satisfactory accuracy. Visual sensor based welding guidance technology has played an essential part in intelligentized robotic welding. However, scanning trajectory dependence of the laser vision sensor (LVS) and low accuracy of the passive vision sensor limit the application of single visual sensor based guidance methods in practical welding guidance. Aiming at this problem, a global camera is introduced to LVS system, and a novel initial welding point (IWP) guidance framework based on binocular cooperation is put forward. The robot is first controlled by the global camera to approach the workpiece through the proposed extended Kalman filter based uncalibrated visual servoing (UVS). Then, the derived cooperative equation of the LVS and the global camera will lead the robot toward the IWP. Subsequently, the feature extraction algorithms for UVS are designed. A compression strategy for Yolo-v4 is present to realize real-time welding torch detection in small computing platforms, and the estimation method of desired features is analyzed. Finally, the feasibility of the proposed algorithm is verified through experiments. The proposed framework combines the advantages of rich information for global camera and high precision for LVS, and could be a feasible solution to realize fully autonomous welding guidance.

Research and development of weld tracking system based on laser vision

Aiming at the shortcomings of low real-time, low applicability, and low welding precision of automatic welding system, a seam tracking system based on laser vision is designed. Use the laser vision sensor to collect the weld image and transmits it to the industrial control computer for processing. Using a median filter to eliminate noise impacts such as arc and splash. Then, this paper focuses on the combination of an improved image threshold segmentation algorithm is used to solve the optimal threshold to obtain the binary image. And the information of laser stripe and the background are separated, overcomes the problems that the researchers have encountered before, such as the unrecognized global optimal solution, and the inaccuracy of the segmentation caused by system jitter. Finally, combined with the improved upper and lower average method, least square method, and Hough transform, the weld feature points are identified and more ideal real-time weld tracking is realized. The experimental results show that the method can accurately track the weld feature points, and improve the detection speed.

Multiple weld seam laser vision recognition method based on the IPCE algorithm

Robotic weldingis a critical technology in manufacturing. Vision sensor is critical for increasing the flexibility and adaptability of robot welding, and the line-structured laser(LSL) visionsensor, in particular, is a long-standing research hotspot in robot intelligent welding.The key technology of the LSL vision sensor used in robot welding is the weld seam recognition algorithm.However, the common weld seam recognition algorithm is hampered by heavyarc and spatter noise during welding. Furthermore, many common weld seam recognition methods can only recognize specifictypes of weld seams and have limitations in terms of flexibility and robustness. To overcome thesechallenges,in this paper, we present a fast, accurate, and robustmultiple-type weld seam recognition algorithm.The algorithm consists primarily of our proposed dynamic ROI method, local adaptive threshold method, internal propulsioncenter extracting(IPCE)algorithm, a weld seam edge point detection operator, and a center analyzing method. Experiment results show that the proposed algorithm can recognize multiple types of weld seams, including I-groove butt weld seams, V-groove butt weld seams, single bevel groove butt weld seams, lap weld joints, etc., even in the presence of heavy arc and spatter noise.

A real-time method for detecting weld deviation of corrugated plate fillet weld by laser vision sensor

In the process of arc welding of the corrugated plate fillet weld, both the modification of weld position and the change of welding torch attitude are required. In general, it is difficult to accurately detect the weld position with the advanced weld detection method, and there is short of real-time in advanced weld detection. In this paper, a unique new sensor and detection method were proposed. The laser vision sensor mainly includes a camera and two circular spot laser light sources. The two laser light sources are separately fixed at two sides of the welding torch clamping piece. The axes of the two laser sources and the axis of the welding torch are parallel and coplanar, and the laser spots on both sides of corrugated plate fillet weld are symmetrically distributed on both sides of welding point. When the welding torch moves horizontally or vertically, the image position of the two laser spots in the video camera will change. Through analysis and calculation, the direction of the welding torch migration was determined, and the offset was calculated to realize the real-time detection of the position of the corrugated plate fillet weld. The experimental results showed that when the welding torch moved ±4.5 mm horizontally or vertically, the detection accuracy of the weld was 0.5 mm.

Welding seam recognition and tracking for a novel mobile welding robot based on multi-layer sensing strategy

That a mobile welding robot with an adaptive seam tracking ability can greatly improve welding efficiency and quality has been extensively studied. To further improve automation in multiple-station welding, we developed a novel intelligent mobile welding robot consisting of a four-wheeled mobile platform and a collaborative manipulator. With the support of simultaneous localization and mapping (SLAM) technology, the robot is capable of automatically navigating to different stations to perform welding operations. To automatically detect the welding seam, a composite sensor system including an RGB-D camera and a laser vision sensor is creatively applied. Based on the sensor system, a multi-layer sensing strategy is performed to ensure that the welding seam can be detected and tracked with high precision. By applying a hybrid filter to the RGB-D camera measurement, the initial welding seam could be effectively extracted. Next, a novel welding start point detection method is proposed. Meanwhile, to guarantee the tracking quality, a robust welding seam tracking algorithm based on laser vision sensor is presented, to eliminate the tracking discrepancy caused by the platform parking error, through which the tracking trajectory can be corrected in real-time. The experimental results show that the robot can autonomously detect and track the welding seam effectively at different welding stations. Also, multiple-station welding efficiency can be improved and quality can also be guaranteed.

An SLAM Algorithm Based on Laser Radar and Vision Fusion with Loop Detection Optimization

SLAM (Simultaneous Localization And Mapping, SLAM) is the key technology for mobile robots to obtain information about their position and surrounding environment, but both laser radar and vision sensors have limitations in solving SLAM problems, for example: a single laser sensor can’t do mapping effectively for hollowed-out obstacles, while a single vision sensor can’t work well in areas without texture. Besides, the existing loop detection technology still has some problems such as large calculation resources and errors. To solve the above problems, an improved SLAM algorithm laser and vision fusion was presented in this paper. In this algorithm, a Gaussian statistical linear interpolation algorithm was used to interpolate the laser point cloud, and then the interpolated laser raster map and visual raster map are fused by the Bayesian method to obtain a denser and more accurate map. Furthermore, the histogram was used to improve the loop detection part to further improve the positioning accuracy. The improved loop detection algorithm was simulated and verified by running the TUM data set. On this basis, a mobile robot experiment system was built to do mapping in the laboratory scene. The experimental results show that the algorithm proposed in this paper can map the hollowed-out area accurately and effectively, it plays a significant role in improving the drawing effect of the robot, and can provide more accurate location information, which has good practicability.

Convolutional Neural Network Model for the Prediction of Back-Bead Occurrence in GMA Root Pass Welding of V-groove Butt Joint

Gas metal arc (GMA) welding is widely used in the machinery industry. The quality of a welded joint is affected by the penetration of root pass welding in the V-groove joint. Automation using GMA welding is continuously required, and root pass welding automation is required to automate the entire welding process. In particular, the development of a prediction model that can ensure full penetration back-bead is required for the automation of root pass welding. In this study, a convolutional neural network (CNN) model was applied to predict the occurrence of back-bead in V-groove butt joint GMA root pass welding. The bead profile was measured using a laser vision sensor system and it was used as the input data for the prediction model, and the bead occurrence was used as the output data for the model. A total of 12,873 bead profiles were extracted and pre-processed through cutting, resizing, and thresholding. The CNN model consists of nine layers, and performs three convolution and two pooling operations. The accuracy of the prediction model was 99.5%, and through this study, it was demonstrated that the quality of root-pass welding can be controlled by using convolutional neural network and it can contribute to automation.

An adaptive orbital system based on laser vision sensor for pipeline GMAW welding

An adaptive orbital system based on laser vision sensor for pipeline GMAW welding

Autonomous Decision-Making of Welding Position During Multipass GMAW With T-Joints: A Bayesian Network Approach

This work investigates a Bayesian network model (BNM) to implement the autonomous decision-making process of welding positions in gas metal arc multipass welding with T-joints for automated manufacturing. The laser vision sensor is used to profile the weld seam, and the weld seam profile (WSP) is extracted with a novel scheme based on scale-invariant feature transform and orientation feature detection. The feature points of the extracted WSP are effectively identified through slope mutation detection. A BNM is built with these points and the determined welding state online, and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">priori</i> probabilities of the model variables are acquired in a computational manner integrated with welding experience. The feature point with the maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">posteriori</i> probability is selected as the current welding position using the evidence prepropagation importance sampling inference algorithm. The analytic hierarchy process and the C4.5 decision tree algorithm are used to compare with the proposed BNM regarding decision effectiveness. Experimental results show that the proposed BNM can give the effective decision-making results of welding positions with T-joints of different thicknesses and show great potential for higher manufacturing efficiency and automatic levels.