Weld Deviation Research Articles

Intelligent seam tracking in foils joining based on spatial–temporal deep learning from molten pool serial images

Vision-based weld seam tracking has become one of the key technologies to realize intelligent robotic welding, and weld deviation detection is an essential step. However, accurate and robust detection of weld deviations during the microwelding of ultrathin metal foils remains a significant challenge. This challenge can be attributed to the fusion zone at the mesoscopic scale and the complex time-varying interference (pulsed arcs and reflected light from the workpiece surface). In this paper, an intelligent seam tracking approach for foils joining based on spatial–temporal deep learning from molten pool serial images is proposed. More specifically, a microscopic passive vision sensor is designed to capture molten pool and seam trajectory images under pulsed arc lights. A 3D convolutional neural network (3DCNN) and long short-term memory (LSTM)-based welding torch offset prediction network (WTOP-net) is established to implement highly accurate deviation prediction by capturing long-term dependence of spatial–temporal features. Then, expert knowledge is further incorporated into the spatio-temporal features to improve the robustness of the model. In addition, the slime mould algorithm (SMA) is used to prevent local optima and improve accuracy, efficiency of WTOP-net. The experimental results indicate that the maximum error detected by our method fluctuates within ± 0.08 mm and the average error is within ± 0.011 mm when joining two 0.12 mm thickness stainless steel diaphragms. The proposed approach provides a basis for automated robotic seam tracking and intelligent precision manufacturing of ultrathin sheets welded components in aerospace and other fields.

Vision-based algorithm for online TIG welding deviation detection of stainless steel welded pipes

Tungsten inert gas (TIG) welding is the main welding process in the production of stainless steel welded pipe. According to the morphological characteristics of the welding molten pool image during the TIG welding process of stainless steel welded pipes, the exact position of the tungsten needle tip is calculated using image moments. Extract the weld region in the contour of the molten pool, interpolate the contour curve based on the cubic B-spline curve interpolation method, utilize the characteristics of the S-G filter, remove the interference coordinates in the contour curve through the detrending of the contour curve, extract the weld feature points, and realize the accurate identification of weld seams. The experimental results show that the method can accurately calculate the welding deviation in the welding process.

Online identification of burn-through and weld deviation in sheet lap MIG welding based on YOLOv5

Online monitoring of the welding process is very important to the development of the modern manufacturing industry. However, monitoring welding defects in real-time and with high accuracy is challenging because welding is inherently a dynamic, non-linear process. In this paper, the forming defects generated during the melt Inert-gas welding of sheet steel as the object of research. We propose a multi-task simultaneous monitoring system for welding defects based on the YOLOv5 model, which can achieve highly accurate simultaneous detection of burn-through and weld deviation defects. Firstly, a passive vision system has been designed that can filter out strong arc light interference. In combination with a self-made camera bracket with flexible adjustment and repeatable positioning, clear images of the molten pool can be acquired. Compared to common methods that require a laser for weld deviation detection, all monitoring tasks can be performed with just one charge coupled device camera, significantly reducing the cost of deploying the system. Secondly, to obtain experimental data closer to the industrial field environment. We simulated different degrees of local and continuous burn-through by changing three parameters: welding current, plate thickness, and lap width. Four different deviations: left, right, and tilt were designed to obtain rich weld deviation image data. Finally, based on the above data, we compared several aspects such as detection speed, recognition accuracy, and loss function. The YOLOv5s model with the smallest model parameters was finally selected as the base model. The results show that our proposed multi-task simultaneous monitoring system achieves an average accuracy of 98.84% for the identification of four molten pool states. Burn-through defect and weld deviation detection with more than 50 frames per second can be achieved in the online detection state. This paper can provide some guidance for the online monitoring of the welding manufacturing process.

Variation Propagation Analysis of Hull Flat Block Construction Process Based on State Space Model

_ Hull flat block (HFB) construction is a typical discrete manufacturing process, during which block deviation is accumulated. To satisfy the dimensional requirements of shipowners, the variation propagation during HFB construction must be modeled and analyzed. Based on the stream of variation theory, the construction process is analyzed to provide a reference for controlling the deviation. In the hull construction process, the deviation is categorized into part deviation, location deviation, and welding deviation. By analyzing the causes of the different deviations, the different deviations and their accumulations can be calculated. A HFB is used to verify the proposed model, and the results show that the method can be used to calculate the deviation to the HFB. Introduction As a typical large-scale complex equipment, the hull construction process is extremely time-consuming and laborious. The final hull is composed of intermediate products, such as subassemblies, unit assemblies, and grand-assemblies (Cho et al. 1999). Since the group technology was applied in hull construction, a series of production lines has been established based on the process similarity of parts, components, blocks, and grand blocks. The actual dimension of parts would inevitably deviate from the theoretical dimension (Mandal 2017), i.e., dimensional and shape deviations will occur owing to many factors (Okumoto 2002). In the hull construction process, the deviation from the previous process will be propagated to the current process and coupled with the deviation in the current process, thereby affecting the accuracy of the product. Hence, the product must be repaired to satisfy the design requirements (Takechi et al. 1998). This will prolong the production cycle and reduce the production efficiency (Heo et al. 2015). It was discovered that the man-hours of the main operations during hull assembly constituted only one-sixth of the total assembly manhours, whereas the man-hours used for adjustment constituted one-third (Chen et al. 2020). Currently, shipyards need to eliminate the adjustment operation to avoid unnecessary rework, as well as reduce human and material resources in subsequent processes. Tanigawa (2003) estimated that an ideal ship construction accuracy control plan can reduce the production cost by 5%. Therefore, the accuracy of the hull construction process must be controlled and improved in shipyards, and a reasonable accuracy control plan should be devised to improve the product quality.

Welding seam detection and location: Deep learning network-based approach

There are many kinds of surface defects, large difference in size and strong randomness in the process of material welding, which makes it difficult for some conventional target detection algorithms to quickly and accurately locate and identify. This paper works out an improved Faster R–CNN network model. By adding FPN pyramid structure, variable convolution network and background suppression function to the traditional Faster R–CNN network, the path between the lower layer and the upper layer is greatly shortened, and the location information is better preserved. The simulation test shows that the model above improves the detection accuracy of the network, shortens the detection time, and reduces the loss value. The mAP is increased by 0.243, and the detection time of a single picture is shortened by 0.019s. The detection accuracy of five weld defects is significantly improved, and the accuracy of weld deviation is improved by 0.234, which is the one that improved the most significantly. It shows that the improved model is effective and has good convergence, and can meet the requirements of the detection of weld defects in the actual production process.

Intelligent Seam Tracking of an Ultranarrow Gap During K-TIG Welding: A Hybrid CNN and Adaptive ROI Operation Algorithm

Deep-penetration keyhole tungsten inert gas (K-TIG) welding uses a large current for welding. The extraction of the deviation between the position of the welding torch and the centerline of the weld is the prerequisite for the realization of seam tracking. However, K-TIG welded workpieces are often assembled without a groove and the gap between each weld is super narrow (0.2–1 mm), making it difficult to accurately identify the seam. In addition, traditional methods of identifying seams based on a fixed region of interest (ROI) are likely to lead to seam tracking failures when the type of weld changes. A narrow gap seam tracking algorithm suitable for different weld shapes is developed to address these factors and increase the robustness. To minimize the interference caused by strong arc lights, the welding images are captured by an high dynamic range (HDR) camera. A you only look once (YOLOv5)-based object detection algorithm is used to determine the position of the torch by the center of the keyhole entrance. Then, an image processing algorithm based on the adaptive ROI operation is used to adaptively extract the weld centerline and obtain the weld deviations. The experimental results proved that the welding deviation detected by the algorithm fluctuated within ±0.122 mm and that the average error was within ±0.043 mm. Moreover, the real-time performance of the algorithm can reach up to 45 frames per second (FPS), which is sufficient to meet the real-time and accuracy requirements of the K-TIG seam tracking system.

Infrared Visual Sensing Detection Approach of Swing Arc Narrow Gap Weld Deviation Based on Outlier Data Filtering

Narrow gap arc welding interferences pose notable challenges to accurate weld deviation detection in infrared visual sensing. An outlier data filtering (ODF) approach is proposed to precisely detect weld deviation between the groove and the torch centers after eliminating the welding-induced intense disturbances in images. The ODF approach develops a global pattern recognition (GPR) based algorithm to detect the groove center, and an optimal-estimation based position consistency filtering (PCF) algorithm to locate the torch center. In addition, ODF also modifies a welding image processing procedure to prepare the groove and the wire edges for the center detections. Experimental results show that the GPR algorithm removes the simulated welding spatters of ≤ 2.48 mm diameters on the groove edge, and the PCF algorithm accurately gains the torch center with the detection error of−0.022 ~ +0.035 mm. Finally, the ODF approach realizes a precise detection of the weld deviation with the detection error of−0.107 ~ +0.079 mm, demonstrating the effectiveness of the proposed approach regardless of welding interferences.

A real-time method for detecting weld deviation of corrugated plate fillet weld by laser vision sensor

In the process of arc welding of the corrugated plate fillet weld, both the modification of weld position and the change of welding torch attitude are required. In general, it is difficult to accurately detect the weld position with the advanced weld detection method, and there is short of real-time in advanced weld detection. In this paper, a unique new sensor and detection method were proposed. The laser vision sensor mainly includes a camera and two circular spot laser light sources. The two laser light sources are separately fixed at two sides of the welding torch clamping piece. The axes of the two laser sources and the axis of the welding torch are parallel and coplanar, and the laser spots on both sides of corrugated plate fillet weld are symmetrically distributed on both sides of welding point. When the welding torch moves horizontally or vertically, the image position of the two laser spots in the video camera will change. Through analysis and calculation, the direction of the welding torch migration was determined, and the offset was calculated to realize the real-time detection of the position of the corrugated plate fillet weld. The experimental results showed that when the welding torch moved ±4.5 mm horizontally or vertically, the detection accuracy of the weld was 0.5 mm.

Online Welding Deviation Detection and Burn-Through Identification of Yolov5 Sheet Lap Mig Welding Based on Passive Vision

Online Welding Deviation Detection and Burn-Through Identification of Yolov5 Sheet Lap Mig Welding Based on Passive Vision

Narrow gap deviation detection in Keyhole TIG welding using image processing method based on Mask-RCNN model

In the process of K-TIG deep penetration welding, the workpiece does not need to be bevelled; therefore, the welding method is butt welding, and the gap to be welded is very narrow (0.2~1 mm). Because of the large welding current, the welding arc light intensity is very strong. These factors cause difficulties in the K-TIG weld seam tracking process. To realize weld seam tracking in the K-TIG welding process, it is necessary to extract the keyhole entrance centre and weld centreline accurately. To reduce the interference of strong arc light in the process of K-TIG welding, the information of keyhole entrance and weld seam in the process of K-TIG welding is obtained by using a high-dynamic-range camera. An image processing algorithm based on Mask-RCNN is proposed to extract the centre of the keyhole entrance accurately. An image processing algorithm based on Hough line fitting is used to accurately identify the weld centreline in the welding image and extract the welding deviation. In welding experiments, it is verified that the welding deviation extracted by the method proposed in this paper fluctuates within ± 0.133 mm, which meets the requirements of actual K-TIG welding seam tracking.

A Fast and Robust Seam Tracking Method for Spatial Circular Weld Based on Laser Visual Sensor

Spatial circular welding is an important application in industrial production. However, some feature points extraction methods for spatial circular welds have disadvantages such as large amounts of calculation and poor anti-interference ability. In addition, due to the welding thermal deformation, the spatial circular weld needs to be tracked. Therefore, a fast and robust seam tracking method for spatial circular weld based on laser visual sensors is proposed in this article. First of all, a self-updating template matching (SUTM) method is proposed to get weld feature points. Compared with the method based on geometric features, the proposed SUTM method has the advantages of rapidity and robustness. What is more, an algorithm for spatial circle center fitting based on random sampling consensus (RANSAC) is proposed, which overcomes the interference of outlier points and improves the robustness of circle center detection. At the same time, the tracking error model of the spatial circular weld is established and the welding deviation from the $x$ -, $y$ -, and $z$ -directions is obtained. By correcting the deviation in real-time, the fast and robust real-time seam tracking of the spatial circular weld is realized. Through welding test, the proposed method can obtain the position of the spatial fillet circular weld quickly and robustly and complete the high-precision seam tracking.

Research on the seam tracking of narrow gap P-GMAW based on arc sound sensing

Arc sensor is important in the automation of narrow gap welding process because only the arc signal can reflect the position of the arc in the groove in real time, as well as the heat input to the side wall. Except the current and voltage of the arc, the deviation state of the welding arc can be identified by experienced welders by the associated arc sound, therefore, the arc sound signal presents a potential welding seam tracking signal. In this study, arc sounds were acquired and processed to examine the correlation with weld deviation during narrow gap pulsed gas metal arc welding (P-GMAW). The influences of groove sidewall on the arc sound signal and the sensitive of arc sound features to welding torch deviation were studied, and the short-time energy was determined as the best parameter to reflect the arc deviation in the narrow gap grooves. Furthermore, a system was established for seam tracking using the arc sound short-time energy and the results show that the arc sound can be used as the control signal to realize seam tracking to a certain degree of precision, thus, providing a new control signal for narrow gap P-GMAW welding seam tracking based on multi-information fusion.

Identification of the deviation of seam tracking and weld cross type for the derusting of ship hulls using a wall-climbing robot based on three-line laser structural light

To realize fully automatic rust removal from the large splicing weld of a ship hull section, we proposed a laser vision identification method based on three-line laser structural light. First, we calibrated the three-line laser vision sensor and obtained the internal and external parameters of the sensor and the laser plane equation of the three-line stripes. On this basis, to accurately obtain the feature points of the weld, we proposed a stepwise piecewise segmenting method for the laser stripe to approximate the true weld curve near the feature point, and we used a Freeman chain code method to mark the weld feature line to eliminate the abnormal points of the laser stripe. Furthermore, for the vertical and horizontal intersections of large splicing welds, we established a uniform recognition model of the cross-shaped or T-shaped weld based on the height characteristics of the three laser stripes and the distances between adjacent laser stripes. The experimental verification showed that the three-line laser method has better anti-interference ability and more abundant information, and it can accurately calculate the weld deviation and identify the cross-shaped or T-shaped weld types under various working conditions, which laid the foundation for real-time seam tracking and trajectory planning.

Study on Arc Welding Seam Tracking of TIG Welding

In this paper, the welding seam tracking system based on rotating arc TIG welding is studied. The design of pid controller for the hardware, software, signal acquisition, filtering and weld deviation identification of the system is deeply studied. And the conclusion is obtained, which lays a foundation for the tracking of rotating arc weld in TIG welding.

Visual Neural Network Model for Welding Deviation Prediction Based on Weld Pool Centroid

To solve the problem in the process of weld seam tracking, a new prediction model for welding deviation based on the weld pool image centroid has been proposed in the paper. First, some weld images under different weld currents were captured by a vision sensor. A composite filter system, which is composed of narrow-band and neutral filters, is used to reduce the disturbance of weld arc. So, several clear weld pool images can be obtained. Then a frontier of weld pool is chosen to be the processing region. Median filter and gray transformation operations are used to enhance the contrast of processing region. On this basis, the variation trend of centroid difference [Formula: see text] and welding deviation [Formula: see text] were analyzed. The centroid difference value [Formula: see text] and the weld current [Formula: see text] were determined to be welding status parameters. Moreover, a BP neural network was set up, which was composed of three layers. Next, elastic gradient descent method was used to be the training function. So a prediction model between the welding status parameters [Formula: see text] and [Formula: see text] and the welding deviation [Formula: see text] was set up. In the end of the paper, several experiments were performed to test the accuracy of the setup prediction model. The results showed that prediction values of welding deviation calculated by the vision model are fit to the real measured values. The final errors of the vision model under the weld current 70[Formula: see text]A and 73[Formula: see text]A were 0.033[Formula: see text]mm and 0.027[Formula: see text]mm, which showed excellent accuracy, environmental suitability and intelligence of the model.

An infrared visual sensing detection approach for swing arc narrow gap weld deviation

The present work develops a weld deviation detection system based on infrared visual sensing for swing arc narrow gap weld tracking, proposes a local pattern recognition (LPR) algorithm to detect groove edge position, and experimentally investigates their adaptabilities. This approach captures welding image synchronously with the base current period of pulsed arc as the arc swings to stay at groove sidewall, and extracts weld deviation feature from the relative distances of electrode wire center to the opposite sidewall of arc staying position, thus reducing arc light disturbance. The LPR algorithm seeks the straightest line segment by finding the minimum standard variance of groove edge pixel distribution in moving LPR window, and can adaptively acquire sidewall edge true position from the straightest segment regardless of size-varying spatters. It is shown that this approach is obviously superior to conventional mean algorithm in the precision and efficiency of weld deviation detection.

A novel dual-channel weld seam tracking system for aircraft T-joint welds

Laser welding technology for skin-stringer joints is widely used in aircraft manufacturing. Double-sided laser beam welding of skin-stringer joints is an approved method for producing defect-free welds. The main drawback for the use of single-sided welding method for T-joints is the occurrence of weld defects. To achieve dual-beam laser-welded T-joint, a dual-beam laser welding platform based on multi-axis CNC machine is set up in this paper. A novel control method of double-sided seam tracking and error compensation based on vision sensor is proposed. The weld seams on both sides of the T-joint are detected simultaneously by two vision sensors. At the same time, the welding deviation on both sides of the T-joint are compensated at real time by an independent error compensation control method based on self-adaptive fuzzy-PID controller. The result shows that the dual-beam laser welding system can achieve high precision dynamic deviation compensation in the double-sided welding process at a high speed.

Weld deviation detection based on wide dynamic range vision sensor in MAG welding process

Weld deviation detection is a precondition for welding automation. Capturing high-quality welding images and extracting deviation information using image processing methods are two steps for weld deviation detection. In this paper, based on the analysis of the imaging characteristics during metal active gas (MAG) welding process, real-time welding images are acquired clearly and steadily using a wide dynamic range vision sensor. According to the connection relationship between the top of the arc and the end of the wire during MAG welding process, a method for determining the wire centreline is proposed. After extracting the precise arc region, the region of interest (ROI) is segmented along the welding direction. To detect the two edges of the V-groove, an improved Canny algorithm is developed. Furthermore, both the Hough Transform and a screening method based on prior knowledge are used to connect V-groove edges. The V-groove centreline is then determined. As a result, the weld deviation between the wire centreline and the V-groove centreline is calculated. Further experiments showed that the precision range of deviation detection can be controlled within ±0.25 mm, which can meet the requirement of real-time welding. This deviation can be used as the input variable for a welding robot, thus laying the foundation for MAG welding automation.

Welding deviation detection algorithm based on extremum of molten pool image contour

The welding deviation detection is the basis of robotic tracking welding, but the on-line real-time measurement of welding deviation is still not well solved by the existing methods. There is plenty of information in the gas metal arc welding(GMAW) molten pool images that is very important for the control of welding seam tracking. The physical meaning for the curvature extremum of molten pool contour is revealed by researching the molten pool images, that is, the deviation information points of welding wire center and the molten tip center are the maxima and the local maxima of the contour curvature, and the horizontal welding deviation is the position difference of these two extremum points. A new method of weld deviation detection is presented, including the process of preprocessing molten pool images, extracting and segmenting the contours, obtaining the contour extremum points, and calculating the welding deviation, etc. Extracting the contours is the premise, segmenting the contour lines is the foundation, and obtaining the contour extremum points is the key. The contour images can be extracted with the method of discrete dyadic wavelet transform, which is divided into two sub contours including welding wire and molten tip separately. The curvature value of each point of the two sub contour lines is calculated based on the approximate curvature formula of multi-points for plane curve, and the two points of the curvature extremum are the characteristics needed for the welding deviation calculation. The results of the tests and analyses show that the maximum error of the obtained on-line welding deviation is 2 pixels(0.16 mm), and the algorithm is stable enough to meet the requirements of the pipeline in real-time control at a speed of less than 500 mm/min. The method can be applied to the on-line automatic welding deviation detection.

Rough set based modeling for welding groove bottom state in narrow gap MAG welding

Purpose – This paper aim to build an information fusion model that can predict the bottom shape of welding groove for better welding quality control. Arc sensor is widely used in seam tracking due to its simplicity and good accessibility, but it heavily relies on the bottom shape of the groove. It is necessary to identify the welding groove bottom state. Therefore, arc sensor information and vision sensing information were fused by the rough set (RS) method to predict the groove state, which will lay the foundation for better welding quality control. Design/methodology/approach – First, a multi-sensor information system was established, which included an arc sensing component and a vision sensing component. For the arc sensing system, the current waveform in each rotating period was obtained and divided into 12 parts to calculate variables representing the variation of arc length. For the vision sensing system, images were obtained by passive vision when the arc was near the groove sidewall. The positions of the sidewall and the arc were calculated to get the weld deviation which was unrelated with the bottom groove state. Second, experimental data were generated by workpiece with various bottom shapes. At last, the RS method was adopted to fuse the arc sensing and the vision information, and a rule-based model with good prediction ability was obtained. Findings – By fusing arc sensing and vision sensing information, an RS-based model was built to predict the welding groove state. Originality/value – The RS modeling method was used to fuse arc sensing information and vision sensing information to build a model that predicts groove bottom state. The arc sensing information represented the arc length variation, while the vision sensing information contains the seam deviation which was unrelated with the bottom groove state. The RS model gives satisfactory prediction results and can be applied to weld quality control.