Monitoring Of Welding Research Articles

Investigating the ability of deep learning to predict welding depth and pore volume in hairpin welding

To advance quality assurance in the welding process, this study presents a deep learning (DL) model that enables the prediction of two critical welds’ key performance characteristics (KPCs): welding depth and average pore volume. In the proposed approach, a wide range of laser welding key input characteristics (KICs) is utilized, including welding beam geometries, welding feed rates, path repetitions for weld beam geometries, and bright light weld ratios for all paths, all of which were obtained from hairpin welding experiments. Two DL networks are employed with multiple hidden dense layers and linear activation functions to investigate the capabilities of deep neural networks in capturing the complex nonlinear relationships between the welding input and output variables (KPCs and KICs). Applying DL networks to the small numerical experimental hairpin welding dataset has shown promising results, achieving mean absolute error values of 0.1079 for predicting welding depth and 0.0641 for average pore volume. This, in turn, promises significant advantages in controlling welding outcomes, moving beyond the current trend of relying only on defect classification in weld monitoring to capture the correlation between the weld parameters and weld geometries.

Penetration State Recognition during Laser Welding Process Control Based on Two-Stage Temporal Convolutional Networks.

Vision-based laser penetration control has become an important research area in the field of welding quality control. Due to the complexity and large number of parameters in the monitoring model, control of the welding process based on deep learning and the reliance on long-term information for penetration identification are challenges. In this study, a penetration recognition method based on a two-stage temporal convolutional network is proposed to realize the online process control of laser welding. In this paper, a coaxial vision welding monitoring system is built. A lightweight segmentation model, based on channel pruning, is proposed to extract the key features of the molten pool and the keyhole from the clear molten pool keyhole image. Using these molten pool and keyhole features, a temporal convolutional network based on attention mechanism is established. The recognition method can effectively predict the laser welding penetration state, which depends on long-term information. In addition, the penetration identification experiment and closed-loop control experiment of unequal thickness plates are designed. The proposed method in this study has an accuracy of 98.96% and an average inference speed of 20.4 ms. The experimental results demonstrate that the proposed method exhibits significant performance in recognizing the penetration state from long sequences of welding image signals, adjusting welding power, and stabilizing welding quality.

In-situ monitoring method of femtosecond laser welding between glass and copper with acoustic emission

The use of femtosecond laser welding has created new possibilities for connecting glasses and metals, revolutionizing the fabrication and assembly of advanced electronic components. However, monitoring the connection process during femtosecond laser welding is challenging due to the small heat-affected zone and brief interaction time. This study utilizes acoustic emission to monitor femtosecond laser welding and introduces the “Sum of Relative RMS” feature to correlate connection formation with acoustic emission signals in glass-copper welding. The mechanism and signal occurrence trends during welding are explained through in-situ signal monitoring and offline scanning electron microscopy characterization. Additionally, a Convolutional Neural Network is enhanced by increasing the convolutional kernel size, integrating multi-dimensional features, and incorporating prior knowledge via a customized loss function, resulting in an increase in the accuracy of successful welding detection from 89% to 95%. This research provides new insights into the monitoring of femtosecond laser glass-metal heterogeneous welding.

Machine learning-based in-process monitoring for laser deep penetration welding: A survey

In-process monitoring (IPM) of laser deep penetration welding (LDPW) has witnessed a rapid growth in approaches that embrace machine learning algorithms, utilizing raw sensor input to generate various weld quality evaluations, instead of concentrating on thermomechanical modeling that is hypotheses-driven and hence biased by it. Benefitting from the capability to unravel hidden interactions in the complex laser welding process, numerous data-driven IPM methods have been proposed to address different problems in this area. In this survey, we present a comprehensive analysis of both classical and recent studies, covering the unique physical mechanisms, sensing techniques, methodologies, strengths, and limitations of machine learning-based IPM-LDPW systems. We delve into several critical tasks, including mechanical performance prediction, weld penetration estimation, and weld defects detection. Meanwhile, we explore the latest developments in deep learning and how to incorporate these techniques into IPM systems for LDPW. To inspire future research, we outline unresolved challenges and explore potential opportunities and new perspectives for addressing these challenges.

Laser welding monitoring techniques based on optical diagnosis and artificial intelligence: a review

Laser welding monitoring techniques based on optical diagnosis and artificial intelligence: a review

AF-FTTSnet: An end-to-end two-stream convolutional neural network for online quality monitoring of robotic welding

Online welding quality monitoring (WQM) is crucial for intelligent welding, and deep learning approaches considering spatiotemporal features for WQM tasks show great potential. However, one of the important challenges for existing approaches is to balance the spatiotemporal representation learning capability and computational efficiency, which makes it challenging to adapt welding processes with complex and drastic molten pool dynamic behavior. This paper proposes a novel approach for WQM using molten pool visual sensing and deep learning considering spatiotemporal features, the proposed deep learning network called attention fusion based frame-temporality two-stream network (AF-FTTSnet). Firstly, a passive vision sensor is used to acquire continuous dynamic molten pool images. Meanwhile, temporal difference images are computed to provide novel features and temporal representations. Then, a two-stream feature extraction module is designed to concurrently extract rich spatiotemporal features from molten pool images and temporal difference images. Finally, an attention fusion module with the ability to automatically identify and weight the most relevant features is designed to achieve optimal fusion of the two-stream features. The shop welding experimental results indicate that the proposed AF-FTTSnet model can effectively and robustly recognize five typical welding states during helium arc welding, with an accuracy of 99.26%. This model has been demonstrated to exhibit significant performance improvements compared to mainstream temporal sequence models. Available: https://github.com/Just199806/TSCNN/tree/master.

Spectrometer as a quantitative sensor for predicting the weld depth in laser welding

Spectrometers have been used for quality assessment or as auxiliary sensors in multisensor systems to record emissions at different wavelengths. This study developed a single-spectrometer sensor to quantitatively predict the laser penetration depth in Cu/Al overlap and dual-phase (DP) steel bead-on-plate welding joints. The spectrometer signals were sampled at 100 Hz, and an optical coherence tomography (OCT) sensor was used to estimate the in situ penetration depth. A one-dimensional (1D) convolution neural network (CNN) model was developed using the current step of the spectrometer signal. Furthermore, a two-dimensional (2D) CNN model was developed using 10 recent steps of the spectrometer signals. Consequently, 1732 and 3000 data points for DP steel and Al/Cu overlap welding were used for the penetration-depth modeling. Through model training, mean absolute errors of 22.67 and 52.96 µm were achieved for the Al/Cu overlap and DP steel welding, respectively, corresponding to approximately 1.5 % of the substrate thickness. The 2D-CNN models outperformed the 1D-CNN models for both material combinations. However, a slight delay was observed in the response of the models when confronted with abrupt output changes. This study confirmed that spectroscopy can serve as the sole quantitative sensing method for laser welding monitoring and control.

Inline failure detection in laser beam welding of battery cells: Acoustic and spectral emission analysis for quality monitoring

The mobility sector is considered a major contributor to global greenhouse gas emissions and air pollution. As a result, many countries have initiated the transition from fossil fuel-powered to electrified powertrains. This transformation of the powertrain concept will lead to a rapid increase in the production of electric vehicles and, therefore, to a high demand for so-called traction batteries. As a production step of the traction batteries, a connection between the cell connector and the terminal of the battery cell has to be manufactured. For this purpose, laser beam welding is a reliable and efficient joining technique. In order to ensure continuous quality of the welding process during production and to detect defects in real time, reliable process monitoring is required. In this study, spectral and acoustic emissions during laser beam welding were recorded using a laser welding monitor and an optical microphone. For determining possible correlations between the signals and weld defects, various failure cases were generated by the systematic placement of disturbance elements. These elements included a contaminated surface, a gap between the cell connector and the battery cell, and a misalignment of the cell connector. Based on the recorded signals, statistical metrics were calculated. Finally, weld seams with and without defects were compared to assess the capability of both sensor systems for detecting the weld defects.

Review on Laser Welding Technology on Aluminum and Copper Dissimilar Materials for Secondary Batteries Assembly

Electric vehicles have recently taken center stage, driven by growing environmental concerns, with a key focus on ensuring the reliability of their batteries. The materials used in the electrical connections of secondary batteries predominantly consist of Al and Cu alloys, necessitating the joining of both similar and dissimilar materials. This paper presents a technical review on laser welding technology applied to the challenging Al/Cu dissimilar metal combination. The complexity arises from the formation of brittle phases in intermetallic compounds and the low absorption of laser beams by the base materials. The characteristics of various laser welding modes and their effective monitoring through deep learning are explained in detail. The development of real-time laser welding monitoring technology is crucial to increase productivity and enable the mass production of reliable electric vehicle battery systems.

Laser welding monitoring with multisensory data fusion: A brief review

As digital manufacturing is being implemented across industries, the automation of the laser welding process is a crucial step to enhance production efficiency. To monitor the in-situ welding process, there are several approaches to detect the electromagnetic and mechanical waves on various frequencies for comprehending laser beam-material interaction. Five sensing techniques, namely the optical microphone, welding camera, inline coherent imaging, infrared camera, and heat flux sensor, can be employed to identify distinct features in the laser welding process. These features include pore formation, melt pool geometry, weld bead topography, keyhole depth, and thermal distribution. The discussion of a proposed welding system designed with compatibility for multisensory data fusion is included, both on its capabilities and potential challenges, to offer guidance of welding monitoring.

One camera-based laser keyhole welding monitoring system using deep learning

The laser-beam absorptance changes dynamically during laser keyhole welding due to unstable keyhole movements, and monitoring the absorptance can provide a deep understanding of the process. Recently, Kim et al. [1,2] developed a deep-learning-based method to monitor the absorptance by detecting the top and bottom keyhole apertures and estimating the absorptance from the reconstructed keyhole shape based on the detected apertures. However, this method was limited in that it required simultaneously observing the top and bottom keyhole apertures using two cameras. In this study, we proposed a novel deep-learning-based method to monitor the laser-beam absorptance in a keyhole using only one camera during laser keyhole welding of Al 5052-H32 alloy. In this method, both the top and bottom keyhole apertures were simultaneously detected from the images coaxially obtained from the top side. Although part of the bottom apertures may be sometimes obscured when viewed from above, this study demonstrated that the predicted absorptance was accurate enough and sufficient for monitoring laser welding processes of aluminum alloys. Using the developed method, changes in welding mode and generation of welding defects were successfully detected.

Development of an Arc Image Classification Algorithm in Tip-Rotating Arc Welding using Camera Image Monitoring and Deep Learning

In this study, we developed a welding system and welding monitoring system for GMA (Gas Metal Arc) aluminum welding using a tip-rotation arc welding torch. Welding was performed under various conditions, including suitable, low-heat, and high-heat input conditions. For welding monitoring, current and voltage were measured, and arc images were obtained. After the experiments, arc images were analyzed and classified based on the bead quality and cross-sectional analysis data. To develop a welding quality prediction model using arc images, a CNN-based model was developed. A total of 5,203 images were trained, and the quality was predicted for 631 images. The prediction accuracy was 99.88% for training data, and the possibility of assessing the quality of welded joints based on arc images during welding was confirmed.

EXPERIMENTAL REMOTE DETERMINATION OF THE STATIC CHARACTERISTIC OF THE ARC IN TIG WELDING

In the conditions of TIG welding, the magnitude of the current is set, but not the voltage of the arc. To determine the arc voltage at a set current, arc length, type of electrode, sharpening angle and others, it is necessary to know the static characteristics of the welding arc. This dependence facilitates the use of simulation modelling methods of welding processes. In the proposed study, the static characteristics of the welding arc in TIG process with 4 types of electrodes were determined experimentally. The magnitude of the current was set in the range of 20 to 200A and the arc voltage was measured. All parameters are measured remotely by a real-time welding monitoring system and transferred to a remote computer via the Internet. Graphs were built based on the recordings, and the numerical results were processed and regression relationships were obtained.

Online visual monitoring method for liquid rocket engine nozzle welding based on a multi-task deep learning model

Online quality monitoring is important in liquid rocket engine nozzle (LREN) welding procedures. Convolutional neural networks (CNNs) are widely applied in welding quality monitoring. However, most CNNs perform single visual tasks and provide limited information. They are not competent in LREN welding quality monitoring. This paper presents an online visual sensing method for LREN welding quality monitoring based on a multi-task deep learning model. Weld-detection-segmentation network (WDS-net), a multi-task CNN for simultaneous detection and segmentation is proposed for weld defect detection and weld width measurement. WDS-net obtains more information from welding images and depicts the welding process more adequately compared with single-task models. A machine vision system is constructed to monitor the LREN welding process. Then, experiments are conducted to obtain LREN welding images and construct a dataset. WDS-net is trained and tested on the dataset and obtains a 95.85% mean average precision (mAP) for weld defect detection and a 92.63% intersection over union (IoU) for weld zone segmentation. Finally, WDS-net is tested on LREN welding image sequences, and the results are compared with the offline measurement results of corresponding welds. WDS-net detects, locates the defects without omission, and measures weld width with a mean error within 0.1 mm compared with the actual weld. WDS-net’s monitoring performance and its average inference rate of 147 frames per second meet the requirements of online monitoring of LREN welding.

Robust and Secure Quality Monitoring for Welding through Platform-as-a-Service: A Resistance and Submerged Arc Welding Study

For smart manufacturing systems, quality monitoring of welding has already started to shift from empirical modeling to knowledge integration directly from the captured data by utilizing one of the most promising Industry 4.0’s key enabling technologies, artificial intelligence (AI)/machine learning (ML). However, beyond the advantages that they bring, AI/ML introduces new types of security threats, which are related to their very nature and eventually, they will pose real threats to the production cost and quality of products. These types of security threats, such as adversarial attacks, are causing the targeted AI system to produce incorrect or malicious outputs. This may undermine the performance (and the efficiency) of the quality monitoring systems. Herein, a software platform servicing quality monitoring for welding is presented in the context of resistance and submerged arc welding. The hosted ML classification models that are trained to perform quality monitoring are subjected to two different types of untargeted, black-box, adversarial attacks. The first one is based on a purely statistical approach and the second one is based on process knowledge for crafting these adversarial inputs that can compromise the models’ accuracy. Finally, the mechanisms upon which these adversarial attacks are inflicting damage are discussed to identify which process features or defects are replicated. This way, a roadmap is sketched toward testing the vulnerability and robustness of an AI-based quality monitoring system.

Recognition of penetration states based on arc sound of interest using VGG-SE network during pulsed GTAW process

Real-time welding monitoring is of great importance for intelligent welding, and the recognition of penetration states based on acoustic signal has been a hot topic. However, previous studies have relied on expert knowledge to extract sound features and ignored that acoustic features have distinct effects on the penetration state. In this paper, based on convolution neuron networks (CNNs), an adaptively feature extraction model for arc sound has been proposed. The time-frequency spectrum of arc sound was obtained by short-time Fourier transform (STFT) and served as the input of proposed model. The fully connected layers are replaced with global average pooling (GAP) or global maximum pooling (GMP) to alleviate the overfitting. To research the interest of arc sound during GTAW process, the squeeze and excitation network (SE-Net) is embedded into CNNs to pay more attention to the region of interest in the time-frequency spectrum. The experimental results have demonstrated that the proposed model outperforms the conventional methods and yield high accuracy of 98.25 %. Furthermore, the interested region of arc sound under different penetration states is visualized by Grad-CAM and guided Grad-CAM. This study extends the application of attention mechanism in intelligent welding and achieved remarkable performance.

Quality monitoring of projection welding using machine learning with small data sets

Capacitor discharge welding is an efficient, cost-effective and stable process. It is mostly used for projection welding. Real-time monitoring is desired to ensure quality. Until this point, measured process quantities were evaluated through expert systems. This method is strongly restricted to specific welding tasks and needs deep understanding of the process. Another possibility is quality prediction based on process data with machine learning. This requires classified welding experiments to achieve a high prediction probability. In industrial manufacturing, it is rarely possible to generate big sets classified data. Therefore, semi-supervised learning is investigated to enable model development on small data sets. Supervised learning models on large amounts of data are used as a comparison to the semi-supervised models. A total of 389 classified weld tests were performed. With semi-supervised learning methods, the amount of training data necessary was reduced to 31 classified data sets.

Use of ResNet modelling for TIG weld feature digitization and correlation : a technique for AI based welding system

TIG Welding is being practiced in the manufacturing industry and it demands highly skilled labour. Artificial Intelligence (AI) is developing rapidly as researchers are constantly finding new ways in which intelligent machines can add value to their industry. An AI-based welding system stands to add value by increasing production rates, improving safety, and decreasing the human input required. Weld monitoring is a key activity in the TIG welding process and successful use of AI system will enable failure prediction and the proactive corrective actions. The aim of this project is to explore, test, and compare ResNet modelling based machine learning algorithms and examine their ability to monitor welds. In this project the weld monitoring process includes collecting images of weld joint for weld feature digitization. Also, the study enables predicting whether the weld shows good quality, contamination, burn through, misalignment, lack of fusion, or lack of penetration through a ResNet modelling based image analysis.

Image Processing and Feature Extraction for Hull Structure GMAW Based on Weld Pool Visual Sensing

Image processing and feature information extraction based on the visual perception of the weld pool are considered essential components of intelligent welding quality monitoring of hull structure gas metal arc welding (GMAW). The unstable characteristics, such as large spatter, much smoke, and strong arc light during hull structure GMAW, lead to the blurring of image acquisition and the difficulty of contour extraction of the weld pool. The present study is aimed at addressing the practical issues from two perspectives, i.e., a spectrum‐visual‐sensing acquisition system and an image‐processing and feature extraction algorithm. First of all, by analyzing the light energy distribution law and acquiring the optical parameters relevant to the cut‐off composite dimming and near‐infrared narrowband filtering, spectral sensing is employed in establishing models of arc light radiation to detect the strength of continuously distinctive spectral lines. Besides, an appropriate high‐speed charge‐coupled device (CCD) camera is selected to build a visual acquisition system, which can reduce the external interference of the arc light on the image acquisition of the weld pool. Afterwards, the implementation of an image‐processing fusion model based on the spatial information fuzzy C‐means (FCM) clustering analysis and Sobel edge detection operator accompanies the investigation of the geometric aspects of the weld pool image. In terms of clear segmentation of the interest region, the edge detection and accurate extraction of the target contour are successfully obtained. In the subsequent section, the Hough transform analysis is adopted to establish the geometric feature extraction model of the weld pool, with corner detection, conversion, and camera calibration as the core technology. Additionally, the left and right views of the image contour are calibrated to achieve the lossless conversion of corner pixels and physical coordinates. Finally, three other distinct image‐processing methods are designed to compare the segmentation effect of the edge contour with the fusion model, and then, the extraction accuracy of the geometric features of the weld pool is verified. The interference of the arc light and smoke has been demonstrated to be substantially diminished, which is attributable to the visual‐sensing system during image acquisition of the weld pool. The results of edge fusion of the weld pool image show that based on the GMAW using the FCM‐Sobel fusion method, a superior extraction accuracy of geometric features characterized by smoothness, continuousness, no breakpoints, and less noise has fulfilled the engineering requirements.

Cross-attention-based multi-sensing signals fusion for penetration state monitoring during laser welding of aluminum alloy

A precision multi-sensor monitoring strategy is required to meet the challenges posed by increasingly complex products and manufacturing processes during laser welding. In this work, an acoustic sensor and a photoelectric sensor were adopted to collect the signals during the laser welding of aluminum alloy. The dataset was divided into three categories according to the morphologies of the top and back sides. The cross-attention fusion neural network (CAFNet) was proposed to interactively capture photoelectric and acoustic information for effective quality classification without prior time–frequency analysis and feature learning. Its effectiveness and superiority were compared with the five types of deep learning (DL) based methods. It demonstrates that the proposed CAFNet method achieved a mean testing accuracy of 99.73% and a standard deviation of 0.37%, which outperforms other compared models. At the same time, the proposed CAFNet achieved the highest average testing accuracy of 94.34% when utilizing limited and imbalanced data, which suggested that the proposed method has stronger robustness than other methods. This approach is a new paradigm in the monitoring of laser welding and can be exploited to provide feedback in a closed-loop quality control system.