Keyhole Imaging Research Articles

Penetration State Recognition during Laser Welding Process Control Based on Two-Stage Temporal Convolutional Networks.

Vision-based laser penetration control has become an important research area in the field of welding quality control. Due to the complexity and large number of parameters in the monitoring model, control of the welding process based on deep learning and the reliance on long-term information for penetration identification are challenges. In this study, a penetration recognition method based on a two-stage temporal convolutional network is proposed to realize the online process control of laser welding. In this paper, a coaxial vision welding monitoring system is built. A lightweight segmentation model, based on channel pruning, is proposed to extract the key features of the molten pool and the keyhole from the clear molten pool keyhole image. Using these molten pool and keyhole features, a temporal convolutional network based on attention mechanism is established. The recognition method can effectively predict the laser welding penetration state, which depends on long-term information. In addition, the penetration identification experiment and closed-loop control experiment of unequal thickness plates are designed. The proposed method in this study has an accuracy of 98.96% and an average inference speed of 20.4 ms. The experimental results demonstrate that the proposed method exhibits significant performance in recognizing the penetration state from long sequences of welding image signals, adjusting welding power, and stabilizing welding quality.

Study on the keyhole oscillation mechanism of laser welding based on electro-mechano-acoustical analogy theory

The acoustic emission phenomena generated by the laser welding process are closely related to the keyhole's forced oscillation and the dynamic pressure balance at the keyhole bottom. In this study, we constructed the mechanical equation of keyhole bottom pressure balance and used the “electro-mechano-acoustical” analogy theory to convert it into an acoustic equation, the acoustic resonant frequency as well as the magnification amplification factor of the keyhole oscillation in different penetration states were derived. A monitoring platform combining acoustic sensor and high-speed photography was established to obtain acoustic emission signals and lateral keyhole images. The acoustic and image features extracted from the monitoring data validated the derived formulas. Additionally, the acoustic phenomena of different frequency-energy components were explained according to the derived formulas in turn. The results demonstrated that the derived theory is compatible with experimental comparisons, showcasing its great interpretability and applicability. The introduced analogy theory effectively establishes a bridge between keyhole forced pressure and acoustic emission vibration, aiding in the understanding of keyhole oscillation characteristics. A novel model of keyhole oscillation was proposed and verified using process monitoring data in this study, which provides a new approach for the investigation of keyhole dynamic.

Using Keyhole Images to Map Soil Liquefaction Induced by the 1966 Xingtai Ms 6.8 and 7.2 Earthquakes, North China

In March 1966, Ms 6.8 and 7.2 earthquakes occurred in Xingtai, North China, resulting in widespread soil liquefaction that caused severe infrastructure damage and economic losses. Using Keyhole satellite imagery combined with aerial images and fieldwork records, we interpreted and identified 66,442 liquefaction points and analyzed the coseismic liquefaction distribution characteristics and possible factors that influenced the Xingtai earthquakes. The interpreted coseismic liquefaction was mainly concentrated above the IX-degree zone, accounting for 80% of all liquefaction points. High-density liquefaction zones (point density > 75 pieces/km2) accounted for 22% of the total liquefaction points. Most of the interpreted liquefaction points were located at the region with a peak ground acceleration (PGA) of >0.46 g. The liquefaction area on 22 March was significantly larger than that on 8 March. The region of liquefaction was mainly limited by sandy soil conditions, water system conditions, and seismic geological conditions and distributed in areas with loose fine sand and silt deposits, a high water table (groundwater level increases before both mainshocks corresponding to the liquefaction intensive regions), rivers, and ancient river channels. Liquefaction exhibited a repeating characteristic in the same region. Further understanding of the liquefaction characteristics of Xingtai can provide a reference for the prevention of liquefaction in northern China.

Dynamic evolution of keyhole and weld pool throughout the thickness during keyhole plasma arc welding

Keyhole plasma arc welding (K-PAW) of medium-thick plates often faces instability in the initial keyhole penetration stage, so a thorough investigation of the keyhole and weld pool behavior in this stage is needed. In this paper, a special "sandwich visual sensing structure" was adopted to obtain images of weld pool and keyhole of the SUS304 stainless-steel plate during the plasma arc welding. This method allows direct observation of keyhole growth and weld pool flow throughout the thickness of the workpiece under different welding conditions. When the welding current is reduced, the plasma arc cannot melt through the metal, and the arc sweeps through the keyhole bottom like a “roller coaster”. The trends of keyhole dimensions along different directions during various welding processes were quantitatively investigated. As the weld proceeds to half of the plate thickness, the mutation zone of the rear keyhole wall becomes very common, and the rear keyhole wall's morphology frequently transforms between type I and type II. Weld pool flow characteristics during constant-speed welding were classified into four types. Last, the molten metal flow cycle and the molten metal cloud cluster have been observed by the new visual sensing method. This paper provides the industry with a more intuitive way to study the keyhole and the weld pool for K-PAW, as well as offers important guidance for the first normal penetration in the medium-thickness plate K-PAW process.

Fundamental understanding of open keyhole effect in plasma arc welding

The keyhole arc welding technique has the advantage of improving welding efficiency by utilizing a stable keyhole mode. Accurate understanding of the keyhole effect is necessary to enhance the welding quality. Due to the high temperature and strong arc force involved, the complex gas–liquid–solid interactions in the complete keyhole process need to be explored. In order to fully demonstrate open keyhole mode welding, a three-tier sandwiched model based on multiphysics and multiphase effects was developed. The top layer of the model is filled with plasma arc, which gradually fuses and penetrates through the middle metal layer. Finally, it enters the third layer, resulting in an open keyhole mode. Multiphysics phenomena due to the plasma arc are fully included in the model, and the gas–liquid–solid interactions are calculated by combining the Volume of Fluid technique and the Enthalpy-porous technique. Arc ignition and dynamic open keyhole effect are demonstrated, and an arc discharge is shown from the open keyhole exit. The arc reflection phenomenon is observed as the arc is blocked by the weld pool frontier. The electric current path varies with the welding movement, and most of the current comes from the weld pool frontier. An experiment was conducted to obtain weld pool and keyhole images, which basically agree with the calculated results. Additionally, the calculated open keyhole time and electric potential drops also coincide well with experimental data.

Online Detection of Keyhole Status in a Laser-MIG Hybrid Welding Process

During laser-metal inert gas (MIG) hybrid welding, a large amount of welding status information is generated in droplet transfer, keyhole and molten pool. In this paper, austenitic stainless steel was adopted as an experimental object, with a dual high-speed camera system used to obtain real-time images of droplet transfer, keyhole and molten pool in a laser-MIG hybrid welding process. The changing regulation of a keyhole in three different penetration states (i.e., non-penetration, partial penetration and normal penetration) was analyzed by extracting the morphological characteristics of a keyhole shape, and combining the droplet transition information and the shape of the weld pool. Experimental results show that the proposed method could effectively reflect the variation characteristics of the keyhole, and the correlation among the keyhole characteristics, the droplet transfer information, the weld pool shape and the welding status, and provide a new perspective for online detection of the laser-MIG welding quality.

Real time 4D Radiation Gel Dosimetry on the Australian MRI-Linac

4D radiation dosimetry using a highly radiation-sensitive polymer gel dosimeter with real-time quantitative MRI readout is presented as a technique to acquire the accumulated radiation dose distribution during image guided radiotherapy (IGRT) on an MRI-Linac. Optimized T2 weighted TSE scans are converted into quantitative ΔR2 maps and subsequently to radiation dose maps. The potential of real-time 4D radiation dosimetry in a theragnostic MRI-Linac is demonstrated in test tubes, for a square beam in a cylindrical gel phantom, for a simple step-and-shoot irradiation in a head phantom and a dynamic arc treatment on a cylindrical gel phantom using a rotating couch. The optimal sequence parameters for maximal dose resolution in the dynamic MRI acquisition will be presented and the trade off between MRI scanning speed and dose resolution will be discussed. A further improvement in temporal resolution using a keyhole imaging approach is the focus of future research.

On-line monitoring of penetration state in laser-arc hybrid welding based on keyhole and arc features

Incomplete penetration is a typical defect in laser-arc hybrid welding. On-line monitoring of welding process is an important method to assess welding quality. In laser-arc hybrid welding, there is a strong correlation between keyhole, arc and incomplete penetration. Therefore, an on-line monitoring method of penetration state based on keyhole and arc features is proposed in this paper. In the proposed method, the images of keyhole and arc in laser-arc hybrid welding are captured by high-speed camera, and then the keyhole and arc features are extracted by image processing algorithm. Finally, the features are used as input to classify the penetration state using SVM model. The results show that the SVM model based on keyhole and arc features can accurately identify the penetration state.

Experimental sensing of molten flow velocity, weld pool and keyhole geometries in ultrasonic-assisted plasma arc welding

• Comparative experiments were performed to investigate the ultrasonic effect in plasma arc welding. • The molten flow velocity on the weld pool surface was measured by tracer particle approach. • With ultrasonic vibration, the front wall and the back wall of the keyhole became steeper. • With ultrasonic vibration, the backward flow velocity of the molten metal decreased obviously. • Ultrasonic vibration causes small sizes of weld pool and keyhole but enhanced keyholing/penetrating capability. Comparative experiments were performed to investigate how the ultrasonic vibration exerted on the tungsten electrode affect the molten metal flow on weld pool surface and the sizes of weld pool and keyhole in plasma arc welding (PAW). The fluid flow velocity on the molten pool surface was measured by tracer particle approach. The images of front molten pool and keyhole entrance were captured synchronously from different directions with two CCD cameras. Special algorithms were developed to calibrate and integrate the weld pool and keyhole images into a single picture in one coordinate system. It was found that when ultrasonic vibration was applied, the keyhole shape was changed, the front wall and the back wall of the keyhole became steeper, and the backward flow velocity of the molten metal decreased obviously. The size of the molten pool and the keyhole entrance in ultrasonic-assisted PAW was smaller than that in PAW, and the keyholing/penetrating capability was enhanced.

Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.

Video-rate lensless endoscope with self-calibration using wavefront shaping.

Lensless fiber endoscopes are of great importance for keyhole imaging. Coherent fiber bundles (CFB) can be used in endoscopes as remote phased arrays to capture images. One challenge is to image at high speed while correcting aberrations induced by the CFB. We propose the combination of digital optical phase conjugation, using a spatial light modulator, with fast scanning, for which a 2D galvo scanner and an adaptive lens are employed. We achieve the transmission of laser and image scanning through the CFB. Video-rate imaging at 20 Hz in 2D with subcellular resolution is demonstrated in 3D with 1 Hz. The sub-millimeter-diameter scanning endoscope has a great potential in biomedicine, for manipulation, e.g., in optogenetics, as well as in imaging.

Monitoring method of weld penetration in laser keyhole welding irradiated by laser auxiliary illuminant

In the field of automotive manufacturing, it is pivotal to obtain a moderate penetration status when joining automotive parts by laser keyhole welding. As a typical characteristic of laser keyhole welding, keyhole behavior can directly reflect the penetration status of the weld bead. In this paper, a coaxial vision monitoring system with a laser auxiliary illuminant is established to collect the keyhole images from the top-face of the weldment during laser welding. The interferences of the bright spot caused by mirror reflection and metal vapors in the collected images are fully considered. Then, a hybrid adaptive keyhole detection algorithm is proposed to accurately segment the keyhole region. Five static image features are extracted from the perspective of the average gray value, area, and perimeter in the keyhole region. The feature selection method of Wrapper, which combines a sequential forward searching algorithm with a random forest classifier, is used to select ten penetration status features (PSFs). The constructed prediction model of weld penetration based on the selected PSFs has excellent performance. The proposed monitoring method in laser keyhole welding irradiated by a laser auxiliary illuminant is contributed to the closed-loop control of weld penetration to obtain a moderate penetration status.

Optimization of Keyhole Imaging Parameters for Glutamate Chemical Exchange Saturation Transfer MRI at 7.0T.

To evaluate the effects of a reference image and keyhole factor (Kf) selections for high-frequency substitution on keyhole imaging technique for applications in glutamate chemical exchange saturation transfer (GluCEST) imaging. The CEST data were obtained using a 7.0T MRI scanner. We used varied Kf ranges that constituted from 16.67 to 75% of the full k-space. The reference image was respectively selected for - 3 and + 3ppm images that associated with the GluCEST calculation and the unsaturated image. The zero-padding algorithm was applied for the missing k-space lines in the low-frequency data collected to compare the results obtained from using the keyhole imaging technique. All the techniques were evaluated using a healthy rat group and extended to the status epilepticus rat group to explore their applicability and usability. The calculated GluCEST signals and visually inspected results from the reconstructed GluCEST maps indicated that the combination of unsaturated image as a reference image, and over 50% of Kf showed consistent signals and image quality compared with the fully sampled CEST data. Combining the keyhole imaging technique with GluCEST imaging enables stable image reconstruction and quantitative evaluation, and this approach is potentially applicable in various CEST imaging applications.

Evaluation of image quality of pituitary dynamic contrast-enhanced MRI using time-resolved angiography with interleaved stochastic trajectories (TWIST) and iterative reconstruction TWIST (IT-TWIST).

Time-resolved angiography with interleaved stochastic trajectories (TWIST) is a keyhole imaging with frequent sampling of central k-space data and view-sharing for the peripheral k-space of dynamic phases. IT-TWIST is a technique to reconstruct images with a smaller temporal footprint using the same raw data obtained with TWIST by iterative reconstruction. To compare image quality between TWIST and IT-TWIST in 3D pituitary DCE-MRI. Retrospective observation study. Fifty-one patients (23 men, 28 women) who underwent 3D pituitary DCE-MRI using TWIST between July 2016 and April 2017. 3T/TWIST and IT-TWIST. Visual evaluation was conducted for image quality of delineation of the pituitary stalk and posterior lobe during the early arterial phase, cerebral white matter near the sella turcica, and the mass lesion. Bolus sharpness of the pituitary stalk, posterior lobe, and bilateral cavernous sinus was evaluated on the enhancement slope map calculated from TWIST and IT-TWIST. Temporal stability of intensity of the nonenhanced area was evaluated on temporal standard deviation (SD) maps calculated from TWIST and IT-TWIST. Paired t-test or Wilcoxon rank-sum test was used to test the differences between TWIST and IT-TWIST in both visual evaluation and region of interest evaluation. Scores of visual evaluations for IT-TWIST were significantly better than those for TWIST (P < 0.001) in all items. Enhancement slope for IT-TWIST was significantly higher than that for TWIST in posterior lobe, and right and left cavernous sinus (P < 0.001). Temporal SD for IT-TWIST was significantly lower than that for TWIST in all items, with statistical significance (P < 0.001). IT-TWIST yielded better visualization, and better enhancement slope, and less temporal SD compared with TWIST in 3D pituitary DCE-MRI. Evidence Level: 4 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2020;51:1497-1506.

Technical Note: Real-time 3D MRI in the presence of motion for MRI-guided radiotherapy: 3D Dynamic keyhole imaging with super-resolution.

The purpose of this study was to present real-time three-dimensional (3D) magnetic resonance imaging (MRI) in the presence of motion for MRI-guided radiotherapy (MRgRT) using dynamic keyhole imaging for high-temporal acquisition and super-resolution generative (SRG) model for high-spatial reconstruction. We propose a unique real-time 3D MRI technique by combining a data sharing technique (3D dynamic keyhole imaging) with a SRG model using cascaded deep learning technique. 3D dynamic keyhole imaging utilizes the data sharing mechanism by combining keyhole central k-space data acquired in real-time with high-spatial, high-temporal resolution prior peripheral k-space data at various motion positions prepared by the SRG model. The efficacy of the 3D dynamic keyhole imaging with super-resolution (SR_dKeyhole) was compared to the ground-truth super-resolution images with the original full k-space data. It was also compared with the zero-filling reconstruction (zero-filling), conventional keyhole reconstruction with low-spatial high-temporal prior data (LR_cKeyhole), and conventional keyhole reconstruction with super-resolution prior data (SR_cKeyhole). High-spatial, high-temporal resolution 3D MRI datasets (1.5×1.5×6mm3 ) were generated from low-spatial, high-temporal resolution 3D MRI datasets (6×6×6mm3 ) using the cascaded deep learning SRG framework (<100ms/volume). 3D dynamic keyhole imaging with the SRG model provided high-spatial, high-temporal resolution images (1.5×1.5×6mm3 , 455ms) with the highest similarity to the ground-truth SR images without any noticeable artifacts. Structural similarity indices (SSIM) of the reconstructed 3D MRI to the original SR 3D MRI were 0.65, 0.66, 0.86, and 0.89 for zero-filling, LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole, respectively (1 for identical image). In addition, average value of image relative error (IRE) of the reconstructed 3D MRI to the original SR 3D MRI were 0.169, 0.191, 0.079, and 0.067 for zero-filling, LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole, respectively (0 for identical image). We demonstrated that high-spatial, high-temporal resolution 3D MRI was feasible by combing 3D dynamic keyhole imaging with a SRG model in terms of image quality and imaging time. The proposed technique can be utilized for real-time 3D MRgRT.

Weld penetration in situ prediction from keyhole dynamic behavior under time-varying VPPAW pools via the OS-ELM model

In situ monitoring and accurate detecting of welding quality have been one of the common challenges of automatic welding process. This paper contributes an intelligent decision-making framework for the weld penetration prediction from the keyhole dynamic behavior under time-varying VPPAW pools. Initially, a series of dynamic experiments under different welding conditions were conducted to acquire the backside images of keyhole and corresponding backside bead width. Then, the geometry appearance of keyhole was described by the supervised descent method (SDM)–based image processing algorithm. Subsequently, the internal correlation between the keyhole characteristics and the backside width was further derived to help understand the nonlinear and time-varying VPPAW process. Finally, a novel dynamic model based on an online sequential extreme learning machine (OS-ELM) was designed to predict the weld penetration as measured by the backside bead width in real time. Extensive experiment results further verify and validate that the proposed dynamic OS-ELM model is significantly better than other state-of-the-art algorithms in terms of predicting accuracy, efficiency, and robustness.

Mapping and correcting hyperpolarized magnetization decay with radial keyhole imaging.

Hyperpolarized (HP) media enable biomedical imaging applications that cannot be achieved with conventional MRI contrast agents. Unfortunately, quantifying HP images is challenging, because relaxation and radio-frequency pulsing generate spatially varying signal decay during acquisition. We demonstrate that, by combining center-out k-space sampling with postacquisition keyhole reconstruction, voxel-by-voxel maps of regional HP magnetization decay can be generated with no additional data collection. Digital phantom, HP 129 Xe phantom, and in vivo 129 Xe human (N = 4 healthy; N = 2 with cystic fibrosis) imaging was performed using radial sampling. Datasets were reconstructed using a postacquisition keyhole approach in which 2 temporally resolved images were created and used to generate maps of regional magnetization decay following a simple analytical model. Mean, keyhole-derived decay terms showed excellent agreement with the decay used in simulations (R2 = 0.996) and with global attenuation terms in HP 129 Xe phantom imaging (R2 > 0.97). Mean regional decay from in vivo imaging agreed well with global decay values and displayed spatial heterogeneity that matched expected variations in flip angle and oxygen partial pressure. Moreover, these maps could be used to correct variable signal decay across the image volume. We have demonstrated that center-out trajectories combined with keyhole reconstruction can be used to map regional HP signal decay and to quantitatively correct images. This approach may be used to improve the accuracy of quantitative measures obtained from hyperpolarized media. Although validated with gaseous HP 129 Xe in this work, this technique can be generalized to any hyperpolarized agent.

Update on multiparametric MRI of urinary bladder cancer.

3 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018;48:882-896.

Effects of pulse parameters on porosity rate and flow characteristics of molten pool in pulsed laser welding

Flow behavior determines the formation and evolution of porosity and other defects in laser welding, but the existing investigations of liquidity of molten pool are restricted by qualitative analysis methods, so it is of great significance to quantitatively characterize the flow patterns and flow characteristics of the pulsed laser welding for understanding the porosity suppression mechanism. A series of welding experiments was carried out by using 1.5 kW slab CO2 laser under a variety of pulse parameters, and the images of molten pool and keyhole were captured by using a high-speed camera during welding. Porosity rate was also obtained by image processing of x-ray inspection of weld joints. The flow characteristics of molten pools including speed and vortices from these images were extracted and analyzed quantitatively. Effects of different pulse frequency and duty ratios on penetration depth, weld width, and flow characteristics of the molten pool were investigated and compared with each other. A three-stage flow pattern of molten pool for pulse laser welding and the respective characteristics of each stage were revealed. The analysis result shows that the velocity at the starting time in a pulse impact phase is about three times of that in the steady-state phase and the primary mechanism for suppression of pores lies in the impact and stirring on molten pool in pulse laser welding. The other suppression mechanisms of pores for welding joints are also discussed.

Automatic gap tracking during high power laser welding based on particle filtering method and BP neural network

A numerical simulation model was established to investigate characteristics of keyhole and molten pool during the laser butt welding. The sharp point in front of the keyhole and molten pool revealed the position of the gap, and its deviation in transversal direction between the centroid of the keyhole demonstrated the real-time deviation between the laser beam and real gap. A visual system was designed to capture real-time infrared images of keyhole and molten pool, and the real-time deviation data between the laser beam and gap was extracted from these images. The state and measurement equations of real gap position prediction were developed based on the welding system. The particle filtering (PF) method was employed to improve the accuracy of the prediction of the gap position. Considering the non-linear and unknown distribution of system error and measurement error, a Back Propagation (BP) neural network was developed to compensate for these errors. The effectiveness of the established PF method combined with BP network was validated by experimental results, and higher prediction accuracy of gap position tracking was achieved.