Defect Segmentation Research Articles

Deep-learning based artificial intelligence tool for melt pools and defect segmentation

Deep-learning based artificial intelligence tool for melt pools and defect segmentation

Materials data science using CRADLE: A distributed, data-centric approach

There is a paradigm shift towards data-centric AI, where model efficacy relies on quality, unified data. The common research analytics and data lifecycle environment (CRADLE™) is an infrastructure and framework that supports a data-centric paradigm and materials data science at scale through heterogeneous data management, elastic scaling, and accessible interfaces. We demonstrate CRADLE’s capabilities through five materials science studies: phase identification in X-ray diffraction, defect segmentation in X-ray computed tomography, polymer crystallization analysis in atomic force microscopy, feature extraction from additive manufacturing, and geospatial data fusion. CRADLE catalyzes scalable, reproducible insights to transform how data is captured, stored, and analyzed.Graphical abstract

Trans-DCN: A High-Efficiency and Adaptive Deep Network for Bridge Cable Surface Defect Segmentation

Cables are vital load-bearing components of cable-stayed bridges. Surface defects can lead to internal corrosion and fracturing, significantly impacting the stability of the bridge structure. The detection of surface defects from bridge cable images faces numerous challenges, including shadow disturbances due to uneven lighting and difficulties in addressing multiscale defect features. To address these challenges, this paper proposes a novel and cost-effective deep learning segmentation network, named Trans-DCN, to detect defects in the surface of the bridge cable. The network leverages an efficient Transformer-based encoder and integrates multiscale features to overcome the limitations associated with local feature inadequacy. The decoder implements an atrous Deformable Convolution (DCN) pyramid and dynamically fuses low-level feature information to perceive the complex distribution of defects. The effectiveness of Trans-DCN is evaluated by comparing it with state-of-the-art segmentation baseline models using a dataset comprising cable bridge defect images. Experimental results demonstrate that our network outperforms the state-of-the-art network SegFormer, achieving a 27.1% reduction in GFLOPs, a 1.2% increase in mean Intersection over Union, and a 1.5% increase in the F1 score. Ablation experiments confirmed the effectiveness of each module within our network, further substantiating the significant validity and advantages of Trans-DCN in the task of bridge cable defect segmentation. The network proposed in this paper provides an effective solution for downstream cable bridge image analysis.

Low-contrast X-ray image defect segmentation via a novel core-profile decomposition network

Accurate X-ray image defect segmentation is of paramount importance in industrial contexts, as it is the foundation for product quality control and production safety. Deep learning (DL) has demonstrated powerful image scene understanding capabilities and has achieved unprecedented performance in defect segmentation tasks. However, existing DL methods suffer from significant performance degradation when facing low-contrast X-ray images, as the core information of defects is often obscured and the profile details are ambiguous. To address this issue, this paper explicitly decomposes the X-ray defect segmentation task into two subtasks: core feature learning and elasticity profile refinement, allowing task “serial” decomposition and performance “parallel” improvement. On this basis, a novel core-profile decomposition network (CPDNet) is developed to achieve accurate defect segmentation of X-ray images. Specifically, the core feature learning module is designed to construct the effective feature space from two views, discriminative and structural, to extract defect-related core features from X-ray images. Subsequently, the elasticity profile refinement module is developed to further improve the defect segmentation performance, which makes the first attempt to define the profile refinement as an out-of-distribution detection and leverage the elasticity score to refine the profile details at the pixel level. To fully evaluate the presented method, we conduct a series of experiments using two real-world X-ray defect datasets, and the results demonstrate that the CPDNet outperforms state-of-the-art methods.

Detecting Internal Defects in FRP-Reinforced Concrete Structures through the Integration of Infrared Thermography and Deep Learning.

This study represents a significant advancement in structural health monitoring by integrating infrared thermography (IRT) with cutting-edge deep learning techniques, specifically through the use of the Mask R-CNN neural network. This approach targets the precise detection and segmentation of hidden defects within the interfacial layers of Fiber-Reinforced Polymer (FRP)-reinforced concrete structures. Employing a dual RGB and thermal camera setup, we captured and meticulously aligned image data, which were then annotated for semantic segmentation to train the deep learning model. The fusion of the RGB and thermal imaging significantly enhanced the model's capabilities, achieving an average accuracy of 96.28% across a 5-fold cross-validation. The model demonstrated robust performance, consistently identifying true negatives with an average specificity of 96.78% and maintaining high precision at 96.42% in accurately delineating damaged areas. It also showed a high recall rate of 96.91%, effectively recognizing almost all actual cases of damage, which is crucial for the maintenance of structural integrity. The balanced precision and recall culminated in an average F1-score of 96.78%, highlighting the model's effectiveness in comprehensive damage assessment. Overall, this synergistic approach of combining IRT and deep learning provides a powerful tool for the automated inspection and preservation of critical infrastructure components.

Attention Similarity Outlier Suppression with Hierarchical Feature Integration for Defect Segmentation

Defect segmentation holds significant importance in industrial applications. However, most existing algorithms are tailored to specific scenarios, lacking a unified defect segmentation algorithm for defect segmentation. Based on the analysis of the characteristics of defects, we aim to propose a unified transformer‐based defect segmentation algorithm. On the one hand, the defect foreground exhibits significant differences from the normal background. Nevertheless, the attention mechanism in the transformer involves all feature nodes in the feature map to the similarity score of one patch with all other patches. This approach may compromise the features of potential defect areas where the defect foreground significantly contrasts with the background. We introduce the Attention Similarity Outlier Suppression (ASOS) strategy to address this challenge. Our method is designed to select valid attention scores and filter out outlier scores for appropriate attention fusion, in which the feature of potential defect areas is not easily affected by background. On the other hand, the defect regions often have relatively small areas. We incorporate a Hierarchical Feature Integration (HFI) module in our framework to facilitate defect recognition at different scales. By progressively up‐sampling, features can achieve better integration to generate the final segmentation result. Extensive experiments have verified the effectiveness of our proposed method in defect segmentation tasks. Moreover, thorough ablation studies have demonstrated the efficacy of our algorithm, reinforcing its robust performance. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

A graph learning approach for automatic subsurface defects segmentation in building façade using RGB and infrared images

Infrared thermal imaging technology provides a non-destructive and automated inspection solution for analyzing building facades. However, conventional methods that rely on detecting subsurface defects through abnormal heat loss often suffer from the limitation of misidentifying other materials present on the building facade. To address this challenge, this study proposes a graph learning approach for segmentation of subsurface defects by integrating both visual and thermal information. The proposed methodology encompasses several key steps. Firstly, the infrared image is partitioned into internally connected superpixels with similar characteristics using a linear iterative clustering algorithm. Subsequently, these superpixels are employed as nodes to construct a weighted undirected graph. The edges of the graph are defined based on the connectivity of superpixels in the image plane, while the weights are determined by the Wasserstein distance between each node. Furthermore, a multimodal graph reuse technique is introduced to construct a new graph based on an RGB image, thereby incorporating additional visual information. By defining the defect as tightly connected in the RGB graph but loosely connected in the infrared graph, the segmentation of defects involving multiple materials is formulated as a multi-graph N-cut problem. The feasibility and effectiveness of the proposed method are demonstrated through laboratory testing conducted on concrete specimens, as well as on-site monitoring carried out on a tiled building façade.

MFANet: Multifeature Aggregation Network for Cross-Granularity Few-Shot Seamless Steel Tubes Surface Defect Segmentation

MFANet: Multifeature Aggregation Network for Cross-Granularity Few-Shot Seamless Steel Tubes Surface Defect Segmentation

Progressive Complementary Knowledge Aggregation for CdZnTe Defect Segmentation

Progressive Complementary Knowledge Aggregation for CdZnTe Defect Segmentation

Unsupervised learning-enabled pulsed infrared thermographic microscopy of subsurface defects in stainless steel

Metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing method (AM) frequently contain microscopic porosity defects, with typical approximate size distribution from one to 100 microns. Presence of such defects could lead to premature failure of the structure. In principle, structural integrity assessment of LPBF metals can be accomplished with nondestructive evaluation (NDE). Pulsed infrared thermography (PIT) is a non-contact, one-sided NDE method that allows for imaging of internal defects in arbitrary size and shape metallic structures using heat transfer. PIT imaging is performed using compact instrumentation consisting of a flash lamp for deposition of a heat pulse, and a fast frame infrared (IR) camera for measuring surface temperature transients. However, limitations of imaging resolution with PIT include blurring due to heat diffusion, sensitivity limit of the IR camera. We demonstrate enhancement of PIT imaging capability with unsupervised learning (UL), which enables PIT microscopy of subsurface defects in high strength corrosion resistant stainless steel 316 alloy. PIT images were processed with UL spatial–temporal separation-based clustering segmentation (STSCS) algorithm, refined by morphology image processing methods to enhance visibility of defects. The STSCS algorithm starts with wavelet decomposition to spatially de-noise thermograms, followed by UL principal component analysis (PCA), fine-tuning optimization, and neural learning-based independent component analysis (ICA) algorithms to temporally compress de-noised thermograms. The compressed thermograms were further processed with UL-based graph thresholding K-means clustering algorithm for defects segmentation. The STSCS algorithm also includes online learning feature for efficient re-training of the model with new data. For this study, metallic specimens with calibrated microscopic flat bottom hole defects, with diameters in the range from 203 to 76 µm, were produced using electro discharge machining (EDM) drilling. While the raw thermograms do not show any material defects, using STSCS algorithm to process PIT images reveals defects as small as 101 µm in diameter. To the best of our knowledge, this is the smallest reported size of a sub-surface defect in a metal imaged with PIT, which demonstrates the PIT capability of detecting defects in the size range relevant to quality control requirements of LPBF-printed high-strength metals.

SPMUNet: Semantic segmentation of citrus surface defects driven by superpixel feature

The segmentation of surface defects in citrus fruit makes a significant contribution to quality assessment and post-harvest processing. However, semantic segmentation of citrus surface defects remains challenging owing to the complexity of defect features, unclear defect boundaries and semantic gaps in models. Therefore, a superpixel driven and multiscale channel cross-fusion transformer (MCCT)-based UNet (named SPMUNet) was proposed to segment citrus surface defects. The model first created a multiscale superpixel region feature (MSRF) branch using the simple linear iterative cluster (SLIC) algorithm to form the key features embedded in the downsampling layer. Then, the proposed Superpixel Loss optimized the loss function based on superpixel grid to provide error information for boundary segmentation. In addition, the MCCT module was used to improve the skip connection layers, and feature fusion was achieved by collaborative learning. Finally, the performance of the model was validated through ablation studies, feature map visualization, and comparison experiments. The results showed that the mPA, mIoU, and dice coefficients of SPMUNet for citrus surface segmentation were 89.1%, 83.8%, and 88.6%, respectively. Compared with the baseline, the proposed MSRF branch, MCCT, and Superpixel Loss improved mIoU by 2.6%, 1.3%, and 4.2%, respectively. And the effectiveness of the proposed methods was also validated by the feature map visualization. Furthermore, the proposed SPMUNet achieved optimal segmentation performance compared to other semantic segmentation models (UNet, ResUNet++, DeepLabv3+, SwinUNet, TransUNet). SPMUNet also excelled in the segmentation of small targets and boundaries, improving by 10.2% and 7.0%, respectively, over the baseline. The improvement in segmentation performance demonstrated the effectiveness of the selected superpixel features. The proposed citrus surface defect segmentation model achieved high-precision segmentation and serves as a technical reference for citrus sorting and quality assessment.

Detection and segmentation of wire rope surface deficiency based on YOLOv8 and U-Net

Abstract The presence of surface defects in wire ropes (WR) may lead to potential safety hazards and performance degradation, necessitating timely detection and repair. Hence, this paper proposes a method for detecting surface defects in WR based on the deep learning models YOLOv8s and U-Net, aiming to identify surface defects in real-time and extract defect data, thereby enhancing the efficiency of surface defect detection. Firstly, the ECA attention mechanism is incorporated into the YOLOv8 algorithm to enhance detection performance, achieving real-time localization and identification of surface defects in WR. Secondly, in order to obtain detailed defect data, the U-Net semantic segmentation algorithm is employed for morphological segmentation of defects, thereby obtaining the contour features of surface defects. Finally, in conjunction with OpenCV technology, the segmentation results of the defects are quantified to extract data, obtaining parameters such as the area and perimeter of the surface defects in the WR. Experimental results demonstrate that the improved YOLOv8-ECA model exhibits good accuracy and robustness, with the model’s mAP@0.5 reaching 84.78%, an increase of 1.13% compared to the base model, an accuracy rate of 90.70%, and an FPS of 65. The U-Net model can efficiently perform segmentation processing on surface defects of WR, with an mIOU of 83.54% and an mPA of 90.78%. This method can rapidly, accurately, and specifically detect surface defects in WR, which is of significant importance in preventing industrial production safety accidents.

Uncertainty inspired domain adaptation network for rail surface defect segmentation

Rail surface defect detection is an essential part of railroad maintenance to prevent safety accidents. Recently, deep learning-based defect detection algorithms have shown impressive detection performance. However, the performance improvement from deep neural networks is based on a strong assumption that the training images have the same data distribution as the test images. This requirement is unrealistic in practical railroad surface defect detection. The appearance of rails varies significantly in different railroad sections due to changes in working conditions and maintenance status. In this paper, we propose a novel uncertainty-inspired unsupervised domain adaptation framework (UIUDA) to improve the generalization of deep learning models on data with distribution differences. Specifically, domain adversarial learning is used to align the feature distributions of training and test data. To mitigate the negative transfer caused by global alignment, we design a locally aligned domain adaptation strategy. In addition, self-training is introduced to refine the decision boundaries of the model in the test data. In this process, we propose an uncertainty-inspired evaluation method to remove noisy pseudo labels. We conducted domain adaptation experiments with RSDD and NEU-RSDD as the source domain and FRSD dataset as the target domain. The results show that UIUDA can effectively improve the generalization ability of the model, and the mIoU metric of defect segmentation is 80.17%.

SIIF: Semantic information interactive fusion network for photovoltaic defect segmentation

Any defects or damage on photovoltaic panels significantly reduce photoelectric conversion efficiency and service life. Therefore, the refined defect segmentation technology is the key to the quality inspection of photovoltaic equipment. However, the existing methods ignore the information interaction between multi-levels, causing semantic features to be easily lost and difficult to fuse. In this paper, we propose a segmentation network based on an interactive fusion of semantic information for solar cell defect identification, which can effectively interact with multi-level rich information and enhance the positioning capabilities of high-level semantics. Specifically, we propose a novel Multi-Level Pyramid Strategy (MLPS) to reduce the information gap between multi-level features. MLPS adopts a pooling distribution mechanism to enable multi-level features to interact effectively in the feature pyramid. In addition, to effectively integrate high-level positioning semantics into the neighbor level, we design a new Neighbor Attention Fusion module (NAF) to propagate high-level semantic information gradually. NAF learns spatial neighbor-level similarity combined with high-level attention to balance location information and contextual features. We perform comprehensive experiments to demonstrate the advancement of our approach. Our method achieves 84.3% MIOU on our solar cell dataset, outperforming existing state-of-the-art (SOTA) methods. To verify the generalization ability of our network, we train and evaluate the PSCDE dataset. The results show that our model achieves an excellent 98.35%MIOU.

Automated weld defect segmentation from phased array ultrasonic data based on U-net architecture

Ultrasonic inspection is an environmentally friendly and easily deployable nondestructive testing (NDT) method widely used for defect detection of critical components in the industry. Phased array ultrasonic testing (PAUT) is one of the most advanced ultrasonic inspection methods, which gives volume inspection with increased resolution and coverage, improving inspection efficiency. Because of the weld structure echoes and the abstract nature of ultrasound images, especially facing meters and feature-changing weld joints with different thicknesses and welding methods in shipbuilding, the analysis of the PAUT weld data still relies on experienced rater random inspections. Rating complex welded products process is lengthy, costly, and prone to introduce human error during the manual rating but challenges automatic detection. To automatically segment defects in PAUT data, this work shows a combination of PAUT data of ship weld and three-dimensional (3D) U-net architecture. Combining PAUT imaging principles with welding and scan processes, a PAUT volumetric image dataset, including different thicknesses and scan angles, is established. We pioneered the application of 3D U-net architecture to segment defects in PAUT volume data. We found that U-net architecture with two encoding stages will perform better in segmenting defects in PAUT data, and region-based loss mainly improves the accuracy. Furthermore, a lightweight U-net architecture containing skip-connection and residual blocks is proposed with precision and efficiency improvement. The validation results show that the proposed U-net architecture offers a feasible solution to the problem of segmenting defects from PAUT data with a Dice accuracy of 90.9 %. Segmentation results help to locate and measure defects. This method makes locating and sizing defects in PAUT weld data possible within a fraction of a second.

Image-Based Fracture Surface Defect Characterization Methods for Additively Manufactured Ti-6Al-4V Tested in Fatigue

Fatigue initiation in additively manufactured samples/parts often occurs at processed-induced defects such as lack-of-fusion (LoF), keyhole, or other morphological/microstructural defects that have unique characteristics and measurable qualities. Attempts at identifying and minimizing such defects have utilized optimized processing conditions along with in situ and ex situ characterization that includes metallography and/or X-ray computed tomography (XCT). This paper highlights the benefits of using fracture surface analyses to detect and quantify defects that may not be detected by metallography/XCT due to sectioning and resolution limits. In addition to using manual quantification of fatigue initiating LoF and keyhole defects on fracture surfaces, image-based machine learning using convolutional neural networks such as U-Net were also used to automate the process. Statistical analyses were used to identify the extreme cases of defects that initiated and accelerated fatigue and to model the distribution of defect size and shape characteristics to distinguish the type of defect. Initial results show agreement between trained machine learning models and ground truth data in defect segmentation, and the distributions of defect characteristics are distinguishable to particular process-induced defect types.

Detection and Segmentation of Defects in CNC Machine Inserts Using Transfer Learning with Dataset Similarity Evaluation

Detection and Segmentation of Defects in CNC Machine Inserts Using Transfer Learning with Dataset Similarity Evaluation

An Adaptive Image Segmentation Network for Surface Defect Detection.

Surface defect detection plays an essential role in industry, and it is challenging due to the following problems: 1) the similarity between defect and nondefect texture is very high, which eventually leads to recognition or classification errors and 2) the size of defects is tiny, which are much more difficult to be detected than larger ones. To address such problems, this article proposes an adaptive image segmentation network (AIS-Net) for pixelwise segmentation of surface defects. It consists of three main parts: multishuffle-block dilated convolution (MSDC), dual attention context guidance (DACG), and adaptive category prediction (ACP) modules, where MSDC is designed to merge the multiscale defect features for avoiding the loss of tiny defect feature caused by model depth, DACG is designed to capture more contextual information from the defect feature map for locating defect regions and obtaining clear segmentation boundaries, and ACP is used to make classification and regression for predicting defect categories. Experimental results show that the proposed AIS-Net is superior to the state-of-the-art approaches on four actual surface defect datasets (NEU-DET: 98.38% ± 0.03%, DAGM: 99.25% ± 0.02%, Magnetic-tile: 98.73% ± 0.13%, and MVTec: 99.72% ± 0.02%).

ViTR-Net: An unsupervised lightweight transformer network for cable surface defect detection and adaptive classification

As a crucial load-bearing component of the cable-stayed bridge, the cable requires surface defect detection to maintain its safety. The current deep-learning-based stay-cable surface defect detection still faces challenges, including insufficient actual defect samples, cumbersome labeling processes, and an effective method for further evaluation and classification of defects. To address these problems, an unsupervised vision transformer-based lightweight reconstruction network called ViTR-Net is proposed, which only uses normal stay-cable surface images to achieve end-to-end localization of defect areas in the image. ViTR-Net is a generative-based method that uses a hybrid network of convolutional neural networks and visual transformers as a feature extraction network. A feature correction fusion module is designed by integrating differential convolution and attention mechanisms to compensate for potential detail loss caused by feature extraction. Subsequently, a severity index for defects is established to assess the severity of various defects, and the detected defects are adaptively classified using the affinity propagation clustering algorithm. ViTR-Net achieves the highest defect detection accuracy and exquisite defect segmentation results on a realistic cable defect dataset through experimental validation. The proposed model can realize the surface defect assessment and adaptive classification of the stay cable, which has a good application prospect in the actual maintenance process of the stay cable.

Automatic PCB Sample Generation and Defect Detection Based on ControlNet and Swin Transformer.

In order to improve the efficiency and accuracy of multitarget detection of soldering defects on surface-mounted components in Printed Circuit Board (PCB) fabrication, we propose a sample generation method using Stable Diffusion Model and ControlNet, as well as a defect detection method based on the Swin Transformer. The method consists of two stages: First, high-definition original images collected in industrial production and the corresponding prompts are input to Stable Diffusion Model and ControlNet for automatic generation of nonindependent samples. Subsequently, we integrate Swin Transformer as the backbone into the Cascade Mask R-CNN to improve the quality of defect features extracted from the samples for accurate detection box localization and segmentation. Instead of segmenting individual components on the PCB, the method inspects all components in the field of view simultaneously over a larger area. The experimental results demonstrate the effectiveness of our method in scaling up nonindependent sample datasets, thereby enabling the generation of high-quality datasets. The method accurately recognizes targets and detects defect types when performing multitarget inspection on printed circuit boards. The analysis against other models shows that our improved defect detection and segmentation method improves the Average Recall (AR) by 2.8% and the mean Average Precision (mAP) by 1.9%.