Surface Defect Detection Research Articles

An unknown wafer surface defect detection approach based on Incremental Learning for reliability analysis

Wafer maps include information about multiple defect patterns on the wafer surface. Intelligent categorization of the defective wafer is essential for investigating the underlying causes and improving the reliability and safety of the entire system. Recently, convolutional neural networks (CNNs) have been widely employed to construct successful defect detectors by learning from offline defect datasets. However, traditional CNN-based detectors are costly and incapable of unknown production defect detection despite the accurate performance. In this paper, we propose a novel IL-based method, called PIRB, for online unknown wafer defect detection. Specifically, we leverage the neural networks to remember old defect patterns by selectively restricting learning on the important weights. A tiny reference buffer is applied to preserve the experienced wafer defect patterns in the learning process to facilitate the detection accuracy. The experimental results show that the proposed method works well for classifying unknown defects, with a 60% reduction in training time compared to offline learning and a 10% increase in total accuracy compared to the state-of-the-art methods.

Swin-Transformer -YOLOv5 for lightweight hot-rolled steel strips surface defect detection algorithm.

An essential industrial application is the examination of surface flaws in hot-rolled steel strips. While automatic visual inspection tools must meet strict real-time performance criteria for inspecting hot-rolled steel strips, their capabilities are constrained by the accuracy and processing speed of the algorithm used to identify defects. To solve the problems of poor detection accuracy, low detection efficiency, and unsuitability of low computing power platforms of the hot-rolled strip surface defect detection algorithm The Swin-Transformer-YOLOv5 model based on the improved one-stage detector is proposed. By employing GhostNet, the model's lightweight design, and guaranteed detection accuracy are both achieved. The C3 module introduces Swin-Transformer to address the issues of cluttered backdrops of defect photos and easily confused defect categories. With the addition of the CoordAttention module, the model's capacity to extract defective features is improved, and its performance keeps getting better. The issue of huge differences in different scales and poor detection of small flaws is resolved by employing BiFPN for feature fusion, and the detector's capacity to adapt to targets of different scales is improved. The experimental results demonstrate that the improved Swin-Transformer-Yolov5 model significantly outperforms the industry-standard target detection algorithms, and the model's mAP value still improves by 8.39% over the original model while reducing the number of parameters, GFLOPs, and weight by 36.6%, 40.0%, and 34.7%, respectively. The model is better suited for use on low-arithmetic platforms as a result.

MPFANet: a multipath feature aggregation network for steel surface defect detection

As a crucial component of product quality surveillance in the industrial manufacturing field, surface defect detection plays a pivotal role in achieving industrial automation. Although numerous defect detection methods have been widely used in steel production, complicated defect characteristics still severely affect the accuracy and efficiency of defect detection methods. To solve this problem, we propose a multi-path feature aggregation network that can significantly improve steel defect detection performance. Firstly, we design an efficient multi-path aggregation feature extraction module combining gradient path with data path propagation design paradigm as the backbone feature extraction unit based on YOLOv5. Secondly, in the branch path of the backbone, deformable convolution is introduced to expand the effective receptive field and enhance the capability of extracting irregularly distributed defect features. Furthermore, we propose a chained feature enhanced pooling module that refines feature information to obtain feature maps containing more complicated defect details in the pooling layer. Experimental results on the publicly available steel surface defect datasets show that our proposed model achieves a mean average precision of 81.9 % on Northeastern University-DET and 72.1 % on GC10-DET improving by 4.6 % and 3.4 % over the baseline, respectively. The proposed defect detection network can achieve a balance between accuracy and efficiency, and meet the demand in practical production. The detection speed of our work reaches 41.1 frames per second (FPS) and 40.0 FPS on the above two datasets, which demonstrates that our method has achieved comprehensive performance in steel surface defect detection.

Sketch-guided spatial adaptive normalization and high-level feature constraints based GAN image synthesis for steel strip defect detection data augmentation

Deep learning methods have made remarkable strides in surface defect detection. But, they heavily rely on large amount of training data, which can be a costly endeavor, especially for specific applications like steel strip surface defect detection, where acquiring and labeling large-scale data is impractical due to the rarity of certain defective categories in production environment. Hence, realistic defect image synthesis can greatly alleviate this issue. However, training image generation networks also demand substantial data, making image data augmentation merely an auxiliary effort. In this work, we propose a Generative Adversarial Network (GAN)-based image synthesis framework. We selectively extract the defect edges of the original image as well as the background texture information, and use them as network input through the spatially-adaptive (de)normalization (SPADE) module. This enriches the input information, thus significantly reducing the amount of training data for GAN network in image generation, and enhancing the background details as well as the defect boundaries in the generated images. Additionally, we introduce a novel generator loss term that balances the similarity and perceptual fidelity between synthetic and real images by constraining high-level features at different feature levels. This provides more valuable information for data augmentation in training object detection models using synthetic images. Our experimental results demonstrate the sophistication of the proposed image synthesis method and its effectiveness in data augmentation for steel strip surface defect detection tasks.

Unleashing the power of AI in detecting metal surface defects: an optimized YOLOv7-tiny model approach.

The detection of surface defects on metal products during the production process is crucial for ensuring high-quality products. These defects also lead to significant losses in the high-tech industry. To address the issues of slow detection speed and low accuracy in traditional metal surface defect detection, an improved algorithm based on the YOLOv7-tiny model is proposed. Firstly, to enhance the feature extraction and fusion capabilities of the model, the depth aware convolution module (DAC) is introduced to replace all ELAN-T modules in the network. Secondly, the AWFP-Add module is added after the Concat module in the network's Head section to strengthen the network's ability to adaptively distinguish the importance of different features. Finally, in order to expedite model convergence and alleviate the problem of imbalanced positive and negative samples in the study, a new loss function called Focal-SIoU is used to replace the original model's CIoU loss function. To validate the effectiveness of the proposed model, two industrial metal surface defect datasets, GC10-DET and NEU-DET, were employed in our experiments. Experimental results demonstrate that the improved algorithm achieved detection frame rates exceeding 100 fps on both datasets. Furthermore, the enhanced model achieved an mAP of 81% on the GC10-DET dataset and 80.1% on the NEU-DET dataset. Compared to the original YOLOv7-tiny algorithm, this represents an increase in mAP of nearly 11% and 9.2%, respectively. Moreover, when compared to other novel algorithms, our improved model demonstrated enhanced detection accuracy and significantly improved detection speed. These results collectively indicate that our proposed enhanced model effectively fulfills the industry's demand for rapid and efficient detection and recognition of metal surface defects.

Research on surface defect detection algorithm of pipeline weld based on YOLOv7

Aiming at the problems of low target detection accuracy and high leakage rate of the current traditional weld surface defect detection methods and existing detection models, an improved YOLOv7 pipeline weld surface defect detection model is proposed to improve detection results. In the improved model, a Le-HorBlock module is designed, and it is introduced into the back of fourth CBS module of the backbone network, which preserves the characteristics of high-order information by realizing second-order spatial interaction, thus enhancing the ability of the network to extract features in weld defect images. The coordinate attention (CoordAtt) block is introduced to enhance the representation ability of target features, suppress interference. The CIoU loss function in YOLOv7 network model is replaced by the SIoU, so as to optimize the loss function, reduce the freedom of the loss function, and accelerate convergence. And a new large-scale pipeline weld surface defect dataset containing 2000 images of pipeline welds with weld defects is used in the proposed model. In the experimental comparison, the improved YOLOv7 network model has greatly improved the missed detection rate compared with the original network. The experimental results show that the improved YOLOv7 network model mAP@80.5 can reach 78.6%, which is 15.9% higher than the original model, and the detection effect is better than the original network and other classical target detection networks.

Improved Deep Learning Model for Workpieces of Rectangular Pipeline Surface Defect Detection

This study introduces a novel approach to address challenges in workpiece surface defect identification. It presents an enhanced Single Shot MultiBox Detector model, incorporating attention mechanisms and multi-feature fusion. The research methodology involves carefully curating a dataset from authentic on-site factory production, enabling the training of a model with robust real-world generalization. Leveraging the Single Shot MultiBox Detector model lead to improvements integrating channel and spatial attention mechanisms in the feature extraction network. Diverse feature extraction methods enhance the network’s focus on crucial information, improving its defect detection efficacy. The proposed model achieves a significant Mean Average Precision (mAP) improvement, reaching 99.98% precision, a substantial 3% advancement over existing methodologies. Notably, the proposed model exhibits a tendency for the values of the P-R curves in object detection for each category to approach 1, which allows a better balance between the requirements of real-time detection and precision. Within the threshold range of 0.2 to 1, the model maintains a stable level of precision, consistently remaining between 0.99 and 1. In addition, the average running speed is 2 fps lower compared to other models, and the reduction in detection speed after the model improvement is kept within 1%. The experimental results indicate that the model excels in pixel-level defect identification, which is crucial for precise defect localization. Empirical experiments validate the algorithm’s superior performance. This research represents a pivotal advancement in workpiece surface defect identification, combining technological innovation with practical efficacy.

Robustness defect detection: Improving the performance of surface defect detection in interference environment

In industrial production environments, the performance of surface defect detection methods for products is affected by disturbances in the equipment, manufacturing processes, and the environment, resulting in poor robustness of the training model. In this paper, an improved defect detection method based on feature extraction fusion and dynamic label assignment is proposed. Three improved network structures, i.e., the adaptive feature recognition convolution (AFRC), path aggregation balanced feature pyramid, and dynamic label assignment, are proposed for the feature extraction, feature fusion, and detection phases of the existing Faster RCNN model, respectively. The AFRC is employed to improve the feature extraction ability of the model for irregular defects, the path balance feature pyramid is utilized to enhance the recognition ability of the model for different levels of feature maps, and the dynamic label assignment is employed to improve the quality of the candidate frames. Under disturbed environments, the results of the proposed improvements outperform the benchmark faster RCNN model in terms of average precision (AP) by 4.1%, 3.7%, and 4.2% on an aluminum surface defect dataset and by 1.8%, 2.2%, and 3.7% AP on a magnetic tile surface defect dataset. The experimental results demonstrate that the proposed scheme effectively improves the robustness of the baseline model in environments with various disturbances.

Electrolytic capacitor surface defect detection based on deep convolution neural network

The existing methods for detecting surface defects in electrolytic capacitors are typically based on conventional machine vision, with limited feature extraction capabilities, poor versatility, slow detection speed, and the inability to achieve accurate and real-time defect detection. In this study, a real-time object detection algorithm based on an improved single shot multibox detector (SSD) is proposed to achieve omnidirectional surface defect detection of electrolytic capacitors. First, an electrolytic capacitor surface image acquisition device was established to capture omnidirectional surface images of the capacitors, and an electrolytic capacitor surface defect dataset was created. Next, the visual geometry group (VGG)-16 network structure was replaced with the MobileNetv2 network structure, effectively reducing the model’s parameter count and improving inference speed. Moreover, the Multibox Loss function was replaced with the Focal Loss function to increase the model’s attention to difficult-to-classify samples and improve model accuracy. Additionally, a transfer learning network model was designed to apply the model to electrolytic capacitors of different colors using small sample learning. Finally, the performance of the improved network model was tested on a dataset of electrolytic capacitor surface defects. The experimental results demonstrate that the parameters quantity of improved model is 3.50 M, the mAP value reaches 92.67 %, which is improved by 2.54 %, and the Macro-F1 value reaches 92.15 %, which is 11.32 % higher than that before improvement. Thus, the proposed improved SSD model provides a theoretical basis and technical prerequisites for automated and intelligent surface defect detection in electrolytic capacitors.

A steel surface defect detection model based on graph neural networks

Steel is an indispensable raw material in the construction industry. To avert catastrophic events such as building collapse, it is essential to detect minute defects on steel surfaces during production. However, this has been a persistent challenge due to the minuscule and dense nature of these defects. To this end, we propose an efficient defect detector called Vision Grapher with Hadamard, which employs a novel attention mechanism (HDmA) to establish local-to-local relationships within an image and integrates global relationships by graph convolution. With the HDmA module, we can not only fuse information under the same field of view, but also under different fields of view, which significantly enhances the richness of the acquired features. In addition, compared to convolutional neural networks, graph neural networks can utilize the contextual information in the image more effectively and resulting in better performance. We evaluate our model on the NEU-DET and GC-10 benchmark datasets, which encompass six and ten types of defects on the surfaces of hot-rolled and cold-rolled steel, and our model achieves a mean average precision of 79.04% and 66.93% on the two datasets, respectively. The results demonstrate that our model significantly improves the accuracy of defect detection compared to existing methods.

Application of laser scanning thermography and regression analysis to determine characteristics of defects in polymer composite materials

The method of laser point scanning thermography is highly sensitive and allows for reliable detection of surface and subsurface defects in products made of polymer composite materials. When implementing this method, the use of robotic manipulators as a scanning device makes it possible to inspect small objects with a curved surface or to further examine questionable areas identified by other methods. The article provides information about the layout of a robotic complex for laser scanning thermography based on a five-axis robotic manipulator, laser power up to 3 W and wavelength 405 nm, as well as a COX CG640 thermal imager. A technique for processing experimental data has been proposed and regression models have been developed to make it possible to measure the size of defects along the trajectory and determine their location. To test the protocol, a control sample was made from fiberglass laminate, including artificial defects of the “delamination” type, in the form of squares of various sizes. The coefficient of determination R2 of regression models turned out to be no worse than 0.94, the root mean square error of the defect model and the transverse size were no worse than ±0.2 and ±1.5 mm2, respectively.

A real-time anchor-free defect detector with global and local feature enhancement for surface defect detection

Industrial surface defect detection (ISDD) is vital for manufacturing enterprises to control product quality. Many general object detection (GOD) methods are utilized in this field. However, they rarely take into full account the characteristics of industrial defects. We identify three crucial characteristics in ISDD: complex background, small size defect and irregular shape. To copy with it, in this paper, we proposed a novel real-time anchor-free defect detector for ISDD. Firstly, to reduce noise interfere from complex background, we proposed global feature enhancement module (GFEM) to enhance high-level feature’s ability in modeling global information so that background noises are alleviated. Secondly, to enhance small size defect’s feature, we introduced local feature enhancement module (LFEM). It enhances small size defect’s feature by amplifying local peaks in low-level features. Thirdly, we introduced box refinement module (BRM) to capture defect’s shape information to provide more accurate prediction result. Lastly, we evaluated the proposed defect detector’s effectiveness using three public ISDD datasets. The experimental results are promising: our detector achieves a mAP of 92.0% on PVEL_AD, 99.6% on the PCB defect dataset, and 81.6% on NEU-DET. These scores outperform state-of-the-art methods, proving the superiority of our proposed detector. Additionally, it reached 46.1 FPS on the PVEL_AD dataset, showing its capability for real-time detection.

Pyramid cross attention network for pixel-wise surface defect detection

Surface defect detection plays a crucial role in improving the overall quality of industrial production processes and ensuring that the resulting products meet the required quality standards. However, detecting these defects can be challenging due to various issues, including low image contrast, significant background noise, variable defect scales, and blurred boundaries. Although several segmentation networks have been developed to address these challenges, they still face difficulties in preserving fine-grained details during the encoding process and maintaining global features during the decoding process. Moreover, the simple skip connection that combines global and local information for feature fusion fails to consider their discrepancies and varying levels of significance. To overcome these limitations, this paper proposes a Pyramid Cross Attention Network (PCANet) for pixel-level surface defect detection. The encoder extracts multiresolution features, and the pyramid adaptive selection module (PASM) is introduced to supplement lost information and adaptively select information based on its importance. Furthermore, the cross attention fusion module (CAFM) is designed to address the incompatibility between the features extracted by the encoder and decoder in the raw skip connection, while strengthening defect areas and suppressing irrelevant noise using cross attention mechanisms. Finally, extensive experimental results demonstrate that the proposed PCANet outperforms other state-of-the-art methods in terms of mean Intersection over Union (mIoU). Specifically, it achieves an mIoU of 84.57 % in NEU-Seg and 79.37 % in MT_defect, respectively.

A flexible permanent magnetic field perturbation testing probe based on through-slot flexible magnetic sheet

In the field of non-destructive testing (NDT), the detection of surface cracks using permanent magnetic field perturbation (PMFP) testing has emerged as a novel approach. It is particularly well-suited for detecting ferromagnetic materials due to its capability to identify open cracks, facilitated by a compact and straightforward probe structure. However, the existing rigid PMFP probes pose challenges in terms of shape-conforming, resulting in large lift-off fluctuations during sweeping, and are prone to wear and tear. Furthermore, their suitability is limited when inspecting specimens with various specifications and irregular shapes. To address these issues, this paper proposes a novel PMFP detection method based on the magnetization of a Flexible Magnetic Sheet (FMS). For further optimization, the machining of a through-slot in the FMS, with the magnetic sensor embedded within, serves to capture the magnetic field perturbation. This configuration is employed for the detection of surface defects. Compared to the conventional PMFP method that relies on rigid permanent magnetization, its flexibility makes the proposed method offer notable advantages such as streamlined processing, lightweight construction, and inherent adaptability to various shapes. Moreover, compared to the traditional PMFP structure where the sensor is placed beneath the permanent magnet, the through-plot FMS design exhibits a larger and more uniform magnetic field. Consequently, this feature significantly improves the signal-to-noise ratio by approximately 30%. The flexible testing method has a high application value and can effectively detect rectangular slots of size 1 mm * 1 mm* 12.5 mm and flat-bottom holes of size Φ2mm* 0.5 mm in pipeline testing.

Identification of surface defects on solar PV panels and wind turbine blades using attention based deep learning model

The global generation of renewable energy has rapidly increased, primarily due to the installation of large-scale renewable energy power plants. However, monitoring renewable energy assets in these large plants remains challenging due to environmental factors that could result in reduced power generation, malfunctioning, and degradation of asset life. Therefore, the detection of surface defects on renewable energy assets is crucial for maintaining the performance and efficiency of these plants. This paper proposes an innovative detection framework to achieve an economical surface monitoring system for renewable energy assets. High-resolution images of the assets are captured regularly and inspected to identify surface or structural damages on solar panels and wind turbine blades. Vision transformer (ViT), one of the latest attention-based deep learning (DL) models in computer vision, is proposed in this work to classify surface defects. The ViT model outperforms other DL models, including MobileNet, VGG16, Xception, EfficientNetB7, and ResNet50, achieving high accuracy scores above 97% for both wind and solar plant assets. From the results, our proposed model demonstrates its potential for monitoring and detecting damages in renewable energy assets for efficient and reliable operation of renewable power plants.

Diffuse Reflection Imaging for Detection of Surface Defect Based on Machine Learning

Diffuse reflection imaging relies on the fundamental principles of light interaction with rough or matte surfaces, enabling light to scatter in various directions. Unlike specular reflection, which directs light in a single path, diffuse reflection occurs sporadically. Its distinctive quality lies in its effectiveness for imaging objects with irregular or nonreflective surfaces, offering comprehensive information challenging to attain through alternative imaging methods. As a burgeoning subfield within optical and computer vision technologies, diffuse reflection imaging holds immense potential across scientific, industrial, and medical domains. This abstract succinctly outlines the key aspects of diffuse reflection imaging and underscores its diverse applications, marking it as a promising and newly developed field with substantial implications.

AnomalySeg: Deep Learning-Based Fast Anomaly Segmentation Approach for Surface Defect Detection

Product quality inspection is a crucial element of industrial manufacturing, yet flaws such as blemishes and stains frequently emerge after the product is completed. Most research has utilized detection models and avoided segmenting networks due to the unequal distribution of faulty information. To overcome this challenge, this work presents a rapid segmentation-based technique for surface defect detection. The proposed model is based on a modified U-Net, which introduces a hybrid residual module (SAFM), combining an improved spatial attention mechanism and a feedforward neural network in place of the remaining downsampling layers, except for the first layer of downsampling in the encoder, and applies this residual module to the decoder structure. Dilated convolutions are also incorporated in the decoder to obtain more spatial information about the feature defects and to reduce the gradient vanishing problem of the model. An improved hybrid loss function with Dice and focal loss is introduced to alleviate the small defect segmentation problem. Comparative experiments were conducted on different segmentation-based inspection methods, revealing that the Dice coefficient (DSC) evaluated by the proposed approach is better than previous generic segmentation benchmarks on KolektorSDD, KolektorSDD2, and RSDD datasets, with fewer parameters and FLOPs. Additionally, the detection network displays higher precision in recognizing the characteristics of minor flaws. This paper proposes a practical and effective technique for anomaly segmentation in surface defect identification, delivering considerable improvements over previous methods.

YOLOv5s‐GC‐Based Surface Defect Detection Method of Strip Steel

Detecting surface defects in strip steel is significantly important for improving production efficiency and product quality. Herein, a novel surface defect detection of strip steel method based on You Only Look Once (YOLO)v5s‐GC is proposed. First, a ResNet‐Mini is developed to preclassify the original dataset to reduce the number of calculations. Subsequently, image preprocessing is conducted to enhance the defect features, which consists of two steps: the first is combining the ResNet‐Mini network weights with Grad‐CAM to crop defective areas and remove background interference and the second is applying the OTSU and normal distribution enhancement algorithm to extract the feature grayscale. Furthermore, a size enhancement strategy is adopted to obtain larger data sizes that could simulate an actual application scenario with a large‐area saccade of strip steel. Finally, the cross‐stage partial module of YOLOv5s is replaced with GhostBottleneck, and the convolutional block attention module is added to the detection neck. The average accuracy of YOLOv5s‐GC for detecting six types of strip steel is 82.4%, marking an increase of 7.0% over YOLOv5s. In addition, the calculation amount of YOLOv5s‐GC is reduced by 48%, and the inference speed is increased by 24%.

Casting-DETR: An End-to-End Network for Casting Surface Defect Detection

Casting-DETR: An End-to-End Network for Casting Surface Defect Detection

Detection of surface defects for maize seeds based on YOLOv5

The external appearance of maize seeds is one of important quality evaluation indicators for maize seeds. For traditional maize seed appearance quality detection, which mainly relies on naked eyes to inspect surface defects. Although computer vision as a relative mature way can be used for sample appearance quality inspection, manual feature extraction is still required. Meanwhile, machine learning technology, especially deep learning, has developed rapidly in the last decades. The maize seed surface defect detection coupled with deep learning can effectively replace traditional detection methods, reduce manual intervention, and decrease costs. In this paper, the collection and preprocessing of maize seed images, as well as the surface defects evaluation methods of maize seeds using a deep learning framework YOLOv5 were proposed. Firstly, in terms of image acquisition, maize seed batch surface defect detection system was established to obtain images. Then, the quality of maize seed images was improved by filtering, segmentation, and enhancement, which could significantly reduce noise in the images, separate the targets from the background and replace the background. Finally, ECA-Improved-YOLOv5S-Mobilenet model, which was established to improve the feature learning performance, could extracted the features from the maize seeds image and detect defects quickly at different levels. The experimental results showed that the precision was 92.8%, the recall rate was 98.9%, and the mPA0.5 was 95.5% with 8.8 MB of model size. In general, the proposed maize seeds surface defect detection method combined with deep learning could provide a theoretical support and technical basis for future development of seed grading and plantation.