Surface Defect Detection Research Articles

Small-modulus worms surface defect detection method based on YOLOv7

Small-modulus worms are widely used in precision transmission mechanisms. The detection of surface defects on processed small-modulus worms mainly relies on manual inspection. However, this detection method has low detection efficiency and low accuracy. In response to this issue, this paper proposed a surface defect detection method for small-modulus worms based on deep learning. Firstly, based on the geometric features and material properties of small-modulus worms, designed a two-cameras worms image acquisition system to capture small-modulus worm images. And the captured images were annotated and classified to construct a small-modulus worms defect dataset. Secondly, the you only look once version 7 (YOLOv7) network model algorithm was studied and improved. A three-stage image preprocessing algorithm was added to the YOLOV7 model to reduce the interference in distorted areas. Next, Ghost-convolution, batch normalization, Silu activation (CBS) structures were employed to replace some of the conventional CBS structures, reducing computational complexity. And we also added a width–height balance weights and alpha structure to the efficient-IoU (EIOU) loss function. Improve the fitting performance of the model on the bounding box with large aspect ratio. Furthermore, three SimAM attention modules were added to the backbone, increasing the network’s focus on key areas. Finally, experimental validation was conducted, and the results demonstrated that the performance of our proposed method is better than other existing methods. The detection accuracy reached 95.7%, with a map@50 of 94.6%. Overall, the performance met the requirements for worm defect detection.

An Algorithm for Detecting Surface Defects in Steel Strips Based on an Improved Lightweight Network

In recent years, surface defect detection methods based on deep learning have been widely applied to steel plate surface defect detection. By locating and classifying defects on the surface of steel plates, production efficiency can be improved. However, there is still a conflict between speed and accuracy in the defect detection process. To address this issue, we propose a high-precision, low-latency surface defect detection algorithm called the GhostConv-ECA-YOLOv5 Network (GEA-Net). The GEA-Net model can predict defect categories without compromising classification and detection accuracy. Experimental results show that our proposed improved model has higher performance compared to other comparative models, achieving a 75.6% mAP on the NEU-DET dataset.

Experimental study on the surface defects of materials in bar rolling with a processing map applicable to billet-to-oval groove rolling

This paper proposes a systematic method for detecting surface defects (SDs) on specimens by performing a pilot hot (850°C–950°C) bar rolling test and suggests a processing map for billet-to-oval groove pass in bar rolling mills. Unlike previous studies, the specimens for the rolling test were prepared such that the billet surface became the surface of the specimens. A series of the rolling tests were performed at different temperature and area reduction ratios. Physical-chemical nondestructive testing was employed to detect SDs in the specimens before and after the rolling test. The depth of the SDs of the rolled specimens was measured using an optical microscope. The results revealed that the SDs depended both on the rolling parameters (temperature and reduction ratios) and the contact condition between the specimen and the roll groove during rolling, which is one of the characteristics of bar rolling that distinguishes it from flat rolling. The decrease in temperature caused an increase in the defect depth above the appropriate reduction ratio of 24.61% when the part of the specimen was in contact with the roll shoulder curve and the relief line of the oval roll groove. The proposed processing map presents a guiding path for operators in actual bar rolling mills to quickly determine a rolling condition (combination of temperature and reduction ratio), avoid SDs, and minimize defect depth within an acceptable range.

Collaborative Filter Pruning for Efficient Automatic Surface Defect Detection

Surface defect detection is a critical task in industrial production, and numerous methods have been proposed to achieve high detection accuracy. Although deep learning-based approaches have achieved state-of-the-art performances, their vast computational cost and high memory footprint prevent their deployment in resource-constrained environments. To address this problem, we propose a collaborative filter pruning method for the defect detection model, which significantly reduces the number of required calculations and parameters while maintaining high performance, even in cases with tasks suffering from the class imbalance problem. Our method aims to obtain lightweight pruned models by removing unimportant filters according to their importance evaluated by both structural similarity and detail richness of corresponding feature maps. Moreover, to improve the performance of pruned models, we propose a knowledge-fused fine-tuning approach that fuses the knowledge derived from two teacher networks to look after both representation learning and classifier learning, alleviating the class imbalance problem. Experimental results on four public datasets demonstrate that the proposed approach performs favorably relative to the state-of-the-art methods. In particular, the proposed method achieves 39× and 59× parameter compression for VGG-16 and ResNet-50, respectively, on the NEU-CLS dataset, with a very small detection accuracy loss (< 0.2%). The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/HXuan-Wang/CFP</uri> .

An unknown wafer surface defect detection approach based on Incremental Learning for reliability analysis

Wafer maps include information about multiple defect patterns on the wafer surface. Intelligent categorization of the defective wafer is essential for investigating the underlying causes and improving the reliability and safety of the entire system. Recently, convolutional neural networks (CNNs) have been widely employed to construct successful defect detectors by learning from offline defect datasets. However, traditional CNN-based detectors are costly and incapable of unknown production defect detection despite the accurate performance. In this paper, we propose a novel IL-based method, called PIRB, for online unknown wafer defect detection. Specifically, we leverage the neural networks to remember old defect patterns by selectively restricting learning on the important weights. A tiny reference buffer is applied to preserve the experienced wafer defect patterns in the learning process to facilitate the detection accuracy. The experimental results show that the proposed method works well for classifying unknown defects, with a 60% reduction in training time compared to offline learning and a 10% increase in total accuracy compared to the state-of-the-art methods.

Swin-Transformer -YOLOv5 for lightweight hot-rolled steel strips surface defect detection algorithm.

An essential industrial application is the examination of surface flaws in hot-rolled steel strips. While automatic visual inspection tools must meet strict real-time performance criteria for inspecting hot-rolled steel strips, their capabilities are constrained by the accuracy and processing speed of the algorithm used to identify defects. To solve the problems of poor detection accuracy, low detection efficiency, and unsuitability of low computing power platforms of the hot-rolled strip surface defect detection algorithm The Swin-Transformer-YOLOv5 model based on the improved one-stage detector is proposed. By employing GhostNet, the model's lightweight design, and guaranteed detection accuracy are both achieved. The C3 module introduces Swin-Transformer to address the issues of cluttered backdrops of defect photos and easily confused defect categories. With the addition of the CoordAttention module, the model's capacity to extract defective features is improved, and its performance keeps getting better. The issue of huge differences in different scales and poor detection of small flaws is resolved by employing BiFPN for feature fusion, and the detector's capacity to adapt to targets of different scales is improved. The experimental results demonstrate that the improved Swin-Transformer-Yolov5 model significantly outperforms the industry-standard target detection algorithms, and the model's mAP value still improves by 8.39% over the original model while reducing the number of parameters, GFLOPs, and weight by 36.6%, 40.0%, and 34.7%, respectively. The model is better suited for use on low-arithmetic platforms as a result.

MPFANet: a multipath feature aggregation network for steel surface defect detection

As a crucial component of product quality surveillance in the industrial manufacturing field, surface defect detection plays a pivotal role in achieving industrial automation. Although numerous defect detection methods have been widely used in steel production, complicated defect characteristics still severely affect the accuracy and efficiency of defect detection methods. To solve this problem, we propose a multi-path feature aggregation network that can significantly improve steel defect detection performance. Firstly, we design an efficient multi-path aggregation feature extraction module combining gradient path with data path propagation design paradigm as the backbone feature extraction unit based on YOLOv5. Secondly, in the branch path of the backbone, deformable convolution is introduced to expand the effective receptive field and enhance the capability of extracting irregularly distributed defect features. Furthermore, we propose a chained feature enhanced pooling module that refines feature information to obtain feature maps containing more complicated defect details in the pooling layer. Experimental results on the publicly available steel surface defect datasets show that our proposed model achieves a mean average precision of 81.9 % on Northeastern University-DET and 72.1 % on GC10-DET improving by 4.6 % and 3.4 % over the baseline, respectively. The proposed defect detection network can achieve a balance between accuracy and efficiency, and meet the demand in practical production. The detection speed of our work reaches 41.1 frames per second (FPS) and 40.0 FPS on the above two datasets, which demonstrates that our method has achieved comprehensive performance in steel surface defect detection.

Sketch-guided spatial adaptive normalization and high-level feature constraints based GAN image synthesis for steel strip defect detection data augmentation

Deep learning methods have made remarkable strides in surface defect detection. But, they heavily rely on large amount of training data, which can be a costly endeavor, especially for specific applications like steel strip surface defect detection, where acquiring and labeling large-scale data is impractical due to the rarity of certain defective categories in production environment. Hence, realistic defect image synthesis can greatly alleviate this issue. However, training image generation networks also demand substantial data, making image data augmentation merely an auxiliary effort. In this work, we propose a Generative Adversarial Network (GAN)-based image synthesis framework. We selectively extract the defect edges of the original image as well as the background texture information, and use them as network input through the spatially-adaptive (de)normalization (SPADE) module. This enriches the input information, thus significantly reducing the amount of training data for GAN network in image generation, and enhancing the background details as well as the defect boundaries in the generated images. Additionally, we introduce a novel generator loss term that balances the similarity and perceptual fidelity between synthetic and real images by constraining high-level features at different feature levels. This provides more valuable information for data augmentation in training object detection models using synthetic images. Our experimental results demonstrate the sophistication of the proposed image synthesis method and its effectiveness in data augmentation for steel strip surface defect detection tasks.

Unleashing the power of AI in detecting metal surface defects: an optimized YOLOv7-tiny model approach.

The detection of surface defects on metal products during the production process is crucial for ensuring high-quality products. These defects also lead to significant losses in the high-tech industry. To address the issues of slow detection speed and low accuracy in traditional metal surface defect detection, an improved algorithm based on the YOLOv7-tiny model is proposed. Firstly, to enhance the feature extraction and fusion capabilities of the model, the depth aware convolution module (DAC) is introduced to replace all ELAN-T modules in the network. Secondly, the AWFP-Add module is added after the Concat module in the network's Head section to strengthen the network's ability to adaptively distinguish the importance of different features. Finally, in order to expedite model convergence and alleviate the problem of imbalanced positive and negative samples in the study, a new loss function called Focal-SIoU is used to replace the original model's CIoU loss function. To validate the effectiveness of the proposed model, two industrial metal surface defect datasets, GC10-DET and NEU-DET, were employed in our experiments. Experimental results demonstrate that the improved algorithm achieved detection frame rates exceeding 100 fps on both datasets. Furthermore, the enhanced model achieved an mAP of 81% on the GC10-DET dataset and 80.1% on the NEU-DET dataset. Compared to the original YOLOv7-tiny algorithm, this represents an increase in mAP of nearly 11% and 9.2%, respectively. Moreover, when compared to other novel algorithms, our improved model demonstrated enhanced detection accuracy and significantly improved detection speed. These results collectively indicate that our proposed enhanced model effectively fulfills the industry's demand for rapid and efficient detection and recognition of metal surface defects.

Research on surface defect detection algorithm of pipeline weld based on YOLOv7

Aiming at the problems of low target detection accuracy and high leakage rate of the current traditional weld surface defect detection methods and existing detection models, an improved YOLOv7 pipeline weld surface defect detection model is proposed to improve detection results. In the improved model, a Le-HorBlock module is designed, and it is introduced into the back of fourth CBS module of the backbone network, which preserves the characteristics of high-order information by realizing second-order spatial interaction, thus enhancing the ability of the network to extract features in weld defect images. The coordinate attention (CoordAtt) block is introduced to enhance the representation ability of target features, suppress interference. The CIoU loss function in YOLOv7 network model is replaced by the SIoU, so as to optimize the loss function, reduce the freedom of the loss function, and accelerate convergence. And a new large-scale pipeline weld surface defect dataset containing 2000 images of pipeline welds with weld defects is used in the proposed model. In the experimental comparison, the improved YOLOv7 network model has greatly improved the missed detection rate compared with the original network. The experimental results show that the improved YOLOv7 network model mAP@80.5 can reach 78.6%, which is 15.9% higher than the original model, and the detection effect is better than the original network and other classical target detection networks.

Improved Deep Learning Model for Workpieces of Rectangular Pipeline Surface Defect Detection

This study introduces a novel approach to address challenges in workpiece surface defect identification. It presents an enhanced Single Shot MultiBox Detector model, incorporating attention mechanisms and multi-feature fusion. The research methodology involves carefully curating a dataset from authentic on-site factory production, enabling the training of a model with robust real-world generalization. Leveraging the Single Shot MultiBox Detector model lead to improvements integrating channel and spatial attention mechanisms in the feature extraction network. Diverse feature extraction methods enhance the network’s focus on crucial information, improving its defect detection efficacy. The proposed model achieves a significant Mean Average Precision (mAP) improvement, reaching 99.98% precision, a substantial 3% advancement over existing methodologies. Notably, the proposed model exhibits a tendency for the values of the P-R curves in object detection for each category to approach 1, which allows a better balance between the requirements of real-time detection and precision. Within the threshold range of 0.2 to 1, the model maintains a stable level of precision, consistently remaining between 0.99 and 1. In addition, the average running speed is 2 fps lower compared to other models, and the reduction in detection speed after the model improvement is kept within 1%. The experimental results indicate that the model excels in pixel-level defect identification, which is crucial for precise defect localization. Empirical experiments validate the algorithm’s superior performance. This research represents a pivotal advancement in workpiece surface defect identification, combining technological innovation with practical efficacy.

Robustness defect detection: Improving the performance of surface defect detection in interference environment

In industrial production environments, the performance of surface defect detection methods for products is affected by disturbances in the equipment, manufacturing processes, and the environment, resulting in poor robustness of the training model. In this paper, an improved defect detection method based on feature extraction fusion and dynamic label assignment is proposed. Three improved network structures, i.e., the adaptive feature recognition convolution (AFRC), path aggregation balanced feature pyramid, and dynamic label assignment, are proposed for the feature extraction, feature fusion, and detection phases of the existing Faster RCNN model, respectively. The AFRC is employed to improve the feature extraction ability of the model for irregular defects, the path balance feature pyramid is utilized to enhance the recognition ability of the model for different levels of feature maps, and the dynamic label assignment is employed to improve the quality of the candidate frames. Under disturbed environments, the results of the proposed improvements outperform the benchmark faster RCNN model in terms of average precision (AP) by 4.1%, 3.7%, and 4.2% on an aluminum surface defect dataset and by 1.8%, 2.2%, and 3.7% AP on a magnetic tile surface defect dataset. The experimental results demonstrate that the proposed scheme effectively improves the robustness of the baseline model in environments with various disturbances.

Electrolytic capacitor surface defect detection based on deep convolution neural network

The existing methods for detecting surface defects in electrolytic capacitors are typically based on conventional machine vision, with limited feature extraction capabilities, poor versatility, slow detection speed, and the inability to achieve accurate and real-time defect detection. In this study, a real-time object detection algorithm based on an improved single shot multibox detector (SSD) is proposed to achieve omnidirectional surface defect detection of electrolytic capacitors. First, an electrolytic capacitor surface image acquisition device was established to capture omnidirectional surface images of the capacitors, and an electrolytic capacitor surface defect dataset was created. Next, the visual geometry group (VGG)-16 network structure was replaced with the MobileNetv2 network structure, effectively reducing the model’s parameter count and improving inference speed. Moreover, the Multibox Loss function was replaced with the Focal Loss function to increase the model’s attention to difficult-to-classify samples and improve model accuracy. Additionally, a transfer learning network model was designed to apply the model to electrolytic capacitors of different colors using small sample learning. Finally, the performance of the improved network model was tested on a dataset of electrolytic capacitor surface defects. The experimental results demonstrate that the parameters quantity of improved model is 3.50 M, the mAP value reaches 92.67 %, which is improved by 2.54 %, and the Macro-F1 value reaches 92.15 %, which is 11.32 % higher than that before improvement. Thus, the proposed improved SSD model provides a theoretical basis and technical prerequisites for automated and intelligent surface defect detection in electrolytic capacitors.

A steel surface defect detection model based on graph neural networks

Steel is an indispensable raw material in the construction industry. To avert catastrophic events such as building collapse, it is essential to detect minute defects on steel surfaces during production. However, this has been a persistent challenge due to the minuscule and dense nature of these defects. To this end, we propose an efficient defect detector called Vision Grapher with Hadamard, which employs a novel attention mechanism (HDmA) to establish local-to-local relationships within an image and integrates global relationships by graph convolution. With the HDmA module, we can not only fuse information under the same field of view, but also under different fields of view, which significantly enhances the richness of the acquired features. In addition, compared to convolutional neural networks, graph neural networks can utilize the contextual information in the image more effectively and resulting in better performance. We evaluate our model on the NEU-DET and GC-10 benchmark datasets, which encompass six and ten types of defects on the surfaces of hot-rolled and cold-rolled steel, and our model achieves a mean average precision of 79.04% and 66.93% on the two datasets, respectively. The results demonstrate that our model significantly improves the accuracy of defect detection compared to existing methods.

Application of laser scanning thermography and regression analysis to determine characteristics of defects in polymer composite materials

The method of laser point scanning thermography is highly sensitive and allows for reliable detection of surface and subsurface defects in products made of polymer composite materials. When implementing this method, the use of robotic manipulators as a scanning device makes it possible to inspect small objects with a curved surface or to further examine questionable areas identified by other methods. The article provides information about the layout of a robotic complex for laser scanning thermography based on a five-axis robotic manipulator, laser power up to 3 W and wavelength 405 nm, as well as a COX CG640 thermal imager. A technique for processing experimental data has been proposed and regression models have been developed to make it possible to measure the size of defects along the trajectory and determine their location. To test the protocol, a control sample was made from fiberglass laminate, including artificial defects of the “delamination” type, in the form of squares of various sizes. The coefficient of determination R2 of regression models turned out to be no worse than 0.94, the root mean square error of the defect model and the transverse size were no worse than ±0.2 and ±1.5 mm2, respectively.

A real-time anchor-free defect detector with global and local feature enhancement for surface defect detection

Industrial surface defect detection (ISDD) is vital for manufacturing enterprises to control product quality. Many general object detection (GOD) methods are utilized in this field. However, they rarely take into full account the characteristics of industrial defects. We identify three crucial characteristics in ISDD: complex background, small size defect and irregular shape. To copy with it, in this paper, we proposed a novel real-time anchor-free defect detector for ISDD. Firstly, to reduce noise interfere from complex background, we proposed global feature enhancement module (GFEM) to enhance high-level feature’s ability in modeling global information so that background noises are alleviated. Secondly, to enhance small size defect’s feature, we introduced local feature enhancement module (LFEM). It enhances small size defect’s feature by amplifying local peaks in low-level features. Thirdly, we introduced box refinement module (BRM) to capture defect’s shape information to provide more accurate prediction result. Lastly, we evaluated the proposed defect detector’s effectiveness using three public ISDD datasets. The experimental results are promising: our detector achieves a mAP of 92.0% on PVEL_AD, 99.6% on the PCB defect dataset, and 81.6% on NEU-DET. These scores outperform state-of-the-art methods, proving the superiority of our proposed detector. Additionally, it reached 46.1 FPS on the PVEL_AD dataset, showing its capability for real-time detection.

Pyramid cross attention network for pixel-wise surface defect detection

Surface defect detection plays a crucial role in improving the overall quality of industrial production processes and ensuring that the resulting products meet the required quality standards. However, detecting these defects can be challenging due to various issues, including low image contrast, significant background noise, variable defect scales, and blurred boundaries. Although several segmentation networks have been developed to address these challenges, they still face difficulties in preserving fine-grained details during the encoding process and maintaining global features during the decoding process. Moreover, the simple skip connection that combines global and local information for feature fusion fails to consider their discrepancies and varying levels of significance. To overcome these limitations, this paper proposes a Pyramid Cross Attention Network (PCANet) for pixel-level surface defect detection. The encoder extracts multiresolution features, and the pyramid adaptive selection module (PASM) is introduced to supplement lost information and adaptively select information based on its importance. Furthermore, the cross attention fusion module (CAFM) is designed to address the incompatibility between the features extracted by the encoder and decoder in the raw skip connection, while strengthening defect areas and suppressing irrelevant noise using cross attention mechanisms. Finally, extensive experimental results demonstrate that the proposed PCANet outperforms other state-of-the-art methods in terms of mean Intersection over Union (mIoU). Specifically, it achieves an mIoU of 84.57 % in NEU-Seg and 79.37 % in MT_defect, respectively.

A flexible permanent magnetic field perturbation testing probe based on through-slot flexible magnetic sheet

In the field of non-destructive testing (NDT), the detection of surface cracks using permanent magnetic field perturbation (PMFP) testing has emerged as a novel approach. It is particularly well-suited for detecting ferromagnetic materials due to its capability to identify open cracks, facilitated by a compact and straightforward probe structure. However, the existing rigid PMFP probes pose challenges in terms of shape-conforming, resulting in large lift-off fluctuations during sweeping, and are prone to wear and tear. Furthermore, their suitability is limited when inspecting specimens with various specifications and irregular shapes. To address these issues, this paper proposes a novel PMFP detection method based on the magnetization of a Flexible Magnetic Sheet (FMS). For further optimization, the machining of a through-slot in the FMS, with the magnetic sensor embedded within, serves to capture the magnetic field perturbation. This configuration is employed for the detection of surface defects. Compared to the conventional PMFP method that relies on rigid permanent magnetization, its flexibility makes the proposed method offer notable advantages such as streamlined processing, lightweight construction, and inherent adaptability to various shapes. Moreover, compared to the traditional PMFP structure where the sensor is placed beneath the permanent magnet, the through-plot FMS design exhibits a larger and more uniform magnetic field. Consequently, this feature significantly improves the signal-to-noise ratio by approximately 30%. The flexible testing method has a high application value and can effectively detect rectangular slots of size 1 mm * 1 mm* 12.5 mm and flat-bottom holes of size Φ2mm* 0.5 mm in pipeline testing.

Identification of surface defects on solar PV panels and wind turbine blades using attention based deep learning model

The global generation of renewable energy has rapidly increased, primarily due to the installation of large-scale renewable energy power plants. However, monitoring renewable energy assets in these large plants remains challenging due to environmental factors that could result in reduced power generation, malfunctioning, and degradation of asset life. Therefore, the detection of surface defects on renewable energy assets is crucial for maintaining the performance and efficiency of these plants. This paper proposes an innovative detection framework to achieve an economical surface monitoring system for renewable energy assets. High-resolution images of the assets are captured regularly and inspected to identify surface or structural damages on solar panels and wind turbine blades. Vision transformer (ViT), one of the latest attention-based deep learning (DL) models in computer vision, is proposed in this work to classify surface defects. The ViT model outperforms other DL models, including MobileNet, VGG16, Xception, EfficientNetB7, and ResNet50, achieving high accuracy scores above 97% for both wind and solar plant assets. From the results, our proposed model demonstrates its potential for monitoring and detecting damages in renewable energy assets for efficient and reliable operation of renewable power plants.

Diffuse Reflection Imaging for Detection of Surface Defect Based on Machine Learning

Diffuse reflection imaging relies on the fundamental principles of light interaction with rough or matte surfaces, enabling light to scatter in various directions. Unlike specular reflection, which directs light in a single path, diffuse reflection occurs sporadically. Its distinctive quality lies in its effectiveness for imaging objects with irregular or nonreflective surfaces, offering comprehensive information challenging to attain through alternative imaging methods. As a burgeoning subfield within optical and computer vision technologies, diffuse reflection imaging holds immense potential across scientific, industrial, and medical domains. This abstract succinctly outlines the key aspects of diffuse reflection imaging and underscores its diverse applications, marking it as a promising and newly developed field with substantial implications.