Surface Defect Classification Research Articles

Research on Steel Surface Defect Detection Algorithm Based on Improved Deep Learning

With the development of industrial economy, more and more enterprises use machine vision and artificial intelligence to replace manual detection. Therefore, the research of steel surface defect detection based on artificial intelligence is of great significance to promote the rapid development of intelligent factory and intelligent manufacturing system. In this paper, Yolov5 deep learning algorithm is used to build a classification model of steel surface defects to realize the classification and detection of steel surface defects. At the same time, on the basis of Yolov5, combined with the attention mechanism, the backbone network is improved to further improve the classification model of steel surface defects. The experiment shows that the Recall and mAP of improved Yolov5 perform better on the steel surface defect data set. Compared with Yolov5, the number of C3CA-Yolov5 parameters decreased by 13.02%, and the size of pt files decreased by 12.72%; the number of C3ECA-Yolov5 parameters decreased by 13.36%, and the size of pt files decreased by 13.22%.

Intelligent Detection and Classification of Surface Defects on Cold-Rolled Galvanized Steel Strips Using a Data-Driven Faulty Model With Attention Mechanism

Abstract In the production of cold-rolled galvanized steel strips used for stamping car body parts, the in-situ and real-time defect detection is crucial for quality control, in which various types of defects inevitably occur. It is challenging to improve the accuracy of defect detection and classification by appropriate means to assist the manual screening process better. Defects under actual production conditions are often not prominent enough in defect characteristics, and there may be a significant similarity between different defect categories. To eliminate this weakness, we propose a data-driven deep learning approach named steel surface faulty detection attention net (SSFDANet) that uses images of the galvanized steel surfaces as input to identify whether the product is qualified and automatic classification of defect types instantaneously. This method can shorten product inspection time and improve the production line automation efficiency. In addition, the attention mechanism is utilized to enhance the performance of SSFDANet. Compared with the baseline ResNet, SSFDANet achieves a noticeable improvement in classification accuracy on test data. The well-trained model can successfully show an improved performance than the baseline models on the multiple types of faulty. Enhanced by SSFDANet with high classification accuracy, the defect rate of products is significantly reduced, and the production speed of the production line is significantly improved. Future prospective studies that are inspired by this article are also discussed.

Small samples data augmentation and improved MobileNet for surface defects classification of hot-rolled steel strips

Surface defects in hot-rolled steel strips are one of the common product problems for the steel industry, which harm the product appearance, affect the corrosion and wear resistance, and shorten the product service life. The natural defect samples are sparse, category imbalanced, and expensive manual annotations. Therefore, it is crucial to study the data augmentation and classification methods for small sample surface defects. To solve the above problems and improve the accuracy and real-time performance of defect classification, we propose a random offline data augmentation algorithm (Random-CutMix) and an improved MobileNet architecture (SP-MobileNet). The Random-CutMix algorithm expands the dataset by random sampling to balance the number of each defect class. The SP-MobileNet combines the inverse residual module with the channel shuffle mechanism (CSIn-Module) and pyramid split attention (PSA) module, which facilitates cross-group information flow and improves model representation capability and generalization performance with low computational cost. The accuracy, recall, F1 score, parameter, computational complexity, and frame rate of SP-MobileNet with Random-CutMix on the X-SDD dataset were 95.97%, 95.22%, 95.46%, 6.5 M, and 0.54 G, 72 FPS, respectively. The experiment results indicate that our method outperforms the state-of-the-art methods and provides an effective trade-off between accuracy and instantaneity in actual industrial production.

Automated steel surface defect detection and classification using a new deep learning-based approach

Automated steel surface defect detection and classification using a new deep learning-based approach

MVGCN: Multi-View Graph Convolutional Neural Network for Surface Defect Identification Using Three-Dimensional Point Cloud

Abstract Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

A New Cycle-consistent Adversarial Networks With Attention Mechanism for Surface Defect Classification With Small Samples

Surface defect detection is the essential process to ensure the quality of products. Surface defect classification (SDC) based on deep learning (DL) has shown its great potential. However, the well-trained SDC model usually requires large training data, and the small intraclass differences between the defect and normal samples also degrades the performance of SDC model. To overcome these drawbacks, this article proposed a new cycle-consistent adversarial networks with attention mechanism (AttenCGAN). First, AttenCGAN is used for synthesizing defect samples to enlarge the samples volume. Second, the attention mechanism is adopted for the feature enhancement by finding the discriminative parts of the samples and enlarging the differences among the samples. AttenCGAN is tested on KolektorSDD and DAGM2007 datasets, and its accuracies are 98.53% and 99.57% with only a few samples. The experiment results show that AttenCGAN outperforms other published SDC methods based on DL and machine learning, which validates its potential.

Research on aluminum defect classification algorithm based on deep learning with attention mechanism

Product quality is an important indicator for determining the quality of industrial products. Defects on the surface of aluminum profiles are inevitably caused in the actual production process due to the influence of various factors such as environment and equipment, and these defects seriously affect the quality of aluminum profiles. The focus and difficulty of research have shifted to how to quickly and accurately identify and classify surface defects in aluminum profiles. To address this issue, this paper proposes an aluminum defect classification algorithm that uses an attention mechanism in conjunction with the traditional Inception V4 network model for deep learning image classification, to accurately identify and classify aluminum defect areas. Experiments and comparative analysis are performed on the aluminum defect recognition dataset from the Alias Tianchi platform, and the results show that the algorithm with the addition of the attention mechanism improves accuracy by 1.24% over the original model.

NSLNet: An improved deep learning model for steel surface defect classification utilizing small training datasets

Manufacturing industries contemplate integrating computer vision and artificial intelligence into shop floor operations, such as steel surface defect identification, to realize smart manufacturing goals. However, inadequate annotated training datasets and reduced prediction abilities with image perturbations restrict the practical implementation. This paper introduces NSLNet framework utilizing ImageNet as a feature-extractor combined with adversarial training in the extracted feature space through Neural Structure Learning to address these barriers. The experiments on public (NEU) and synthetically generated datasets (ENEU) showed that the NSLNet could learn with few training samples maintaining resilience against image perturbations outperforming conventional models significantly and nearest deep learning competitors marginally.

Research on Steel Surface Defect Detection Based on YOLOv5 with Attention Mechanism

Due to the irresistible factors of material properties and processing technology in the steel production, there may be different types of defects on the steel surface, such as rolling scale, patches and so on, which seriously affect the quality of steel, and thus have a negative impact on the economic efficiency of the enterprises. Different from the general target detection tasks, the defect detection tasks have small targets and extreme aspect ratio targets. The contradiction of high positioning accuracy for targets and their inconspicuous features makes the defect detection tasks difficult. Therefore, the original YOLOv5 algorithm was improved in this paper to enhance the accuracy and efficiency of detecting defects on steel surfaces. Firstly, an attention mechanism module was added in the process of transmitting the shallow feature map from the backbone structure to the neck structure, aiming at improving the algorithm attention to small targets information in the feature map and suppressing the influence of irrelevant information on the algorithm, so as to improve the detection accuracy of the algorithm for small targets. Secondly, in order to improve the algorithm effectiveness in detecting extreme aspect ratio targets, K-means algorithm was used to cluster and analyze the marked steel surface defect dataset, so that the anchor boxes can be adapted to all types of sizes, especially for extreme aspect ratio defects. The experimental results showed that the improved algorithms were better than the original YOLOv5 algorithm in terms of the average precision and the mean average precision. The mean average precision, demonstrating the largest increase among the improved YOLOv5 algorithms, was increased by 4.57% in the YOLOv5+CBAM algorithm. In particular, the YOLOv5+CBAM algorithm had a significant increase in the average precision for small targets and extreme aspect ratio targets. Therefore, the YOLOv5+CBAM algorithm could make the accurate localization and classification of steel surface defects, which can provide a reference for the automatic detection of steel defects.

An in situ surface defect detection method based on improved you only look once algorithm for wire and arc additive manufacturing

Purpose Wire and arc additive manufacturing (WAAM) is a widely used advanced manufacturing technology. If the surface defects occurred during welding process cannot be detected and repaired in time, it will form the internal defects. To address this problem, this study aims to develop an in situ monitoring system for the welding process with a high-dynamic range imaging (HDR) melt pool camera. Design/methodology/approach An improved you only look once version 3 (YOLOv3) model was proposed for online surface defects detection and classification. In this paper, improvements were mainly made in the bounding box clustering algorithm, bounding box loss function, classification loss function and network structure. Findings The results showed that the improved model outperforms the Faster regions with convolutional neural network features, single shot multibox detector, RetinaNet and YOLOv3 models with mAP value of 98.0% and a recognition rate of 59 frames per second. And it was indicated that the improved YOLOv3 model satisfied the requirements of real-time monitoring well in both efficiency and accuracy. Originality/value Experimental results show that the improved YOLOv3 model can solve the problem of poor performance of traditional defect detection models and other deep learning models. And the proposed model can meet the requirements of WAAM quality monitoring.

A novel CNN ensemble framework for bearing surface defects classification based on transfer learning

Bearing surface defect detection and classification methods based on machine vision have been widely used in bearing quality inspection due to their high-speed, high-precision, and non-contact advantages. However, traditional machine vision algorithms have low reusability and their development processes are often expensive and time consuming. Several deep learning-based bearing surface defect detection methods have been proposed. However, these deep learning models often require a large number of datasets, which is often difficult to achieve in the actual industry. Transfer learning provides a promising solution to the small sample difficulties associated with deep learning. However, the complexity of the illumination conditions and the huge differences between bearing dataset and ImageNet dataset make it impossible to use the current single model-based transfer learning for bearing defect detection. In this study, we propose a novel transitive transfer learning convolutional neural network (CNN) ensemble framework for classifying bearing surface defects. Only small-scale datasets are needed in this framework. A transfer path and transfer method selection strategy for transitive transfer learning is then proposed to train the deep learning models, which enhances the feature extraction ability of the CNN models on the basis of multiple illuminations. Ablation experiments are conducted to verify the effectiveness of the proposed method. Experimental results show that the proposed transitive transfer learning CNN ensemble framework has the accuracy rate of 97.51%. The average time for detecting each bearing is 155 ms, which can meet the requirements of industrial online detection.

On enhancing prediction abilities of vision-based metallic surface defect classification through adversarial training

Computer vision is augmented in various manufacturing industries to perform automated inspection operations accurately and efficiently. It has been observed that the performance of vision-based inspection approaches degrades considerably upon utilizing images captured under shop-floor conditions. This work proposes utilizing Histogram Equalization and adversarial training through Neural Structure Learning (NSL) for developing a robust vision-based Surface Defect Classification framework. A novel deep neural network architecture obtains adversarial samples in the extracted feature space instead of obtaining the same in the original input image space. The architecture can be easily integrated and employed with various machine learning models. A commonly employed steel surface defect dataset (NEU) with practical relevance to industrial cases is selected for the model training and experimental studies. The robustness of the proposed approach is evaluated over the Extended Diversity Enhanced (ENEU) dataset derived by simulating image acquisition variations similar to shop floor conditions. The results reveal that the proposed approach enhances the recognition accuracy of the baseline method from 87.7% to 92.4% over ENEU. The prediction accuracy of the proposed approach is considerably better than the traditional methods and deep learning competitors over ENEU. The qualitative and quantitative comparison of results obtained using the present approach with methods reported in the literature demonstrates the effectiveness of adversarial training in improving the generalization abilities of machine learning models.

An Effective Surface Defect Classification Method Based on RepVGG with CBAM Attention Mechanism (RepVGG-CBAM) for Aluminum Profiles

The automatic classification of aluminum profile surface defects is of great significance in improving the surface quality of aluminum profiles in practical production. This classification is influenced by the small and unbalanced number of samples and lack of uniformity in the size and spatial distribution of aluminum profile surface defects. It is difficult to achieve high classification accuracy by directly using the current advanced classification algorithms. In this paper, digital image processing methods such as rotation, flipping, contrast, and luminance transformation were used to augment the number of samples and imitate the complex imaging environment in actual practice. A RepVGG with CBAM attention mechanism (RepVGG-CBAM) model was proposed and applied to classify ten types of aluminum profile surface defects. The classification accuracy reached 99.41%, in particular, the proposed method can perfectly classify six types of defects: concave line (cl), exposed bottom (eb), exposed corner bottom (ecb), mixed color (mc), non-conductivity (nc) and orange peel (op), with 100% precision, recall, and F1. Compared with the existing advanced classification algorithms VGG16, VGG19, ResNet34, ResNet50, ShuffleNet_v2, and basic RepVGG, our model is the best in terms of accuracy, macro precision, macro recall and macro F1, and the accuracy was improved by 4.85% over basic RepVGG. Finally, an ablation experiment proved that the classification ability was strongest when the CBAM attention mechanism was added following Stage 1 to Stage 4 of RepVGG. Overall, the method we proposed in this paper has a significant reference value for classifying aluminum profile surface defects.

Casting Defect Detection and Classification of Convolutional Neural Network Based on Recursive Attention Model

The method based on deep learning shows excellent performance in the recognition and classification of surface defects of some industrial products. The method based on deep learning has high efficiency in the identification and classification of surface defects of industrial products, and the false detection rate and missed detection rate are relatively low. However, the recognition accuracy of defect detection and classification of most industrial products needs to be improved, especially for those with similar contours and relatively large structural different casting. This paper takes casting defect detection as the goal and proposes a convolutional neural network casting defect detection and classification (RCNN-DC) algorithm based on the recursive attention model. Through this model, the casting can be better identified and detected, and casting defects can be avoided as much as possible, which is of great significance to the technological development of the industry. First, use a large amount of readily available defect-free sample data to detect anomalous defects. Next, we compare the accuracy and performance of the detection model and the general recognition model. The research results show that the test effect of the RCNN-DC casting defect detection network model is significantly better than the traditional detection model, with a classification accuracy of 96.67%. Then, we compare the RCNN-DC network with three classic popular networks, GooGleNet, ResNet-50, and AlexNet. Among them, AlexNet and ResNet-50 achieved 95.00% and 95.56% classification accuracy, respectively, while GooGleNet achieved slightly better results of 96.38%. In contrast, the accuracy of RCNN-DC is 1.67% higher than that of AlexNet, while the number of FLOPs is reduced by 17.2 times, and the accuracy is 1.09% higher than that of ResNet-50, while the number of FLOPs is reduced by 99.7 times, and the accuracy is higher than GooGleNet 0.29% while FLOPs whose number has been reduced by 36.5 times.

Surface Defect Detection of Fresh-Cut Cauliflowers Based on Convolutional Neural Network with Transfer Learning.

A fresh-cut cauliflower surface defect detection and classification model based on a convolutional neural network with transfer learning is proposed to address the low efficiency of the traditional manual detection of fresh-cut cauliflower surface defects. Four thousand, seven hundred and ninety images of fresh-cut cauliflower were collected in four categories including healthy, diseased, browning, and mildewed. In this study, the pre-trained MobileNet model was fine-tuned to improve training speed and accuracy. The model optimization was achieved by selecting the optimal combination of training hyper-parameters and adjusting the different number of frozen layers; the parameters downloaded from ImageNet were optimally integrated with the parameters trained on our own model. A comparison of test results was presented by combining VGG19, InceptionV3, and NASNetMobile. Experimental results showed that the MobileNet model’s loss value was 0.033, its accuracy was 99.27%, and the F1 score was 99.24% on the test set when the learning rate was set as 0.001, dropout was set as 0.5, and the frozen layer was set as 80. This model had better capability and stronger robustness and was more suitable for the surface defect detection of fresh-cut cauliflower when compared with other models, and the experiment’s results demonstrated the method’s feasibility.

Internal defects detection and classification in hollow cylindrical surfaces using single shot detection and MobileNet

Surface defect inspection, detection, and classification in hollow cylindrical surfaces such as pipes and barrels have a significant impact on the structural integrity of various industrial products. Regular inspection and identification of the faults reduces the likelihood of faults’ aggravation, limits the damaging effects, and increases the product life. However, most of the defect detection algorithms for cylindrical surfaces rely heavily on handcrafted feature extraction limiting the ability to recognize the defects effectively. This research work proposes an image processing-based automatic defect detection and classification approach for cylindrical hollow surfaces. The proposed system uses a single shot multi-box detection (SSD) algorithm for localization and a customized lightweight deep convolutional neural network as a backbone network to classify defects generally found in industrial pipes and gun barrels. First, the image dataset is acquired from a real-time working environment using an indigenously developed borescope featuring a rotating camera and special hardware features. Later, the bounding boxes are calculated using extracted features to localize defects with SSD which takes a single shot to detect multiple objects within the image. Finally, the defected regions are classified into five classes of commonly found issues of pitting, chipping, rusting, dirt, and thermal cracking by utilizing deep learning architecture of 53 layers. It is found that the proposed approach can indicate the exact location of the classified defect in terms of angle and distance from a reference point. Also, the proposed method improves the detection and classification accuracy significantly compared to other existing methods. To encourage the development and evaluation of new methods for cylindrical surface defect detection, the dataset is also made publicly available.

Hemispherical Surface Defect Classification of Valve Core with a Noval Semi-supervised Learning

Background: In the industrial manufacturing process, manually labeling enough datasets is time-consuming, which hinders the training and deployment of defect detection models. Therefore, automatic defect detection and its classification is the premise of industrial production quality. Objectives: The study mainly discusses about the detection of the Hemispherical Surface of the valve core by machine vision method. Methods: The paper put forward a novel semi-supervised algorithm to detect the Hemispherical Surface of the Valve Core. Under the condition of the lack of labeled datasets, the paper used labeled and unlabeled samples for model training. This thesis proposed, for the first time, using the Mean Teacher semisupervised learning framework and then making changes to the model; firstly, this paper proposed to use the Stochastic Weight Average (SWA) algorithm to update the weight of the teaching model to enhance this model’s generalization ability. Furthermore, in order to select reliable datasets and calculate the consistency loss, this study also proposed an Uncertainty Filter (UF) method. Thirdly, the selection of hard-ware equipment, since the hemispherical surface is anisotropic, ring light source is used, which can lit the surface from top to bottom. Results: Experimental results show that in two different conditions, the classification accuracy can raise. On one hand, under the condition of training with a small amount of labeled datasets, the proposed semi-supervised learning model can achieve a classification accuracy of 90.51%; whereas, under the condition of the semi-supervised learning mechanism and a large amount of unlabeled datasets, the accuracy increases from 93.7% to 98.1%. Conclusion: This paper uses hemispherical metal surface as the dataset for the first time, and also innovatively optimizes the semi-supervised model. On the other hand, experimental comparative analysis indicates that the model proposed in this paper is significantly better than the comparison model, which lays the basic position for the defect detection of the hemispherical surface’s metal. At the same time, the novel semi-supervised algorithm can also be used to detect other metal part’s hemispherical surfaces.

ILCS: An Improved Lightweight Convolution Structure and Mixed Interactive Attention for Steel Surface Defect Classification.

The classification method of steel surface defects based on deep learning provides a basis for quality control of industrial steel manufacturing. Due to a large number of interference in the steel production area and the limited computing resources of the edge equipment deployed in the production area, it is a challenge to develop a lightweight model to achieve rapid and accurate classification in the case of limited computing resources. In this article, an improved lightweight convolution structure (LCS) is proposed, which combines the separable structure of convolution and introduces depth convolution and point direction convolution instead of the traditional convolutional module, so as to realize the lightweight of the model. In order to ensure the classification accuracy, spatial attention and channel attention are combined to compensate for the accuracy loss after deep convolution and point direction convolution respectively. Further, in order to improve the classification accuracy, a mixed interactive attention module (MIAM) is proposed to enhance the extracted feature information after LCS. The experimental results show that the recognition accuracy of our method exceeds that of the traditional model, and the number of parameters and the amount of calculation are greatly reduced, which realizes the lightweight of the steel surface defect classification model.

Quantifying the atomistic free-volume morphology of materials with graph theory

We introduce a new computational methodology for the identification and characterization of free volume within/around atomistic configurations. This scheme employs a three-stage workflow, by which spheres are iteratively grown inside of voxels, and ultimately converted to planar graphs, which are then characterized via a graph-based order parameter. Our approach is computationally efficient, physically intuitive, and universally transferable to any material system. Validation of our methodology is performed on several sets of materials problems: (1) classification of unique free volumes in various crystal phases, (2) autonomous detection and classification of complex surface defects during epitaxial growth simulations, (3) characterization of free volume defects in metals/alloys, and (4) quantification of the spatio-temporal behavior of nano-scale free volume morphologies as a function of both temperature and free-volume size. Our method accurately identifies and characterizes unique free volumes over a multitude of systems and length scales, indicating its potential for future use in understanding the relationship between free volume morphology and material properties under both static and dynamic conditions.

A nondestructive testing method for detecting surface defects of Si3N4-Bearing cylindrical rollers based on an optimized convolutional neural network

Aiming at solving the problems of low detection efficiency, poor accuracy rate, and low applicability of the traditional detection methods for surface defects on the silicon nitride (Si3N4) bearing cylindrical rollers. In this study, to detect the surface defects on Si3N4-bearing cylindrical rollers, a nondestructive testing (NDT) method based on an optimized convolutional neural network is proposed, which uses machine vision system to detect the surface defects. The optimized convolutional neural network is a two-stage network. It combines a semantic segmentation sub-network and a decision sub-network. The semantic segmentation sub-network performs an end-to-end learning based on the features of the original image. It completes defect feature extraction to segment the surface defect area from the normal area. The decision sub-network classifies the defects in the segmented surface defect area. The experimental results show that the detection time of surface defects on Si3N4-bearing cylindrical rollers by the proposed network was 72 ms. In addition, its Accuracy (Acc), Percision (Pre), Sensitivity (Sen), and Specificity (Spe) were 97.5%, 99.9%, 99.0%, 98.6%, respectively, which significantly improved the Acc and Pre of the detection of surface defects on Si3N4-bearing cylindrical rollers compared with the traditional detection method. On the other hand, its mIoU, mPA were 84.4%, 92.9%, respectively, meaning brilliant ability of image segmentation for this network. To sum up, the ability of the proposed network to detect and classify surface defects on Si3N4-bearing cylindrical rollers is excellent.