Steel Surface Defect Detection Research Articles

Segmentation-based Decision Networks for Steel Surface Defect Detection

<p>With the advent of the Industrial 4.0 era, deep learning has been continuously applied to the task of surface defect detection, and effective progress has been made. However, the limited number of training samples and high labelling costs are considerable obstacles to the vigorous development of this task. Thus, we explore the use of different numbers of labels with various accuracies during training to achieve the maximum detection accuracy with the lowest cost. Our proposed method includes improved segmentation and decision networks. An attention mechanism is integrated into the segmentation subnetwork. Moreover, atrous convolutions are used in the segmentation and decision subnetworks. In addition, the original loss function is improved. Several experiments are carried out on the Severstal Steel Defect dataset collected in Germany, and the results show that each component improves the detection accuracy by 1% to 2%. Finally, when we add an appropriate number of pixel-level labels in the weakly supervised learning mode, the detection accuracy reaches that of the fully supervised mode with a significantly reduced annotation cost.</p> <p> </p>

Faster RCNN based on Feature Fusion and its Application in Steel Surface Defect Detection

Surface defects are an inevitable problem in steel production and processing. To solve the problems of error detection, leakage detection and low accuracy of traditional manual methods for steel surface defect detection, this paper proposes a steel surface defect detection algorithm based on Faster RCNN, a classical model in the field of computer vision object detection. The proposed model uses ResNet50 as its backbone and introduces FPN to fuse the multi-layer feature maps of the backbone to improve the detection capability for defects of different scales. The experimental results on NEU-DET dataset show that the proposed model in this paper has different detection accuracies for six different types of defects in the dataset, and the overall mAP reaches 0.708. In addition, the model execution speed reaches 43.3 frame/s, which meets the applications in industrial scenarios.

Review of surface defect detection of steel products based on machine vision

Steel plays an important role in industry, and the surface defect detection for steel products based on machine vision has been widely used during the last two decades. This paper attempts to review state-of-art of vision-based surface defect inspection technology of steel products by investigating about 170 publications. This review covers the overall aspects of vision-based surface defect inspection for steel products including hardware system, automated vision-based inspection method, existing problems and latest development. The types of steel product surface defects composition of visual inspection system are briefly described, and image acquisition system is introduced as well. The image processing algorithms for surface defect detection of steel products are reviewed, including image pre-processing, region of interest (ROI) detection, image segmentation for ROI, feature extraction and selection and defect classification. The important problems such as small sample and real time of steel surface defect detection are discussed. Finally, the challenge and development trend of steel surface defect detection are prospected.

The Amalgamation of the Object Detection and Semantic Segmentation for Steel Surface Defect Detection

Steel surface defect detection is challenging because it contains various atypical defects. Many studies have attempted to detect metal surface defects using deep learning and had success in applying deep learning. Despite many previous studies to solve the steel surface defect detection, it remains a difficult problem. To resolve the atypical defects problem, we introduce a hierarchical approach for the classification and detection of defects on the steel surface. The proposed approach has a hierarchical structure of the binary classifier at the first stage and the object detection and semantic segmentation algorithms at the second stage. It shows 98.6% accuracy in scratch and other types of defect classification and 77.12% mean average precision (mAP) in defect detection using the Northeastern University (NEU) surface defect detection dataset. A comparative analysis with the previous studies shows that the proposed approach achieves excellent results on the NEU dataset.

Improving the Algorithm Study of YOLO in Steel Surface Defect Detection

To solve the problem of low detection accuracy caused by background interference and diverse target forms, a series of improvements are proposed to improve the detection accuracy. According to the various characteristics of steel surface defects, this paper presents the K-Means clustering algorithm to optimize the clustering results and quickly and accurately obtain the size of the prior box. In view of the small proportion of the target defect area in the overall image and background interference, a two-way attention module (TWA-Block) is proposed to establish the long-distance dependence of the spatial domain and channel domain features, and a background suppression function is designed to realize the division of defect areas. Experiments of the proposed improvements in the NEU-DET dataset based on the YOLO series model show that the detection accuracy of all the improved YOLO series models has improved, and the number of parameters will not increase substantially

CASPPNet: a chained atrous spatial pyramid pooling network for steel defect detection

Automatic machine vision-based defect detection has been successfully applied to many industrial visual inspection applications. However, automatic steel surface defect detection is still a challenging task due to diverse defect categories, low-contrast between defect and complex texture background. To address these challenges, a chained atrous spatial pyramid pooling network (CASPPNet) is proposed for steel surface defect detection. In CASPPNet, chained atrous spatial pyramid pooling is designed to enlarge receptive field and obtain enrich semantic information. An improved global attention feature fusion module is introduced to achieve feature interaction and salience. Moreover, residual boundary refinement block is introduced to get more complete defect boundary. Comparative experimental results verify that our method is superior to the state-of-the-art segmentation methods on public accessible SD-saliency-900 datasets and can meet the requirement of real-time online detection (the detection efficiency is at over 47 FPS on a single GPU).

Deep learning-based defect detection for hot-rolled strip steel

Defects in the strip’s surface can have an impact on the strip company’s product sales. As a result, it’s crucial to spot flaws on the strip steel’s surface. The Faster R-CNN network is structurally upgraded for the issue of strip steel surface defect detection by employing FPN for feature fusion in the feature extraction layer, RoI Align instead of RoI Pooling in the pooling layer, and Softer-NMS in the final prediction fully connected layer. In addition, for training, transfer learning is employed. The experimental findings demonstrate that the suggested technique mAP outperforms the original Faster R-CNN by 7.6%, and the detection time is fast enough for strip steel detection online.

Efficient Fused-Attention Model for Steel Surface Defect Detection

Steel surface defect detection is an essential quality control task in manufacturing. As patterns of defects may be viewed as an object, some current defect detection methods, which have achieved promising performance, have been developed based on object-detection models. However, most of these defect detection methods simply incorporate additional heavy modules to improve the accuracy. These methods do not consider the efficiency of the models or the characteristics of the defects. In this paper, we focus on three challenges of steel surface defect detection, which are scale variations, shape variations, and detection efficiency. To address these challenges, we propose a fused-attention network (FANet) for detecting various steel surface defects. Specifically, we propose a fused-attention framework for efficiently detecting defects. This framework applies an attention mechanism to a single balanced feature map, rather than multiple feature maps. This can improve the accuracy and preserve the detection speed of the detection network. To handle defects with multiple scales, we propose an adaptively balanced feature fusion (ABFF) method that can fuse features with suitable weights. It can enhance the discriminative power of the feature maps for detecting defects of different scales. Moreover, we propose a fused-attention module (FAM) to deal with the shape variations of defects. This module can enhance the channel and spatial feature information to perform precise localization and classification of defects with shape variations. Experimental results on two steel surface defect detection datasets, NEU-DET and GC10-DET, demonstrate that our proposed method can achieve state-of-the-art performance.

Efficient Detection Model of Steel Strip Surface Defects Based on YOLO-V7

During the production of strip steel, there are often some defects on the surface of the product. Therefore, the detection of such defect is the key to produce high-quality products. At the same time, the surface defects of the steel cause huge economic losses to the high-tech industry. A steel surface defect detection algorithm based on improved YOLO-V7 is proposed to address the problems of low detection speed and poor detection accuracy of traditional steel surface defect detection methods. First, we use the de-weighted BiFPN structure to make full use of the deep, shallow and original feature information to strengthen feature fusion, reduce the loss of feature information during the convolution process, and improve the detection accuracy. Secondly, the ECA attention mechanism is combined in the backbone part to strengthen the important feature channels. Finally, the original bounding box loss function is replaced by the SIoU loss function, where the penalty term is redefined by taking the vector angle between the required regressions into account. The experimental results show that the improved model proposed in this paper has higher performance compared with other comparison models. Based on our experiments, the proposed model yields 80.2% mAP on the GC10-DET dataset and 81.9% mAP on the NEU-DET dataset with high detection speed, which is better than other existing models.

Few-Shot Steel Surface Defect Detection

Deep learning-based algorithms have been widely employed to build reliable steel surface defect detection systems, which are important for manufacturing. The performance of deep learning models relies heavily on abundant annotated data. Nevertheless, the labeled image volume in industrial datasets is often limited. The scarcity of training data would lead to poor detection precision. To tackle this issue, we propose the first few-shot defect detection framework. Through pre-training models using data relevant to the target task, the proposed framework can produce well-trained networks with a few labeled images. Meanwhile, we release the first publicly available few-shot defect detection dataset, namely few-shot NEU-DET (FS-ND). This dataset will serve as a fair benchmark for various contrasting methods. Afterward, we analyze the characteristics of steel surface defect detection. It is observed that the limited amount of training data can hardly cover the data distributions in practical applications. Given this observation, we develop two domain generalization strategies that enhance the appearance and scale diversity of extracted features. Furthermore, it is found that noise existing in industrial images could result in the collapse of models. To address this problem, we devise a noise regularization strategy that improves the robustness of trained models significantly. We have conducted extensive experiments to evaluate the effectiveness of our framework. The results indicate that our framework outperforms the contrasted baseline by around 15 mAP and achieves comparable performance with models trained using abundant data.

A Real-Time Steel Surface Defect Detection Approach With High Accuracy

Surface defect inspection is a key step to ensure the quality of the hot rolled steel surface. However, current advanced detection (DET) methods have high precision but low detection speed, which hinders the application of the detector in actual production. In this work, a real-time detection network (RDN) focusing on both speed and accuracy is proposed to solve the problem of steel surface defect detection. RDN takes ResNet-dcn, a modular encoding, and decoding network with light weights, as the basic convolutional architecture whose backbone is pretrained on ImageNet. To improve the detection accuracy, a skip layer connection module (SCM) and a pyramid feature fusion module (PFM) are involved into RDN. On the standard dataset NEU-DET, the proposed method can achieve the state-of-the-art recognition speed of 64 frames per second (FPS) and the mean average precision of 80.0% on a single GPU, which fully meets the requirements of the detection accuracy and speed in the actual production line.

A pixel-wise framework based on convolutional neural network for surface defect detection.

The automatic surface defect detection system supports the real-time surface defect detection by reducing the information and high-lighting the critical defect regions for high level image under-standing. However, the defects exhibit low contrast, different textures and geometric structures, and several defects making the surface defect detection more difficult. In this paper, a pixel-wise detection framework based on convolutional neural network (CNN) for strip steel surface defect detection is proposed. First we extract the salient features by a pre-trained backbone network. Secondly, contextual weighting module, with different convolutional kernels, is used to extract multi-scale context features to achieve overall defect perception. Finally, the cross integrate is employed to make the full use of these context information and decoded the information to realize feature information complementation. The experimental results of this study demonstrate that the proposed method outperforms against the previous state-of-the-art methods on strip steel surface defect dataset (MAE: 0.0396; Fβ: 0.8485).

Steel Surface Defect Detection Using an Ensemble of Deep Residual Neural Networks

Abstract Steel defect diagnostics is important for industry task as it is tied to the product quality and production efficiency. The aim of this paper is evaluating the application of residual neural networks for recognition of industrial steel defects of three classes. Developed and investigated models based on deep residual neural networks for the recognition and classification of surface defects of rolled steel. Investigated the influence of various loss functions, optimizers and hyperparameters on the obtained result and selected optimal model parameters. Based on an ensemble of two deep residual neural networks ResNet50 and ResNet152, a classifier was constructed to detect defects of three classes on flat metal surfaces. The proposed technique allows classifying images with high accuracy. The average binary accuracy of classifying the test data is 96.7% for all images (including defect-free ones). The fields of neuron activation in the convolutional layers of the model were investigated. Feature maps formed in the process were found to reflect the position, size, and shape of the objects of interest very well. The proposed ensemble model has proven to be robust and able to accurately recognize steel surface defects. Erroneous recognition cases of the classifier application are investigated. It was shown that errors most often occur in ambiguous situations, where surface artifacts of different types are similar.

A strip steel surface defect detection method based on attention mechanism and multi-scale maxpooling

In industry, defect detection involves two kinds of tasks: defect classification and location, which make it difficult to ensure the accuracy of both, and also make the task still challenging in practical application. Based on the analysis of the advantages and disadvantages of the current defect detection method, this paper proposes a defect detection method based on attention mechanism and multi-scale maxpooling (MSMP). In order to effectively improve the detection accuracy of the model, we use Resnet50 as the pre-training network construct two-stage detection model which is used to be the baseline network, and introduce the attention mechanism and MSMP module on this basis. The attention mechanism can enhance the features of the feature map extracted in each stage of Resnet50, so that the network concentrates on the effective areas for the final detection results, and ignores the background areas that are invalid or even unfavorable for detection. The proposed MSMP can incrementally enhance the receptive field, distinguish the most significant context features, and effectively improve detection precision. The proposed method is used to train and test on the NEU-DET dataset. Compared with the baseline network without any improvement, the proposed method in this paper achieves 3.65% mAP performance improvement. Meanwhile, our method achieves a performance improvement of 3.65% mAP. In addition, compared with the feature fusion mechanism, our method improves 4.03% mAP. Moreover, compared with the attention mechanisms such as spatial attention and SE block, our method improves 1.51%/1.03% mAP. Furthermore, compared with the one-stage detection algorithm SSD/YOLO-V4, the proposed method improves 5.01%/4.92% mAP. In addition, the classification accuracy of our model is as high as 94.73%.

The Performance Analysis of Transfer Learning for Steel Defect Detection by Using Deep Learning

Detecting steel surface defects is one of the challenging problems for industries worldwide that had been used in manufacturing quality management. Manual inspection of the steel surface detects is a time-consuming process. This work aims at developing deep learning models that can perform steel defect detection and evaluating the potential of transfer learning for this task. In this paper, four types of transfer learning methods: ResNet, VGG, MobileNet, and DenseNet are experimentally evaluated to develop models for steel surface defect detection. The models were developed for binary classification (defect and no-defect) using the SEVERSTAL dataset from that contains 12,568 images of the steel surface. Then, these models were also assessed for multiclass classification using NEU dataset with 1800 images. In this work, image pre-processing is included to improve the result of steel defects detection. The experimental results have shown that the model developed by using the MobileNet method have the highest detection rate with 80.41% for the SEVERSTAL dataset and 96.94% for the NEU dataset compare to ResNet, VGG, and DenseNet transfer learning.

A New Steel Defect Detection Algorithm Based on Deep Learning.

In recent years, more and more scholars devoted themselves to the research of the target detection algorithm due to the continuous development of deep learning. Among them, the detection and recognition of small and complex targets are still a problem to be solved. The authors of this article have understood the shortcomings of the deep learning detection algorithm in detecting small and complex defect targets and would like to share a new improved target detection algorithm in steel surface defect detection. The steel surface defects will affect the quality of steel seriously. We find that most of the current detection algorithms for NEU-DET dataset detection accuracy are low, so we choose to verify a steel surface defect detection algorithm based on machine vision on this dataset for the problem of defect detection in steel production. A series of improvement measures are carried out in the traditional Faster R-CNN algorithm, such as reconstructing the network structure of Faster R-CNN. Based on the small features of the target, we train the network with multiscale fusion. For the complex features of the target, we replace part of the conventional convolution network with a deformable convolution network. The experimental results show that the deep learning network model trained by the proposed method has good detection performance, and the mean average precision is 0.752, which is 0.128 higher than the original algorithm. Among them, the average precision of crazing, inclusion, patches, pitted surface, rolled in scale and scratches is 0.501, 0.791, 0.792, 0.874, 0.649, and 0.905, respectively. The detection method is able to identify small target defects on the steel surface effectively, which can provide a reference for the automatic detection of steel defects.

Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network

Automatic inspection methods based on machine vision have been widely employed for steel surface defect detection. The central purpose of these methods is to extract features to represent different defects. However, current methods depend on machine learning that demands handcrafted features and overlooks the domain shift. In this paper, we propose a new method combining domain adaptation (DA) and adaptive convolutional neural network (ACNN), called DA-ACNN, to achieve steel surface defect detection. The convolutional neural network (CNN) is used as the backbone. To account for the lack of labels in a new domain, we introduce an additional domain classifier and a constraint on label probability distribution to achieve the cross-domain and cross-task recognition. The normal distribution and the quadratic function are used to optimize the loss to improve the network performance. Adaptive learning rates based on the loss and the weight, respectively, are proposed to minimize the losses of DA and classification. We conducted experiments on steel surface defect datasets to validate the effectiveness of DA-ACNN. Compared with the classical CNN and other approaches, the results demonstrate the superiority of the proposed method.

RetinaNet With Difference Channel Attention and Adaptively Spatial Feature Fusion for Steel Surface Defect Detection

Surface defect detection of products is an important process to guarantee the quality of industrial production. A defect detection task aims to identify the specific category and precise position of defect in an image. It is hard to take into account the accuracy of both, which makes it be challenging in practice. In this study, a new deep neural network (DNN), RetinaNet with difference channel attention and adaptively spatial feature fusion (DEA_RetinaNet), is proposed for steel surface defect detection. First, a differential evolution search-based anchor optimization is performed to improve the detection accuracy of DEA_RetinaNet. Second, a novel channel attention mechanism is embedded in DEA_RetinaNet to reduce information loss. Finally, the adaptive spatial feature fusion (ASFF) module is used for an effective fusion of shallow and deep features extracted by convolutional kernels. The experimental results on a steel surface defect data set (NEU-DET) show that DEA_RetinaNet achieved 78.25 mAP and improved by 2.92% over RetinaNet. It has better recognition performance compared with other famous DNN-based detectors.

Multi-class classification method for steel surface defects with feature noise

Defect classification is the key task of a steel surface defect detection system. The current defect classification algorithms have not taken the feature noise into consideration. In order to reduce the adverse impact of feature noise, an anti-noise multi-class classification method was proposed for steel surface defects. On the one hand, a novel anti-noise support vector hyper-spheres (ASVHs) classifier was formulated. For N types of defects, the ASVHs classifier built N hyper-spheres. These hyper-spheres were insensitive to feature and label noise. On the other hand, in order to reduce the costs of online time and storage space, the defect samples were pruned by support vector data description with parameter iteration adjustment strategy. In the end, the ASVHs classifier was built with sparse defect samples set and auxiliary information. Experimental results show that the novel multi-class classification method has high efficiency and accuracy for corrupted defect samples in steel surface.

Application of Machine Vision in the Inspection System of Steel Plate Surface Defects

Although my country is a big country in steel production, the development of domestic steel plate surface defect detection technology is backward, and most small and medium-sized enterprises are still at the stage of manual visual inspection. This paper investigated by machine vision defects in the steel sheet surface detection system application, in order to improve the steel surface defect detection rate. In this paper, a set of steel plate surface defect detection system is proposed based on machine vision, the hardware modules and selection methods of the system are designed, and four surface defects of holes, scratches, zinc scars and cracks are verified, and good experimental results are obtained. Using three different speeds for detection, the detection rate of steel surface defects reached more than 90%, and the recognition rate reached more than 80%. Research shows that the system in this paper can detect defects on the surface of steel plates well.