Defect Datasets Research Articles

Printing roller image salient object detection based on multi-scale feature fusion

In this study, aiming at the problem that the surface defect image of the printing roller has different sizes and a large number of defects, a salient object detection method based on stepwise multi-scale feature fusion is proposed, which consists of a stepwise multi-scale feature guidance module, a multi-resolution information fusion module, and a deeply supervised side output. Specifically, the stepwise multi-scale feature guidance module contains two paths, top-down and bottom-up, which strengthen the transmission of multi-scale features in the network, and fuse multi-level features through the multi-resolution information fusion module to suppress the interference of background noise and redundant features. In addition, deeply supervised side outputs are used to eliminate differences caused by background and foreground imbalance. The experimental results on the roller surface defect dataset show that the method can achieve Fmax, MAE of 0.7959, 0.0144, and the detection rate can reach 24.98fps, which has better detection performance than other algorithms.

DA-CNN-based similar terahertz signal identification for intelligent characterization of internal debonding defects of composites under high-resolution mode

With the prevalent occupation of glass fiber reinforced polymer (GFRP) composites in engineering structures, quality inspection of GFRPs is particularly urgent to evaluate their health state. As a typical damage form during the manufacturing and lifetime service of GFRP, debonding defects not only degrades the structural strength and remaining performance of composite materials, but also brings about unpredictable challenge to overall safety of the system. Recently, the combination of terahertz (THz) spectroscopy and artificial intelligence (AI) technique has emerged great potential for automatic defect identification inside composites. However, conventional AI algorithms are difficult to classify similar THz signals and may degrade THz detection accuracy of defects due to limited feature extraction capability. Here we propose a deformable attention convolutional neural network (DA-CNN) framework-based THz characterization system, in which the defect datasets are established firstly by the THz time domain spectroscopy (THz-TDS), and then the DA-CNN framework is adopted to realize the automatic defect location and imaging by accurate THz signals classification. It is worth noting that the proposed DA-CNN framework has powerful feature extraction capability to automatically identify internal GFRP defects, especially for similar THz signals at the edge of debonding defects. A series of experiments have been performed to validate the effectiveness of proposed system, which will provide a new solution for intelligent and automatic THz characterization of internal debonding defects of composites.

Surface defect classification of hot-rolled steel strip based on mixed attention mechanism

PurposeThis paper aims to propose a classification method for steel strip surface defects based on a mixed attention mechanism to achieve fast and accurate classification performance. The traditional method of classifying surface defects of hot-rolled steel strips has the problems of low recognition accuracy and low efficiency in the industrial complex production environment.Design/methodology/approachThe authors selected min–max scaling comparison method to filter the training results of multiple network models on the steel strip surface defect data set. Then, the best comprehensive performance model EfficientNet-B0 was refined. Based on this, the authors proposed two mixed attention addition methods, which include squeeze-excitation spatial mixed module and multilayer mixed attention mechanism (MMAM) module, respectively.FindingsWith these two methods, the authors achieved 96.72% and 97.70% recognition accuracy on the steel strip data set after data augmentation for adapting to the complex production environment, respectively. Using the transfer learning method, the EfficientNet-B0 based on MMAM obtained 100% recognition accuracy.Originality/valueThis study not only focuses on improving the recognition accuracy of the network model itself but also considers other performance indicators of the network, which are rarely considered by many researchers. The authors further improve the intelligent production technique and address this issue. Both methods proposed in this paper can be applied to embedded equipment, which can effectively improve steel strip factory production efficiency and reduce material and time loss.

YOLO-Xray: A Bubble Defect Detection Algorithm for Chip X-ray Images Based on Improved YOLOv5

In the manufacturing of chips, the accurate and effective detection of internal bubble defects of chips is essential to maintain product reliability. In general, the inspection is performed manually by viewing X-ray images, which is time-consuming and less reliable. To solve the above problems, an improved bubble defect detection model YOLO-Xray based on the YOLOv5 algorithm for chip X-ray images is proposed. First, the chip X-ray images are preprocessed by image segmentation to construct the chip X-ray defect dataset, namely, CXray. Then, in the input stage, the K-means++ algorithm is used to re-cluster the CXray dataset to generate the anchors suitable for our dataset. In the backbone network, a micro-scale detection head is added to improve the capabilities for small defect detection. In the neck network, the bi-direction feature fusion idea of BiFPN is used to construct a new feature fusion network based on the improved backbone to fuse the semantic features of different layers. In addition, the Quality Focal Loss function is used to replace the cross-entropy loss function to solve the imbalance of positive and negative samples. The experimental results show that the mean average precision (mAP) of the YOLO-Xray algorithm on the CXray dataset reaches 93.5%, which is 5.1% higher than the original YOLOv5. Meanwhile, the YOLO-Xray algorithm achieves state-of-the-art detection accuracy and speed compared with other mainstream object detection models. This shows the proposed YOLO-Xray algorithm can provide technical support for bubble defect detection in chip X-ray images. The CXray dataset is also open and available at CXray.

EW-YOLOv7: A Lightweight and Effective Detection Model for Small Defects in Electrowetting Display

In order to overcome the shortcomings of existing electrowetting display defect detection models in terms of computational complexity, structural complexity, detection speed, and detection accuracy, this article proposes an improved YOLOv7-based electrowetting display defect detection model. The model effectively optimizes the detection performance of display defects, especially small target defects, by integrating GhostNetV2 modules, Acmix attention mechanisms, and NGWD (Normalized Gaussian Wasserstein Distance) Loss. At the same time, it reduces the parameter size of the network model and improves the inference efficiency of the network. This article evaluates the performance of an improved model using a self-constructed electrowetting display defect dataset. The experimental results show that the proposed improved model achieves an average detection rate (mAP) of 89.5% and an average inference time of 35.9 ms. Compared to the original network, the number of parameters and computational costs are reduced by 19.2% and 64.3%, respectively. Compared with current state-of-the-art detection network models, the proposed EW-YOLOv7 exhibits superior performance in detecting electrowetting display defects. This model helps to solve the problem of defect detection in industrial production of electrowetting display and assists the research team in quickly identifying the causes and locations of defects.

Automatic Fabric Defect Detection Method Using AC-YOLOv5

In the face of detection problems posed by complex textile texture backgrounds, different sizes, and different types of defects, commonly used object detection networks have limitations in handling target sizes. Furthermore, their stability and anti-jamming capabilities are relatively weak. Therefore, when the target types are more diverse, false detections or missed detections are likely to occur. In order to meet the stringent requirements of textile defect detection, we propose a novel AC-YOLOv5-based textile defect detection method. This method fully considers the optical properties, texture distribution, imaging properties, and detection requirements specific to textiles. First, the Atrous Spatial Pyramid Pooling (ASPP) module is introduced into the YOLOv5 backbone network, and the feature map is pooled using convolution cores with different expansion rates. Multiscale feature information is obtained from feature maps of different receptive fields, which improves the detection of defects of different sizes without changing the resolution of the input image. Secondly, a convolution squeeze-and-excitation (CSE) channel attention module is proposed, and the CSE module is introduced into the YOLOv5 backbone network. The weights of each feature channel are obtained through self-learning to further improve the defect detection and anti-jamming capability. Finally, a large number of fabric images were collected using an inspection system built on a circular knitting machine at an industrial site, and a large number of experiments were conducted using a self-built fabric defect dataset. The experimental results showed that AC-YOLOv5 can achieve an overall detection accuracy of 99.1% for fabric defect datasets, satisfying the requirements for applications in industrial areas.

A Semantic Information Decomposition Network for Accurate Segmentation of Texture Defects

Defect detection on textured surfaces remains a challenging task in the field of industrial automation due to the wide range of textures and defects. Current unsupervised learning-based texture defect detection methods based on texture background reconstruction, cannot detect texture defects with high precision because it is difficult to guarantee a high-precision reconstruction of the texture background while suppressing the defect foreground. In this study, we propose a novel semantic information decomposition network (SIDN) for accurate texture defect segmentation. The SIDN is trained on artificial defective images produced by a defect generation module (DGM). First, the SIDN uses a feature extraction module (FEM) to extract latent features with both texture semantic information and defect semantic information. Then, a novel feature separation extraction module (FSEM) for decomposing the texture semantic information and defect semantic information from the feature map generated by the FEM is proposed, preventing the coupling of the texture and defect semantic information from affecting the final segmentation accuracy. Next, a novel global semantic relation module (GSRM) is proposed to determine the relevance of the global semantic information to comprehensively consider the context and improve the feature representation. Finally, a segmentation module (SM) that directly segments the textures and defects instead of reconstructing the texture background is proposed. The final detection result is obtained by calculating a weighted average of the texture and defect segmentation results. The extensive experimental tests with the most popular and most challenging texture defect dataset demonstrate that the SIDN achieves accurate segmentation of various texture defects without using real defect samples.

Coating Defect Detection Method Based on Data Augmentation and Network Optimization Design

Coating defect detection is a critical aspect of ensuring product quality in the manufacturing process. However, due to the variety of coating defects and the complex detection background in actual production, detecting these defects can be challenging. To improve the accuracy and robustness of coating defect detection, a coating defect detection method based on data augmentation and network optimization design is proposed. First, a feature image random adaptive weighted mapping (FIRAWM) strategy is proposed, considering the prior accuracy, quantity and context information of each category. Then, several improvements are made to the YOLOv5 network. Specifically, to mitigate the aliasing effects and enhance feature richness during the feature fusion process, an additional detection layer is added, and the coordinate attention module and the adaptively spatial feature fusion (ASFF) module are introduced. Finally, ablation and comparison experiments are performed to demonstrate the effectiveness of the proposed method. The results show that the method achieves a 96.7 mAP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</sub> with a processing speed of 61 FPS on the coating defect dataset, outperforming other popular detectors. Furthermore, the method is versatile and can be applied to detection tasks in various scenarios.

PHAM-YOLO: A Parallel Hybrid Attention Mechanism Network for Defect Detection of Meter in Substation

Accurate detection and timely treatment of component defects in substations is an important measure to ensure the safe operation of power systems. In this study, taking substation meters as an example, a dataset of common meter defects, such as a fuzzy or damaged dial on the meter and broken meter housing, is constructed from the images of manual inspection in power systems. There are several challenges involved in accurately detecting defects in substation meter images, such as the complex background, different meter sizes and large differences in the shapes of meter defects. Therefore, this paper proposes the PHAM-YOLO (Parallel Hybrid Attention Mechanism You Only Look Once) network for automatic detection of substation meter defects. In order to make the network pay attention to the key areas against the complex background of the meter defect images and the differences between different defect features, a Parallel Hybrid Attention Mechanism (PHAM) module is designed and added to the backbone of YOLOv5. PHAM integration of local and non-local correlation information can highlight these differences while remaining focused on the meter defect features. To improve the expressive ability of the feature map, a Spatial Pyramid Pooling Fast (SPPF) module is introduced, which pools the input feature map using a continuous fixed convolution kernel, fusing the feature maps of different receptive fields. Bounding box regression (BBR) is the key way to determine object positioning performance in defect detection. EIOU (Efficient Intersection over Union) is, therefore, introduced as a boundary loss function to solve the ambiguity of the CIOU (Complete Intersection Over Union) loss function, making the BBR regression more accurate. The experimental results show that the Average Precision Mean (mAP), Precision (P) and Recall (R) of the proposed PHAM-YOLO network in the dataset are 78.3%, 78.3%, and 79.9%, respectively, with mAP being improved by 2.7% compared to the original model and higher than SSD, Fast R-CNN, etc.

Steel surface defect detection based on self-supervised contrastive representation learning with matching metric

Defect detection is crucial in the quality control of industrial applications. Existing supervised methods are heavily reliant on the large amounts of labeled data. However, labeled data in some specific fields are still scarce, and it requires professionals to do expensive manual annotations. In this paper, we construct a novel self-supervised steel surface defect detection model by learning better embedding feature representation of the defect on large amounts of unlabeled data, which can achieve excellent results in downstream detection tasks. Commonly used image embeddings strategies in self-supervised contrastive learning methods destroy the spatial structures of the image and are not suitable for pre-training of object detection. To address the aforementioned issue, we preserve convolutional feature maps to mine robust data structures and local features, which can enhance the representation capability of the upstream model and make it applicable for transfer to object detection tasks. Besides, in order to eliminate the effect of random augmentations of contrastive learning, which can introduce noise on multi-target coexistence datasets, the Earth Mover’s Distance (EMD) metric is employed to evaluate the contrastive matching similarity. Finally, a Self-supervised Contrastive Representation Learning framework with EMD (SCRL-EMD) is constructed through learning on large-scale unlabeled data and then transferred to Faster R-CNN and RetinaNet for detection performance validation on two public steel defect datasets. Comparative experimental results show that our method can achieve superior results than the state-of-the-art approaches. Compared to the baseline model, it achieves 4.1% and 6.8% mAP improvement on the two datasets, respectively. More importantly, a further improvement can be achieved on a smaller downstream dataset, revealing the meaningful potential of our method in exploiting more readily available unlabeled data.

STMS-YOLOv5: A Lightweight Algorithm for Gear Surface Defect Detection

Most deep-learning-based object detection algorithms exhibit low speeds and accuracy in gear surface defect detection due to their high computational costs and complex structures. To solve this problem, a lightweight model for gear surface defect detection, namely STMS-YOLOv5, is proposed in this paper. Firstly, the ShuffleNetv2 module is employed as the backbone to reduce the giga floating-point operations per second and the number of parameters. Secondly, transposed convolution upsampling is used to enhance the learning capability of the network. Thirdly, the max efficient channel attention mechanism is embedded in the neck to compensate for the accuracy loss caused by the lightweight backbone. Finally, the SIOU_Loss is adopted as the bounding box regression loss function in the prediction part to speed up the model convergence. Experiments show that STMS-YOLOv5 achieves frames per second of 130.4 and 133.5 on the gear and NEU-DET steel surface defect datasets, respectively. The number of parameters and GFLOPs are reduced by 44.4% and 50.31%, respectively, while the mAP@0.5 reaches 98.6% and 73.5%, respectively. Extensive ablation and comparative experiments validate the effectiveness and generalization capability of the model in industrial defect detection.

Deep Lifelong Learning Optimization Algorithm in Dense Region Fusion

Deep lifelong learning models can learn new information continuously while minimizing the impact on previously acquired knowledge, and thus adapt to changing data. However, existing optimization approaches for deep lifelong learning cannot simultaneously satisfy the following conditions: unrestricted learning of new data, no use of old data, and no increase in model parameters. To address this problem, a deep lifelong learning optimization algorithm based on dense region fusion (DLLO-DRF) is proposed. This algorithm first obtains models for each stage of lifelong learning, and divides the model parameters for each stage into multiple regions based on the parameter values. Then, based on the dispersion of the parameter distribution, dense regions are dynamically obtained from the divided regions, and the parameters of the dense regions are averaged and fused to optimize the model. Finally, extensive experiments are conducted on the self-labeled transmission line defect dataset, and the results show that DLLO-DRF has the best performance among various comparative algorithms.

Batch-balanced focal loss: a hybrid solution to class imbalance in deep learning.

To validate the effectiveness of an approach called batch-balanced focal loss (BBFL) in enhancing convolutional neural network (CNN) classification performance on imbalanced datasets. BBFL combines two strategies to tackle class imbalance: (1)batch-balancing to equalize model learning of class samples and (2) focal loss to add hard-sample importance to the learning gradient. BBFL was validated on two imbalanced fundus image datasets: a binary retinal nerve fiber layer defect (RNFLD) dataset () and a multiclass glaucoma dataset (). BBFL was compared to several imbalanced learning techniques, including random oversampling (ROS), cost-sensitive learning, and thresholding, based on three state-of-the-art CNNs. Accuracy, F1-score, and the area under the receiver operator characteristic curve (AUC) were used as the performance metrics for binary classification. Mean accuracy and mean F1-score were used for multiclass classification. Confusion matrices, t-distributed neighbor embedding plots, and GradCAM were used for the visual assessment of performance. In binary classification of RNFLD, BBFL with InceptionV3 (93.0% accuracy, 84.7% F1, 0.971 AUC) outperformed ROS (92.6% accuracy, 83.7% F1, 0.964 AUC), cost-sensitive learning (92.5% accuracy, 83.8% F1, 0.962 AUC), and thresholding (91.9% accuracy, 83.0% F1, 0.962 AUC) and others. In multiclass classification of glaucoma, BBFL with MobileNetV2 (79.7% accuracy, 69.6% average F1 score) outperformed ROS (76.8% accuracy, 64.7% F1), cost-sensitive learning (78.3% accuracy, 67.8.8% F1), and random undersampling (76.5% accuracy, 66.5% F1). The BBFL-based learning method can improve the performance of a CNN model in both binary and multiclass disease classification when the data are imbalanced.

Surface defect detection and classification of steel using an efficient Swin Transformer

Detecting steel-surface defects is a crucial phase in steel manufacturing; however, accurately completing the detection task is challenging. The Swin Transformer, a self-attention-based model, has shown strong performance in the field of computer vision to enhance the adaptability of the Swin Transformer to the task of steel-surface defect detection, and a new network architecture called the LSwin Transformer is proposed in this study. First, in the downsampling process, we propose a convolutional embedding module and an attention patch merging module, which simultaneously strengthen the connections between the feature map channels, reduce the resolution, and increase image information retention. Second, we propose an effective window shift strategy and a convenient computation approach to make a complete defect between patches have more opportunity to obtain interactive computing. Finally, to combine the feature extraction capability of convolutional neural networks with the global dependency building capability of the Swin Transformer, we propose a depth multilayer perceptron module. Numerous experiments were conducted on a steel-surface defect dataset. The results demonstrated that the detection effect of our model outperformed competing methods, with a mean average precision of 81.2 %. In the ablation study, we verified the effectiveness of each module and initialized the parameters of the model through transfer learning to accelerate the convergence of the model. Therefore, the proposed LSwin Transformer has significant potential for detecting steel-surface defects.

A Hybrid Software Defects Prediction Model for Imbalance Datasets Using Machine Learning Techniques: (S-SVM Model)

Software defect prediction (SDP) is an essential task for developing quality software, and various models have been developed for this purpose. However, the imbalanced nature of software defect datasets has challenged these models, resulting in decreased performance. To address this challenge, the author has proposed a hybrid machine learning model that combines Synthetic Minority Oversampling Technique (SMOTE) with Support Vector Machine (SVM)—SMOTE-SVM (S-SVM) model. The author has empirically examined SDP using multiple datasets (CM1, PC1, JM1, PC3, KC1, EQ and JDT) from the PROMISE and AEEEM repositories. The experimental study indicates that the S-SVM model involved training and compared with previously developed balanced and imbalanced test datasets using four evaluation metrics: Precision, Recall, F1 score, and Accuracy. For the balanced dataset, the S-SVM model achieved precision values ranging from 70 to 96, recall values ranging from 52 to 94, F1-score values ranging from 67 to 90, and accuracy values ranging from 69 to 98. For the imbalanced dataset, the S-SVM model achieved precision values ranging from 60 to 93, recall values ranging from 64 to 97, F1-score values ranging from 69 to 91, and accuracy values ranging from 67 to 87. The proposed S-SVM model outperforms other models’ ability to classify and predict software defects. Therefore, the hybridisation of SMOTE and SVM improved the model’s ability to categories and predict balanced and imbalanced datasets when sufficient defective and non-defective data is provided.

A data generation method with dual discriminators and regularization for surface defect detection under limited data

Surface defect detection is a critical aspect of industrial production as it directly impacts product quality. However, advancements in manufacturing have made it challenging to collect surface defect images, which often limits the performance of deep learning-based defect detection models. Although generative adversarial network (GAN)-based image augmentation methods can help alleviate the problem of insufficient data, they are not effectively used when there is limited defect data. To address these challenges, our paper proposes a data generation method that can generate surface defect images under limited data with stability. Firstly, we propose a dual discriminator architecture based on GAN to facilitate the deployment of the regularization scheme. Next, we introduce a regularization scheme into the dual discriminator, which enhances the quality and diversity of the defect images generated by making it difficult for the discriminator to overfit under limited data. Finally, we introduce an overfitting heuristic that can adaptively adjust the strength of the regularization algorithm based on the degree of overfitting. The experimental results demonstrate that the defect images generated by our proposed data generation method have high quality and rich diversity, significantly improving the performance of defect detection models in various scenarios. For instance, the generated defect images increased the mean average precision at the intersection over union threshold of 0.5 (mAP@.5) value of the defect detection model from 82.26 to 97.59 on the Northeastern University (NEU) dataset. Additionally, we propose a new metric for measuring the quality of generated defect images and construct a surface defect dataset of solar aluminum profile frames collected in a real industrial scene to accelerate the development of defect detection technology.

Class Imbalance Reduction and Centroid based Relevant Project Selection for Cross Project Defect Prediction

Cross-Project Defect Prediction (CPDP) is the process of predicting defects in a target project using information from other projects. This can assist developers in prioritizing their testing efforts and finding flaws. Transfer Learning (TL) has been frequently used at CPDP to improve prediction performance by reducing the disparity in data distribution between the source and target projects. Software Defect Prediction (SDP) is a common study topic in software engineering that plays a critical role in software quality assurance. To address the cross-project class imbalance problem, Centroid-based PF-SMOTE for Imbalanced data is used. In this paper, we used a Centroid-based PF-SMOTE to balance the datasets and Centroid based relevant data selection for Cross Project Defect Prediction. These methods use the mean of all attributes in a dataset and calculating the difference between mean of all datasets. For experimentation, the open source software defect datasets namely, AEEM, Re-Link, and NASA, are considered.

Multi-scale attention and dilation network for small defect detection

Although object detection methods have got surprising performance in simple natural scenes, it is still a challenging task to apply them to complex scenes, especially when the detected objects are small which is common in defect detection tasks of industrial scenarios. Motivated by the higher resolution feature maps, the better detection performance on small objects, we increase the resolution of the feature layers in this paper, and introduce convolutional block attention module (CBAM) into the feature aggregation network to better integrate features at different scales and make better use of the information of small defects. For solving the problem of global context information loss caused by limiting the perceptual field by increasing the resolution of the feature layers, we use dilated convolution blocks to expand the perceptual field. Experimental results show that our method improves the mAP(0.5) on the Industrial Surface Defect Dataset from 88.3% in Yolov5 to 89.2%, mAP(0.75) on the DAGM 2007 increases from 76.56% in Yolov5 to 77.65%, mAP on small object increases by 2.6% on Industrial Surface Defect Dataset and 5.8% on DAGM 2007, which proves the effectiveness of our method for small defect detection.

A TEDE Algorithm Studies the Effect of Dataset Grouping on Supervised Learning Accuracy

Datasets are the basis for research on deep learning methods in computer vision. The impact of the percentage of training sets in a dataset on the performance of neural network models needs to be further explored. In this paper, a twice equal difference enumeration (TEDE) algorithm is proposed to investigate the effect of different training set percentages in the dataset on the performance of the network model, and the optimal training set percentage is determined. By selecting the Pascal VOC dataset and dividing it into six different datasets from largest to smallest, and then dividing each dataset into the datasets to be analyzed according to five different training set percentages, the YOLOv5 convolutional neural network is used to train and test the 30 datasets to determine the optimal neural network model corresponding to the training set percentages. Finally, tests were conducted using the Udacity Self-Driving dataset with a self-made Tire Tread Defects (TTD) dataset. The results show that the network model performance is superior when the training set accounts for between 85% and 90% of the overall dataset. The results of dataset partitioning obtained by the TEDE algorithm can provide a reference for deep learning research.

P‐98: Novel Solution for High‐Speed Detecting Very Small Defects in Display

There are various inspection stages in the process of display manufacturing. Accurate detection of defects in various sizes is crucial for high yield rate. Detecting tiny defects in an image is a challenging task for object detection techniques based on convolutional neural networks because their feature information can be lost in deep convolutional layers. Also, our task requires defects in various sizes are detected in a high‐speed to achieve high yield rate. We tackled this challenging issues in our object detection problem by establishing three theories: Slicing and Robust of Outlier, Multi‐Inference. For the defect dataset that occurs in mass production, mAP@0.5 78.5% for Robust Dataset is increased to 89.5%, and the existing method, which take 4.7 sec with SAHI method, is reduced to 0.2 sec.