Defect Detection Research Articles

Quantifying the effect of X-ray scattering for data generation in real-time defect detection.

X-ray imaging is widely used for the non-destructive detection of defects in industrial products on a conveyor belt. In-line detection requires highly accurate, robust, and fast algorithms. Deep Convolutional Neural Networks (DCNNs) satisfy these requirements when a large amount of labeled data is available. To overcome the challenge of collecting these data, different methods of X-ray image generation are considered. Depending on the desired degree of similarity to real data, different physical effects should either be simulated or can be ignored. X-ray scattering is known to be computationally expensive to simulate, and this effect can greatly affect the accuracy of a generated X-ray image. We aim to quantitatively evaluate the effect of scattering on defect detection. Monte-Carlo simulation is used to generate X-ray scattering distribution. DCNNs are trained on the data with and without scattering and applied to the same test datasets. Probability of Detection (POD) curves are computed to compare their performance, characterized by the size of the smallest detectable defect. We apply the methodology to a model problem of defect detection in cylinders. When trained on data without scattering, DCNNs reliably detect defects larger than 1.3mm, and using data with scattering improves performance by less than 5%. If the analysis is performed on the cases with large scattering-to-primary ratio (1<SPR<5), the difference in performance could reach 15% (approx. 0.4mm). Excluding the scattering signal from the training data has the largest effect on the smallest detectable defects, and the difference decreases for larger defects. The scattering-to-primary ratio has a significant effect on detection performance and the required accuracy of data generation.

Performance Analysis of ResNet50 and Inception-V3 Image Classification for Defect Detection in 3D Food Printing

In the future, 3D food printing may be a fruitful area for the food industry. The presence of this technology opens the public's eyes to a revolution in the food sector, where food processing is limited to the conventional. The existence of this technology will also make food production much more effective and efficient. However, there is a problem when the 3D Food Printing process is carried out; when object defects occur, there will be material waste. This research proposes deep learning-based defect detection for defect and non-defect objects. This model mainly uses CNN as a deep learning method. The dataset is taken from the image of the print process performed at the time of object creation to be trained and validated to check the effectiveness of the proposed model. The architecture used uses pre-trained CNN models namely Inception-V3 and ResNet50 with the hope of classifying images that have a higher viscosity of the material. Where the model has been tested with previous datasets and applied with 3D Food Printing datasets with a dataset division ratio of 85% which is training data and 15% is validation data. After testing the two proposed scenarios, the accuracy result obtained in the test model scenario 1, Inception-V3, is 84.62% and for the test model scenario 2, ResNet50, the accuracy is 93.83%. The outcomes also demonstrate that improved accuracy, loss, and classification time may be obtained by applying CNN in conjunction with data augmentation.

Deep learning-assisted automatic quality assessment of concrete surfaces with cracks and bugholes

Cracks and bugholes are common factors affecting the quality of concrete surface, thus automatic detection of apparent defects is essential for concrete surface quality assessment. This study proposes an automatic quality assessment method based on deep learning for concrete surfaces with different scales of cracks and bugholes. In particular, a modified cascade mask region-based convolutional neural network (MC Mask R-CNN) with ResNet and feature pyramid network (FPN) is presented. Furthermore, a novel quantitative analysis algorithm for instance segmentation and size quantification of cracks and bugholes is proposed. Multi-scale features extracted by ResNet can be integrated using FPN. Closed operations and fixed elliptical kernels are adopted to reduce inaccuracies associated with crack discontinuities. Datasets collected at construction sites are used for training and testing of the network after data augmentation. Comparison work was carried out to evaluate the proposed model, and the box mean average precision (APbbox) and the mask mean average precision (APmask) index can reach up to 0.588 and 0.553. The overall quantization errors of cracks and bugholes were within acceptable limits. Validation experiments were conducted to illustrate the validity of the quality assessment method.

CNC machine gearbox fault detection with convolutional neural network

This study focuses on the development of a deep learning-based approach of gearbox monitoring and fault detection. The project aims to create a solution for early detection of defects in dynamic equipment based on data from vibration sensor by building a binary classifier with convolutional neural network implemented. The gearboxes condition of which is being assessed is stored in three similar computer numerically controlled (CNC) milling machines. Data is collected during 15 milling operations of different duration and with different tool’s speed and feed. Vibration is measured by an accelerometer stored on the body of each gearbox. Convolutional neural network takes vibration spectra as inputs and whether fault is detected makes a prediction of a gearbox condition. To make the whole solution autonomous and be able to embed it into manufacture the project is integrated into a server with an edge-to-cloud architecture. As an end product deep learning fault classifier stored on a server is to detect possible gearbox faults, draw conclusions on condition of dynamic equipment and automate the process of fault detection.

Spot scanning imaging calibration method based on deviation model for wafer inspection

The spot scanning defect detection system has good detection performance for wafer defects. However, due to the existence of system assembly errors, the wafer defect inspection system could not scan following the planned trace, which resulted in distortion of the reconstructed image and had a greater impact on defect feature extraction and accurate defect location. The system deviation calibration and adjustment can further facilitate the accuracy of the system position and image reconstruction. To address this issue, the causes and shapes of imaging distortion were analyzed and simulated, and a simple and effective calibration method based on a scanning deviation model for the spot scanning defect detection system was proposed. First, a scanning deviation model was established following the characteristics of the spot scanning system, and different degrees of reconstructed image distortion were simulated and analyzed as the system deviation changes. Then, an 8-inch wafer with a grid was used as the calibrator and scanned using the developed spot scanning system. Serveral grid corners in the reconstructed image are selected and used as the calibration feature points. The constraint function is constructed using the equidistant features between adjacent grid points, and then the quasi-Newton method is used to optimize the parameters of the function to obtain the best deviation estimation of the spot scanning system. To prove the validity of the estimated deviation parameters, another patterned wafer was scanned and the distortion image was used for reconstruction. The experiment demonstrated that an accurate and distortion-free wafer image can be reconstructed using the calibrated deviation parameters, and the root mean square error of the measured distance between two points of interest was only 0.61 pixels, which proved the effectiveness of the proposed calibration method.

Identification and classification of surface defects for digital twin models of the workpiece.

Workpiece surface defect detection is an indispensable part of intelligent production. The surface information obtained by traditional 2D image detection has some limitations due to the influence of environmental light factors and part complexity. However, the digital twin model has the characteristics of high fidelity and scalability, and the digital twin surface can be obtained by a device with a scanning accuracy of 0.02mm to achieve the representation of the real surface of the workpiece. The surface defect detection system for digital twin models is proposed based on the improved YOLOv5 model in this paper. Firstly, the digital twin model of the workpiece is reconstructed by the point cloud data obtained by the scanning device, and the surface features with defects are captured. Subsequently, the training dataset is calibrated based on the defect surface, where the defect types include Inclusion, Perforation, pitting surface and Rolled-in scale. Finally, the improved YOLOv5 model with CBAM mechanism and BiFPN module was used to identify the surface defects of the digital twin model and compare it with the original YOLOv5 model and other common models. The results show that the improved YOLOv5 model can realize the identification and classification of surface defects. Compared with the original YOLOv5 model, the mAP value of the improved YOLOv5 model has increased by 0.2%, and the model has high precision. On the basis of the same data set, the improved YOLOv5 model has higher recognition accuracy than other models, improving 11.7%, 3.4%, 6.2%, 33.5%, respectively. As a result, this study provides a practical and systematic detection method for digital twin model surface during the intelligent production process, and realizes the rapid screening of the workpiece with defects.

Detection and Classification of Rolling Bearing Defects Using Direct Signal Processing with Deep Convolutional Neural Network

Currently, great emphasis is being placed on the electrification of means of transportation, including aviation. The use of electric motors reduces operating and maintenance costs. Electric motors are subjected to various types of damage during operation, of which rolling bearing defects are statistically the most common. This article focuses on presenting a diagnostic tool for bearing conditions based on mechanic vibration signals using convolutional neural networks (CNN). This article presents an alternative to the well-known classical diagnostic tools based on advanced signal processing methods such as the short-time Fourier transform, the Hilbert–Huang transform, etc. The approach described in the article provides fault detection and classification in less than 0.03 s. The proposed structures achieved a classification accuracy of 99.8% on the test set. Special attention was paid to the process of optimizing the CNN structure to achieve the highest possible accuracy with the fewest number of network parameters.

Применение нейросетевого подхода для обнаружения дефектов текстильных материалов классов «слет» и «затек воды»

At present, the automation of production processes, including the use of computer vision, machine learning and artificial intelligence methods, is of relevance at light industry enterprises due to the fourth industrial revolution. The key role in the production processes is played by the quality of manufactured products – textile fabrics, which is directly affected by the process of defectoscopy. Due to the development of digital technologies and the growth of computing power, it is possible to automate the process of defectoscopy of textile fabrics using computer vision to reduce labor costs and increase the accuracy of defect detection. The purpose of this paper is to conduct experimental studies of the marking and detection of specific classes of textile defects using a hardware-software complex of computer vision and using a neural network approach. To achieve this goal, the paper describes the existing classification of textile web defects, describes the used hardware-software system, and presents the application of the neural network model of the Mask R-CNN architecture to solve the problem of exemplar defect segmentation. As part of the study, a manual partitioning of more than 400 tissue photographs into two classes of defects was performed as an extension of the training sample: “weft crack” and “water damage”, the obtained results of the neural network model were evaluated by IoU metrics: the best result for the class “weft crack” DIoU = 0.2, for the class “water damage” DIoU = 0.87. Based on the results of the experimental studies, conclusions are made about the existing potential of using neural network approach for defectoscopy of similar classes of defects. The presented results can be used for training and retraining of various models of object detection, the gained experience can be applied in other spheres of industry.

Artificial intelligence powered real-time quality monitoring for additive manufacturing in construction

In the manufacturing process of 3D Concrete Printing (3DCP), defects and anomalies have a significant impact on both the success rate and the quality of the final products, underscoring the need for real-time monitoring. Currently, monitoring is primarily based on manual observation and existing automated methods are limited in real-time performance and accuracy. This study introduced a real-time and highly accurate defect detection and measurement system for using deep learning (DL) and computer vision (CV) techniques. A range of improvement methods were applied in YOLOv7, showing better capacities of accuracy and speed for detecting defects in 3DCP than current cutting-edge detectors such as YOLOv8. Notably, the virtual high-fidelity data were produced by DL based data augmentation strategy and their effects were assessed. Replacing real data as the training dataset, the generated virtual data were used in the models to improve measurement accuracy. Applying the proposed method, the comprehensive insights into 3DCP defects were obtained. Consequently, the relationship formula between defect frequency and printer parameters was investigated by the proposed method, guiding operators in effectively controlling printer parameters and preventing breakpoint defects during the printing process.

Defect Diagnosis of Rigid Catenary System Based on Pantograph Vibration Performance

The condition of a catenary is significant to ensure a high current collection quality. Owing to the dynamic interaction between the pantograph and the catenary system, the vibration of the pantograph can be used to analyze the condition of the catenary system. Therefore, we developed a novel diagnosis system based on the correlation between catenary defects and pantograph vibration. The proposed system is capable of detecting the type and location of commonly encountered defects in rigid support catenary systems. Catenary positioning coefficient and enhanced sample entropy methods were proposed for the extraction of defect features, and subsequently, linear discriminate analysis was used to diagnose the type and location of the catenary defects. Finally, the proposed defect detection and diagnosis system was applied to a commercial metro line, and the results verified the reliability and effectiveness of this diagnosis system.

Automated defect detection in printed circuit boards based on the YOLOv5 neural network

Automated defect detection in printed circuit boards based on the YOLOv5 neural network

CNN BASED REALTIME AIRCRAFT DETECTION

Deep learning techniques such as Convolutional Neural Networks (CNN) and Transfer Learning are being used to detect and identify fighter aircraft or jets in a dataset consisting of 21 different aircraft with 20,000 images. The principle of "pooling" in Convolutional Neural Networks (CNN) involves progressively reducing the spatial size of the model to decrease the number of parameters and computations in the network. These techniques have been applied to various aspects of aircraft recognition, including object detection and engine defect detection. Convolutional Neural Networks (CNNs) are widely employed in various domains, including defence, agriculture, business, and face recognition technology, for image detection tasks. Transfer learning is a machine learning method that involves using a pre-trained model as the initial point for a new task, allowing for faster training and improved performance. This technique is particularly useful in deep learning, where large amounts of data are required for training complex models. The dataset is processed using Python libraries such as pandas, seaborn, and sci-kit-learn to find pre-trained patterns and insights. The data is then split into training and testing datasets, with 80% and 20% of the total data, respectively. A model is built using the TensorFlow library for CNN, and the metric used is "accuracy". Additionally, a transfer learning model is built to compare the accuracy results and adopt the best-fitting one. Keywords: Convolutional Neural Network, Deep Learning, Artificial Neural Network, Support Vector Machines.

Object and defect detection in additive manufacturing using deep learning algorithms

Object and defect detection in additive manufacturing using deep learning algorithms

Automatic flaw detection in sectoral scans using machine learning

Phased array ultrasonic testing (PAUT) requires highly trained and qualified personnel to interpret and analyze images. It takes a solid understanding of wave propagation physics to comprehend the generated images. As such, the inspector's judgment and level of experience have a significant impact on the analysis's outcome. In addition, the procedure is prone to error and laborious. AI had shown to be effective in computer vision in a variety of classification and detection tasks. Regarding PAUT, studies have also demonstrated that machine learning may be able to identify defects with a level of accuracy that is on par or even superior to that of trained and qualified inspectors. Nonetheless, the use of computer vision in PAUT remains very limited. The primary cause of this is the challenge accessing large databases of labelled inspections. In fact, a considerable amount of training data is required for machine learning. While it is easy to access sizeable, labelled databases of MRI scans or photographs for instance, that is not the case in PAUT because inspection results are usually confidential. In this project, a large database was generated using mock-ups commonly used to train and evaluate inspectors. The different defects contained in these mock-ups were used to train a machine learning model. The data was acquired with several different probes centered at different frequencies. Each acquisition was performed using Full Matrix Capture (FMC). The post-processing of the data contained in the FMC allows to compute any sectoral scan from its focal laws. As a result, a comprehensive database composed of hundreds of thousands of sectoral scans was generated from these few FMC acquisitions. The completeness of this database facilitated robust training of a defect detection model for PAUT sectoral scans. The evaluation of the model demonstrated its ability to generalize even to defect types it had never been trained on. Furthermore, the detection performance remained consistent even in high noise conditions where the Contrast-to-Noise Ratio (CNR) was very low.

Deep learning-based method for defect detection in electroluminescent images of polycrystalline silicon solar cells

To achieve defect detection in bare polycrystalline silicon solar cells under electroluminescence (EL) conditions, we have proposed ASDD-Net, a deep learning algorithm evaluated offline on EL images. The model integrates strategies such as downsampling adjustment, feature fusion optimization, and detection head improvement. The ASDD-Net utilizes the Space to Depth (SPD) module to effectively extract edge and fine-grained information. The proposed Enhanced Cross-Stage Partial Network Fusion (EC2f) and Hybrid Attention CSP Net (HAC3) modules are placed at different positions to enhance feature extraction capability and improve feature fusion effects, thereby enhancing the model's ability to perceive defects of different sizes and shapes. Furthermore, placing the MobileViT_CA module before the second detection head balances global and local information perception, further enhancing the performance of the detection heads. The experimental results show that the ASDD-Net model achieves a mAP value of 88.81% on the publicly available PVEL-AD dataset, and the detection performance is better than the current SOTA model. The experimental results on the ELPV and NEU-DET datasets verify that the model has some generalization ability. Moreover, the proposed model achieves a processing frame rate of 69 frames per second, meeting the real-time defect detection requirements for solar cell surface defects.

Magnetic focusing sensor and its characterizations of defect non-destructive testing for ferromagnetic steel plate

The fast development in precision manufacturing and product quality inspection of ferromagnetic material desperately demands that the defect determining the performance, service life and reliability of the mechanical part should be accurately sensed, detected and evaluated. However, conventional defect detection methods and related magnetic induction sensors are usually characterized with low accuracy in small defect detection, which makes the non-destructive testing (NDT) for ferromagnetic material full of challenges. Thus, a new magnetic focusing sensor is proposed and designed for the precision defect detection of ferromagnetic steel plate here. First, the theoretical principle analysis of the magnetic focusing sensor is introduced, and the preliminary finite element modeling (FEM) is conducted to demonstrate the feasibility of the magnetic focusing sensor in tiny defect inspection. Then, both the static and coupling dynamic characterizations of the magnetic focusing sensor are investigated using the FEM method. The optimal structural parameters of the magnetic focusing (MF) sensor in static conditions are summarized. Furthermore, the effects of the dynamic movement of the MF sensor on the multi-physical fields of the magnetization field, electric field, electromagnetic attraction force field and thermal field are all revealed. Finally, experiments regarding the performance evaluation and comparison between the MF sensor and other two magnetic flux leakage (MFL) sensors are conducted, and the prominent capabilities of accurate and precise defect detection for the MF sensor is demonstrated through the defect detection with different sizes. The effectiveness and feasibility of the MF sensor in detecting tiny defects on ferromagnetic object, as well as its great potential for practical application is therefore validated. Additionally, the main conclusions, advantages of the MF sensor are summarized and the future work is discussed.

Rail-STrans: A Rail Surface Defect Segmentation Method Based on Improved Swin Transformer

With the continuous expansion of the transport network, the safe operation of high-speed railway rails has become a crucial issue. Defect detection on the surface of rails is a key part of ensuring the safe operation of trains. Despite the progress of deep learning techniques in defect detection on the rails’ surface, there are still challenges related to various problems, such as small datasets and the varying scales of defects. Based on this, this paper proposes an improved encoder–decoder architecture based on Swin Transformer network, named Rail-STrans, which is specifically designed for intelligent segmentation of high-speed rail surface defects. The problem of a small and black-and-white rail dataset is solved using self-made large and multiple rail surface defect datasets through field shooting, data labelling, and data expansion. In this paper, two Local Perception Modules (LPMs) are added to the encoding network, which helps to obtain local context information and improve the accuracy of detection. Then, the Multiscale Feature Fusion Module (MFFM) is added to the decoding network, which helps to effectively fuse the feature information of defects at different scales in the decoding process and improves the accuracy of defect detection at multiple scales. Meanwhile, the Spatial Detail Extraction Module (SDEM) is added to the decoding network, which helps to retain the spatial detail information in the decoding process and further improves the detection accuracy of small-scale defects. The experimental results show that the mean accuracy of the semantic segmentation of the method proposed in this paper can reach 90.1%, the mean dice coefficient can reach 89.5%, and the segmentation speed can reach 37.83 FPS, which is higher than other networks’ segmentation accuracy. And, at the same time, it can achieve higher efficiency.

Automated void detection in high resolution x-ray printed circuit boards (PCBs) images with deep segmentation neural network

Defect detection and quantification on printed circuit board (PCB) assemblies are crucial for ensuring the quality of electronic products. X-ray inspection is commonly used to detect internal defects, in which high-resolution gray-scale images are captured to reveal the internal structure, particularly the defect features, of an assembled PCB. To detect defect features, conventional computer vision techniques require hand-crafted feature filters and usually have a narrow application scope. This study introduces an automated method for detecting voids using supervised machine learning. The method employs end-to-end segmentation models to identify pixels that belong to the void and integrated circuit (IC) regions in X-ray PCB images. The segmentation results are then used to compute the percentage of voids within each IC, which is a key measure of the quality of the inspected PCB. Accurate predictions with an intersection over union of over 0.95 have achieved. A challenge in this development is the large size of the images, which can reach to millions of pixels, and the small scale of defect features. Training a deep segmentation model for large-scale images using conventional methods requires a lot of memory. Downscaling high-resolution images result in the loss of defect features. In this study, a unique training method utilizing the local nature of the defects is proposed, which reduces memory consumption and speeds up the training process. Additionally, to overcome the limited training data, various augmentation methods are applied and transfer learning is employed to construct segmentation models for different types of PCBs.

On the enhancement of semi-supervised deep learning-based railway defect detection using pseudo-labels

In recent years, the use of deep learning methods in railway defect detection has expanded rapidly due to the technology’s ability to improve accuracy and efficiency in fault diagnosis. However, modeling deep learning applications requires a large amount of data, and the labeling process for such datasets demands significant manpower. To address this gap this paper introduces a semi-supervised learning approach using a student–teacher model with YOLOv4 for automatically labeling images, aiming to reduce the manual effort required for image classification. We also present a novel dataset generated from images provided by the TCDD (The State Railways of the Republic of Turkey), which includes five distinct railway defects. Results of the experiments on the same test dataset show that the proposed student–teacher model not only improves YOLOv4’s detection performance according to several decision metrics but also extends the training set with high-confidence pseudo-labeled images.

An Instance Segmentation Method for Insulator Defects Based on an Attention Mechanism and Feature Fusion Network

Among the existing insulator defect detection methods, the automatic detection of inspection robots based on the instance segmentation algorithm is relatively more efficient, but the problem of the limited accuracy of the segmentation algorithm is still a bottleneck for increasing inspection efficiency. Therefore, we propose a single-stage insulator instance defect segmentation method based on both an attention mechanism and improved feature fusion network. YOLACT is selected as the basic instance segmentation model. Firstly, to improve the segmentation speed, MobileNetV2 embedded with an scSE attention mechanism is introduced as the backbone network. Secondly, a new feature map that combines semantic and positional information is obtained by improving the FPN module and fusing the feature maps of each layer, during which, an attention mechanism is introduced to further improve the quality of the feature map. Thirdly, in view of the problems that affect the insulator segmentation, a Restrained-IoU (RIoU) bounding box loss function which covers the area deviation, center deviation, and shape deviation is designed for object detection. Finally, for the validity evaluation of the proposed method, experiments are performed on the insulator defect data set. It is shown in the results that the improved algorithm achieves a mask accuracy improvement of 5.82% and a detection speed of 37.4 FPS, which better complete the instance segmentation of insulator defect images.