Surface Defect Inspection Research Articles

Dynamic graph CNN based semantic segmentation of concrete defects and as-inspected modeling

Obtaining accurate information of defective areas of infrastructures helps to perform repair actions more efficiently. Recently, LiDAR scanners have been used for the inspection of surface defects. Moreover, machine learning methods have attracted the attention of researchers for semantic segmentation and classification based on point cloud data. Although much work has been done for processing visual information with images, research on machine learning methods for semantic segmentation of raw point cloud data is still in its early stages. Moreover, LiDAR technology is commonly used to create as-is BIM models. Therefore, the BIM model needs to be integrated with the results of defect semantic segmentation after the LiDAR-based inspection. Addressing the above issues, this paper has the following objectives: (1) Developing a method for point cloud-based concrete surface defects semantic segmentation; and (2) Developing a semi-automated process for as-inspected modeling. The challenges related to the size of the dataset and imbalanced classes are studied. Sensitivity analysis is applied to capture the best combination of hyperparameters and investigate their effects on the network performance. The proposed method resulted in 98.56% and 96.50% recalls for semantic segmentation of cracks and spalls, respectively. Furthermore, post-processing of the results of the concrete surface defects semantic segmentation is done to semi-automate the process of as-inspected modeling. As-inspected BIM includes the updated information of the facilities at the time of data collection. This semi-automated process made it possible to manage and visualize the detected defects by extracting their dimensions and identifying the conditions on the 3D model.

Research on Visual Inspection of Surface Defects of Metal Parts Based on Machine Vision

Research on Visual Inspection of Surface Defects of Metal Parts Based on Machine Vision

Performances of an in-line deep learning-based inspection system for surface defects of die-cast components for hybrid vehicles

The detection of surface defects is a critical activity for quality assurance of automotive products. Deep learning (DL) methods range among the most suitable tools to support this task. They allow the inspection system to learn how to detect surface defects on the basis of a series of sample images. However, most of the past studies regard laboratory experiments. This study demonstrates the actual integration of an inspection system into an industrial production plant to perform surface defect inspection operations on machined and unmachined areas of aluminium components for hybrid cars. The proposed DL method is practical for industry, where the number of available defect samples is low. Experimental results showed a True Positive Rate of 98%.

Cross-factory Polarizer Sheet Surface Defect Inspection System Based on Multi-Teacher Knowledge Amalgamation

Cross-factory Polarizer Sheet Surface Defect Inspection System Based on Multi-Teacher Knowledge Amalgamation

DES-YOLO: a novel model for real-time detection of casting surface defects.

Surface defect inspection methods have proven effective in addressing casting quality control tasks. However, traditional inspection methods often struggle to achieve high-precision detection of surface defects in castings with similar characteristics and minor scales. The study introduces DES-YOLO, a novel real-time method for detecting castings' surface defects. In the DES-YOLO model, we incorporate the DSC-Darknet backbone network and global attention mechanism (GAM) module to enhance the identification of defect target features. These additions are essential for overcoming the challenge posed by the high similarity among defect characteristics, such as shrinkage holes and slag holes, which can result in decreased detection accuracy. An enhanced pyramid pooling module is also introduced to improve feature representation for small defective parts through multi-layer pooling. We integrate Slim-Neck and SIoU bounding box regression loss functions for real-time detection in actual production scenarios. These functions reduce memory overhead and enable real-time detection of surface defects in castings. Experimental findings demonstrate that the DES-YOLO model achieves a mean average precision (mAP) of 92.6% on the CSD-DET dataset and a single-image inference speed of 3.9 milliseconds. The proposed method proves capable of swiftly and accurately accomplishing real-time detection of surface defects in castings.

Erasing-inpainting-based data augmentation using denoising diffusion probabilistic models with limited samples for generalized surface defect inspection

Surface defect inspection aims to identify defective regions in product surface images to ensure product quality. Existing deep learning methods have developed rapidly on surface defect inspection. However, their excellent performances rely on a large number of training samples, which are hard to acquire in practical industrial scenarios due to the continuous improvements of the production line. To solve this issue, we propose an erasing-inpainting-based data augmentation method using a denoising diffusion probabilistic model (DDPM) with limited samples for generalized surface defect inspection. Our method is based on the idea that a defect image is difficult to recover to its previous state after undergoing a large-scale erasure operation, thus diverse defect images can be generated using different settings in the inpainting model. Specifically, we first train a DDPM model using limited defective images. Then, we erase large-scale parts of an input image to obtain a degraded image and restore the erased areas using the trained DDPM. Finally, the repaired images are further used for updating the DDPM. The main advantage of our method is to generate diverse images by only being trained using limited training samples. On the one hand, our method fundamentally avoids the dimension inconsistency between the sampled noise and the generated image by sampling from a two-dimensional noise map with the same resolution as the output image based on DDPM. On the other hand, the proposed erasing-inpainting operation promotes the recombination of the real features from the training set and the learned features from the trained DDPM to fully use the limited defective samples and the easily obtainable defect-free samples. Extensive experiments demonstrated the effectiveness and advantagesof our model on data augmentation for generalized surface defect inspection.

A method of concrete damage detection and localization based on weakly supervised learning

AbstractAutomatic inspection of concrete surface defects based on visual elements is crucial for the timely detection of security risks in infrastructure. Moreover, accurate determination of the geographical location of the detected defects is critical for subsequent maintenance and reinforcement tasks. This study employed convolutional neural network (CNN) training methods for detection and localization. This approach employs bounding boxes to confine damaged pixels and utilizes projection loss to foster similarity learning between pixels. In addition, geotags are automatically indexed through the vectorization of invariant features in a scene, which maintains high model accuracy and reduces training costs. The proposed method can detect and classify typical concrete defects in complex scenarios and accurately locate them without the use of external sensors. In addition, the proposed model can achieve pixel‐level defect detection and geographic location determination through the cost of bounding box annotation and automatic indexing. The proposed model was evaluated using 10,691 images of four typical concrete defects in various complex environments. The results demonstrate that the proposed method achieves a pixel‐level detection accuracy of 48.75 and a location accuracy of 83.69, showing a better performance than other methods.

Residual shape adaptive dense-nested Unet: Redesign the long lateral skip connections for metal surface tiny defect inspection

The state-of-the-art metal surface defect inspection methods have two problems: (1) they are sensitive to tiny defects because of their extreme small sizes, and (2) they cannot accurately locate the random appeared defects whose semantic relationship with the background context is weak. To solve these problems, Residual Shape Adaptive Dense-nested Unet, a pixel-based defect inspection method is proposed, to obtain the exact shape and location of the defect, by (1) assembling different depth Unet branches with dense skip connections as the feature extractor to combine multi-semantic level visual features; (2) adding Residual Shape Adaptive modules on the dense skip connections to help the model locate the defect regions; and (3) introducing the multi-branch training method which enables model pruning to reduce redundant parameters and accelerate the inspection speed. Experiments are conducted and demonstrated that the Residual Shape Adaptive Dense-nested Unet achieves the best performance among the state-of-the-art defect inspection methods.

Investigations into the negative sensitivity phenomenon in the detection of hidden defects using coplanar capacitive sensors

Non-destructive Evaluation and Testing techniques play a crucial role in the inspection of materials and structures, and among them, the capacitive technique has emerged as a promising approach. Previous studies have highlighted the presence of a significant challenge known as the negative sensitivity phenomenon in coplanar capacitive sensors, which affects the interpretation of surface defect inspection results. In this study, investigations were conducted, combining simulations and experiments, to explore the existence and composition of the negative sensitivity phenomenon and its application in detecting hidden defects. The simulations and experimental findings unequivocally demonstrate that the negative sensitivity phenomenon is an inherent characteristic within the sensitivity distribution of a coplanar capacitive sensor. It was found out that the negative sensitivity phenomenon not only exists but also constitutes the largest portion of the sensitivity distribution, with two critical points influencing its behavior. Additionally, it was observed that the thickness of the material under test, along with its properties, significantly influences the sensitivity distribution. Furthermore, this study reveals that the pre-defect peak-trough effect observed is a result of the composition and perturbation of the sensitivity distribution caused by the defects themselves. Through experimental validation, the feasibility of utilizing the negative sensitivity phenomenon region for the detection of hidden defects is validated.

Reinforcement learning-based approach for plastic texture surface defects inspection

Reinforcement learning-based approach for plastic texture surface defects inspection

Automated defect detection on inductive thermography images using supervised and semi-supervised Deep Learning methods

Inductive infrared thermography has been proven as an interesting solution for the inspection of surface defects. To automate the inspection, defect detection methods based on convolutional neural network proved their efficiency for complex detection tasks compared to traditional methods. Both supervised and semi-supervised learning approaches have been proposed for the inspection task. While the supervised approach remains the most common one, it requires images of both defective and non-defective parts during the training phase. Unfortunately, in many industries where the scrap rate is low, acquiring images of defective parts is difficult and requires time which can delay the deployment of such solutions. This paper compares these two learning approaches by illustrating the advantages and disadvantages of each approach from an industrial point of view. In conclusion, we describe an inspection deployment strategy, which combines the two approaches to ensure robust inspection with rapid deployment.

Synthetic‐to‐realistic domain adaptation for cold‐start of rail inspection systems

AbstractRail surface defects are potential danger factors for railway systems, and visual inspection of surface defects plays a vital role in rail maintenance. Recently, the methods based on deep learning have been widely used in rail inspection systems, but such systems often face the problem of a lack of defect samples for training deep learning models at start‐up, which is called the cold‐start problem. It is challenging to obtain sufficient defect samples since defects are sparse and even non‐existent for a running railway system. Therefore, a synthetic‐to‐realistic domain adaptation (SRDA) method is proposed for real‐world rail inspection. SRDA adapts synthetic images to look more realistic for reducing the domain gap between synthetic images and realistic rail surface images and obtains translated images that consist of synthetic defect information and realistic rail background. After that, the translated images are used to train a detector for rail inspection of rail surface defects. In order to make the detector more robust to complex backgrounds, SRDA generates images with the same defect‐level semantics but with different texture appearances and makes the detector align these images in the learned feature space. In addition, the synthetic and realistic rail surface defects (SRRSD) dataset containing 20,662 images is built. The experimental results on SRRSD show that SRDA achieves higher detection performance than other established domain adaption methods with 19.0% for and 26.7% for average precision.

Structural Design and Simulation of a Multi-Channel and Dual Working Condition Wafer Defect Inspection Prototype.

Detecting and classifying defects on unpatterned wafers is a key part of wafer front-end inspection. Defect inspection schemes vary depending on the type and location of the defects. In this paper, the structure of the prototype is designed to meet the requirements of wafer surface and edge defect inspection. This prototype has four inspection channels: scattering, reflection, phase, and contour, with two working conditions: surface and edge inspection. The key structure of the prototype was simulated using Ansys. The simulation results show that the maximum deformation of the optical detection subsystem is 19.5 μm and the fundamental frequency of the prototype is 96.9 Hz; thus, these results meet the requirements of optical performance stability and structural design. The experimental results show that the prototype meets the requirements of the inspection sensitivity better than 200 nm equivalent PSL spherical defects.

A Two-Stage Focal Transformer for Human–Robot Collaboration-Based Surface Defect Inspection

Abstract Human–robot collaboration has become a hotspot in smart manufacturing, and it also has shown the potential for surface defect inspection. The robot can release workload, while human collaboration can help to recheck the uncertain defects. However, the human–robot collaboration-based defect inspection can be hardly realized unless some bottlenecks have been solved, and one of them is that the current methods cannot decide which samples to be rechecked, and the workers can only recheck all of the samples to improve inspection results. To overcome this problem and realize the human–robot collaboration-based surface defect inspection, a two-stage Transformer model with focal loss is proposed. The proposed method divides the traditional inspection process into detection and recognition, designs a collaboration rule to allow workers to collaborate and recheck the defects, and introduces the focal loss into the model to improve the recognition results. With these improvements, the proposed method can collaborate with workers by rechecking the defects and improve surface quality. The experimental results on the public dataset have shown the effectiveness of the proposed method, the accuracies are significantly improved by the human collaboration, which are 1.70%∼4.18%. Moreover, the proposed method has been implemented into a human–robot collaboration-based prototype to inspect the carton surface defects, and the results also verify the effectiveness. Meanwhile, the proposed method has a good ability for visualization to find the defect area, and it is also conducive to defect analysis and rechecking.

Physical Coupling Fusion Sensing of MFL-EMAT for Synchronous Surface and Internal Defects Inspection

Non-destructive testing (NDT) techniques are widely used for inspection and evaluation of conductive materials. However, a single physical mechanism sensing cannot simultaneously satisfy multiple inspection requirements. In this paper, we propose a hybrid transducer based on magnetic flux leakage (MFL) and electromagnetic acoustic transducer (EMAT) for synchronous detection of surface and internal defects. In the proposed hybrid transducer, both MFL and EMAT share a common magnetic field. The magnetic circuit characteristics of the MFL and EMAT principles are fully exploited, and the magnetic field provided by a single permanent magnet is used to excite ultrasonic waves that can detect internal defects within the sample. Iron is used to introduce magnetic field into the sample and provide a horizontal magnetic field to detect discontinuities on the near-surface of the sample. The huge frequency difference between the MFL signal and the EMAT signal effectively suppresses interference between the two signals. Simulations and experiments have been undertaken to show that the proposed transducer can overcome the detection limitations associated with the MFL and the dead detection zone of the EMAT, while simultaneously detecting the surface defects, internal defects and bottom thinning defects in the specimen.

GAN-based statistical modeling with adaptive schemes for surface defect inspection of IC metal packages

GAN-based statistical modeling with adaptive schemes for surface defect inspection of IC metal packages

A survey of real-time surface defect inspection methods based on deep learning

A survey of real-time surface defect inspection methods based on deep learning

A review of the automated timber defect identification approach

Timber quality control is undoubtedly a very laborious process in the secondary wood industry. Manual inspections by operators are prone to human error, thereby resulting in poor timber quality inspections and low production volumes. The automation of this process using an automated vision inspection (AVI) system integrated with artificial intelligence appears to be the most plausible approach due to its ease of use and minimal operating costs. This paper provides an overview of previous works on the automated inspection of timber surface defects as well as various machine learning and deep learning approaches that have been implemented for the identification of timber defects. Contemporary algorithms and techniques used in both machine learning and deep learning are discussed and outlined in this review paper. Furthermore, the paper also highlighted the possible limitation of employing both approaches in the identification of the timber defect along with several future directions that may be further explored.

A domain adaptation YOLOv5 model for industrial defect inspection

In the process of industrial production, manufactured products are prone to surface defects for a variety of reasons. To overcome the problem of high time cost and the strong demand for large sample data sets, a detector based on transfer learning is commonly utilized. In this paper, a domain adaptation YOLOv5 model, named DAYOLOv5, is proposed for automatic surface defect inspection. The hyperparameter α in DAYOLOv5 for knowledge transfer can be designed specially to achieve better generalization in real-world industrial applications. Meanwhile, in the field of magnetic tile surface defect detection, our DAYOLOv5 outperforms traditional mixed training and pretrain-finetune methods with limited data sets and has great robustness. Overall, the experimental results demonstrate that our DAYOLOv5 model can indeed boost performance on small-scale target data sets and is applicable to practical industrial scenarios.

FDD: a deep learning-based steel defect detectors.

Surface defects are a common issue that affects product quality in the industrial manufacturing process. Many companies put a lot of effort into developing automated inspection systems to handle this issue. In this work, we propose a novel deep learning-based surface defect inspection system called the forceful steel defect detector (FDD), especially for steel surface defect detection. Our model adopts the state-of-the-art cascade R-CNN as the baseline architecture and improves it with the deformable convolution and the deformable RoI pooling to adapt to the geometric shape of defects. Besides, our model adopts the guided anchoring region proposal to generate bounding boxes with higher accuracies. Moreover, to enrich the point of view of input images, we propose the random scaling and the ultimate scaling techniques in the training and inference process, respectively. The experimental studies on the Severstal steel dataset, NEU steel dataset, and DAGM dataset demonstrate that our proposed model effectively improved the detection accuracy in terms of the average recall (AR) and the mean average precision (mAP) compared to state-of-the-art defect detection methods. We expect our innovation to accelerate the automation of industrial manufacturing process by increasing the productivity and by sustaining high product qualities.