Real-time Defect Research Articles

Building Surface Defect Detection Using Machine Learning and 3D Scanning Techniques in the Construction Domain

The rapid growth of the real estate market has led to the appearance of more and more residential areas and large apartment buildings that need to be managed and maintained by a single real estate developer or company. This scientific article details the development of a novel method for inspecting buildings in a semi-automated manner, thereby reducing the time needed to assess the requirements for the maintenance of a building. This paper focuses on the development of an application which has the purpose of detecting imperfections in a range of building sections using a combination of machine learning techniques and 3D scanning methodologies. This research focuses on the design and development of a machine learning-based application that utilizes the Python programming language and the PyTorch library; it builds on the team′s previous study, in which they investigated the possibility of applying their expertise in creating construction-related applications for real-life situations. Using the Zed camera system, real-life pictures of various building components were used, along with stock images when needed, to train an artificial intelligence model that could identify surface damage or defects such as cracks and differentiate between naturally occurring elements such as shadows or stains. One of the goals is to develop an application that can identify defects in real time while using readily available tools in order to ensure a practical and affordable solution. The findings of this study have the potential to greatly enhance the availability of defect detection procedures in the construction sector, which will result in better building maintenance and structural integrity.

Detection and Classification of Fabric Defects Using Deep Learning Algorithms

The textile industry primarily relies on fabric as a crucial raw material, the production of which involves multiple complex stages. Due to the multitude and complexity of these stages, fabric defects can frequently occur. With the modern fabric production process being nearly fully automated, and given the variety of potential defects, detecting errors on fabrics has become increasingly challenging. The rapid pace of production and the substantial market share of the sector mean that relying on human inspection for error detection can lead to significant time losses and can reduce the accuracy of defect detection to around 60%. Consequently, recent years have seen a shift towards the development of intelligent systems for fabric defect detection in parallel with technological advancements. With the rapid progression of artificial intelligence, the application of image processing techniques has commenced in this field. This study has developed a real-time defect detection system for fabrics using deep learning techniques. Initially, a network model was created using an open-source neural network library, CNN, achieving 89% accuracy. Subsequent implementations using the VGG16 and InceptionV3 architectures reached accuracies of 89% and 86%, respectively. To further improve the study, fabrics were classified into two categories: defective and non-defective, and the pre-trained Convolutional Neural Networks model ResNet50-v2 was employed as a feature extractor. This approach yielded an approximate accuracy of 95%.

Real Time Defect Identification using Advanced Artificial Intelligence based Material Strain Sensors for Environmental Safety

Advanced materials find applications in various fields of engineering and medicine. Many advanced materials are synthesized characterised to design sensors for various application including environmental protection. This work explores the possibility of real time defect identification using advanced materials such as graphene composite based sensor for Environment safety. Advanced materials like Graphene composite based sensor were prepared using solvent casting technique. Possibility of using graphene-PVDF based sensors as non- destructive probes to identify surface defects of materials were evaluated by probing on materials with pre-machined defects. Defects were induced artificially in different materials such as (SS316 LN stainless steel) flat plate and cylindrical disc plate. Defects vary both in size and depth. Nanocomposite films were cut in to small pieces of equal dimension and pasted over the various defects. The variation in size and depth of the defects is measured in terms of variation in resistance and suitably calibration graphs were obtained. Using advanced materials for environmental protection, Industry environment safety can be monitored and essential measures can be carried out. This work also explores the change in absorption property of carbon based nanocomposite films on addition of different weight percentage of advanced material based fillers such as Multiwalled carbon nanotubes and graphene. Filler weight percentage increases in matrix which is indicated by red shift in the absorption spectra.In this research work the use of advanced materials for environmental protection in the Metallurgical Industry is explored . Development of advanced functional materials were carried out . Characterization of these advanced material were also carried out . Sensitivity of these advanced material based sensors were analyzed and calibarated. By using these advanced material based senors catastrophic disorders can be avoided and environmental safety is ensured.The non-destructive evaluation (NDE) method that is being presented uses a Carbon Nanotube (CNT) and graphene-based sensor to identify defects in pre-machined stainless steel plates is innovative and straightforward. Utilizing nano composites is meant to combine the advantageous qualities of the filler and matrix. It was decided to use PVDF, a piezoelectric polymer, because of its benefits including low cost, versatility, and quick response time. A small amount (weight%) of MWCNTs and graphene are employed as filler to improve the material's shortcomings, such as delayed electrical response, and to make it behave as a piezoresistive and conductive material.

Electrolytic capacitor surface defect detection based on deep convolution neural network

The existing methods for detecting surface defects in electrolytic capacitors are typically based on conventional machine vision, with limited feature extraction capabilities, poor versatility, slow detection speed, and the inability to achieve accurate and real-time defect detection. In this study, a real-time object detection algorithm based on an improved single shot multibox detector (SSD) is proposed to achieve omnidirectional surface defect detection of electrolytic capacitors. First, an electrolytic capacitor surface image acquisition device was established to capture omnidirectional surface images of the capacitors, and an electrolytic capacitor surface defect dataset was created. Next, the visual geometry group (VGG)-16 network structure was replaced with the MobileNetv2 network structure, effectively reducing the model’s parameter count and improving inference speed. Moreover, the Multibox Loss function was replaced with the Focal Loss function to increase the model’s attention to difficult-to-classify samples and improve model accuracy. Additionally, a transfer learning network model was designed to apply the model to electrolytic capacitors of different colors using small sample learning. Finally, the performance of the improved network model was tested on a dataset of electrolytic capacitor surface defects. The experimental results demonstrate that the parameters quantity of improved model is 3.50 M, the mAP value reaches 92.67 %, which is improved by 2.54 %, and the Macro-F1 value reaches 92.15 %, which is 11.32 % higher than that before improvement. Thus, the proposed improved SSD model provides a theoretical basis and technical prerequisites for automated and intelligent surface defect detection in electrolytic capacitors.

A bilateral feature fusion network for defect detection on mobile cameras

Defect detection in mobile phone cameras constitutes a critical aspect of the manufacturing process. Nonetheless, this task remains challenging due to the complexities introduced by intricate backgrounds and low-contrast defects, such as minor scratches and subtle dust particles. To address these issues, a Bilateral Feature Fusion Network (BFFN) has been proposed. This network incorporates a bilateral feature fusion module, engineered to enrich feature representation by fusing feature maps from multiple scales. Such fusion allows the capture of both fine and coarse-grained details inherent in the images. Additionally, a Self-Attention Mechanism is deployed to garner more comprehensive contextual information, thereby enhancing feature discriminability. The proposed Bilateral Feature Fusion Network has been rigorously evaluated on a dataset of 12,018 mobile camera images. Our network surpasses existing state-of-the-art methods, such as U-Net and Deeplab V3+, particularly in mitigating false positive detection caused by complex backgrounds and false negative detection caused by slight defects. It achieves an F1-score of 97.59%, which is 1.16% better than Deeplab V3+ and 0.99% better than U-Net. This high level of accuracy is evidenced by an outstanding precision of 96.93% and recall of 98.26%. Furthermore, our approach realizes a detection speed of 63.8 frames per second (FPS), notably faster than Deeplab V3+ at 57.1 FPS and U-Net at 50.3 FPS. This enhanced computational efficiency makes our network particularly well-suited for real-time defect detection applications within the realm of mobile camera manufacturing.

Real-time grid detection in sheet metal fiber laser cutting through coaxial monitoring

Real-time defects’ estimation and control of the cut quality through the coaxial camera monitoring of the kerf are amongst the most promising developments for laser cutting. In industrial systems, sheets are positioned on a metallic grid creating discontinuities in the cutting process due to unpredictable thickness, blown and resolidified material and the time varying position of the grid. The interaction between laser radiation escaping from the kerf and the grid, along with the difficult outflow of molten material, causes changes in the process emission images. These changes influence the defect estimation approach, and, consequently, the control action. In this work, a real-time grid identification and classification-based machine learning algorithm was developed and tested during the fusion cutting of 6 mm thick Al5754, exploiting a NIR coaxial monitoring system. Real-time control experiments with dross estimates were performed, demonstrating a correct identification of the grid and highlighting a feasible industrial application.

DSA-Net: An Attention-Guided Network for Real-Time Defect Detection of Transmission Line Dampers Applied to UAV Inspections

DSA-Net: An Attention-Guided Network for Real-Time Defect Detection of Transmission Line Dampers Applied to UAV Inspections

A Real-Time Defect Detection Strategy for Additive Manufacturing Processes Based on Deep Learning and Machine Vision Technologies.

Nowadays, additive manufacturing (AM) is advanced to deliver high-value end-use products rather than individual components. This evolution necessitates integrating multiple manufacturing processes to implement multi-material processing, much more complex structures, and the realization of end-user functionality. One significant product category that benefits from such advanced AM technologies is 3D microelectronics. However, the complexity of the entire manufacturing procedure and the various microstructures of 3D microelectronic products significantly intensified the risk of product failure due to fabrication defects. To respond to this challenge, this work presents a defect detection technology based on deep learning and machine vision for real-time monitoring of the AM fabrication process. We have proposed an enhanced YOLOv8 algorithm to train a defect detection model capable of identifying and evaluating defect images. To assess the feasibility of our approach, we took the extrusion 3D printing process as an application object and tailored a dataset comprising a total of 3550 images across four typical defect categories. Test results demonstrated that the improved YOLOv8 model achieved an impressive mean average precision (mAP50) of 91.7% at a frame rate of 71.9 frames per second.

Analysis of Surface Roughness during Surface Polishing of ITO Thin Film Using Acoustic Emission Sensor Monitoring

This study investigates the intricate process of surface polishing for ITO-coated Pyrex glass utilizing magnetic abrasive polishing (MAP) while employing acoustic emission (AE) sensors for real-time defect monitoring. MAP, known for its versatility in achieving nanoscale thickness processing and uniform surfaces, has been widely used in various materials. However, the complexity of the process, influenced by multiple variables like cutting conditions, material properties, and environmental factors, poses challenges to maintaining high surface quality. To address this, a sensor monitoring system, specifically one that uses AE sensors, was integrated into the MAP process to detect and confirm defects, providing real-time insights into machining conditions and outcomes. AE sensors excel in identifying material deterioration, microcrack formation, and wear, even in cases of minor damage. Leveraging AE sensor data, this study aims to minimize surface defects in ITO thin films during MAP while optimizing surface roughness. The investigation involves theoretical validation, magnetic density simulations, and force sensor pressure measurements to identify factors influencing surface roughness. ANOVA analysis is employed to determine optimal processing conditions. Additionally, this study compares the identified optimal roughness conditions with those predicted by AE sensor parameters, aiming to establish a correlation between predicted and achieved surface quality. The integration of AE sensor monitoring within the MAP process offers a promising avenue for enhancing surface quality by effectively identifying and addressing defects in real time. This comprehensive analysis contributes to advancing the understanding of surface polishing methodologies for ITO-coated Pyrex glass, paving the way for improved precision and quality in thin-film surface processes.

Machine Learning in Precision Manufacturing: A Collaborative Computer and Mechanical Engineering Perspective

Precision manufacturing, characterized by the production of intricate components with stringent tolerances, has witnessed a paradigm shift with the integration of machine learning (ML) techniques. This research paper delves into the collaborative endeavors between computer and mechanical engineering disciplines, exploring how the fusion of expertise from both domains contributes to the optimization and advancement of precision manufacturing processes. [1] The motivation behind this research stems from the recognition of ML's transformative potential in revolutionizing precision manufacturing. This paper reviews the application of ML in key areas such as predictive maintenance, process optimization, quality control, and adaptive manufacturing strategies. It emphasizes the synergy between computer and mechanical engineers in designing and implementing intelligent algorithms that harness data from various sensors, enabling real-time adjustments to manufacturing parameters. Predictive maintenance emerges as a critical aspect of the collaborative efforts, where ML algorithms analyze sensor data to predict equipment failures, thereby reducing downtime and maintenance costs. Process optimization is explored through the integration of ML in Computer Numerical Control (CNC) machining, highlighting how algorithms optimize tool paths and enhance machining precision. Quality control in precision manufacturing is addressed by intelligent systems that utilize computer vision and sensor technologies, providing real-time defect detection. [2] The collaborative approach presents challenges, including data security and privacy concerns and the need for skillset integration between computer and mechanical engineering disciplines. The paper concludes by exploring future directions, including the integration of emerging technologies such as edge computing and the Internet of Things (IoT), underscoring the importance of interdisciplinary collaboration for the successful implementation of ML techniques in achieving heightened precision, efficiency, and quality in precision manufacturing.

An inspection network with dynamic feature extractor and task alignment head for steel surface defect

High-precision identification and real-time localization for irregular-shaped steel surface defects are crucial for shipbuilding quality control. Although traditional lightweight networks enable real-time defect inspection, the incurred model cannot achieve precise inspection for the defects with large variations in aspect ratios of ship plates. This paper proposes a lightweight inspection network with a dynamic feature extractor and task alignment detection head (INDT) for multi-class steel plate surface defects to address this obstacle. A dynamic structure expansion training strategy based on a re-parameterization multi-branch block is constructed to achieve real-time inspection containing multi-scale information. Furthermore, fed with multi-scale information, the task alignment head with a preprocess for multi-task to concentrate task-oriented features into specific channels. Besides, a soft-weighted sample assignment algorithm with dynamic priors to irregular defects is developed to supervise high-precision model training. The experiments show that the INDT achieves higher precision among all the benchmark methods with lossless accelerating inference.

Real-time defect detection of saw chains on automatic assembly lines based on residual networks and knowledge coding

The use of machine vision methods has rapidly developed for detecting product defects, but it remains challenging to identify shape and color defects as well as the assembly relationship of parts due to occlusion between components and changes in assembly structures. This paper proposed a method for detecting saw chain assembly defects based on residual networks and knowledge coding to achieve parts segmentation and identification of assembly defects. Firstly, we design an adaptive segmentation algorithm for saw chain parts, which uses the Hough transform to locate rivets and completes the image segmentation based on the inherent assembly features between parts. Secondly, A trained ResNet (Residual Network) based deep learning model was used to identify the individual defects of saw chain parts, in which the transfer learning method was used to improve the detection speed of individual defects. Finally, a parallel inference method based on a knowledge encoding matrix is proposed for detecting part defects and constructing a sparse matrix to locate assembly defects. Evaluating the proposed method in building a saw chain testing platform. Results show that the proposed method can achieve real-time and high-precision detection requirements in different application scenarios. The detection accuracy on individual and assembly defects reaches 94.2% and 89.2%, respectively, which is better than the current state-of-the-art object detection models. Assembly defect detection methods are an important reference for inspecting other types of simple assemblies in the industry.

Comparison of In-situ Nondestructive Testing and Ex-situ Methods in Additive Manufactured Specimens for Internal Feature Detection

ABSTRACT The effectiveness and repeatability of Additive Manufacturing (AM) technologies has been often measured through rigorous testing, both in-situ and ex-situ. In-situ nondestructive testing (NDT) has been used to understand the AM process and detect anomalies during the build. Ex-situ testing has been used for material characterization and internal defect detection. In this work, in-situ NDT and ex-situ testing methods are compared as applied to a Fused Filament Fabrication (FFF) process to detect defects that have formed internally in the part. Intentionally embedded features and defects are monitored and analyzed in-situ and ex-situ. Results from an automated image-based real-time defect detection system, an ultrasound ex-situ system, and a tensile testing system are compared to detect defects and their effects on mechanical properties. Real-time image-based defect detection is found to effectively spot defects as they form and allow the potential for implementing defect correction methods, while ex-situ tensile testing can be used as a form of validation to in-situ results.

DsP-YOLO: An anchor-free network with DsPAN for small object detection of multiscale defects

Industrial defect detection is of great significance to ensure the quality of industrial products. The surface defects of industrial products are characterized by multiple scales, multiple types, abundant small objects, and complex background interference. In particular, small object detection of multiscale defects under complex background interference poses significant challenges for defect detection tasks. How to improve the algorithm’s ability to detect industrial defects, especially in enhancing the detection capabilities of small-sized defects, while ensuring that the inference speed is not overly affected is a long-term prominent challenge. Aiming at achieving accurate and fast detection of industrial defects, this paper proposes an anchor-free network with DsPAN for small object detection. The core of this method is to propose a new idea i.e., feature enhancement in the feature fusion network for the feature information of small-size objects. Firstly, anchor-free YOLOv8 is adopted as the basic framework for detection to eliminate the affections of hyperparameters related to anchors, as well as to improve the detection capability of multi-scale and small-size defects. Secondly, considering the PAN path (top layer of neural networks for feature fusion) is more task-specific, we advocate that the underlying feature map of the PAN path is more vulnerable to small object detection. Hence, we in-depth study the PAN path and point out that the standard PAN will encounter several drawbacks caused by losing local details and position information in deep layer. As an alternative, we propose a lightweight and detail-sensitive PAN (DsPAN) for small object detection of multiscale defects by designing an attention mechanism embedded feature transformation module(LCBHAM) and optimizing the lightweight implementation. Our proposed DsPAN is very easy to be incorporated in YOLO series framework. The proposed method is evaluated on three public datasets, NEU-DET, PCB-DET, and GC10-DET. The mAP of the method is 80.4%, 95.8%, and 76.3%, which are 3.6%, 2.1%, and 3.9% higher than that of YOLOv8 and significantly higher than the state-of-the-art (SOTA) detection methods. Also, the method achieves the second-highest inference speed among the thirteen models tested. The results indicate that DsP-YOLO is expected to be used for real-time defect detection in industry.

Development of an Algorithm for Detecting Real-Time Defects in Steel

The integration of artificial intelligence with steel manufacturing operations holds great potential for enhancing factory efficiency. Object detection algorithms, as a category within the field of artificial intelligence, have been widely adopted for steel defect detection purposes. However, mainstream object detection algorithms often exhibit a low detection accuracy and high false-negative rates when it comes to detecting small and subtle defects in steel materials. In order to enhance the production efficiency of steel factories, one approach could be the development of a novel object detection algorithm to improve the accuracy and speed of defect detection in these facilities. This paper proposes an improved algorithm based on the YOLOv5s-7.0 version, called YOLOv5s-7.0-FCC. YOLOv5s-7.0-FCC integrates the basic operator C3-Faster (C3F) into the C3 module. Its special T-shaped structure reduces the redundant calculation of channel features, increases the attention weight on the central content, and improves the algorithm’s computational speed and feature extraction capability. Furthermore, the spatial pyramid pooling-fast (SPPF) structure is replaced by the Content Augmentation Module (CAM), which enriches the image feature content with different convolution rates to simulate the way humans observe things, resulting in enhanced feature information transfer during the process. Lastly, the upsampling operator Content-Aware ReAssembly of Features (CARAFE) replaces the “nearest” method, transforming the receptive field size based on the difference in feature information. The three modules that act on feature information are distributed reasonably in YOLOv5s-7.0, reducing the loss of feature information during the convolution process. The results show that compared to the original YOLOv5 model, YOLOv5s-7.0-FCC increases the mean average precision (mAP) from 73.1% to 79.5%, achieving a 6.4% improvement. The detection speed also increased from 101.1 f/s to 109.4 f/s, an improvement of 8.3 f/s, further meeting the accuracy requirements for steel defect detection.

Real-Time Defect Detection Model in Industrial Environment Based on Lightweight Deep Learning Network

Surface defect detection in industrial environments is crucial for quality management and has significant research value. General detection networks, such as the YOLO series, have proven effective in various dataset detections. However, due to the complex and varied surface defects of industrial products, many defects occupy a small proportion of the surface and fall into the category of typical small target detection problems. Moreover, the complexity of general detection network architectures relies on high-tech hardware, making it difficult to deploy on devices without GPUs or on edge computing and mobile devices. To meet the practical needs of industrial product defect inspection applications, this paper proposes a lightweight network specifically designed for defect detection in industrial fields. This network is composed of four parts: a backbone network, a multiscale feature aggregation network, a residual enhancement network, and an attention enhancement network. The network includes a backbone network that integrates attention layers for feature extraction, a multiscale feature aggregation network for semantic information, a residual enhancement network for spatial focus, and an attention enhancement network for global–local feature interaction. These components enhance detection performance for diverse defects while maintaining low hardware requirements. Experimental results show that this network outperforms the latest and most popular YOLOv5n and YOLOv8n models in the five indicators P, R, F1, mAP@.5, and GFLOPS when used on four public datasets. It even approaches or surpasses the YOLOv8s and YOLOv5s models with several times the GFLOPS computation. It balances the requirements of lightweight real-time and accuracy in the scenario of industrial product surface defect detection.

Quality control for the recycling of carbon fibre

Quality control for the recycling of carbon fibre Due to the inevitable variability of the end-of-life input material, the resulting recycled fibre and nonwoven or woven material will have a higher degree of variability than virgin material. To make sure such recycled fabric is usable for high-quality applications, additional methods for quality control are required. Existing methods for quality control, such as the Profactor FScan sensor, are used to detect a range of different types of deviations during the manufacturing of carbon fibre fabric. Such sensors can be integrated into weaving machines to cover the whole width of the material (e.g. up to 2.5m) and identify defects in real time.

AYOLOv7-tiny: Towards efficient defect detection in solid color circular weft fabric

This article presents a solid color circular weft fabric defect detection method based on AYOLOv7-tiny. The aim of the development of this network is to enable real-time defect detection in the production of large circular machine fabrics. The network has a good accuracy rate, fast detection speed, and a lightweight model. In the YOLOv7-tiny network, a space-to-depth layer followed by a non-strided convolution layer is introduced to enhance the feature extraction capability, improve image sharpness, address issues such as uneven grayscale and difficult detection of minor defects, and simplify the model complexity while reducing computation. Additionally, we combined the Squeeze and Excitation (SE) and Spatial Attention Module (SAM) based on Convolutional Block Attention Module (CBAM) to construct the Hybrid Attention Module (SC), which is integrated into the YOLOv7-tiny network. The SC module increases the weight of important features, enhances the feature extraction capability, improves image segmentation, and enhances the accuracy of the network's location information. Through extensive experiments with a dataset of large circular machine solid color circular weft fabric defect collected from an industrial site, the results show that AYOLOv7-tiny has a detection accuracy (mean average precision) of 98.7%, a detection speed of up to 333 frames per second, and a computational complexity of only 4.4 GFlops, which is better than the current mainstream surface defect detection models. The detection accuracy, detection speed, and model complexity of the AYOLOv7-tiny network all meet the real-time detection requirements of the industry and have been successfully used for real-time detection of large circular machine fabric defects.

The competing role of defects and surface roughness on the fatigue behavior of additively manufactured AlSi10Mg alloy

The fatigue behavior of AlSi10Mg alloy processed by laser-based powder bed fusion of metals (PBF-LB/M) in as-built (AB) and surface treated conditions, is assessed. A new chemo-mechanical polishing (CMP) surface treatment is applied to AB specimens to achieve an average surface roughness of Sa≈ 0.3 μm which is more than two orders of magnitude lower than that of AB specimens (Sa≈ 48.6 μm). The CMP surface treatment led to a significant improvement in fatigue strength of the alloy as compared to the AB material. The fatigue life of AB + CMP specimens is discovered to be governed by surface or near surface defects, whereas the deepest surface roughness valleys play a dominant role in controlling the fatigue behavior of the AB material. A simplified fracture mechanics-based fatigue life modeling approach was developed by combining real-time defects distribution data from 3D computed tomography (CT) and surface profilometry measurements. The model provided a quick and effective medium for obtaining reasonable first-order fatigue life approximations for AlSi10Mg alloy fabricated using PBF-LB/M.

Inline Bondwave Monitoring for Direct Bonding, Process Optimization and Impact on Post-Bond Distortion

Direct wafer bonding is essential in semiconductor manufacturing, with bondwave propagation dynamics influencing the overall bond quality. The main factor impacting bondwave velocity dynamics are substrate rigidity, fluid viscosity and adhesion energy between the two surfaces. The latter, which depends on substrates properties and process parameters, is key and should be controlled and monitored for fusion bonding applications. The use of an inline infrared (IR) monitoring within an automated fusion bonding equipment, EVG®850LT, facilitates real-time defect detection, root cause analysis, and bond re-workability. The equipment, which features a sealed bonding chamber equipped with an IR camera and a reflective chuck, together with a plasma chamber for pre-processing enables the assessment of various process conditions. Post-bond distortions are then evaluated using high-resolution metrology. Bondwave monitoring use-cases, a detailed analysis of bondwave speed impact on bonded stack distortion and the influence of diverse process parameters on bondwave propagation are presented or discussed in the current paper.