Real-time Fault Detection Research Articles

DsP-YOLO: An anchor-free network with DsPAN for small object detection of multiscale defects

Industrial defect detection is of great significance to ensure the quality of industrial products. The surface defects of industrial products are characterized by multiple scales, multiple types, abundant small objects, and complex background interference. In particular, small object detection of multiscale defects under complex background interference poses significant challenges for defect detection tasks. How to improve the algorithm’s ability to detect industrial defects, especially in enhancing the detection capabilities of small-sized defects, while ensuring that the inference speed is not overly affected is a long-term prominent challenge. Aiming at achieving accurate and fast detection of industrial defects, this paper proposes an anchor-free network with DsPAN for small object detection. The core of this method is to propose a new idea i.e., feature enhancement in the feature fusion network for the feature information of small-size objects. Firstly, anchor-free YOLOv8 is adopted as the basic framework for detection to eliminate the affections of hyperparameters related to anchors, as well as to improve the detection capability of multi-scale and small-size defects. Secondly, considering the PAN path (top layer of neural networks for feature fusion) is more task-specific, we advocate that the underlying feature map of the PAN path is more vulnerable to small object detection. Hence, we in-depth study the PAN path and point out that the standard PAN will encounter several drawbacks caused by losing local details and position information in deep layer. As an alternative, we propose a lightweight and detail-sensitive PAN (DsPAN) for small object detection of multiscale defects by designing an attention mechanism embedded feature transformation module(LCBHAM) and optimizing the lightweight implementation. Our proposed DsPAN is very easy to be incorporated in YOLO series framework. The proposed method is evaluated on three public datasets, NEU-DET, PCB-DET, and GC10-DET. The mAP of the method is 80.4%, 95.8%, and 76.3%, which are 3.6%, 2.1%, and 3.9% higher than that of YOLOv8 and significantly higher than the state-of-the-art (SOTA) detection methods. Also, the method achieves the second-highest inference speed among the thirteen models tested. The results indicate that DsP-YOLO is expected to be used for real-time defect detection in industry.

Circuit defect detection based on AI deep learning

In the milieu of promptly advancing technology and increasing demand for electronic devices, circuit defect detection has become crucial to warranting product quality. This study tackles the cons of traditional defect detection methods, proposing a mind-boggling approach based on AI deep learning. The study intends to establish and enhance deep learning algorithms for the exact and real-time detection of circuit defects. This research encompasses an in-depth review of existing literature on circuit defect detection and AI deep learning, underlining the existing gaps and pitfalls in the field. The study will primarily deploy convolutional neural networks (CNNs) and recurrent neural networks (RNNs) as the primary tools to process various data modalities. The results highlight that the proposed AI deep learning framework depicts grander performance, unlike in traditional manual inspection. The study sets precedence in AI applications in quality control as it contributes to improved manufacturing efficiency, reduced production costs, and delivery of utmost-quality electronic products to consumers.

Constructing a Network Digital Twin through formal modeling: Tackling the virtual–real mapping challenge in IIoT networks

The Industrial Internet of Things (IIoT) presents numerous requirements, such as reliability, energy efficiency, and real-time performance, among others. Digital Twins technology addresses these needs by enabling the simulation, monitoring, and optimization of such systems. Specifically, Network Digital Twins (NDTs) have recently attracted significant attention from both the industrial and academic networking communities. Numerous research efforts focus on the challenge of network modeling in this context, as simulation tools can increase cost and computational complexity. In this paper, we explore the integration of the formal modeling technique, Petri-nets, within the context of NDTs to model the IIoT, enabling data-driven Petri-nets. We present an architecture based on open-source tools as a framework to encourage researchers to investigate this research direction. Furthermore, we validate the proposed architecture through a case study involving a small-scale network modeled with Timed Petri-Nets. The results demonstrate the model’s effectiveness in executing what-if scenarios based on the network’s operational parameters to predict the Packet Delivery Ratio and enable real-time fault detection. Lastly, we conduct a study on the IIoT observability by the NDT to address the challenge of virtual–real systems mapping in such a context.

Improved STMask R-CNN-based defect detection model for automatic visual inspection of an optics lens.

A lens defect is a common quality issue that has seriously harmed the scattering characteristics and performance of optical elements, reducing the quality consistency of the finished products. Furthermore, the energy hotspots coming from the high-energy laser through diffraction of optical component defects are amplified step by step in multi-level laser conduction, causing serious damage to the optical system. Traditional manual detection mainly relies on experienced workers under a special light source environment with high labor intensity, low efficiency, and accuracy. The common machine vision techniques are incapable of detecting low contrast and complex morphological defects. To address these challenges, a deep learning-based method, named STMask R-CNN, is proposed to detect defects on the surface and inside of a lens in complex environments. A Swin Transformer, which focuses on improving the modeling and representation capability of the features in order to improve the detection performance, is incorporated into the Mask R-CNN in this case. A challenge dataset containing more than 3800 images (18000 defect sample targets) with five different types of optical lens defects was created to verify the proposed approach. According to our experiments, the presented STMask R-CNN reached a precision value of 98.2%, recall value of 97.7%, F1 score of 97.9%, mAP@0.5 value of 98.1%, and FPS value of 24 f/s, which outperformed the SSD, Faster R-CNN, and YOLOv5. The experimental results demonstrated that the proposed STMask R-CNN outperformed other popular methods for multiscale targets, low contrast target detection and nesting, stacking, and intersecting defects sample detection, exhibiting good generalizability and robustness, as well as detection speed to meet mechanical equipment production efficiency requirements. In general, this research offers a favorable deep learning-based method for real-time automatic detection of optical lens defects.

Fault Diagnosis of Suspension System Based on Spectrogram Image and Vision Transformer

The suspension system plays a critical role in vehicles, providing both comfort and directional control. Therefore, it is essential to implement a monitoring system to ensure the proper functioning of suspension components, as a failure in any of these components can lead to accidents. Furthermore, monitoring the condition of the suspension system helps in maintaining its performance and minimizes maintenance costs. Traditionally, diagnosing faults in suspension systems has relied on specialized setups and vibration analysis. Alternatively, deep learning-based approaches for fault diagnosis in suspension systems offer a promising solution by enabling faster and more accurate real-time fault detection. This study investigated the use of vision transformers as an innovative approach to fault diagnosis in suspension systems, leveraging spectrogram images. Spectrogram images from vibration signals were extracted and used as inputs for the vision transformer model. Test results showcased a remarkable 99.39% accuracy in fault identification, affirming the system's effectiveness.

Comparative Analysis of Decision Tree (DT) & Deep Neural Network (DNN) Techniques to Diagnose the Stator Winding Fault in the Permanent Magnet Synchronous Motor (PMSM)

Every industry is moving towards automation, and Electrical motors like DC and AC play a significant role in the overall automation scheme. Electrical motors, however, are prone to multiple manufacturing defects, and untimed wear and tear. Automated detection of faults, while the motor is in operations, can prevent automation breakdowns, and unforeseen accidents. The majority of faults in electrical motors are because of overheating of windings, bearing misalignment, shaft mismatch, vibrations, noise, etc. A few hours of Electric Motor running can generate a lot of sensor data measuring temperature, torque etc.. Manual analysis of such data can be extremely time consuming. Machine Learning can be used to perform automated analysis of data and real-time fault detection as suggested by many experts. In this research, we have used a few machine learning techniques to train the model on top of sensor data and use the model to diagnose the electric motor faults, occurring because of heat in the stator winding part of the permanent magnet synchronous motor (pmsm). We have specifically compared Decision Tree (DT) and Deep Neural Network (DNN) techniques on real-life datasets, and compared their accuracy. As per the final results, Decision Tree (DT) and Deep Neural Network (DNN) have 93.82% & 97% accuracy respectively.

A security enabled real time fault detection and classification of power system conditions

A security enabled real time fault detection and classification of power system conditions

Real-Time Defect Detection Model in Industrial Environment Based on Lightweight Deep Learning Network

Surface defect detection in industrial environments is crucial for quality management and has significant research value. General detection networks, such as the YOLO series, have proven effective in various dataset detections. However, due to the complex and varied surface defects of industrial products, many defects occupy a small proportion of the surface and fall into the category of typical small target detection problems. Moreover, the complexity of general detection network architectures relies on high-tech hardware, making it difficult to deploy on devices without GPUs or on edge computing and mobile devices. To meet the practical needs of industrial product defect inspection applications, this paper proposes a lightweight network specifically designed for defect detection in industrial fields. This network is composed of four parts: a backbone network, a multiscale feature aggregation network, a residual enhancement network, and an attention enhancement network. The network includes a backbone network that integrates attention layers for feature extraction, a multiscale feature aggregation network for semantic information, a residual enhancement network for spatial focus, and an attention enhancement network for global–local feature interaction. These components enhance detection performance for diverse defects while maintaining low hardware requirements. Experimental results show that this network outperforms the latest and most popular YOLOv5n and YOLOv8n models in the five indicators P, R, F1, mAP@.5, and GFLOPS when used on four public datasets. It even approaches or surpasses the YOLOv8s and YOLOv5s models with several times the GFLOPS computation. It balances the requirements of lightweight real-time and accuracy in the scenario of industrial product surface defect detection.

AYOLOv7-tiny: Towards efficient defect detection in solid color circular weft fabric

This article presents a solid color circular weft fabric defect detection method based on AYOLOv7-tiny. The aim of the development of this network is to enable real-time defect detection in the production of large circular machine fabrics. The network has a good accuracy rate, fast detection speed, and a lightweight model. In the YOLOv7-tiny network, a space-to-depth layer followed by a non-strided convolution layer is introduced to enhance the feature extraction capability, improve image sharpness, address issues such as uneven grayscale and difficult detection of minor defects, and simplify the model complexity while reducing computation. Additionally, we combined the Squeeze and Excitation (SE) and Spatial Attention Module (SAM) based on Convolutional Block Attention Module (CBAM) to construct the Hybrid Attention Module (SC), which is integrated into the YOLOv7-tiny network. The SC module increases the weight of important features, enhances the feature extraction capability, improves image segmentation, and enhances the accuracy of the network's location information. Through extensive experiments with a dataset of large circular machine solid color circular weft fabric defect collected from an industrial site, the results show that AYOLOv7-tiny has a detection accuracy (mean average precision) of 98.7%, a detection speed of up to 333 frames per second, and a computational complexity of only 4.4 GFlops, which is better than the current mainstream surface defect detection models. The detection accuracy, detection speed, and model complexity of the AYOLOv7-tiny network all meet the real-time detection requirements of the industry and have been successfully used for real-time detection of large circular machine fabric defects.

Real-Time Optical Detection of Artificial Coating Defects in PBF-LB/P Using a Low-Cost Camera Solution and Convolutional Neural Networks

Additive manufacturing plays a decisive role in the field of industrial manufacturing in a wide range of application areas today. However, process monitoring, and especially the real-time detection of defects, is still an area where there is a lot of potential for improvement. High defect rates should be avoided in order to save costs and shorten product development times. Most of the time, effective process controls fail because of the given process parameters, such as high process temperatures in a laser-based powder bed fusion, or simply because of the very cost-intensive measuring equipment. This paper proposes a novel approach for the real-time and high-efficiency detection of coating defects on the powder bed surface during the powder bed fusion of polyamide (PBF-LB/P/PA12) by using a low-cost RGB camera system and image recognition via convolutional neural networks (CNN). The use of a CNN enables the automated detection and segmentation of objects by learning the spatial hierarchies of features from low to high-level patterns. Artificial coating defects were successfully induced in a reproducible and sustainable way via an experimental mechanical setup mounted on the coating blade, allowing the in-process simulation of particle drag, part shifting, and powder contamination. The intensity of the defect could be continuously varied using stepper motors. A low-cost camera was used to record several build processes with different part geometries. Installing the camera inside the machine allows the entire powder bed to be captured without distortion at the best possible angle for evaluation using CNN. After several training and tuning iterations of the custom CNN architecture, the accuracy, precision, and recall consistently reached >99%. Even defects that resembled the geometry of components were correctly classified. Subsequent gradient-weighted class activation mapping (Grad-CAM) analysis confirmed the classification results.

A New Approach to Detect Power Quality Disturbances in Smart Cities Using Scaling-Based Chirplet Transform with Strategically Placed Smart Meters

The growth of Internet of Things (IoT)-enabled devices has increased the amount of data created by the distribution network’s periphery nodes, requiring more data transfer capacity. Recent applications’ real-time requirements have strained standard computing paradigms, and data processing has struggled to keep up. Edge computing is employed in this research to detect distribution network faults, allowing for instant sensing and real-time reaction to the control room for faster investigation of distribution problems and power outages, making the system more reliable. Moreover, to overcome the challenges of fault detection, advanced signal processing methods need to be integrated with the Adaboost classifier. An Adaboost-based edge device, suitable for installation on top of a power pole, is proposed in this research as a means of real-time fault detection. To increase throughput, decrease latency and offload network traffic, data collecting, feature extraction and Adaboost-based problem identification are all performed in an integrated edge node. Enhanced detection accuracy (98.67%) and decreased latency (115.2 ms) verify the effectiveness of the suggested approach. In this research, we enhance the classical chirplets transform to create the scaling-basis chirplet transform (SBCT) for time–frequency (TF) analysis. This approach modulates the TF basis around the relevant time function to modify the chirp rate with frequency and time. By carefully selecting the sampling frequency, it is possible to discriminate between short circuit fault and high-impedance fault (HIF) by calculating spectral entropy. The TF representation obtained with the SBCT provides considerably higher energy concentrations, even for signals with numerous components, closely spaced frequencies and heavy background noise.

Real-Time DC Series Arc Fault Detection Based on Noise Pattern Analysis in Photovoltaic System

This paper presents a method for detecting series arc faults based on noise pattern analysis in photovoltaic systems. The arc detection circuit is required to detect the series arc quickly and not falsely detect the system noise as arc noise. This paper proposes noise pattern analysis to distinguish between system noise and arc noise and prevent false detection. First, periodic feature analysis prevents false detection by periodic noise, such as inverter noise, using the difference in periodic characteristics. Second, zero-range density analysis detects series arc noise using the difference in fluctuation characteristics. An integrated arc fault detection algorithm comprising discrete wavelet transform decomposition, periodic feature analysis, zero-range density, and an iterative loop algorithm is developed. The proposed methods are verified using noise distinction experiments, wherein the injected signal and arc noise are distinguished with high classification precision using the proposed noise pattern analysis method. The developed arc detection algorithm is applied to a real-time series arc detection board based on the TMS320F28335 DSP, and the performance of the series arc detection and false detection prevention is verified using the UL1699B arc detection test circuit and a real PV system.

Integration of Artificial Intelligence for Real-Time Fault Detection in Semiconductor Packaging

Semiconductor manufacturing involves a complex sequence of unit processes, where even a minor error can disrupt the entire production chain. Present-day manufacturing setups rely on continuous data monitoring of equipment health, wafer measurements, and inspections to identify any abnormalities that could impact the quality and performance of the final chip. The primary aim is fault detection and classification (FDC), for which a range of techniques including statistical analysis and machine learning algorithms are commonly employed. In this study, we introduce an innovative approach utilizing an artificial immune system (AIS), inspired by biological mechanisms, for FDC in semiconductor equipment. The main culprits behind process failures are shifts caused by the aging of parts and modules over time. Our methodology integrates state variable identification (SVID) data, reflecting current equipment conditions, and optical emission spectroscopy (OES) data, capturing plasma process information under faulty scenarios induced by deliberate gas flow rate adjustments in semiconductor fabrication. Our results demonstrate a modeling prediction accuracy of 94.69% when incorporating selected SVID and OES data, and 93.68% accuracy using OES data alone. In conclusion, we suggest the potential application of AIS in semiconductor process decision-making, offering promising avenues for enhancing fault detection in semiconductor equipment.

Inline Bondwave Monitoring for Direct Bonding, Process Optimization and Impact on Post-Bond Distortion

Direct wafer bonding is essential in semiconductor manufacturing, with bondwave propagation dynamics influencing the overall bond quality. The main factor impacting bondwave velocity dynamics are substrate rigidity, fluid viscosity and adhesion energy between the two surfaces. The latter, which depends on substrates properties and process parameters, is key and should be controlled and monitored for fusion bonding applications. The use of an inline infrared (IR) monitoring within an automated fusion bonding equipment, EVG®850LT, facilitates real-time defect detection, root cause analysis, and bond re-workability. The equipment, which features a sealed bonding chamber equipped with an IR camera and a reflective chuck, together with a plasma chamber for pre-processing enables the assessment of various process conditions. Post-bond distortions are then evaluated using high-resolution metrology. Bondwave monitoring use-cases, a detailed analysis of bondwave speed impact on bonded stack distortion and the influence of diverse process parameters on bondwave propagation are presented or discussed in the current paper.

Explainable deep neural network for in-plain defect detection during additive manufacturing

PurposeThe purpose of this study is to develop a deep learning framework for additive manufacturing (AM), that can detect different defect types without being trained on specific defect data sets and can be applied for real-time process control.Design/methodology/approachThis study develops an explainable artificial intelligence (AI) framework, a zero-bias deep neural network (DNN) model for real-time defect detection during the AM process. In this method, the last dense layer of the DNN is replaced by two consecutive parts, a regular dense layer denoted (L1) for dimensional reduction, and a similarity matching layer (L2) for equal weight and non-biased cosine similarity matching. Grayscale images of 3D printed samples acquired during printing were used as the input to the zero-bias DNN.FindingsThis study demonstrates that the approach is capable of successfully detecting multiple types of defects such as cracks, stringing and warping with high accuracy without any prior training on defective data sets, with an accuracy of 99.5%.Practical implicationsOnce the model is set up, the computational time for anomaly detection is lower than the speed of image acquisition indicating the potential for real-time process control. It can also be used to minimize manual processing in AI-enabled AM.Originality/valueTo the best of the authors’ knowledge, this is the first study to use zero-bias DNN, an explainable AI approach for defect detection in AM.

Weight-guided feature fusion and non-local balance model for aluminum surface defect detection

Aluminum surface defect detection plays a crucial role in the manufacturing industry. Due to the complexity of aluminum surface defects, the existing defect detection methods have false and missed detection problems. To address the characteristics of aluminum surface defects and the problems of existing methods, we propose a weight-guided feature fusion and non-local balance model to improve the detection effect. Firstly, we design the feature extraction network cross-stage partial ConvNeXt, which achieves adequate feature extraction while reducing the model’s size. In addition, we propose a weight-guided feature fusion and non-local balanced feature pyramid (WBFPN). Specifically, we design a weight-guided feature fusion module to replace the simple feature fusion method so that the WBFPN can suppress interference information when fusing feature maps at different scales. The non-local balancing module captures the long-range dependencies of image features and effectively balances small target defects’ detail and semantic information. Finally, the confidence loss was redefined to effectively solve the problem of poor detection effect caused by the imbalance of positive and negative samples. Experimental results show that the average accuracy of the proposed model reaches 91.9%, and the detection speed is high, which meets the requirement of real-time defect detection.

Aquaculture defects recognition via multi-scale semantic segmentation

Aquaculture net pen defects such as biofouling, vegetation, and holes are key challenges to efficient and sustainable fish production in aquaculture. These defects must be monitored to avoid problems with aquaculture net safety as well as fish growth. Recently, deep learning methods have been adopted to solve detection and classification problems in different applications. However, the conventional methods are challenging to meet the demands of high precision and real-time detection of aquaculture net defects in a complex marine environment. Towards this end, this paper proposes an autonomous net pen defect detection system that contains a novel multi-scale semantic segmentation topology for detecting the biofouling, vegetation, and hole problems in the aquaculture environment. In particular, we emphasize fusing the attention maps obtained across different decomposition levels of the network to generate rich feature distributions that enable the accurate extraction of biofouling, vegetation, and holes within aquaculture nets. Moreover, the proposed model is thoroughly tested on two private datasets and two public datasets which were acquired from a real-field fish farms. Across all four datasets, the proposed framework showed remarkable biofouling, vegetation, and hole detection performance, where it outperformed state-of-the-art methods by 6.58%, 3.69%, 6.44%, and 4.78% in terms of mean average precision across LABUST, KU, NDv1, and NDv2 datasets, respectively.

EFFNet: Element-wise feature fusion network for defect detection of display panels

Online or real-time defect detection of display panels after array process is of paramount importance for quality control and yield rate improvement of products in display industry. However, owing to the limitation in feature representation, the performances of traditional defect detection methods are not satisfactory. This paper develops a novel element-wise feature fusion network (EFFNet) to solve the issue and achieve high-accuracy real-time defect detection of display panels. The method adopts a transfer learning and fine-tuning strategy for feature extraction layers and a decoder with relatively less computational complexity. Particularly, a feature fusion module based on element-wise addition of pyramid features is proposed in skip connection to improve detection efficiency and accuracy. Our method is compared with many state-of-the-art CNN-based models. Additionally, the effects of training dataset size, motion blur, and different backgrounds on the performance of the proposed method are investigated. Extensive experiments, including the ablation study, demonstrate that the developed network can accurately detect defects with complex textures, ambiguous boundaries and low contrast. It also has good robustness against motion blur. It outperforms state-of-the-art methods in terms of mIoU, mPA, and F1-Measure. Moreover, it is able to detect defects at speeds of up to 159 fps with input images of size 256 × 256 pixels.

A New Deep Learning Framework for Imbalance Detection of a Rotating Shaft.

Rotor unbalance is the most common cause of vibration in industrial machines. The unbalance can result in efficiency losses and decreased lifetime of bearings and other components, leading to system failure and significant safety risk. Many complex analytical techniques and specific classifiers algorithms have been developed to study rotor imbalance. The classifier algorithms, though simple to use, lack the flexibility to be used efficiently for both low and high numbers of classes. Therefore, a robust multiclass prediction algorithm is needed to efficiently classify the rotor imbalance problem during runtime and avoid the problem's escalation to failure. In this work, a new deep learning (DL) algorithm was developed for detecting the unbalance of a rotating shaft for both binary and multiclass identification. The model was developed by utilizing the depth and efficacy of ResNet and the feature extraction property of Convolutional Neural Network (CNN). The new algorithm outperforms both ResNet and CNN. Accelerometer data collected by a vibration sensor were used to train the algorithm. This time series data were preprocessed to extract important vibration signatures such as Fast Fourier Transform (FFT) and Short-Time Fourier Transform (STFT). STFT, being a feature-rich characteristic, performs better on our model. Two types of analyses were carried out: (i) balanced vs. unbalanced case detection (two output classes) and (ii) the level of unbalance detection (five output classes). The developed model gave a testing accuracy of 99.23% for the two-class classification and 95.15% for the multilevel unbalance classification. The results suggest that the proposed deep learning framework is robust for both binary and multiclass classification problems. This study provides a robust framework for detecting shaft unbalance of rotating machinery and can serve as a real-time fault detection mechanism in industrial applications.

Infrared in-line monitoring of flaws in steel welded joints: a preliminary approach with SMAW and GMAW processes

The non-destructive full-field non-contact thermographic technique is applied for non-destructive flaw detection of the welded joints, in real-time and offline configuration. In this paper, a thermographic procedure for real-time flaw detection in manual arc welding process is presented. Surface temperature acquisitions by means of an IR camera were performed during arc welding process of 8 specimen both for calibration and validation of the numerical model. The investigated variables are the technique (manual stick arc (SMAW) and gas arc (GMAW) welding) and the joint shape (butt and T joint) for steel joints, in sound conditions and with artificial flaws. Numerical simulation of welding thermal transients was run to obtain the expected surface temperature fields and thermal behavior for different welding parameter configurations. Hardness measurement and micro-graphic analysis were performed to validate numerical simulation results. The real-time thermographic study of the weld pool gives direct indications of anomalies; local studies of the thermal transient and thermal profiles can detect some kind of flaws; microstructural analysis of Heat-Affected Zone (HAZ) and surrounding areas higlights the presence of austenite and martensite distribution which justifies the thermal transients and thermal profiles for different welding configurations. Comparing real-time IR acquisition of the welding process with simulated thermal contours of sound processes provides information of presence of some kind of flaws. Since most of the flaws are generated in the weld pool, it is possible to recognize anomalies directly from the thermal acquisitions or with post-processing the acquired data.