Real-time Fault Detection Research Articles

A new key performance indicator oriented industrial process monitoring and operating performance assessment method based on improved Hessian locally linear embedding

ABSTRACT The industrial process monitoring and operating performance assessment techniques are of great significance to ensure the safety and efficiency of the production and to improve the comprehensive economic benefits for the modern enterprises. In this paper, a new key performance indicator (KPI) oriented nonlinear process monitoring and operating performance assessment method is proposed based on the improved Hessian locally linear embedding (HLLE), in view of the problems of strong nonlinearity, high dimension and information redundancy in actual industrial process data. Firstly, in order to characterise the similarities of samples in both temporal and spatial dimensions, a new measurement, based on Finite Markov theory, is defined to replace the Euclidean distance in traditional HLLE. Secondly, by mining the relationships between process variables and the key performance indicator, the KPI oriented feature extraction method is developed. On this basis, the monitoring statistics is constructed and the corresponding control limit is determined for the real-time fault detection. After that, a new operating performance assessment approach based on sliding window Kullback–Leibler divergence is put forward to facilitate maintenance or adjustments. Finally, the proposed method is applied to the hot strip mill process, and the results show the effectiveness.

Lightweight Hot-Spot Fault Detection Model of Photovoltaic Panels in UAV Remote-Sensing Image.

Photovoltaic panels exposed to harsh environments such as mountains and deserts (e.g., the Gobi desert) for a long time are prone to hot-spot failures, which can affect power generation efficiency and even cause fires. The existing hot-spot fault detection methods of photovoltaic panels cannot adequately complete the real-time detection task; hence, a detection model considering both detection accuracy and speed is proposed. In this paper, the feature extraction part of YOLOv5 is replaced by the more lightweight Focus structure and the basic unit of ShuffleNetv2, and then the original feature fusion method is simplified. Considering that there is no publicly available infrared photovoltaic panel image dataset, this paper generates an infrared photovoltaic image dataset through frame extraction processing and manual annotation of a publicly available video. Consequently, the number of parameters of the model was 3.71 M, mAP was 98.1%, and detection speed was 49 f/s. A comprehensive comparison of the accuracy, detection speed, and model parameters of each model showed that the indicators of the new model are superior to other detection models; thus, the new model is more suitable to be deployed on the UAV platform for real-time photovoltaic panel hot-spot fault detection.

Fabric Defect Detection using Image Processing

Fabric defect is one of the most important and serious matters of quality control in textile industry in Bangladesh. This task takes a lot of time and money. For this reason we have introduced a simple process to find defects on fabric based on edge detection. This process is mainly focused on image processing which can be integrated with fabric defect detection automation system. In this paper we have tried a new approach using the filter method with edge detection and found good results. Our algorithm can detect defected fabric area successfully. It can be also used in real-time defect detection considering light intensity, zoom, fabric width, camera resolution etc. As our algorithm mainly works on the principle of edge detection, it cannot detect defect on multicoloured or patterned fabric. It works well on single coloured fabric without any fold or edge.

Improved Faster R-CNN Based Surface Defect Detection Algorithm for Plates

Defect recognition plays an important part of panel inspection, and most of the current manual inspection methods are used, but the recognition efficiency and recognition accuracy are low. The Fast-Convolutional Neural Network (Faster R-CNN) algorithm is improved, and a surface defect detection algorithm based on the improved Faster R-CNN is proposed. Firstly, the algorithm improves the bilateral filtering algorithm to smooth the image texture background. Subsequently, a feature pyramid network with a shape-variable convolutional ResNet50 network can be applied to acquire defect semantic feature maps to improve the network's ability to express the features of multiscale defects while solving the difficulty problem of many types of defects and variable shapes. To obtain more accurate defect localization information, the algorithm in this paper uses the Region of Interest Align (ROI Align) algorithm instead of the crude Region of Interest Pooling (ROI Pooling) algorithm. Then, an improved attention region recommendation network is used to improve the focus of the model on plate defects and suppress the features of complex background. Finally, a K-means algorithm is added to cluster the defect data to derive anchor frames that are better adapted to the plate defects. In this paper, a dataset containing 3216 images of surface defects of plate metal is made by acquiring surface defect images from the production site of the plate metal factory, which mainly include various defect types. This dataset is used to train and test the algorithm model of this paper, and the results of detection accuracy and detection speed are compared with those of other algorithms, which prove that the algorithm of this paper can achieve real-time detection of plate defects with high detection accuracy.

Defect Detection for Metal Base of TO-Can Packaged Laser Diode Based on Improved YOLO Algorithm

Defect detection is an important part of the manufacturing process of mechanical products. In order to detect the appearance defects quickly and accurately, a method of defect detection for the metal base of TO-can packaged laser diode (metal TO-base) based on the improved You Only Look Once (YOLO) algorithm named YOLO-SO is proposed in this study. Firstly, convolutional block attention mechanism (CBAM) module was added to the convolutional layer of the backbone network. Then, a random-paste-mosaic (RPM) small object data augmentation module was proposed on the basis of Mosaic algorithm in YOLO-V5. Finally, the K-means++ clustering algorithm was applied to reduce the sensitivity to the initial clustering center, making the positioning more accurate and reducing the network loss. The proposed YOLO-SO model was compared with other object detection algorithms such as YOLO-V3, YOLO-V4, and Faster R-CNN. Experimental results demonstrated that the YOLO-SO model reaches 84.0% mAP, 5.5% higher than the original YOLO-V5 algorithm. Moreover, the YOLO-SO model had clear advantages in terms of the smallest weight size and detection speed of 25 FPS. These advantages make the YOLO-SO model more suitable for the real-time detection of metal TO-base appearance defects.

Generalized fault diagnosis method of transmission lines using transfer learning technique

Recent artificial intelligence-based methods have shown great promise in the use of neural networks for real-time detection of transmission line faults and estimation of their locations. The expansion of power systems including transmission lines with various lengths have made the fault detection, classification, and location estimation process more challenging. Transmission line datasets are stream data which are continuously collected by various sensors and hence, require generalized and fast fault diagnosis approaches. Newly collected datasets including voltages and currents for faulty and non-faulty situations might not have adequate and accurate labels that are useful to train neural networks. In this paper, a novel transfer learning framework based on a pre-trained LeNet-5 convolutional neural network is proposed. This method is able to diagnose faults for different transmission line lengths and impedances by transferring the knowledge from a source convolutional neural network to predict a dissimilar target dataset. By transferring this knowledge, faults from various transmission lines, even without sufficient data samples with labels, can be diagnosed faster and more efficiently than the existing methods. To prove the feasibility and effectiveness of this methodology, seven different datasets that include various lengths of transmission lines are used. The robustness of the proposed methodology against the generator voltage fluctuations, variations in fault locations, fault inception angle, fault resistance, and phase difference between the two generators are well studied to prove the reliability of this technique for fault diagnosis of transmission lines.

Real-Time Detection of Incipient Inter-Turn Short Circuit and Sensor Faults in Permanent Magnet Synchronous Motor Drives Based on Generalized Likelihood Ratio Test and Structural Analysis.

This paper presents a robust model-based technique to detect multiple faults in permanent magnet synchronous motors (PMSMs), namely inter-turn short circuit (ITSC) and encoder faults. The proposed model is based on a structural analysis, which uses the dynamic mathematical model of a PMSM in an frame to evaluate the system’s structural model in matrix form. The just-determined and over-determined parts of the system are separated by a Dulmage–Mendelsohn decomposition tool. Subsequently, the analytical redundant relations obtained using the over-determined part of the system are used to form smaller redundant testable sub-models based on the number of defined fault terms. Furthermore, four structured residuals are designed based on the acquired redundant sub-models to detect measurement faults in the encoder and ITSC faults, which are applied in different levels of each phase winding. The effectiveness of the proposed detection method is validated by an in-house test setup of an inverter-fed PMSM, where ITSC and encoder faults are applied to the system in different time intervals using controllable relays. Finally, a statistical detector, namely a generalized likelihood ratio test algorithm, is implemented in the decision-making diagnostic system resulting in the ability to detect ITSC faults as small as one single short-circuited turn out of 102, i.e., when less than 1% of the PMSM phase winding is short-circuited.

Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture

Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture

An Object Detection Model for Paint Surface Detection Based on Improved YOLOv3

To solve the problem of poor performance of the target detection algorithm and false detection in the detection of paint surface defects of office chairs five-star feet, we propose a defect detection method based on the improved YOLOv3 algorithm. Firstly, a new feature fusion structure is designed to reduce the missed detection rate of small targets. Then we used the CIOU loss function to improve the positioning accuracy. At the same time, a parallel version of the k-means++ initialization algorithm (K-means||) is used to optimize and determine the parameters of the a priori anchor so as to improve the matching degree between the a priori anchor and the feature layer. We constructed a dataset of paint surface defects on the five-star feet of office chairs and performed optimization training, and used multiple algorithms and different datasets to conduct comparative experiments to validate the algorithm. The experimental results show that the improved YOLOv3 algorithm is effective in that the average precision on the self-made dataset reaches 88.3%, which is 5.8% higher than the original algorithm. At the same time, it has also been verified based on the Aliyun Tianchi competition aluminum dataset, and the average precision has reached 89.2%. This method realizes the real-time detection of the paint surface defects of the five-star feet of the office chair very well.

Correlating in-situ sensor data to defect locations and part quality for additively manufactured parts using machine learning

In this work, process monitoring data, including layerwise imagery, multi-spectral emissions, and laser scan vector data, were collected during laser-based powder bed fusion additive manufacturing and correlated to fatigue performance. All parts were X-ray CT scanned post-build, and internal flaws were identified via an automated defect recognition software. Convolutional neural networks were trained to discriminate flaws from nominal build conditions using in situ data modalities only. Trained classifiers were then tested against a previously unseen data set collected from an independent build, and classification performance and metrics for information content provided by each individual modality were formally established. Correlations were drawn between the detected flaw populations and the corresponding fatigue properties, demonstrating that fatigue critical lack-of-fusion flaws can be detected via machine learning of in situ sensor data. The present results also show that, at least from a classification accuracy perspective, flaw detection via ML on process monitoring data is a viable path forward for real-time flaw detection and automated, interlayer repair strategies. However, strategies for extracting and analyzing sensor data in real-time without incurring excessive increases in build time must first be developed. These developments represent necessary components to draw direct correlations between in situ data modalities, internal part quality, and fatigue performance.

A real-time defect detection in printed circuit boards applying deep learning

Inspection of defects in the printed circuit boards (PCBs) has both safety and economic significance in the 4.0 industrial manufacturing. Nevertheless, it is still a challenging problem to be studied in-depth due to the complexity of the PCB layouts and the shrinking down tendency of the electronic component size. In this paper, a real-time automated supervision algorithm is proposed to test the PCBs quality among different scenarios. The density of the PCBs layout and the complexity on the surface are analyzed based on deep learning and image feature extraction algorithms. To be more detailed, the ORB feature and the Brute-force matching method are utilized to match perfectly the input images with the PCB templates. After transferring images by aiding the RANSAC algorithm, a hybrid method using modern computer vision algorithms is developed to segment defective areas on the PCBs surface. Then, by applying the enhanced Residual Network –50, the proposed algorithm can classify the groove defects on the surface mount technology electronic components which minimum size up to 1x3 mm. After the training process, the proposed system is capable to categorize various types of overproduced, recycled, and cloned PCBs. The speed of the quality testing operation maintains at a high level with an average precision rate up to 96.29 % in case of good brightness conditions. Finally, the computational experiments demonstrate that the proposed system based on deep learning can obtain superior results and it outperforms several existing works in terms of speed, precision, and robustness

Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network

In order to realize on-line detection of defective apples on a two-lane fruit sorting machine, an inspection module was constructed using NIR cameras and a diffuse illumination chamber. A real-time apple defects inspection method was proposed based on YOLO V4 deep learning algorithm. The input images were generated by combining NIR images in three consecutive rubber roller stations. Channel pruning and layer pruning methods were used to simplify the YOLO V4 network and accelerate the detection speed. A non-maximum suppression (NMS) method based on L1 norm is proposed to remove redundant prediction box after fine-tuning the pruned network. The test results indicated that the model size and inference time of the pruning-based YOLO V4 network was decreased by 241.24 megabyte (MB) and 10.82 ms, respectively, and the mean average precision (mAP) was increased from 91.82% to 93.74%, compared with the YOLO V4 network before pruning. The pruning-based YOLO V4 network based on NIR images was not affected by the variation of skin color and suitable for detects identification of different cultivars including ‘Fuji’ apple covered in red-yellow striping and red blush, ‘Golden Delicious’, and ‘Granny Smith’, with the average detection accuracy of 93.9% at the on-line test assessing five fruit per second. The overall results showed that the proposed pruning-based YOLO V4 network combined with the developed inspection module, had great potential to be implemented in commercial fruit packing line for fruit defects identification.

Fault detection of petrochemical process based on space-time compressed matrix and Naive Bayes

Due to the high available and reliable requirements of petrochemical processes, it is critical to develop real-time fault detection approaches with high performance. Some machine learning approaches have shown good results but the learning process is too complicated to meet the requirements of online application, such as plenty of samples or the laborious hyper-parameter optimization is needed. In this paper, a fault detection approach based on space-time compressed matrix (STCM) and Naive Bayes (NB) is proposed to realize the fast learning and prediction. First, the slowly varying features which reflect the inherent dynamic information of petrochemical processes are extracted by slow feature analysis. Second, the accumulative importance of slow features and the reconstructive advantage of slow feature under-sampling are proposed to achieve the space-time compression of data matrix. Finally, the STCM is employed to establish the NB model, which can significantly reduce the learning complexity while ensuring classification performance. Experiments on the Tennessee Eastman benchmark show that the proposed approach reduces the sample-size and feature-size by 75% and 92% respectively. Both the average classification accuracy and F1 score on 21 faults exceed 84%, achieving the state-of-the-art results among the comparative approaches.

Real-time defect detection and classification on wood surfaces using deep learning

This paper proposes a novel method for automatic realtime defect detection and classification on wood surfaces. Our method uses deep convolutional neural network (CNN) based approach Faster R-CNN (Region-based CNN ) as detector and MobileNetV3 as backbone network for feature extraction. The key difference of our approach from the existing methods is that it detects knots and other type of defects efficiently and does the classification in real-time from the input video frames. Speed and accuracy is the main focus of our work. In the case of the industrial quality control and inspection such as defects detection, the task of detection and classification needs to be done in real-time on a computationally limited processing units or commodity processors. Our trained model is a light weight, and it can even be deployed on systems for example mobile and edge devices. We have pre-trained the MobileNet V3 on large image dataset for feature extraction. We use Faster R-CNN for detection and classification of defects. The system does the real-time detection and classification on an average of 37 frames per second from input video frames, using low cost and low memory GPU (Graphics Processing Unit). Our method has achieved an overall accuracy of 99% in detecting and classifying defects.

RoboPV: An integrated software package for autonomous aerial monitoring of large scale PV plants

In this paper, a novel software package, called RoboPV, is introduced for autonomous aerial monitoring of PV plants. RoboPV automatically performs aerial monitoring of PV plants, from optimal trajectory planning to image processing and pattern recognition for real-time fault detection and analysis. RoboPV consists of four integrated components: boundary area detection, path planning, dynamic processing, and fault detection. To design an optimal flight path, aerial images of PV plants, which have been collected from experimental flights, are given as inputs to a developed encoder-decoder deep learning architecture to extract boundary points of PV plants automatically. Then, a novel path planning algorithm is conducted by RoboPV to design an optimal flight path with full coverage of whole regions of the PV plant. Aerial images are analysed in real-time during the flight by a high precise neural network trained for automatic fault detection. In this study, several decision-making and maneuver algorithms were developed for various real-world flight conditions to improve the performance of RoboPV during an autonomous aerial inspection. RoboPV is a modular processing library that can be installed on any micro-computer processor with a low computational power. Moreover, supporting the MAVLink communication protocol enables RoboPV to connect with an intelligent Pixhawk flight autopilot and navigate a wide range of multi-rotors. To demonstrate the performance of RoboPV, a six degrees of freedom dynamic model of a multi-rotor is developed in a SIMULINK environment with a defined aerial monitoring mission on three different real megawatt-scale PV plants. The results prove that RoboPV can execute the autonomous aerial inspection with an overall accuracy of 93% for large-scale PV plants.

CNN and Convolutional Autoencoder (CAE) based real-time sensor fault detection, localization, and correction

Increasing advances in sensing technologies and analytics have led to the proliferation of sensors to monitor structural and infrastructural systems. Accurate sensor data can provide information about structural health, aid in prognosis, and help calculate forces for vibration control. However, sensors are susceptible to faults such as loss of data, random noise, bias, drift, etc., due to the aging of sensors, defects, or environmental factors. Although traditional signal processing techniques can detect and isolate faults and reconstruct corrupt or missing sensor data, they demand significant human intervention. The continuous rise in computational power and demonstrated efficacy in numerous domains motivates the use of deep learning to minimize human-in-the-loop techniques. In this work, we introduce a novel, deep learning framework for linear systems with time-invariant parameters that identifies the presence and type of fault in sensor data, location of the faulty sensor and subsequently reconstructs the correct sensor data for fault detection, fault classification, and reconstruction. In our framework, first, a Convolutional Neural Network (CNN) is used to detect the presence of a fault and identify its type. Next, a suite of individually trained Convolutional Autoencoder (CAE) networks corresponding to each type of fault are employed for reconstruction. We demonstrate the efficacy of our framework to address both single and multiple sensor faults in synthetically generated data of a simple shear-type structure and experimentally measured data from a simplified arch bridge. While the framework is agnostic of fault-type, we demonstrate its use for four types of fault namely, missing, spiky, random, and drift. For both simulated and experimental datasets with a single fault, our models performed well, achieving 100% accuracy in faulty sensor localization, more than 98.7% accuracy in fault type detection, and more than 99% accuracy in reconstruction. Our framework can also address multiple concurrent faults with similar accuracy. We empirically demonstrate that our proposed framework performs better than other state-of-the-art techniques in terms of computational efficiency with comparable accuracy. Adoption of our framework in online structural health monitoring applications can lead to minimal disruption to monitoring processes, reduced downtime for structures and infrastructure while simultaneously reducing uncertainty and improving the quality of sensor data for historical records.

AYOLOv3-Tiny: An improved convolutional neural network architecture for real-time defect detection of PAD light guide plates

Light guide plates (LGPs) are the main component of the backlight unit of liquid crystal display (LCD) devices, and defective LGPs directly affect the display effect of LCDs. In view of the features of portable Android device (PAD) LGP images, such as complex texture background, low contrast, different defect sizes and various defect types and their optical properties, light-spot distribution, defect formation principle and imaging characteristics, this paper proposes an AYOLOv3-Tiny network for the defect detection of LGPs. First, by combining overlapping pooling and the spatial attention mechanism, the overlapping pooling spatial attention module (OSM) is constructed to replace the traditional convolution operation of the YOLOv3-Tiny backbone network. Overlapping pooling can improve the accuracy and prevent overfitting, and the spatial attention mechanism can help the network better extract defect features. Second, a dilated convolution module (DCM) is constructed in the detection branch. The module can expand the receptive field of the convolution kernel and improve the detection ability of large defects by integrating the dilated convolution into the residual network structure. Third, a large number of experiments based on the self-built dataset PAD LGP SDD are carried out. The experimental results show that the mean average precision (mAP) and F1-score of the LGP defect detection system can reach 99.50% and 99.61% respectively, and the detection speed can reach 144 fps. Finally, by testing on the PAD LGP images with defects, it is verified that the proposed network meets the application requirements of high precision and real-time online detection of PAD LGP images.

Real-time Defect Detection in Production of NTC Thermistor Discs

Real-time Defect Detection in Production of NTC Thermistor Discs

Surface Defect Detection in Sanitary Ceramics Based on Lightweight Object Detection Network

Sanitary ceramic products, such as toilet and wash basin, are widely used in our daily life. Sanitary ceramics are expected to have some excellent physical properties, such as corrosion resistance, easy cleaning, and low water absorption. However, surface defects in sanitary ceramics are inevitable due to complex production processes and changing production environment. Therefore, surface defect detection must be performed in the manufacturing process of sanitary ceramics. There are many types of surface defects in sanitary ceramics, and different types of defects have large differences in characteristics and scales. Traditional detection methods with artificially designed features and classifiers are difficult to apply in this context. In addition, there are few studies on surface defect detection methods of sanitary ceramics based on deep neural networks. In this article, a lightweight real-time defect detection network based on the lightweight backbone MobileNetV3 is presented. The proposed network achieves multi-scale detection of surface defects in sanitary ceramics with a multi-layer feature pyramid. Combining region proposal network and anchor-free method, real-time defect detection is achieved. Finally, a detection head with channel attention structure and a low-level mixed feature classification strategy is used to perform defect classification with higher accuracy. Experimental results show that the proposed approach achieves at least 22.9% detection speed improvement and 35.0% average precision improvement while reducing memory consumption by at least 8.4% compared with the classic one-stage SSD, YOLO V3 and two-stage Faster R-CNN methods.

On the Problem of Identification System, Controller Design, Real-Time Fault Detection and Diagnosis for Water Single Tank System

The objective of this paper presents an identification of single water tank system by Genetic Algorithm (GA), a comparison control quality of two controllers sliding mode control (SMC) and fuzzy sliding mode control (FSMC) and a design of real-time fault detection and diagnosis for the system. The entire system has been modeled and tested by Matlab/Simulink toolbox.