Industrial Datasets Research Articles

A novel application of deep transfer learning with audio pre-trained models in pump audio fault detection

The maintenance of industrial systems is a practical and exciting topic in engineering. High accuracy in fault detection is essential since failure to detect faults can cause damage and reduce efficiency. The application of deep transfer learning with audio pre-training for audio fault detection is investigated in this paper. The main novelty of this research is that for the first time, the knowledge learned by pre-trained models on an extensive audio dataset called AudioSet is used to detect pump faults, and the results are compared with models with image pre-training. In the field of fault detection, the audio of industrial equipment provides a valuable source of data on the state of their operation. In this paper, the audio files of industrial pumps recorded with ambient noise in a real industrial environment have been used, so an algorithm that can achieve high accuracy when analyzing this dataset has a high chance of success when working with other industrial datasets. Usually, fault detection datasets are imbalanced because the amount of faulty data is limited. One of the practical methods to deal with this problem is to use deep transfer learning, which can achieve high accuracy with a limited number of labeled data. Also, four balancing techniques have been studied and compared to determine the optimum way to balance the dataset using various measuring criteria and find the best classifier model. These models are evaluated by multiple criteria to be used for different applications. In this research, four mathematical methods have been used for feature extraction showing that the Log-mel spectrogram is the most effective. Furthermore, it has been demonstrated that this accuracy can be increased in the saturation situation by adding appropriate transformations as data fusion. These transformations look at the knowledge contained in the audio components from different points of view. Finally, the value of AUC increased from 96.86 to 97.48, which is a significant amount compared to other studies.

Imputation of Missing Values in Time Series Using an Adaptive-Learned Median-Filled Deep Autoencoder.

Missing values are ubiquitous in industrial data sets because of multisampling rates, sensor faults, and transmission failures. The incomplete data obstruct the effective use of data and degrade the performance of data-driven models. Numerous imputation algorithms have been proposed to deal with missing values, primarily based on supervised learning, that is, imputing the missing values by constructing a prediction model with the remaining complete data. They have limited performance when the amount of incomplete data is overwhelming. Moreover, many methods have not considered the autocorrelation of time-series data. Thus, an adaptive-learned median-filled deep autoencoder (AM-DAE) is proposed in this study, aiming to impute missing values of industrial time-series data in an unsupervised manner. It continuously replaces the missing values by the median of the input data and its reconstruction, which allows the imputation information to be transmitted with the training process. In addition, an adaptive learning strategy is adopted to guide the AM-DAE paying more attention to the reconstruction learning of nonmissing values or missing values in different iteration periods. Finally, two industrial examples are used to verify the superior performance of the proposed method compared with other advanced techniques.

WSAMF-Net: Wavelet Spatial Attention-Based MultiStream Feedback Network for Single Image Dehazing

Single image-based dehazing has achieved remarkable progress with the development of deep learning technologies. End-to-end neural networks have been proposed to learn a direct hazy-to-clear image translation to recover the clear structures and edges cues from the hazy inputs. However, the frequency domain information is explored insufficiently and lots of intermediate structure and texture related cues of current dehazing networks are ignored, which limits the performances of current approaches. To handle these limitations mentioned above, a wavelet spatial attention based multi-stream feedback network (WSAMF-Net) is proposed for effective single image dehazing. Specifically, the proposed wavelet spatial attention utilizes both frequency-domain and spatial-domain information to enhance the extracted features for better structures and edges. Meanwhile, an enhanced multi-stream based cross feature fusion strategy, including vertical and horizontal attentions, is proposed to reweight and fuse the intermediate features of each stream to acquire more meaningful aggregated features, while the weight sharing strategy is used to achieve a good trade-off between performance and parameters. Besides, feedback mechanism is also designed to provide strong reconstruction ability. Furthermore, we propose a critical real-world industrial dataset (IDS) with images captured in real-world industrial quarry scenarios for research uses. Extensive experiments on various benchmarking datasets, including both synthetic and real-world datasets, demonstrate the superiority of our WSAMF-Net over state-of-the-art single image dehazing methods. The IDS dataset will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/XBSong/IDS-Datasethttps://github.com/XBSong/IDS-Dataset</uri> .

Dynamic transfer soft sensor for concept drift adaptation

Data-driven soft sensor technology has been widely used in process monitoring, quality prediction, etc. However, there are dynamic time-varying and concept drift problems in industrial processes, making the accurate modeling of the soft sensor is still a challenging task. To solve the above problems, this paper proposes a concept drift adaptive dynamic partial least squares method. The method maps high-dimensional process data into a low-dimensional latent variable subspace. Dynamic information is introduced by establishing the dynamic regression relationship between process latent variables and quality latent variables. At the same time, transfer learning is used to align second-order statistics between latent variables of data from different distributions to solve the concept drift problem. The experiments on multiple industrial datasets show that the method proposed in this paper can effectively reduce prediction errors and improve the generalization ability of the model.

Masked Swin Transformer Unet for Industrial Anomaly Detection

The intelligent detection process for industrial anomalies employs artificial intelligence methods to classify images that deviate from a normal appearance. Traditional convolutional neural network (CNN)-based anomaly detection algorithms mainly use the network to restructure abnormal areas and detect anomalies by calculating the errors between the original image and reconstructed image. However, the traditional CNNs struggle to extract global context information, resulting in poor anomaly detection performance. Thus, a masked Swin Transformer Unet (MSTUnet) for anomaly detection is proposed. To solve the problem of insufficient abnormal samples in the training phase, an anomaly simulation and mask strategy is first applied on anomaly-free samples to generate a simulated anomaly and, then, the Swin Transformer's powerful global learning ability is used to inpaint the masked area. Finally, a convolution-based Unet network is used for end-to-end anomaly detection. Experimental results on industrial dataset MVTec AD show that MSTUnet achieves superior anomaly detection and localization performance.

PAEDID: Patch Autoencoder-based Deep Image Decomposition for pixel-level defective region segmentation

Unsupervised pixel-level defective region segmentation is an important task in image-based anomaly detection for various industrial applications. The state-of-the-art methods have their own advantages and limitations: matrix-decomposition-based methods are robust to noise, but lack complex background image modeling capability; representation-based methods are good at defective region localization, but lack accuracy in defective region shape contour extraction; reconstruction-based methods detected defective region match well with the ground truth defective region shape contour, but are noisy. To combine the best of both worlds, we present an unsupervised Patch AutoEncoder-based Deep Image Decomposition (PAEDID) method for defective region segmentation. In the training stage, we learn the common background as a deep image prior by a patch autoencoder network. In the inference stage, we formulate anomaly detection as an image decomposition problem with the deep image prior and sparsity regularizations. By adopting the proposed approach, the defective regions in the image can be accurately extracted in an unsupervised fashion. We demonstrate the effectiveness of the PAEDID method in simulation studies and an industrial dataset in the case study.

Hierarchical Image Transformation and Multi-Level Features for Anomaly Defect Detection.

Anomalies are a set of samples that do not follow the normal behavior of the majority of data. In an industrial dataset, anomalies appear in a very small number of samples. Currently, deep learning-based models have achieved important advances in image anomaly detection. However, with general models, real-world application data consisting of non-ideal images, also known as poison images, become a challenge. When the work environment is not conducive to consistently acquiring a good or ideal sample, an additional adaptive learning model is needed. In this work, we design a potential methodology to tackle poison or non-ideal images that commonly appear in industrial production lines by enhancing the existing training data. We propose Hierarchical Image Transformation and Multi-level Features (HIT-MiLF) modules for an anomaly detection network to adapt to perturbances from novelties in testing images. This approach provides a hierarchical process for image transformation during pre-processing and explores the most efficient layer of extracted features from a CNN backbone. The model generates new transformations of training samples that simulate the non-ideal condition and learn the normality in high-dimensional features before applying a Gaussian mixture model to detect the anomalies from new data that it has never seen before. Our experimental results show that hierarchical transformation and multi-level feature exploration improve the baseline performance on industrial metal datasets.

The nature of losses from cyber-related events: risk categories and business sectors

Abstract In this study, we examine the nature of losses from cyber-related events across different risk categories and business sectors. Using a leading industry dataset of cyber events, we evaluate the relationship between the frequency and severity of individual cyber-related events and the number of affected records. We find that the frequency of reported cyber-related events has substantially increased between 2008 and 2016. Furthermore, the frequency and severity of losses depend on the business sector and type of cyber threat: the most significant cyber loss event categories, by number of events, were related to data breaches and the unauthorized disclosure of data, while cyber extortion, phishing, spoofing, and other social engineering practices showed substantial growth rates. Interestingly, we do not find a distinct pattern between the frequency of events, the loss severity, and the number of affected records as often alluded to in the literature. We also analyse the severity distribution of cyber-related events across all risk categories and business sectors. This analysis reveals that cyber risks are heavy-tailed, i.e. cyber risk events have a higher probability to produce extreme losses than events whose severity follows an exponential distribution. Furthermore, we find that the frequency and severity of cyber-related losses exhibit a very dynamic and time-varying nature.

Fault diagnosis model of multi-axis industrial robot based on triplet network

Fault diagnosis is an important link in intelligent development of industrial robots. Aiming at the problem of weak fault diagnosis performance caused by insufficient training samples, a fault diagnosis model based on triplet network is proposed. Firstly, we combine the multiscale convolutional neural network (MSCNN) with channel attention networks (squeeze-and-excitation network, SENet), and use it to construct a triple sub-network structure MS-SECNN, which can adaptively extract features from the original fault signal. Then, the feature similarity is calculated by triplet loss in the low dimensional space to realize the fault classification task. The experiments are based on the real industrial robot operation data set. In this model, we use Few-shot learning strategy to test the diagnostic performance under small samples, and compare it with WDCNN, FDCNN and MSCNN models. Experimental results show that the proposed model has more effective fault classification ability under small samples. In addition, when the training sample size is 1400, the average accuracy of MS-SECNN reaches 99.21%.

Genetic parameter analysis of bareness and tail traits in New Zealand sheep.

Physical traits that improve welfare and disease outcomes for sheep are becoming increasingly important due to both increased climate challenges and societal expectations. Such traits include tail length, the amount of skin (vs. wool) on the underside of the tail, and the area of no-wool (hair) on the belly and breech areas (surrounding the anus) of the animal. An industry dataset consisting of records from individual stud breeders and industry progeny tests was available to estimate the genetic parameters associated with these traits and to investigate the potential for within-breed genetic selection. The heritability estimate for tail length was 0.68 ± 0.01 when breed was not fitted, and 0.63 ± 0.01 when breed was fitted. Similar trends were observed for breech and belly bareness which had heritability estimates around 0.50 (± 0.01). The estimates for these bareness traits are both higher than previous reports from animals of the same age. There was, however, between breed variation in the starting point for these traits, with some breeds having significantly longer tails and a wooly breech and belly, and limited variability. Overall, the results of this study show that flocks exhibiting some variation will be able to make rapid genetic progress in selecting for bareness and tail length traits, and therefore have the potential to make progress towards a sheep that is easier to look after and suffers fewer welfare insults. For those breeds that showed limited within-breed variation, outcrossing may be required to introduce genotypes that exhibit shorter tail length and bareness of belly and breech to increase the rate of genetic gain. Whatever approach is taken by the industry, these results support that genetic improvement can be used to breed "ethically improved sheep".

Industrial Fault Detection using Contrastive Representation Learning on Time-series Data

Deep learning (DL) has been known as one of the effective techniques for building data-driven fault detection methods. The successful DL-based methods require the condition that massive labeled data are available, but this is sometimes an inevitable obstacle in real industrial environments. As one of the solutions, autoencoders (AEs) are widely adopted since AEs can extract features from unlabeled data. However, some challenges in AE-based fault detection methods remain, such as the design of encoder architecture, the computational cost, and the usage of the limited labeled data. This paper proposes a new industrial fault detection method through learning instance-level representation of time-series based on the self-supervised contrastive learning framework (SSCL). The proposed method uses dilated-causal-convolution-based encoder-only architecture to extract the information from industrial time-series data. A new data augmentation method for time-series data is proposed based on the temporal distance distribution, which is used to construct positive pairs in SSCL. Moreover, the encoder is alternately trained by the new weighted contrastive loss and the traditional classification loss. Finally, the experiments are conducted on the industrial data set and a semi-physical system, showing the effectiveness of the proposed method.

Improved deep depth estimation for environments with sparse visual cues

Most deep learning-based depth estimation models that learn scene structure self-supervised from monocular video base their estimation on visual cues such as vanishing points. In the established depth estimation benchmarks depicting, for example, street navigation or indoor offices, these cues can be found consistently, which enables neural networks to predict depth maps from single images. In this work, we are addressing the challenge of depth estimation from a real-world bird’s-eye perspective in an industry environment which contains, conditioned by its special geometry, a minimal amount of visual cues and, hence, requires incorporation of the temporal domain for structure from motion estimation. To enable the system to incorporate structure from motion from pixel translation when facing context-sparse, i.e., visual cue sparse, scenery, we propose a novel architecture built upon the structure from motion learner, which uses temporal pairs of jointly unrotated and stacked images for depth prediction. In order to increase the overall performance and to avoid blurred depth edges that lie in between the edges of the two input images, we integrate a geometric consistency loss into our pipeline. We assess the model’s ability to learn structure from motion by introducing a novel industry dataset whose perspective, orthogonal to the floor, contains only minimal visual cues. Through the evaluation with ground truth depth, we show that our proposed method outperforms the state of the art in difficult context-sparse environments.

Improving Virtual Sensor Models by Censored Online Data

Digital twins are able to bridge the physical and the virtual world, which is especially useful in industrial environments. One small, but rather essential, kind of digital twin is the so called virtual sensor model. This type of model is used to enhance or replace a physical sensor in industrial settings to reduce costs or enable information retrieval from inaccessible locations within a machine or process. The virtual sensor model is usually trained once, whereat only a predefined amount of information without any adaptation possibilities on the clients side is provided. As manufacturers want to provide the possibility of custom adaptations for their clients, the virtual sensor models require to incorporate adaptive features, which also have to handle incomplete data recordings, e.g. uncensored data. Traditional offline machine learning approaches are often insufficient for such adaptive requirements, therefore the usage of online learning approaches is gaining increased attention, to avoid high computational, storage and temporal costs. This paper covers the continued training of such adaptive virtual sensor models, focusing on the handling and integration of censored online data. Different approaches to tackle the problems of catastrophic forgetting in online learning and correction of censored data are presented as well as the handling of censored data in online learning environments. The experiments sections compares various scenarios with and without censored data using an industrial dataset and demonstrates the positive influence of different online learning approaches.

Retracted: Feature Sequencing Method of Industrial Control Data Set Based on Multidimensional Evaluation Parameters.

[This retracts the article DOI: 10.1155/2022/9248267.].

Application of Alarm Correlations in Root Cause Diagnosis of Plant-wide Oscillations*

Plant-wide oscillations are common types of disturbances in industrial processes, whose effects propagate to multiple process units through material and/or information flow paths. Such oscillations cause poor controller performance and thus can impact product quality and overall plant performance. Early detection of oscillations and root cause diagnosis is essential. Constructing an adjacency matrix is an efficient method to locate the root cause of oscillations and uses control loop digraphs to capture controller interactions from the process flowsheet. However, constructing an adjacency matrix is time-consuming and resource-intensive, especially when dealing with complex industrial facilities. This paper proposes an alternative data-driven method to construct an adjacency matrix using alarm correlations extracted from historical Alarm & Event (A&E) data. A component-level adjacency matrix is constructed by consolidating process unit information from alarm configuration attributes. Potential root causes of oscillations are shortlisted for further investigations using a process knowledge-based method. This preliminary diagnosis tool can reduce the computational complexity of the subsequent knowledge-driven stage. The applicability and effectiveness of the proposed method are demonstrated via a case study using a benchmark industrial dataset.

Federated Semisupervised Learning for Attack Detection in Industrial Internet of Things

Security has become a critical issue for Industry4.0 due to different emerging cyber-security threats. Recently, many deep learning (DL) approaches have focused on intrusion detection. However, such approaches often require sending data to a central entity. This in turn raises concerns related to privacy, efficiency, and latency. Despite the huge amount of data generated by the Internet of Things (IoT) devices in Industry 4.0, it is difficult to get labeled data, because data labeling is costly and time-consuming. This poses many challenges for several DL approaches, which require labeled data. In order to deal with these issues, new approaches should be adopted. This article proposes a novel federated semisupervised learning scheme that takes advantage of both unlabeled and labeled data in a federated way. First, an autoencoder (AE) is trained on each device (using unlabeled local/private data) to learn the representative and low-dimensional features. Then, a cloud server aggregates these models into a global AE using federated learning (FL). Finally, the cloud server composes a supervised neural network, by adding fully connected layers (FCN) to the global encoder (the first part of the global AE) and trains the resulting model using publicly available labeled data. Extensive case studies on two real-world industrial datasets demonstrate that our model: (a) ensures that no local private data is exchanged; (b) detects attacks with high classification performance, (c) works even when only a few amounts of labeled data are available; and (d) haslow communication overhead.

Bayesian Optimization Cost-Sensitive XGBoost Learning Algorithm for Imbalanced Data in Semiconductor Industry

This paper proposes an improved ensemble learning model based on extreme gradient boosting (XGBoost) with Bayesian optimization cost-sensitive learning algorithm for dealing with highly imbalanced data in the semiconductor process to achieve the highest possible pass and fail accuracy or recall for the classification performances. Most of the existing models are biased toward the majority class neglecting the minority class. The proposed Bayesian optimization cost-sensitive XGboost model is configured to be applied to the semiconductor dataset. The obtained experimental results - based on benchmarking semiconductor industry dataset - show 91.46% and 23.08% for the pass and fail accuracies, respectively. This confirms that the proposed model is significant for imbalanced cases in semiconductor applications. Moreover, this investigation reveals that the proposed model is able not only to maintain the performance of the majority class, but also to classify well the minority class.

Rain-Like Layer Removal From Hot-Rolled Steel Strip Based on Attentive Dual Residual Generative Adversarial Network

Rain-like layer removal from hot-rolled steel strip surface is proved to be a workable measure for suppressing the false alarms frequently triggered in automated visual inspection (AVI) instrument. This paper extends the scope of “rain-like layer” from dispersed waterdrops to splashing water streaks and tiny white droplets. And a targeted method with both channel-wise and spatial-wise attention, namely attentive dual residual generative adversarial network (ADRGAN), is proposed. Meanwhile, a newly updated steel surface image dataset with typical natures of “rain-like layer” gathered from actual hot-rolling line, Steel_Rain, is opened for the first time. The comparison experimental results between our proposed network and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">eleven</i> prestigious networks show that our ADRGAN-restored images are the closest to the ground-truth images on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">six</i> public datasets, especially on the newly-opened industrial dataset Steel_Rain, it yeilds the best scores of 56.8627 peak signal to noise ratio (PSNR), 0.9980 structural similarity index (SSIM), 0.134 mean-square error (MSE) and 0.006 learned perceptual image patch similarity (LPIPS). In the final verification test, the concept of rain-like layer removal has been proved to perform best in defect inspection, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">four</i> traditional defect detection algorithms are involved. And as expected, defect detection methods assisted by ADRGAN yield the minimum false-alarms <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Compact Dynamic Inner Canonical Correlation Analysis for Nonstationary Dynamic Feature Extraction and Prediction

In this paper, a novel compact dynamic inner canonical correlation analysis (DiCCA) algorithm with autoregressive integrated moving average (ARIMA) inner models is proposed to better capture the latent dynamics of high dimensional time series. It can extract latent factors that capture the underlying dynamics of the data and model them using the ARIMA structure, which has fewer parameters and more flexibility than the potentially high-order AR structure used in the original DiCCA algorithm. The proposed algorithm can also handle nonstationary latent factors by explicitly modeling the unit roots. The algorithm integrates the extraction and the modeling of a latent factor in one step, resulting in a consistent inner and outer model. The algorithm is applied to an industrial dataset and shows better performance than the original DiCCA algorithm.

Anomaly Detection of GAN Industrial Image Based on Attention Feature Fusion

As life becomes richer day by day, the requirement for quality industrial products is becoming greater and greater. Therefore, image anomaly detection on industrial products is of significant importance and has become a research hotspot. Industrial manufacturers are also gradually intellectualizing how product parts may have flaws and defects, and that industrial product image anomalies have characteristics such as category diversity, sample scarcity, and the uncertainty of change; thus, a higher requirement for image anomaly detection has arisen. For this reason, we proposed a method of industrial image anomaly detection that applies a generative adversarial network based on attention feature fusion. For the purpose of capturing richer image channel features, we added attention feature fusion based on an encoder and decoder, and through skip-connection, this performs the feature fusion for the encode and decode vectors in the same dimension. During training, we used random cut-paste image augmentation, which improved the diversity of the datasets. We displayed the results of a wide experiment, which was based on the public industrial detection MVTec dataset. The experiment illustrated that the method we proposed has a higher level AUC and the overall result was increased by 4.1%. Finally, we realized the pixel level anomaly localization of the industrial dataset, which illustrates the feasibility and effectiveness of this method.