Industrial Datasets Research Articles

Compact Dynamic Inner Canonical Correlation Analysis for Nonstationary Dynamic Feature Extraction and Prediction

In this paper, a novel compact dynamic inner canonical correlation analysis (DiCCA) algorithm with autoregressive integrated moving average (ARIMA) inner models is proposed to better capture the latent dynamics of high dimensional time series. It can extract latent factors that capture the underlying dynamics of the data and model them using the ARIMA structure, which has fewer parameters and more flexibility than the potentially high-order AR structure used in the original DiCCA algorithm. The proposed algorithm can also handle nonstationary latent factors by explicitly modeling the unit roots. The algorithm integrates the extraction and the modeling of a latent factor in one step, resulting in a consistent inner and outer model. The algorithm is applied to an industrial dataset and shows better performance than the original DiCCA algorithm.

Anomaly Detection of GAN Industrial Image Based on Attention Feature Fusion

As life becomes richer day by day, the requirement for quality industrial products is becoming greater and greater. Therefore, image anomaly detection on industrial products is of significant importance and has become a research hotspot. Industrial manufacturers are also gradually intellectualizing how product parts may have flaws and defects, and that industrial product image anomalies have characteristics such as category diversity, sample scarcity, and the uncertainty of change; thus, a higher requirement for image anomaly detection has arisen. For this reason, we proposed a method of industrial image anomaly detection that applies a generative adversarial network based on attention feature fusion. For the purpose of capturing richer image channel features, we added attention feature fusion based on an encoder and decoder, and through skip-connection, this performs the feature fusion for the encode and decode vectors in the same dimension. During training, we used random cut-paste image augmentation, which improved the diversity of the datasets. We displayed the results of a wide experiment, which was based on the public industrial detection MVTec dataset. The experiment illustrated that the method we proposed has a higher level AUC and the overall result was increased by 4.1%. Finally, we realized the pixel level anomaly localization of the industrial dataset, which illustrates the feasibility and effectiveness of this method.

Machine Learning-Enabled Smart Industrial Automation Systems Using Internet of Things.

Industrial automation uses robotics and software to operate equipment and procedures across industries. Many applications integrate IoT, machine learning, and other technologies to provide smart features that improve the user experience. The use of such technology offers businesses and people tremendous assistance in successfully achieving commercial and noncommercial requirements. Organizations are expected to automate industrial processes owing to the significant risk management and inefficiency of conventional processes. Hence, we developed an elaborative stepwise stacked artificial neural network (ESSANN) algorithm to greatly improve automation industries in controlling and monitoring the industrial environment. Initially, an industrial dataset provided by KLEEMANN Greece was used. The collected data were then preprocessed. Principal component analysis (PCA) was used to extract features, and feature selection was based on least absolute shrinkage and selection operator (LASSO). Subsequently, the ESSANN approach is proposed to improve automation industries. The performance of the proposed algorithm was also examined and compared with that of existing algorithms. The key factors compared with existing technologies are delay, network bandwidth, scalability, computation time, packet loss, operational cost, accuracy, precision, recall, and mean absolute error (MAE). Compared to traditional algorithms for industrial automation, our proposed techniques achieved high results, such as a delay of approximately 52%, network bandwidth accomplished at 97%, scalability attained at 96%, computation time acquired at 59 s, packet loss achieved at a minimum level of approximately 53%, an operational cost of approximately 59%, accuracy of 98%, precision of 98.95%, recall of 95.02%, and MAE of 80%. By analyzing the results, it can be seen that the proposed system was effectively implemented.

The Paycheck Protection Program and small business performance: Evidence from craft breweries.

The Paycheck Protection Program (PPP) provided approximately US $790 billion in COVID-19 relief funds to small businessesacross the United States. This study merges a verified industry dataset of craft beer producers with government microdata on PPP loan recipients to examine the relationship between PPP funding and small business performance during the pandemic. Results indicate that firms receiving PPP funding were more likely to remain in operation and experience a smaller decline in annual production. However, even within a single industry, COVID-19 had heterogeneous effects on different market segments, demonstrating the importance of a firm's pre-pandemic business model on its flexibility and resiliency during a crisis. Finally, using a quasi-experiment that exploits a natural break in the loan program, the study suggests a positive causal effect of the role of loan approval timing on short-run performance outcomes. These findings provide evidence that the PPP alleviated some losses induced by COVID-19, but questions remain about the program's distribution and long-term impacts. The US federal government created the Paycheck Protection Program (PPP) to minimize the economic damages from COVID-19 on workers and small businesses. One industry hit particularly hard by the pandemic was the craft brewing industry, making it an ideal industry to explore whether the PPP achieved its objectives. The results show that receiving a PPP loan increased the likelihood of remaining in business through the pandemic. Additionally, while most craft breweries experienced a decline in annual production from 2019 to 2020, firms that received a PPP loan experienced a smaller reduction. Breweries that received the earliest funding also performed better, suggesting that loan timing played a key role in performance outcomes. Taken together, the study suggests that the government program helped reduce economic damages associated with COVID-19, but more work is needed to fully understand the program's impact. The online version contains supplementary material available at 10.1007/s11187-022-00717-3.

Molecular Reconstruction of Crude Oil: Novel Structure-Oriented Homologous Series Lumping with a Cloud Model

It is crucial to provide the detailed composition of crude oil for molecular management in refineries. Molecular reconstruction methods have been widely investigated for converting bulk properties to the detailed composition of feedstock by optimization calculations. In this work, a novel structure-oriented homologous series lumping with a cloud model (SOHSL-CM) is proposed for the molecular reconstruction of crude oil to achieve a molecular-level representation of crude oil. A two-dimensional homologous series distribution was applied to construct a molecular library with a clear structure, and a CM was designed to introduce fluctuations in the molecular distribution. Molecular reconstruction was demonstrated by industrial datasets containing 42 sets of crude oil data, with a mean absolute relative error <0.7%. In addition, the computational efficiency of the optimization was acceptable, with a time consumption of <1000 s. The accuracy, reliability, and efficiency of the proposed method indicate substantial potential for practical industrial applications.

Agnostic learning for packing machine stoppage prediction in smart factories

The cyber-physical convergence is opening up new business opportunities for industrial operators. The need for deep integration of the cyber and the physical worlds establishes a rich business agenda towards consolidating new system and network engineering approaches. This revolution would not be possible without the rich and heterogeneous sources of data, as well as the ability of their intelligent exploitation, mainly due to the fact that data will serve as a fundamental resource to promote Industry 4.0. One of the most fruitful research and practice areas emerging from this data-rich, cyber-physical, smart factory environment is the data-driven process monitoring field, which applies machine learning methodologies to enable predictive maintenance applications. In this paper, we examine popular time series forecasting techniques as well as supervised machine learning algorithms in the applied context of Industry 4.0, by transforming and preprocessing the historical industrial dataset of a packing machine's operational state recordings (real data coming from the production line of a manufacturing plant from the food and beverage domain). In our methodology, we use only a single signal concerning the machine's operational status to make our predictions, without considering other operational variables or fault and warning signals, hence its characterization as "agnostic". In this respect, the results demonstrate that the adopted methods achieve a quite promising performance on three targeted use cases.

Working Condition Recognition of a Mineral Flotation Process Using the DSFF-DenseNet-DT

The commonly used working condition recognition method in the mineral flotation process is based on shallow features of flotation froth images. However, the shallow features of flotation froth images frequently have an excessive amount of redundant and noisy information, which has an impact on the recognition effect and prevents the flotation process from being effectively optimized. Therefore, a working condition recognition method for the mineral flotation process based on a deep and shallow feature fusion densely connected network decision tree (DSFF-DenseNet-DT) is proposed in this paper. Firstly, the color texture distribution (CTD) and size distribution (SD) of a flotation froth image obtained in advance are approximated by the nonparametric kernel density estimation method, and a set of kernel function weights is obtained to represent the color texture and size features, while the deep features of the flotation froth image are extracted through the densely connected network (DenseNet). Secondly, a two-stage feature fusion method based on a stacked autoencoder after Concat (Cat-SAE) is proposed to fuse and reduce the dimensionality of the extracted shallow features and deep features so as to maximize the comprehensive description of the features and eliminate redundant and noisy information. Finally, the feature vectors after fusion dimensionality reduction are fed into the densely connected network decision tree (DenseNet-DT) for working condition recognition. Multiple experiments employing self-built industrial datasets reveal that the suggested method’s average recognition accuracy, precision, recall and F1 score reach 92.67%, 93.9%, 94.2% and 0.94, respectively. These results demonstrate the proposed method’s usefulness.

A mixed-integer programming approach for industrial non-intrusive load monitoring

With the development of the smart grid, more load data can be collected and utilized to facilitate bidirectional communications between the supply-side and demand-side. More detailed data can achieve better results in applications, but it is not feasible due to sensor placement and data transmission limitations. Non-intrusive load monitoring (NILM) is an exceptionally low-cost solution for providing appliance-level load information of a building without installing extra sensors. While most current studies focus on NILM in residential buildings, the potential benefit of industrial NILM is preferable due to the grander power consumption scale and more professional management. This paper considers the challenges and strengths of industrial NILM, and a mixed-integer programming NILM approach is proposed for flowline industries. This approach separately models equipment with different load features, resulting in better performance in industrial scenarios containing various loads. Temporal dependencies between equipment are exploited by recognizing the statistical regularity of load fluctuation and are introduced as a novel flowline constraint in the model. A pulse width constraint that restricts the state duration of equipment is also introduced to improve performance. Several cases are designed and tested on two public industrial datasets to evaluate the effectiveness and transferability of the model. The classical factorial hidden Markov model (FHMM), a state-of-the-art matrix factorization method, and a deep learning model are used as benchmarks for comparison.

An iterated greedy algorithm for the planning of yarn‐dyeing boilers

AbstractYarn dyeing is a critical link in the textile production chain that consumes the most time and energy. Today's dyeing shops receive hundreds of demands with thousands of different colors, different due dates, and different production requirements. This situation has made it very difficult for the human brain to create a minimum‐cost production plan by complying with the due dates. In this study, a real‐life problem of a company operating in the textile industry is discussed and a solution has been developed for the planning of yarn‐dyeing boilers. The application was held in Bursalı Textile, which is the major towel manufacturer operating in Turkey. The problem dealt with is basically in the nature of the variable‐size bin‐packing problem (VSBPP). The limited availability of bins (boilers) of different sizes and the packing of the items (yarn work orders) with due date constraints are the original aspects of this study. Multi‐objective mixed integer programming model is developed to minimize two objectives. For the solution, the preemptive method called the lexicographic approach, in which the objectives are solved in order, is preferred. As the first objective, the overcapacity usage is minimized and then the second objective, which is the boiler usage cost, is minimized. Given that the VSBPP is strongly NP‐hard, an iterated greedy algorithm with two different decoding approaches is proposed. Computational experiments were conducted on 20 randomly generated benchmark instances and a real‐world industrial dataset. The numerical results show that good solutions can be obtained in seconds using the proposed approaches.

Semi-supervised LSTM with historical feature fusion attention for temporal sequence dynamic modeling in industrial processes

In modern industrial processes, the data-driven soft sensor technology has been widely used for the prediction of key quality variables. Due to the important of dynamics and nonlinearity in industrial process data, deep learning models like long short-term memory (LSTM) network are well suited for temporal sequence dynamic modeling due to their excellent long-term memory function and feature extraction capability. Furthermore, industrial processes generate a large amount of process data with irregular sampling frequencies. However, traditional LSTM cannot fully utilize the process data with irregular sampling frequency and the guidance value of historical data samples for feature learning. To address these issues, a novel semi-supervised LSTM with history feature fusion attention (HFFA-SSLSTM) model is proposed in this paper. First, the semi-supervised learning strategy is implemented in LSTM to fully utilize the unlabeled data and mine the temporal sequence features of labeled samples and unlabeled samples with irregular sampling frequencies. Then, a novel historical feature fusion attention (HFFA) mechanism is developed, which utilizes historical hidden features to learn attention scores for obtaining weighted historical information-related features. Finally, the extracted features are combined to form the soft sensor model to perform time series prediction tasks for key quality variables in industrial processes. The experimental results on the actual industrial hydrocracking data set demonstrate the effectiveness of the proposed HFFA-SSLSTM model and its possibility of applicating in real industrial processes.

Dynamic Process Safety Assessment Using Adaptive Bayesian Network with Loss Function

Fault detection and diagnosis (FDD) is crucial for dynamic process safety analysis. Integrated with failure prediction models, it enables us to realize how a deviation in process variable(s) can affect system safety (measured as risk). This work aims to overcome the challenges of nonlinear, non-Gaussian, and multimodal behavior of the processing systems to detect abnormal process operations, predict dynamic operational risk, and diagnose root cause of the abnormal situation. A methodology is proposed here by integrating different techniques. The artificial neural network (ANN) is used to identify process modes, while the Bayesian network (BN) is used for fault detection. How a fault will lead to a process failure is modeled using the event tree (ET), whereas time-dependent losses associated with the failure scenarios are assessed using the inverted normal loss function (INLF). A probability adaption mechanism is used to estimate the conditional probabilities in each time slice. The complexity of estimating conditional probabilities is handled using the copula theory. The proposed framework is validated using numerical, simulated, and industrial datasets. The results suggest that the developed framework can provide greater flexibility and wider applications.

Unsupervised Anomaly Detection for Surface Defects With Dual-Siamese Network

Unsupervised anomaly detection in real industrial scenarios is challenging since the small amount of defect-free images contain limited discriminative information, and anomaly defects are unpredictable. Although nowadays image reconstruction-based methods are widely being used in various anomaly detection applications, they cannot effectively learn semantic representation, which leads to imperfect reconstruction. In this article, anomaly detection is formulated as a joint problem of feature reconstruction and inpainting in the dual-siamese framework. The proposed approach forces the network to model the feature distribution from the normal area and capture the semantic context for discriminating normal and abnormal areas. It first uses a Siamese architecture to capture discriminative features of defect-free samples and its corresponding defective samples generated by the defect random generation module. A dense feature fusion module is then employed to obtain the dense feature representation of dual input. The second Siamese network is proposed to reconstruct and inpaint the dual-dense features of the previous stage. Compared to the existing methods that mostly employ single image reconstruction, it is beneficial to simultaneously reconstruct and inpaint the information of dense discriminative features. The experimental results on the MVTec AD datasets and some major real industrial datasets demonstrate that our method achieves state-of-the-art inspection accuracy.

Self-Attention Mechanism Enhanced User Interests Modeling for Personalized Recommendation Services in Cyber-Physical-Social Systems

Cyber-Physical-Social Systems (CPSS) provide great value to our lives, but they also cause data overload problems. Data-driven personalized recommendation service is one of the most efficient means to solve such problems, which is currently receiving wide attention from research and industrial communities. The most important task of personalized recommender systems is to predict the click-through rate of given items, which is especially true for personalized advertisement recommendation systems. Recently, a number of deep click-through models have been proposed, which obtain low-dimensional dense embedding vectors of features, and then concatenate together and input into multi-layer perceptron to learn the nonlinear relationship between the features. However, the existing models don't dig deep enough into the user preferences and habits in users’ behavior history. In this paper, we propose a new model: Self-attention based Deep Neural Network (DeepSA), which addresses this issue by constructing Ad-related graph and training graph embedding vectors to enhance the representation of the advertisement for capturing user interests, and learns the internal correlation between user behaviors via the self-attention mechanism, which better explores interests and preferences hidden in users’ historical behaviors. Experiments on two public datasets and an industrial dataset demonstrate the proposed method outperforms the state-of-the-art models.

Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry

In the process industry, condition monitoring systems with automated fault diagnosis methods assist human experts and thereby improve maintenance efficiency, process sustainability, and workplace safety. Improving the automated fault diagnosis methods using data and machine learning-based models is a central aspect of intelligent fault diagnosis (IFD). A major challenge in IFD is to develop realistic datasets with accurate labels needed to train and validate models, and to transfer models trained with labeled lab data to heterogeneous process industry environments. However, fault descriptions and work-orders written by domain experts are increasingly digitised in modern condition monitoring systems, for example in the context of rotating equipment monitoring. Thus, domain-specific knowledge about fault characteristics and severities exists as technical language annotations in industrial datasets. Furthermore, recent advances in natural language processing enable weakly supervised model optimisation using natural language annotations, most notably in the form of natural language supervision (NLS). This creates a timely opportunity to develop technical language supervision (TLS) solutions for IFD systems grounded in industrial data, for example as a complement to pre-training with lab data to address problems like overfitting and inaccurate out-of-sample generalisation. We surveyed the literature and identify a considerable improvement in the maturity of NLS over the last two years, facilitating applications beyond natural language; a rapid development of weak supervision methods; and transfer learning as a current trend in IFD which can benefit from these developments. Finally we describe a general framework for TLS and implement a TLS case study based on Sentence-BERT and contrastive learning based zero-shot inference on annotated industry data.

Development of a Robust Data-Driven Soft Sensor for Multivariate Industrial Processes with Non-Gaussian Noise and Outliers

Industrial processes are often nonlinear and multivariate and suffer from non-Gaussian noise and outliers in the process data, which cause significant challenges in data-driven modelling. To address these issues, a robust soft-sensing algorithm that integrates Huber’s M-estimation and adaptive regularisations with multilayer perceptron (MLP) is proposed in this paper. The proposed algorithm, called RAdLASSO-MLP, starts with an initially well-trained MLP for nonlinear data-driven modelling. Subsequently, the residuals of the proposed model are robustified with Huber’s M-estimation to improve the resistance to non-Gaussian noise and outliers. Moreover, a double L1-regularisation mechanism is introduced to minimise redundancies in the input and hidden layers of MLP. In addition, the maximal information coefficient (MIC) index is investigated and used to design the adaptive operator for the L1-regularisation of the input neurons to improve biased estimations with L1-regularisation. Including shrinkage parameters and Huber’s M-estimation parameter, the hyperparameters are determined via grid search and cross-validation. To evaluate the proposed algorithm, simulations were conducted with both an artificial dataset and an industrial dataset from a practical gasoline treatment process. The results indicate that the proposed algorithm is superior in terms of predictive accuracy and robustness to the classic MLP and the regularised soft-sensing approaches LASSO-MLP and dLASSO-MLP.

Practical joint human-machine exploration of industrial time series using the matrix profile

Technological advancements and widespread adaptation of new technology in industry have made industrial time series data more available than ever before. With this development grows the need for versatile methods for mining industrial time series data. This paper introduces a practical approach for joint human-machine exploration of industrial time series data using the Matrix Profile, and presents some challenges involved. The approach is demonstrated on three real-life industrial data sets to show how it enables the user to quickly extract semantic information, detect cycles, find deviating patterns, and gain a deeper understanding of the time series. A benchmark test is also presented on ECG (electrocardiogram) data, showing that the approach works well in comparison to previously suggested methods for extracting relevant time series motifs.

Triple Factorization-Like Symmetric NLF Models With Latent Item–Item Relationship

Undirected, high-dimension and sparse (SHiDS) networks which are often encountered in industrial applications usually contain numerous useful information; thus, it is practically important to develop effective collaborative filtering methods to rigidly extract the latent information. In order to improve the performance of nonnegative latent factor (NLF) models based on the double factorization while reducing the computation burden of NLF models in terms of the triple factorization (TF) technique, a novel symmetric NLF (SNLF) model, i.e., TF-like SNLF model, is proposed in this article. A latent factor (LF) matrix that reflects the relationship of items is introduced into the objective function so as to offer more freedom for solutions. Then, a fixed weighting matrix designed by sensitivity experiments is used to replace the aforementioned LF matrix dependent on the update, which greatly helps save the time cost without sacrificing any accuracy. Furthermore, the alteration direction (ADI) method is adopted to design the algorithm, which effectively helps reduce the occurrence of the local minimum while guaranteeing convergence. Finally, experimental results on four real industrial data sets demonstrate that the proposed TF-like SNLF model makes a good compromise between the model performance and the time efficiency.

Multivariate time series prediction of complex systems based on graph neural networks with location embedding graph structure learning

Graph convolutional neural networks (GNNs) have an excellent expression ability for complex systems. However, the smoothing hypothesis based GNNs have certain limitations for complex process industrial systems with high dynamics and noisy environment. In addition, it is difficult to obtain an accurate information about the interconnections of sensor networks in manufacturing systems, which brings challenges to the application of GNNs. This paper introduces a graph convolution filter with a serial alternating structure of low-pass filter and high-pass filter to alleviate the problem of node feature loss. Furthermore, we propose a simple and effective method to learn graph structure information during training. This method combines the advantages of graph structure learning based on metric method and direct optimization method. Finally, a spatiotemporal parallel feature extraction framework for multivariate time series prediction is constructed. Experiments are carried out on real industrial datasets, and the results demonstrate the effectiveness of the model.

Using Domain Adaptation for Incremental SVM Classification of Drift Data

A common assumption in machine learning is that training data is complete, and the data distribution is fixed. However, in many practical applications, this assumption does not hold. Incremental learning was proposed to compensate for this problem. Common approaches include retraining models and incremental learning to compensate for the shortage of training data. Retraining models is time-consuming and computationally expensive, while incremental learning can save time and computational costs. However, the concept drift may affect the performance. Two crucial issues should be considered to address concept drift in incremental learning: gaining new knowledge without forgetting previously acquired knowledge and forgetting obsolete information without corrupting valid information. This paper proposes an incremental support vector machine learning approach with domain adaptation, considering both crucial issues. Firstly, a small amount of new data is used to fine-tune the previous model to generate a model that is sensitive to the new data but retains the previous data information by transferring parameters. Secondly, an ensemble and model selection mechanism based on Bayesian theory is proposed to keep the valid information. The computational experiments indicate that the performance of the proposed model improved as new data was acquired. In addition, the influence of the degree of data drift on the algorithm is also explored. A gain in performance on four out of five industrial datasets and four synthetic datasets has been demonstrated over the support vector machine and incremental support vector machine algorithms.

Polygon generation and video-to-video translation for time-series prediction

This paper proposes an innovative method for time-series prediction in energy-intensive industrial systems characterized by highly dynamic non-linear operations. The proposed method can capture the true distributions of the inputs and outputs of such systems and map these distributions using polygon generation and video-to-video translation techniques. More specifically, the time-series data are represented as polygon streams (videos), then the video-to-video translation is used to transform the input polygon streams into the output ones. This transformation is tuned based on a model trustworthiness metric for optimal video synthesis. Finally, an image processing procedure is used for mapping the output polygon streams back to time-series outputs. The proposed method is based on cycle-consistent generative adversarial networks as an unsupervised approach. This does not need the heavy involvement of the human expert who devotes much effort to labeling the complex industrial data. The performance of the proposed method was validated successfully using a challenging industrial dataset collected from a complex heat exchanger network in a Canadian pulp mill. The results obtained using the proposed method demonstrate better performance than other comparable time-series prediction models. This allows process operators to accurately monitor process key performance indicators (KPIs) and to achieve a more energy-efficient operation.