Industrial Datasets Research Articles

Model construction and optimization for raising the concentration of industrial bioethanol production by using a data-driven ANN model

This work aims to model and predict the bioethanol produced from a conventional fermentation process. An industrial dataset was obtained from a sugar mill in India. These datasets comprised a total of 1300 experimental values acquired from a total 100 days of production of the sugar mill in the year 2021. Using this data, a framework based on deep learning Artificial Neural Network (ANN) technique model was developed and validated with the test data. Specifically, the normalized dataset was passed through an ANN model consisting of one input layer, two hidden layers and one output layer, and the percent model error calculated by using average absolute deviation metric and was found to be 4.48 and 1.99% for training and testing data, respectively. The optimization of the process variables was performed for the first time using a data synthesis technique in which the normalized dataset was first synthesized and then passed through an ANN model to get an optimized input variable set for an increase in bioethanol concentration (BEC) in the final product by 1°GL. The operating parameters which significantly influenced the BEC are concentration of cell in pre-fermentation (PFM), water fermentation pH (WFMpH), and water fermentation hardness (WFH). An increase of 8.45% in BEC was obtained for Sugar mill, India.

System Health Assessment Model Based on Anomaly Detection

Accurately assessing system health and taking timely interventions play a crucial role in promoting intelligent system operation. Traditional system health assessment techniques depend on human-set thresholds, which are only appropriate for tracking a few indicators and do not accurately represent the system’s health overall. Therefore, keeping track of a large amount of data and a diverse variety of indicators will be extremely challenging. To address this issue, we develop the AD_Select anomaly detection algorithm and suggest a system health assessment model based on indicator anomaly detection. By fusing anomaly detection with system health assessment, this algorithm increases the scientific rigor and precision of health assessment. The experiments of AD_Select anomaly detection algorithm and healthiness assessment model are conducted on the 2018AIOps dataset and industrial dataset respectively, and the results show that both the AD_Select algorithm and the healthiness assessment model proposed in this paper achieve good performance.

Personalized Latent Structure Learning for Recommendation.

In recommender systems, users' behavior data are driven by the interactions of user-item latent factors. To improve recommendation effectiveness and robustness, recent advances focus on latent factor disentanglement via variational inference. Despite significant progress, uncovering the underlying interactions, i.e., dependencies of latent factors, remains largely neglected by the literature. To bridge the gap, we investigate the joint disentanglement of user-item latent factors and the dependencies between them, namely latent structure learning. We propose to analyze the problem from the causal perspective, where a latent structure should ideally reproduce observational interaction data, and satisfy the structure acyclicity and dependency constraints, i.e., causal prerequisites. We further identify the recommendation-specific challenges for latent structure learning, i.e., the subjective nature of users' minds and the inaccessibility of private/sensitive user factors causing universally learned latent structure to be suboptimal for individuals. To address these challenges, we propose the personalized latent structure learning framework for recommendation, namely PlanRec, which incorporates 1) differentiable Reconstruction, Dependency, and Acyclicity regularizations to satisfy the causal prerequisites; 2) Personalized Structure Learning (PSL) which personalizes the universally learned dependencies through probabilistic modeling; and 3) uncertainty estimation which explicitly measures the uncertainty of structure personalization, and adaptively balances personalization and shared knowledge for different users. We conduct extensive experiments on two public benchmark datasets from MovieLens and Amazon, and a large-scale industrial dataset from Alipay. Empirical studies validate that PlanRec discovers effective shared/personalized structures, and successfully balances shared knowledge and personalization via rational uncertainty estimation.

Industrial Internet Intrusion Detection Based on Res-CNN-SRU

Nowadays, the industrial Internet is developing rapidly, but at the same time it faces serious information security risks. At present, industrial Internet data generally have the problems of complex attack sample types, large numbers, and high feature dimensions. When training a model, the complexity and quantity of attack samples will result in a long detection time for the intrusion detection algorithm, which will fall short of the system’s real-time performance. Due to the high feature dimension of the data, shallow feature extraction will be unable to extract the data’s more significant features, which will render the model’s overall detection capacity insufficient. Aiming at the above problems, an industrial Internet intrusion detection method based on Res-CNN-SRU is proposed. This method not only considers the temporality of network traffic data but can also effectively capture the local features in the data. The dataset used in the experiment is the gas pipeline industry dataset proposed by Mississippi State University in 2014. Experiments show that the algorithm can effectively improve the recognition rate of the system and reduce the false-alarm rate. At the same time, the training time required for this method is also greatly shortened, and it can perform efficient intrusion detection on the industrial Internet.

Distributed deep learning approach for intrusion detection system in industrial control systems based on big data technique and transfer learning

ABSTRACT Industry 4.0 refers to a new generation of connected and intelligent factories that is driven by the emergence of new technologies such as artificial intelligence, Cloud computing, Big Data and industrial control systems (ICS) in order to automate all phases of industrial operations. The presence of connected systems in industrial environments poses a considerable security challenge, moreover with the huge amount of data generated daily, there are complex attacks that occur in seconds and target production lines and their integrity. But, until now, factories do not have all the necessary tools to protect themselves, they mainly use traditional protection. In order to improve industrial control systems in terms of efficiency and response time, the present paper propose a new distributed intrusion detection approach using artificial intelligence methods, Big Data techniques and deployed in a cloud environment. A variety of Machine Learning and Deep Learning algorithms, basically convolutional neural networks (CNN), have been tested to compare performance and choose the most suitable model for the classification. We test the performance of our model by using the industrial dataset SWat.

An explainable unsupervised learning framework for scalable machine fault detection in Industry 4.0

Despite the diverse number of machine learning algorithms reported in the literature for machine fault detection, their implementation is mainly confined to laboratory-scale demonstrations. The complexity and black-box nature of machine learning models, the processing cost involved in appropriate feature extraction, limited access to labeled data, and varying operating conditions are some of the key reasons that curtail their implementation in practical applications. Furthermore, most such models serve as decision support tools, aiding domain experts in root cause analysis, and are not truly autonomous by themselves. To address these challenges, we present a lightweight autoencoder-based unsupervised learning framework to accurately identify machine faults against the changing operating conditions in a real-world scenario. The fault detection strategy is further strengthened by a model agnostic Shapley Additive exPlanations (SHAP)-based method (kernel SHAP) for identifying the most prominent features contributing to fault detection inference, the findings of which are then explored for identifying trends and correlations among prominent features and various types of faults. The framework is validated using two widely used and publicly available datasets for machine condition monitoring, as well as a large industrial dataset comprising 18 machines installed at three factories in India, monitored for several months.

Semisupervised Fault Diagnosis for Gearboxes: A Novel Method Based on a Hybrid Classification Network and Weighted Pseudo-Labeling

To make the gearboxes reliable and secure, accurate and efficient fault diagnosis has received wide attention. Recently, many data-driven intelligent methods have made great progress due to their powerful feature extraction capability. However, most deep learning based methods can only perform well with sufficient labeled data and the shortage of labeled data has become a common problem in practical industrial applications. Therefore, a semi-supervised approach based on a hybrid classification network and weighted pseudo-labeling is proposed to relieve the above problem. The hybrid classification network is composed of an autoencoder and a softmax classifier. With this structure, the process of latent representation learning of autoencoder is guided by the supervised classification training. Therefore, the features extracted by the autoencoder are more suitable for the classification task. Moreover, a weighted pseudo-labeling method is developed to further enhance the generalization capability of the model. Firstly, pseudo labels for the unlabeled samples are generated according to the model predictions. Then, the proposed sample weighting scheme assigns a confidence-based weight to each pseudo-labeled sample, which is used to filter out the incorrect pseudo labels. Finally, these weighted pseudo-labeled samples are utilized to further optimize the model. Two gearbox datasets are utilized to validate the effectiveness of the proposed method. One is an experimental gearbox dataset and the other is an industrial wind turbine gearbox dataset. The experimental results show that the proposed method can still achieve high diagnostic accuracies for both the experimental and industrial datasets under limited labeled data.

Automated Categorization of Multiclass Welding Defects Using the X-ray Image Augmentation and Convolutional Neural Network.

The detection of weld defects by using X-rays is an important task in the industry. It requires trained specialists with the expertise to conduct a timely inspection, which is costly and cumbersome. Moreover, the process can be erroneous due to fatigue and lack of concentration. In this context, this study proposes an automated approach to identify multi-class welding defects by processing the X-ray images. It is realized by an intelligent hybridization of the data augmentation techniques and convolutional neural network (CNN). The proposed data augmentation mainly performs random rotation, shearing, zooming, brightness adjustment, and horizontal flips on the intended images. This augmentation is beneficial for the realization of a generalized trained CNN model, which can process the multi-class dataset for the identification of welding defects. The effectiveness of the proposed method is confirmed by testing its performance in processing an industrial dataset. The intended dataset contains 4479 X-ray images and belongs to six groups: cavity, cracks, inclusion slag, lack of fusion, shape defects, and normal defects. The devised technique achieved an average accuracy of 92%. This indicates that the approach is promising and can be used in contemporary solutions for the automated detection and categorization of welding defects.

Identifying Differences in the Performance of Machine Learning Models for Off-Targets Trained on Publicly Available and Proprietary Data Sets.

Each year, publicly available databases are updated with new compounds from different research institutions. Positive experimental outcomes are more likely to be reported; therefore, they account for a considerable fraction of these entries. Established publicly available databases such as ChEMBL allow researchers to use information without constrictions and create predictive tools for a broad spectrum of applications in the field of toxicology. Therefore, we investigated the distribution of positive and nonpositive entries within ChEMBL for a set of off-targets and its impact on the performance of classification models when applied to pharmaceutical industry data sets. Results indicate that models trained on publicly available data tend to overpredict positives, and models based on industry data sets predict negatives more often than those built using publicly available data sets. This is strengthened even further by the visualization of the prediction space for a set of 10,000 compounds, which makes it possible to identify regions in the chemical space where predictions converge. Finally, we highlight the utilization of these models for consensus modeling for potential adverse events prediction.

Online visual end-to-end detection monitoring on surface defect of aluminum strip under the industrial few-shot condition

Surface defect detection systems based on deep learning are employed in the manufacturing system, and their good detection performance largely relies on abundant annotated data. Nevertheless, industrial datasets are often difficult to obtain, which hinders the development of defect detection systems to some extent. In an effort to address the issue that the scarcity of training data leads to poor performance, this paper proposes a novel end to end few-shot detection method for industrial real-time detection on aluminum strips. The meta-learning theory is introduced into the multi-scale structure of the YOLOv4 framework, which means that multi-scale meta-feature maps are extracted through the backbone network and each meta-feature map is recalibrated by reweighted vectors to strengthen class-specific features. Subsequently, a multi-scale prediction module with a new parsing strategy is designed to locate and classify the defects at different scales for the reweighted meta-features. All experiments are performed using data collected from the aluminum strip production line in a cold rolling workshop. In addition, the dataset required for training is constructed based on the few-shot training strategy. Pre-training is conducted on a large amount of labeled data from base classes in advance, and the pre-trained weights are then loaded to train the model on a small amount of labeled data from all classes to rapidly deploy the model in multi-task scenarios. The proposed method shows excellent performance, improving the accuracy for novel classes by more than 14.6%, which enhances the industrial practicality of detection systems.

A new regularized and robust LSTM learning algorithm for industrial soft sensors

Modern industrial processes often have nonlinearity, multivariate, time-delay, and measurement outliers, which make accurate data-driven modeling of key performance indicators difficult. To address these issues, a robust and regularized long short-term memory (LSTM) neural network for soft sensors in complex industrial processes was proposed. First, a conventional LSTM architecture was used as the basic model to deal with nonlinearity and time delay. Thereafter, a novel LSTM loss function that combines the excellent resistance to outliers of Huber M-Loss with the superior model reduction capability of ℓ1 regularization was designed. Subsequently, a backpropagation through time training algorithm for the proposed LSTM was developed, including the chain derivative calculation and updating formulas. The adaptive moment estimation was applied to perform the gradient update, while the grid search and moving window cross-validation were used to find the optimal hyperparameters. Finally, nonlinear artificial datasets with time series and outliers, as well as an industrial dataset of a desulfurization process, were applied to investigate the performance of the proposed soft sensor. Simulation results show that the proposed algorithm outperforms other state-of-the-art soft sensors in terms of predictive accuracy and training time. The causal relationship of the data-driven soft sensor trained by the proposed algorithm is consistent with the field operation and chemical reactions of the desulfurization process.

Robust Wafer Classification With Imperfectly Labeled Data Based on Self-Boosting Co-Teaching

Wafer classification is a critical task for semiconductor manufacturing. Most conventional algorithms require a large-scale perfectly-labeled dataset to train accurate classifiers. In practice, it is usually difficult or even impossible to collect perfect labels without errors, and the classification accuracy in the presence of imperfectly labeled data would degrade. To facilitate robust wafer classification with noisy labels, we propose a novel self-boosting co-teaching (SB-CT) approach. Specifically, we iteratively correct the wrong labels by using the predictions of two classifiers that are jointly trained with noisily labeled data. To make the proposed method of practical utility, we develop a novel method to accurately estimate the noise rate, and adopt a probability scaling technique to further improve the classification accuracy. As demonstrated by the experimental results based on two industrial datasets, the proposed SB-CT approach achieves superior accuracy over other conventional methods.

ITran: A novel transformer-based approach for industrial anomaly detection and localization

Anomaly detection is currently an essential quality monitoring process in industrial production. It is often affected by factors such as under or over reconstruction of images and unclear criteria for feature distribution evaluation, thus making it challenging to improve detection performance. To solve the above problems, this paper proposes a novel transformer-based approach, Inductive Transformer (ITran) for industrial anomaly detection and localization, which utilizes a multi-layer pyramid structure and multi-level jump connections to extract multi-scale features of the data, putting the anomaly detection into the feature space and achieving more accurate industrial anomaly detection and localization results. It incorporates inductive bias and convolution operations into the Transformer which helps to break the myth of Transformer being “data hungry”. Compared with the common Transformers, ITran significantly reduces the computational cost and memory usage and makes it work well on small datasets. In addition, we basically eliminate the effect of positional embedding on the proposed Transformer model. Sufficient experiments have been conducted to validate global anomaly detection on three datasets MNIST, Fashion-MNST and Cifar-10, as well as local anomaly detection on the industrial datasets MVTec AD, Concrete Crack Image and BTAD. The proposed ITran achieves outstanding results on all the above datasets.

The arc loss challenge: A novel industrial benchmark for process analytics and machine learning

Rapid development in data-driven process monitoring has provided a rich selection of models and data preprocessing strategies for applications such as fault detection and diagnosis. However, the development, comparison, and selection of process monitoring algorithms can become complicated and unnecessarily onerous. As a result, numerous publicly available benchmark datasets have emerged in the literature. Unfortunately, benchmark literature often suffers from problems such as low fidelity, inconsistent usage, and lack of transparency. This paper presents a benchmark challenge based on a large-scale industrial dataset that aims to enhance the evaluation and comparison of learning algorithms and overall data preprocessing workflows. We introduce the arc loss challenge, a machine learning benchmark with data from a large-scale mining and pyrometallurgy operation. By providing a supervised learning challenge based on large quantities of raw industrial process data with transparent and consistent evaluation procedures, the arc loss challenge is a unique contribution to fault detection benchmarking.

DropMessage: Unifying Random Dropping for Graph Neural Networks

Graph Neural Networks (GNNs) are powerful tools for graph representation learning. Despite their rapid development, GNNs also face some challenges, such as over-fitting, over-smoothing, and non-robustness. Previous works indicate that these problems can be alleviated by random dropping methods, which integrate augmented data into models by randomly masking parts of the input. However, some open problems of random dropping on GNNs remain to be solved. First, it is challenging to find a universal method that are suitable for all cases considering the divergence of different datasets and models. Second, augmented data introduced to GNNs causes the incomplete coverage of parameters and unstable training process. Third, there is no theoretical analysis on the effectiveness of random dropping methods on GNNs. In this paper, we propose a novel random dropping method called DropMessage, which performs dropping operations directly on the propagated messages during the message-passing process. More importantly, we find that DropMessage provides a unified framework for most existing random dropping methods, based on which we give theoretical analysis of their effectiveness. Furthermore, we elaborate the superiority of DropMessage: it stabilizes the training process by reducing sample variance; it keeps information diversity from the perspective of information theory, enabling it become a theoretical upper bound of other methods. To evaluate our proposed method, we conduct experiments that aims for multiple tasks on five public datasets and two industrial datasets with various backbone models. The experimental results show that DropMessage has the advantages of both effectiveness and generalization, and can significantly alleviate the problems mentioned above. A detailed version with full appendix can be found on arXiv: https://arxiv.org/abs/2204.10037.

Set-to-Sequence Ranking-Based Concept-Aware Learning Path Recommendation

With the development of the online education system, personalized education recommendation has played an essential role. In this paper, we focus on developing path recommendation systems that aim to generating and recommending an entire learning path to the given user in each session. Noticing that existing approaches fail to consider the correlations of concepts in the path, we propose a novel framework named Set-to-Sequence Ranking-based Concept-aware Learning Path Recommendation (SRC), which formulates the recommendation task under a set-to-sequence paradigm. Specifically, we first design a concept-aware encoder module which can capture the correlations among the input learning concepts. The outputs are then fed into a decoder module that sequentially generates a path through an attention mechanism that handles correlations between the learning and target concepts. Our recommendation policy is optimized by policy gradient. In addition, we also introduce an auxiliary module based on knowledge tracing to enhance the model’s stability by evaluating students’ learning effects on learning concepts. We conduct extensive experiments on two real-world public datasets and one industrial dataset, and the experimental results demonstrate the superiority and effectiveness of SRC. Code now is available at https://gitee.com/mindspore/models/tree/master/research/recommend/SRC.

Decision-Making Context Interaction Network for Click-Through Rate Prediction

Click-through rate (CTR) prediction is crucial in recommendation and online advertising systems. Existing methods usually model user behaviors, while ignoring the informative context which influences the user to make a click decision, e.g., click pages and pre-ranking candidates that inform inferences about user interests, leading to suboptimal performance. In this paper, we propose a Decision-Making Context Interaction Network (DCIN), which deploys a carefully designed Context Interaction Unit (CIU) to learn decision-making contexts and thus benefits CTR prediction. In addition, the relationship between different decision-making context sources is explored by the proposed Adaptive Interest Aggregation Unit (AIAU) to improve CTR prediction further. In the experiments on public and industrial datasets, DCIN significantly outperforms the state-of-the-art methods. Notably, the model has obtained the improvement of CTR+2.9%/CPM+2.1%/GMV+1.5% for online A/B testing and served the main traffic of Meituan Waimai advertising system.

Quality-Aware Self-Training on Differentiable Synthesis of Rare Relational Data

Data scarcity is a very common real-world problem that poses a major challenge to data-driven analytics. Although a lot of data-balancing approaches have been proposed to mitigate this problem, they may drop some useful information or fall into the overfitting problem. Generative Adversarial Network (GAN) based data synthesis methods can alleviate such a problem but lack of quality control over the generated samples. Moreover, the latent associations between the attribute set and the class labels in a relational data cannot be easily captured by a vanilla GAN. In light of this, we introduce an end-to-end self-training scheme (namely, Quality-Aware Self-Training) for rare relational data synthesis, which generates labeled synthetic data via pseudo labeling on GAN-based synthesis. We design a semantic pseudo labeling module to first control the quality of the generated features/samples, then calibrate their semantic labels via a classifier committee consisting of multiple pre-trained shallow classifiers. The high-confident generated samples with calibrated pseudo labels are then fed into a semantic classification network as augmented samples for self-training. We conduct extensive experiments on 20 benchmark datasets of different domains, including 14 industrial datasets. The results show that our method significantly outperforms state-of-the-art methods, including two recent GAN-based data synthesis schemes. Codes are available at https://github.com/yaxinhou/QAST.

Self-Supervised Interest Transfer Network via Prototypical Contrastive Learning for Recommendation

Cross-domain recommendation has attracted increasing attention from industry and academia recently. However, most existing methods do not exploit the interest invariance between domains, which would yield sub-optimal solutions. In this paper, we propose a cross-domain recommendation method: Self-supervised Interest Transfer Network (SITN), which can effectively transfer invariant knowledge between domains via prototypical contrastive learning. Specifically, we perform two levels of cross-domain contrastive learning: 1) instance-to-instance contrastive learning, 2) instance-to-cluster contrastive learning. Not only that, we also take into account users' multi-granularity and multi-view interests. With this paradigm, SITN can explicitly learn the invariant knowledge of interest clusters between domains and accurately capture users' intents and preferences. We conducted extensive experiments on a public dataset and a large-scale industrial dataset collected from one of the world's leading e-commerce corporations. The experimental results indicate that SITN achieves significant improvements over state-of-the-art recommendation methods. Additionally, SITN has been deployed on a micro-video recommendation platform, and the online A/B testing results further demonstrate its practical value. Supplement is available at: https://github.com/fanqieCoffee/SITN-Supplement.

Enhanced Multi-Relationships Integration Graph Convolutional Network for Inferring Substitutable and Complementary Items

Understanding the relationships between items can improve the accuracy and interpretability of recommender systems. Among these relationships, the substitute and complement relationships attract the most attention in e-commerce platforms. The substitutable items are interchangeable and might be compared with each other before purchasing, while the complementary items are used in conjunction and are usually bought together with the query item. In this paper, we focus on two issues of inferring the substitutable and complementary items: 1) how to model their mutual influence to improve the performance of downstream tasks, 2) how to further discriminate them by considering the strength of relationship for different item pairs. We propose a novel multi-task learning framework named Enhanced Multi-Relationships Integration Graph Convolutional Network (EMRIGCN). We regard the relationship inference task as a link prediction task in heterogeneous graph with different types of edges between nodes (items). To model the mutual influence between substitute and complement, EMRIGCN adopts a two-level integration module, i.e., feature and structure integration, based on experts sharing mechanism during message passing. To obtain the strength of relationship for item pairs, we build an auxiliary loss function to further increase or decrease the distances between embeddings of items with weak or strong relation in latent space. Extensive experiments on both public and industrial datasets prove that EMRIGCN significantly outperforms the state-of-the-art solutions. We also conducted A/B tests on real world recommender systems of Meituan Maicai, an online supermarket platform in China, and obtained 15.3% improvement on VBR and 15.34% improvement on RPM.