Accurate Soft Sensors Research Articles

A data-driven LSTMSCBLS model for soft sensor of industrial process

Abstract In the chemical industry, data-driven soft sensor modeling plays a crucial role in efficiently monitoring product quality and status. Industrial data in these applications typically exhibit significant temporal characteristics, meaning that current information is influenced by data from previous periods. Effectively extracting and utilizing these temporal features is essential for achieving accurate soft sensor modeling in complex chemical scenarios. To address this challenge, this study proposes a data-driven Broad Learning System (BLS) model, which combines Long Short-Term Memory (LSTM) networks with an adaptive algorithm known as the Stochastic Configuration Algorithm (SC), referred to as LSTMSCBLS. The model operates in two stages: temporal feature extraction and final prediction. In the temporal feature extraction stage, the integration of the LSTM network with a feature attention mechanism allows for efficient extraction of temporal features from high-dimensional time-series data. In the final prediction stage, the SC is integrated into the BLS, effectively mitigating issues related to node space redundancy and the determination of the number of nodes. The effectiveness and superiority of the proposed model are demonstrated through two industrial case studies involving a debutanizer column and a sulfur recovery unit.

Metaheuristic Procedures for the Determination of a Bank of Switching Observers toward Soft Sensor Design with Application to an Alcoholic Fermentation Process

The present work focused on the development of soft sensors for single-input single-output (SISO) nonlinear dynamic systems with unknown physical parameters using a switching observer design. Toward the development of more accurate soft sensors, as compared with hard sensors, an extended design methodology for the determination of a bank of operating points satisfying the dense web principle was proposed, where for the determination of the bank of operating points and the observer parameters, a metaheuristic procedure was developed. To validate the results of the metaheuristic algorithm, the case of an alcoholic fermentation process was studied as a special case of the present approach. For the nonlinear model of the process, an observer-based soft sensor was developed using the metaheuristic procedure. First, the accuracy of the linear approximant of the process with respect to the original nonlinear model was investigated. Second, the I/O reconstructability of the linear approximant was verified. Third, based on the linear approximant, an observer was designed for the estimation of the non-measurable variable. Fourth, considering that the observer is designed upon the linear approximant, the linear approximant model parameters are derived through identification, for different operating points, upon the nonlinear model. Fifth, the observers corresponding to the different operating points, constitute a bank of observers. The design was completed using a data-driven rule-based system, performing stepwise switching between the observers of the bank. The efficiency of the proposed metaheuristic algorithm and the performance of the switching scheme were demonstrated through a series of computational experiments, where it was observed that the herein-proposed approach was more than two orders of magnitude more accurate than traditional single-step approaches of transition from one operating point to another.

A soft sensor modeling framework embedded with domain knowledge based on spatio-temporal deep LSTM for process industry

In the process industries, the complex mechanisms, the many variables with complex interactions, the high uncertainty and errors in instrumentation, etc. make it very difficult to build accurate soft sensor models. Domain knowledge plays a very important role in soft sensor modeling. However, current data-driven methods lack domain knowledge of the specific processes. Therefore, a spatio-temporal deep learning soft sensor modeling framework embedded with domain knowledge is proposed in this paper. First, the time-delays between the key variable and the other process variables are analyzed to align the process variables temporally and select secondary variables. A spatio-temporal structure model (STSM) of the process is then constructed by using domain knowledge to decouple the sub-processes. Finally, a deep Long Short-Term Memory (LSTM) model embedded with the STSM is proposed to learn the characteristics of the spatiotemporal nonlinear dynamic features between the sub-process variables and the key variable in industrial processes. The effectiveness of the framework is verified by prediction of the key indices in two mineral processing processes.

Dynamic Graph-Based Adaptive Learning for Online Industrial Soft Sensor With Mutable Spatial Coupling Relations

Accurate online soft sensor in complex industrial processes remains challenging because underlying spatial coupling relations among process variables have not been effectively mined and exploited. Recently, some deep learning-based studies construct static graphs to explicitly represent underlying spatial coupling relations among process variables, but they neglect the fact that spatial coupling relations have mutable characteristics, leading to the poor performance in online soft sensor. Therefore, we propose a novel deep learning model to address this issue to achieve accurate soft sensor. Specifically, a dynamic graph is proposed to realize adaptive learning and automatic inference for mutable spatial coupling relations, so that the proposed model is endowed with the ability to real-timely sense spatial coupling relations in online industrial soft sensor. Then, based on the dynamic graph, a new multi-hop attention graph convolutional network is proposed to systematically aggregate various crucial node feature representations during graph convolution processes to capture fine-grained spatial dependence features, thereby achieving effective modeling for variation patterns of process variables. Finally, a new multivariate incremental training algorithm is designed for deep learning models to further improve the prediction performance. Verification study on a coal mill rig demonstrates the feasibility and effectiveness of the proposed model.

Manufacturing industry based on dynamic soft sensors in integrated with feature representation and classification using fuzzy logic and deep learning architecture.

Soft sensors are data-driven devices that allow for estimates of quantities that are either impossible to measure or prohibitively expensive to do so. DL (deep learning) is a relatively new feature representation method for data with complex structures that has a lot of promise for soft sensing of industrial processes. One of the most important aspects of building accurate soft sensors is feature representation. This research proposed novel technique in automation of manufacturing industry where dynamic soft sensors are used in feature representation and classification of the data. Here the input will be data collected from virtual sensors and their automation-based historical data. This data has been pre-processed to recognize the missing value and usual problems like hardware failures, communication errors, incorrect readings, and process working conditions. After this process, feature representation has been done using fuzzy logic-based stacked data-driven auto-encoder (FL_SDDAE). Using the fuzzy rules, the features of input data have been identified with general automation problems. Then, for this represented features, classification process has been carried out using least square error backpropagation neural network (LSEBPNN) in which the mean square error while classification will be minimized with loss function of the data. The experimental results have been carried out for various datasets in automation of manufacturing industry in terms of computational time of 34%, QoS of 64%, RMSE of 41%, MAE of 35%, prediction performance of 94%, and measurement accuracy of 85% by proposed technique.

A Soft Sensor Model for Predicting Cement-Specific Surface Area Based on Convolution Fast Recurrent Unit Network

Due to the time lag, nonlinearity, and variable coupling in the process sector, production variables are hard to measure directly and require accurate soft sensor models. In light of this, this research proposes a soft sensor convolution neural network fast recurrent unit (CNN-FRU) network model. Convolution is used to get spatial features and get rid of redundancy data in variables. FRU changes the linear coupling constraint within a gated recurrent unit (GRU) and solves the problem of blocked information flow in GRU. It can be used to extract time-varying delay characteristics from complex processes industry data. FRU is compared with GRU and long short-term memory (LSTM) on a standard dataset to demonstrate the advantages of FRU in terms of training speed and accuracy. As a specific application in the process industry, the proposed soft sensor model is trained and validated on the cement-specific surface area dataset. The experimental results demonstrate that the soft sensor model can be generalized better.

Interpretable Soft Sensors using Extremely Randomized Trees and SHAP

Tree-based ensemble models are easy to implement and have been widely used in various fields. However, they have limitations in industrial process applications since the majority of tree-based ensemble models are prone to over-fitting. In addition, the internal structure of tree-based ensemble models is very complex and the output of the model is also difficult to explain, which makes its application in industrial soft sensors very challenging. The purpose of this work is to build accurate and interpretable soft sensors for industrial processes. First, to deal with overfitting, a robust tree-based ensemble model and extremely randomized trees are used to build accurate soft sensors. Then, to improve model interpretability, an interpretable machine learning algorithm, namely Shapely additive explanation, is used to infer the global and local contributions of each feature to the predictions. Finally, the effectiveness of the proposed algorithms is validated on real industrial fluid catalytic cracker unit data.

A New Distributed Echo State Network Integrated With an Auto-Encoder for Dynamic Soft Sensing

As dynamic industrial processes become increasingly complicated, it tends to be difficult to develop accurate soft sensors. Echo state networks (ESNs) as dynamic neural network (NN) models have been broadly used in developing dynamic soft sensors. However, when facing high-dimensional data, the dimension disaster problem might occur in the input space of ESNs. Furthermore, in ESNs, the number of reservoir nodes is large, causing collinearity between the reservoir outputs. To solve these problems, in this article, a new distributed ESN model integrated with auto-encoder (AE-DESNm) is proposed. In particular, AE-DESNm has distributed and independent input subnets. The dimensionality of each input subnet has been reduced, eliminating the dimension disaster problem. To obtain the distributed and independent input subnets of AE-DESNm, the first integer neighbor clustering hierarchy (FINCH) algorithm is used to cluster the input attributes of high-dimensional data. Each cluster corresponds to one input subnet. In addition, seeking to handle the collinearity issue in the reservoir outputs, auto-encoder (AE) is used to extract features from the reservoir outputs. Finally, the extracted features are modeled by an extreme learning machine (ELM) to estimate key variables. To verify the performance of AE-DESNm, a numerical example and a dataset from one real industrial process named purified terephthalic acid (PTA) production are used. Compared with other methods, the experimental results show that the proposed AE-DESNm can achieve much higher accuracy in soft sensor modeling for dynamic industrial processes.

Correntropy long short term memory soft sensor for quality prediction in industrial polyethylene process

A typical challenge for construction of accurate soft sensors in the process industries is that industrial process data often contains various noise and outliers. Inspired by correntropy in tackling non-Gaussian noise effectively, a maximum correntropy criterion-based long short term memory (MCC-LSTM) neural network is proposed to develop a reliable soft sensor model for quality prediction. Without tedious data preprocessing approaches, the MCC-LSTM adopts the objective function using the maximum correntropy criterion (MCC) centered on a Gaussian kernel, which assigns relatively smaller weights to outliers automatically. Once the MCC-LSTM model is constructed, the outliers can be identified and their negative effects on the prediction can be reduced to some extent. Consequently, the prediction performance can be enhanced for modeling of process data with uncertainties. Additionally, an index is introduced to assess the performance of prediction models when data contain outliers and noise. A numerical example and an industrial polyethylene process demonstrate that the MCC-LSTM soft sensor can achieve more reliable predictive performance compared to traditional LSTM and support vector machine candidates.

Transient swelling of cylindrical hydrogels under coupled extension-torsion: Analytical and 3D FEM solutions

To design more accurate soft sensors and actuators, there is a significant requirement for the application of complicated deformation conditions. Coupled extension-torsion is a type of deformation which may be utilized in the property characterization of materials with complex behavior such as soft hydrogels. Hydrogels with coupled diffusion and large deformation behavior have intricate kinetics which should be studied in detail. In this work, a robust semi-analytical procedure is proposed to capture the transient behavior of cylindrical hydrogels in combined loading of extension and torsion in different conditions. Moreover, the whole problem with the same conditions is simulated in COMSOL-Multiphysics as a verification framework of the proposed semi-analytical procedure, where comparing the results shows a good agreement. The results demonstrate that the transient behavior of the cylindrical hydrogel under coupled extension-torsion is significantly dependent on the rate of the deformation and material properties. Besides, since the hydrogel undergoes a highly nonlinear swelling in this deformation regime, the simulations predict a complicated reaction force and moment. Visualizing the effect of different parameters engaged in the material shows the robust behavior of the proposed semi-analytical procedure, which can be employed in constitutive models’ calibration as well as in the design and optimization of hydrogels structures.

Soft Sensor for Melt Index Prediction Based on Long Short-Term Memory Network

This paper presents a soft sensor model for melt index (MI) prediction in an industrial polymerization process based on long short-term memory (LSTM) network. MI is one of the important specifcations that determine the quality and grade of thermoplastic polymers. However, lack of online measurement of MI makes it difcult to monitor and control the quality of polymer products. Thus, there has been a great efort to build accurate soft sensor models to predict MI with easy-to-measure process variables by using black-box modeling approaches. However, real chemical processes have strong nonlinearity and complicated temporal correlations between the process and quality variables, which is very challenging for traditional static black-box models to handle. Recently, LSTM network that is an advanced form of recurrent neural network (RNN) has shown great advantages in capturing and modeling the long-term dynamic nature of complex industrial processes. We develop an LSTM-based MI prediction model for an industrial styrene-acrylonitrile (SAN) polymerization process in Korea. The developed model provides the most accurate predictions compared to other soft sensor models based on partial least squares (PLS), support vector machines (SVM), Gaussian process regression (GPR), and feedforward artificial neural network (ANN).

Novel Stacked Input-Enhanced Supervised Autoencoder Integrated With Gated Recurrent Unit for Soft Sensing

Machine learning techniques have been successfully utilized as effective soft sensing for industrial processes. Unfortunately, as modern industrial processes because increasingly complex in the terms of scale and integration, the collected process data become more complicated in terms of dimensionality. Thus, it becomes a great challenge to establish an accurate soft sensor model for dynamic processes. In this article, a novel stacked input-enhanced supervised autoencoder (SISAE) integrated with gated recurrent unit (GRU) is proposed. In SISAE-GRU, the presented SISAE model is used to extract high-level and useful features from collected high-dimensionality process data. Different from the standard stacked autoencoder (SAE) model, the original input variables are embedded into each autoencoder (AE) unit of the presented SISAE model during the pre-training process by supervised learning, enhancing the feature extraction performance of SAE. Considering the dynamic nature between extracted features and output variables, GRU is utilized to learn the dynamic relationships and a dynamic model of soft sensing is finally constructed using a fully connected (FC) layer. To illustrate the effectiveness and superiority of the proposed SISAE-GRU model, two industrial datasets from the debutanizer column and the purified terephthalic acid (PTA) process are utilized. The experimental results show that the proposed SISAE-GRU can reduce the <i>RMSE</i> by 53.6% and 46.1% on average compared with some other state-of-the-art methods for the debutanizer column and PTA, respectively.

Pseudo-Labeling Optimization Based Ensemble Semi-Supervised Soft Sensor in the Process Industry.

Nowadays, soft sensor techniques have become promising solutions for enabling real-time estimation of difficult-to-measure quality variables in industrial processes. However, labeled data are often scarce in many real-world applications, which poses a significant challenge when building accurate soft sensor models. Therefore, this paper proposes a novel semi-supervised soft sensor method, referred to as ensemble semi-supervised negative correlation learning extreme learning machine (EnSSNCLELM), for industrial processes with limited labeled data. First, an improved supervised regression algorithm called NCLELM is developed, by integrating the philosophy of negative correlation learning into extreme learning machine (ELM). Then, with NCLELM as the base learning technique, a multi-learner pseudo-labeling optimization approach is proposed, by converting the estimation of pseudo labels as an explicit optimization problem, in order to obtain high-confidence pseudo-labeled data. Furthermore, a set of diverse semi-supervised NCLELM models (SSNCLELM) are developed from different enlarged labeled sets, which are obtained by combining the labeled and pseudo-labeled training data. Finally, those SSNCLELM models whose prediction accuracies were not worse than their supervised counterparts were combined using a stacking strategy. The proposed method can not only exploit both labeled and unlabeled data, but also combine the merits of semi-supervised and ensemble learning paradigms, thereby providing superior predictions over traditional supervised and semi-supervised soft sensor methods. The effectiveness and superiority of the proposed method were demonstrated through two chemical applications.

Quality Variable Prediction for Nonlinear Dynamic Industrial Processes Based on Temporal Convolutional Networks

Soft sensors have been extensively developed to estimate the difficult-to-measure quality variables for real-time process monitoring and control. Process nonlinearities and dynamics are two main challenges for accurate soft sensor modeling. To cope with these problems, two temporal convolutional network (TCN)-based soft sensor models are developed in this paper. With a hierarchy of temporal convolution kernels and large receptive fields, TCN is able to describe the nonlinearities and long dynamic dependence of process variables. Thus, a TCN model is first designed for nonlinear dynamic soft sensing. However, the original TCN model neglects the auto-correlations of the quality variable, as well as the cross-correlations between the quality variable and process variables. Hence, an autoregressive TCN (AR-TCN) is further constructed by integrating the lagged quality data in the past neighboring samples for quality prediction of the current sample. Thus, AR-TCN is able to capture the auto-correlations and cross-correlations between the quality variables and process variables, which are more beneficial for quality prediction. The effectiveness of two proposed models is validated on an industrial debutanizer column and an industrial hydrocracking process.

Novel manifold learning based virtual sample generation for optimizing soft sensor with small data

Due to the extremely complex mechanism and strong non-linear characteristics of industrial processes, data-driven soft sensor technologies play a key role in the intelligent measurement of process industries. However, the information of the collected process data in the steady stage is quite limited and unreliable, causing the small sample problem. As a result, it becomes an intractable challenge to catch the nature of the process and build accurate soft sensor models. To solve this problem, this paper proposes a novel manifold learning based virtual sample generation method (Isomap-VSG) to generate feasible virtual samples in the information gaps for supplementing the original small sample space. To find data sparse regions reasonably, one kind of manifold learning methods called Isomap is used to visualize process data with high dimension. Then virtual samples can be generated by the interpolation method and extreme learning machine. The simulation results on a standard dataset and a real-world application demonstrate that, compared with other advanced methods, the proposed Isomap-VSG method can achieve better performance in terms of generating feasible virtual samples and improving the accuracy of soft sensor models using limited samples.

Novel soft sensor development using echo state network integrated with singular value decomposition: Application to complex chemical processes

It is of great importance to develop advanced soft sensors for ensuring the safety and stability of complex industrial processes. Unluckily, with the increasing scale of chemical processes, it becomes more and more demanding to develop soft sensor with high accuracy. In addition, most of industrial processes are dynamic. As a result, the soft sensors developed using static models cannot achieve acceptable performance. In order to handle this problem, the Echo state network (ESN) as a kind of recurrent neural network is selected. However, the output weights of ESN are calculated linearly. On one hand, the collinear in the reserve layer outputs may decrease the performance; on the other hand, the over-fitting problem may occur. To enhance and improve the ESN performance, singular value decomposition based ESN (SVD-ESN) is presented. In the SVD-ESN method, the singular value decomposition instead of the traditional least square is adopted to calculate the weights between the output layer and the reserve layer. Through singular value analysis in the outputs of the reserve layer, appropriate defining parameters are selected to enhance the accuracy and ensure the computing speed. As a result, the collinearity and over-fitting problem is solved; then the performance of ESN is enhanced. To test and validate the performance of SVD-ESN, the proposed SVD-ESN is developed as soft sensor for the High Density Polyethylene (HDPE) production process and Purified Terephthalic Acid (PTA) production process. Compared with the conventional ESN, Extreme Learning Machine (ELM), Dynamic Window based ELM (DW-ELM) and Long Short-Term Memory (LSTM), the simulation results show that the proposed SVD-ESN model obtains better performance in terms of prediction accuracy, which conforms that the proposed SVD-ESN can be used as an effective dynamic model for developing accurate soft sensors.

Development of NVW-SAEs with nonlinear correlation metrics for quality-relevant feature learning in process data modeling

Soft sensors have become reliable tools for predicting difficult-to-measure quality variables in modern industrial process modeling. Feature representation is a key step to construct accurate soft sensor models. In the past decade, deep learning has shown great capacity of feature extractor for soft sensor modeling. However, most existing deep networks cannot capture quality-related features for output prediction. To deal with this problem, a variable-wise weighted stacked autoencoder (VW-SAE) was previously proposed to learn deep quality-related features, in which a variable weighted objective function is designed to learn quality-related features layer by layer. However, only linear correlation is considered for variable weighting, which is insufficient to extract quality-related features fully. In this paper, nonlinear VW-SAEs (NVW-SAEs) are constructed to enhance the learning capability for deep quality-related features, in which three correlation metrics are utilized to measure the nonlinear variable relationships and learn deep quality-related features. The prediction results show that NVW-SAEs can effectively extract quality-related features from process data. Two public data sets and an industrial debutanizer are used to validate the effectiveness of the NVW-SAEs.

Novel Soft Sensor for Measuring and Controlling Product Recovery in a High-Purity, Multicomponent, Side-Draw Distillation Column

The use of soft sensors for monitoring purposes is an established practice in the process industry. In this study, the focus has been on developing a soft sensor that can be used to monitor the product recovery rate of double-ended, high-purity distillation columns with a side-draw. To this end, this study specifically focuses on developing a cost-effective and accurate soft sensor for an industrial methanol distillation unit where a side-draw is used to meet the parts per million level impurity specifications. The novel soft sensor is based on the unique characteristics of the mass balance in this type of column, and uses the density measurement at the side-draw together with the flow rates of the side-draw and product draw to calculate the product recovery rate. The developed soft sensor was validated against real plant data as well as on a process simulation of an industrial methanol distillation column. The soft sensor demonstrated the ability to predict product recovery to an accuracy of 0.05% and sh...

Deep quality-related feature extraction for soft sensing modeling: A deep learning approach with hybrid VW-SAE

Soft sensors have been extensively used to predict difficult-to-measure quality variables for effective modeling, control and optimization of industrial processes. To construct accurate soft sensors, it is significant to carry out feature extraction for massive high-dimensional process data. Recently, deep learning has been introduced for feature representation in process data modeling. However, most of them cannot capture deep quality-related features for output prediction. In this paper, a hybrid variable-wise weighted stacked autoencoder (HVW-SAE) is developed to learn quality-related features for soft sensor modeling. By measuring the linear Pearson and nonlinear Spearman correlations for variables at the input layer with the quality variable at each encoder, a corresponding weighted reconstruction objective function is designed to successively pretrain the deep networks. With the constraint of preferential reconstruction for more quality-related variables, it can ensure that the learned features contain more information for quality prediction. Finally, the effectiveness of the proposed HVW-SAE based soft sensor method is validated on an industrial debutanizer column process.

Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE

In modern industrial processes, soft sensors have played an important role for effective process control, optimization, and monitoring. Feature representation is one of the core factors to construct accurate soft sensors. Recently, deep learning techniques have been developed for high-level abstract feature extraction in pattern recognition areas, which also have great potential for soft sensing applications. Hence, deep stacked autoencoder (SAE) is introduced for soft sensor in this paper. As for output prediction purpose, traditional deep learning algorithms cannot extract high-level output-related features. Thus, a novel variable-wise weighted stacked autoencoder (VW-SAE) is proposed for hierarchical output-related feature representation layer by layer. By correlation analysis with the output variable, important variables are identified from other ones in the input layer of each autoencoder. The variables are assigned with different weights accordingly. Then, variable-wise weighted autoencoders are designed and stacked to form deep networks. An industrial application shows that the proposed VW-SAE can give better prediction performance than the traditional multilayer neural networks and SAE.