Data-driven Soft Sensor Research Articles

Graph semi-supervised soft sensor modeling based on label propagation algorithm

Abstract Data-driven soft sensor modeling methods have become prevalent in the industry. Nonetheless, the complexity of industrial processes often leads to the absence or difficulty in obtaining key labeled data, and existing methods frequently fail to fully utilize the inherent correlations between variables. This paper proposes a novel graph semi-supervised soft sensor modeling method using the label propagation algorithm to address these issues. This method utilizes correlations within the data to assign pseudo-labels to unlabeled data reasonably and employs Graph Convolutional Networks (GCN) to capture spatial relationships between nodes. Additionally, by embedding a Long Short-Term Memory (LSTM) structure, the model can capture temporal dependencies of the data while focusing on spatial structures. Furthermore, the introduction of a residual structure enables the model to directly learn the differences between inputs and outputs, facilitating information transmission, and improving the model's feature extraction ability. Experiments demonstrate the effectiveness of the method.

Evaporating temperature estimation of refrigeration systems based on vibration data-driven soft sensors

The evaluation of the operating conditions of refrigeration compressors once installed in household appliances is challenging due to the need to install pressure transducers, a process which requires system evacuation and refrigerant reintroduction. In addition, changes in the piping modify the characteristics of the original product. This paper proposes a soft-sensing technique based on vibration measurements of the compressor surface to predict the evaporating temperature. Different machine learning (ML) techniques are evaluated as data-driven prediction models, namely multilayer perceptron (MLP) neural networks, least squares boosting, generalized additive model, random forest, extreme learning machine, and random vector functional link neural networks. These techniques were applied to data obtained from a test rig designed to emulate compressor operation in a refrigeration system, with an operating envelope from -30°C to -10°C for the evaporating temperature and from 34°C to 54°C for the condensing temperature. The results showed that, with a single vibration measurement point, it was possible to use an MLP technique to estimate the evaporating temperature with a root mean squared error of 1.74°C in a non-intrusive way. For the other prediction techniques, the errors were a bit higher than for the MLP, but the maximum error value was about 2.5°C in all cases.

Semi-supervised soft sensor method for fermentation processes based on physical monotonicity and variational autoencoders

Data-driven models have shown broad application prospects in soft sensor modeling. However, numerous challenges persist. On the one hand, data-driven soft sensor methods have high requirements on data quality. On the other hand, models relying on limited experimental data often lack physical interpretability. To tackle these challenges, a semi-supervised soft sensor method (PMVAER) for fermentation processes based on physical monotonicity and variational autoencoders (VAEs) is introduced. First, physical monotonicity constraint is incorporated into the loss function of VAEs for regression to ensure that the model's predictions adhere to physical feasibility. Next, considering the disparate sampling frequencies for process and quality variables, this approach is extended to learn from unlabeled data, creating a semi-supervised soft sensor model. The proposed model is validated on simulation and real cases of penicillin fermentation. Comparisons with five other methods verify that the proposed method exhibits exceptional predictive accuracy along with enhanced generalization ability.

Twinning quality sensors in wastewater treatment process via optimized echo state network-based soft sensors

The presence of a large amount of quality-related but hard-to-measure variables usually makes effective monitoring of industrial processes difficult, and even impossible. Soft computing techniques and digital twins can revolutionize standard approaches to solve this issue, even though industrial data are frustrated by nonlinearity, disturbances, and dynamic behaviors. In this light, this paper proposes a data-driven soft sensor with the help of the Fourier amplitude sensitivity test (FAST) and improved probabilistic regularized echo state network (IPRESN), then to twin the quality hardware sensors. Within this framework, the initial step is to rely on the FAST method to measure the collaborated importance of multiple process variables with respect to each quality variable. This quantification facilitates the selection of auxiliary variables for the training of IPRESN. Furthermore, a probabilistic regularization method is learned to improve the robust performance of the echo state network (ESN) by re-designing a new objective function to minimize the mean and variance of the modeling error. Additionally, the whale optimization algorithm (WOA) is improved and then used to optimize the critical hyper-parameters of ESN, thus mitigating the risks associated with suboptimal hyperparameter selection. The effectiveness of the proposed method is verified through a Benchmark Simulation Model 2 (BSM2) and a full-scale wastewater treatment plant data set. The findings show the potential of the proposed method in facilitating digital twin implementations for hardware sensors.

Modelling the influence of tool wear on shape errors in milling using a hybrid soft-sensor

AbstractThe wear of cutting tools during milling processes not only constrains the volume of material that can be removed by a tool but also results in a progressive deterioration of the quality of the workpiece. Although there are established methodologies for predicting tool wear, there is a paucity of knowledge regarding the impact of tool wear on shape error. The authors present a data-driven soft sensor to model the effect of tool wear on shape error, obviating the need for direct tool wear measurement. To evaluate this approach, a milling experiment was conducted, wherein process forces, spindle current, and resulting shape error were measured. Furthermore, a geometrical cutting simulation was conducted in order to obtain cutting conditions, including the volume of material removed. This study examines the contribution of these features to the prediction performance of the proposed soft sensor. Additionally, the transferability from models trained on different tools is investigated to ascertain the impact of tool wear variance on prediction performance. Prediction experiments demonstrate that a soft-sensor based on a combination of simulation and process monitoring data enables a model trained on data from multiple milling tools to account for wear and predict shape error well under varying wear scenarios. The approach presented here has been demonstrated to result in a reduction of the prediction error of up to 60% compared to an average baseline prediction.

Deep learning with local spatiotemporal structure preserving for soft sensor development of complex industrial processes

Data-driven soft sensors have emerged as indispensable tools for predicting quality variables in complex industrial processes because of their cost-effectiveness and ease of maintenance. In particular, soft sensors based on deep learning have been utilized in extensive research and successful applications in recent years. However, traditional deep learning methods capture hierarchical data features by minimizing global fitting errors, neglecting the local structural characteristics implied in the original data. In this paper, we propose a new deep learning method for soft sensor development. Utilizing autoencoders as the foundational architecture of our network, a new semisupervised strategy is adopted for layerwise pretraining optimization. On the one hand, more representative data features are extracted by maintaining the local spatiotemporal structure of the data; on the other hand, layer-by-layer supervised learning is employed to identify the critical features that are aligned with the ultimate task, which aids in obtaining the optimal network parameters and improving the resulting prediction accuracy. Subsequently, a local spatiotemporal structure-preserving stacked semisupervised autoencoder (LSP-SuAE) is established. To evaluate the feasibility and effectiveness of the proposed approach, experiments are carried out in a real industrial process. A soft sensor based on the LSP-SuAE is developed to predict the rate of ethylbenzene conversion during the dehydrogenation of styrene. The experimental results demonstrate that, compared to five other common or similar data-driven modeling methods, LSP-SuAE exhibits higher prediction accuracy and better stability.

Additive dynamic Bayesian networks for enhanced feature learning in soft sensor modeling

Due to the advantages of indicating variable structure and efficient reasoning, Bayesian Networks (BN) have been widely used in data-driven soft sensor applications. However, restricted to linear and conditional Gaussian property, BN-based soft sensors rarely achieve high prediction accuracy. In this paper, an ensemble learning framework – additive dynamic Bayesian networks (ADBN) is proposed for enhanced feature learning, in which Dynamic Bayesian networks are used to learn the conditional independencies among variables and construct feature sets for the following base learners. Additional DBNs are constructed upon the residual information from the past model, to carry out feature learning to fit the residuals. The procedure is repeated and a termination rule from the perspective of feature learning is proposed to end this process, and thus the model complexity can be well restricted. The proposed method is validated on two actual industrial cases. It reveals that the ADBN feature learning method has obtained great improvements. Compared to the single DBN feature engineering method, the root mean square error (RMSE) performance has been improved by 20% and 13% on the two cases, respectively.

A data-driven LSTMSCBLS model for soft sensor of industrial process

Abstract In the chemical industry, data-driven soft sensor modeling plays a crucial role in efficiently monitoring product quality and status. Industrial data in these applications typically exhibit significant temporal characteristics, meaning that current information is influenced by data from previous periods. Effectively extracting and utilizing these temporal features is essential for achieving accurate soft sensor modeling in complex chemical scenarios. To address this challenge, this study proposes a data-driven Broad Learning System (BLS) model, which combines Long Short-Term Memory (LSTM) networks with an adaptive algorithm known as the Stochastic Configuration Algorithm (SC), referred to as LSTMSCBLS. The model operates in two stages: temporal feature extraction and final prediction. In the temporal feature extraction stage, the integration of the LSTM network with a feature attention mechanism allows for efficient extraction of temporal features from high-dimensional time-series data. In the final prediction stage, the SC is integrated into the BLS, effectively mitigating issues related to node space redundancy and the determination of the number of nodes. The effectiveness and superiority of the proposed model are demonstrated through two industrial case studies involving a debutanizer column and a sulfur recovery unit.

Capturing connectivity information from process flow diagrams by sequential-orthogonalized PLS to improve soft-sensor performance

In the development of data-driven soft sensors for product quality assessment in multi-unit manufacturing processes, the only information that is typically used as an input to the model is real-time measurements from field sensors. However, even if detailed knowledge of the mechanistic behavior of the process may not be available, information about the sequence of processing units, and their connectivity, is available, typically in graphical form through process flow diagrams. In this study, we investigate the use of sequential-orthogonalized partial least-squares (SO-PLS) regression as a way to capture connectivity information from a process flow diagram, and transfer it into a data-driven model to be used as a soft sensor in a multi-unit process. Connectivity between units is captured and translated into a block order that establishes a sequence for block regressions. Orthogonalization between two blocks is then carried out with the aim of eliminating overlapping data and retaining information that is unique to each block. Product quality is finally predicted by summing the contributions from each block, and the accuracy of prediction is enhanced due to the embedded dual feature-extraction procedure, which combines orthogonalization and latent-variable extraction. The effectiveness of the proposed approach is illustrated by comparing the quality prediction performance of two soft sensors for a simulated multi-unit continuous process: one using standard PLS and one using SO-PLS. Superior performance of the SO-PLS soft sensor is achieved, even more markedly so when fewer field measurements are available to build the soft sensor.

Semi-supervised soft sensor development based on dynamic dimensionality reduction-assisted large-scale pseudo label optimization and sample-weighted quality-relevant deep learning

Data-driven soft sensors have become popular tools for estimating critical quality variables in the process industry. However, in practical applications, it is very common that the unlabeled data are abundant but the labeled data are scarce, which poses a great challenge for developing high-performance data-based soft sensors. Thus, a dynamic dimensionality reduction-assisted large-scale pseudo label optimization method (DDR-LSPLO) is proposed for achieving sample expansion. This method repeatedly converts the LSPLO issue into a reduced-dimension pseudo label optimization problem with the low-confidence pseudo labels as new decision variables during the evolutionary optimization process. Meanwhile, to tackle the sample imbalance problem resulting from the inclusion of large-scale pseudo-labeled samples, a sample expansion and weighting-based quality-relevant autoencoder (SEWQAE) is developed for semi-supervised soft sensor modeling. The effectiveness and superiority of the proposed DDR-LSPLO and SEWQAE methods are verified through an industrial chlortetracycline (CTC) fermentation process and a simulated Tennessee Eastman (TE) chemical process.

Adaptive Temporal-Spatial Pyramid Variational Autoencoder Model for Multirate Dynamic Chemical Process Soft Sensing Application.

Data-driven soft sensors play an important role in practical processes and have been widely applied. They provide real-time prediction of quality variables and then guide production and improve product quality. In practical chemical production processes, nonlinear dynamic multirate data is widespread and challenging to model. This paper innovatively proposes a temporal-spatial pyramid variational autoencoder (TS-PVAE) model for the nonlinear temporal-spatial feature pyramid extraction from multirate data. This structure not only selectively utilizes multirate data but also handles complex nonlinear time-series data. Based on this, integrated with just-in-time (JIT) learning, an adaptive TS-PVAE (ATS-PVAE) model is developed. In this model, historical data are used for real-time fine-tuning of the model, leading to the development of an adaptive model. Finally, the proposed models are validated by an industrial case of a methanation furnace, demonstrating a superior estimation performance.

A data-driven soft sensor model for coal-fired boiler SO2 concentration prediction with non-stationary characteristic

Environmental issues have received significant global attention, and achieving ultra-low emission has become a crucial goal for thermal power plant. Pollutant prediction is essential to achieving ultra-low emission control. Due to the frequent adjustment of operating conditions, the coal-fired power generation process is a typical non-stationary process with complex changes along the time direction. A single model cannot effectively portray the complex non-stationary characteristics and model mismatch may occur in the prediction, which brings new challenges for modeling coal-fired SO2 emission. Therefore, the condition-driven category boosting (CatBoost) soft sensor model is proposed for real-time prediction of SO2 concentration. First, the thermal process time series is rearranged into different data slices. These data slices vary along the direction of the condition indicator, which can characterize the process characteristics on different slices. Second, the load-based sequential condition division (LSCD) algorithm is designed to realize the division of boiler operation process. The characteristics of process are effectively captured by variational auto-encoder (VAE), and the data slices with similar process characteristics are merged into the same mode. Non-stationary process along the time direction is reorganized into different condition modes along the load direction. Finally, the high-resolution local CatBoost models under different modes are constructed to achieve fine-grained prediction of SO2 concentration. The effectiveness of the proposed condition-driven model is verified by an industrial example of 1000 MW coal-fired boiler. The prediction results show that the proposed model can effectively divide the operation conditions and achieve accurate prediction of SO2 concentration. In the typical models in SO2 predicting, the proposed model has better performance with a determination coefficient of 0.92.

Physics-guided graph learning soft sensor for chemical processes

The surge in data-driven soft sensors for industrial processes is evident. However, most of them suffer from the limitation of being black-box models and this will hamper their widespread use. In response to this challenge, this study proposes a physics-guided graph-learning soft sensor that integrates a physical understanding of industrial processes by incorporating graph-based concepts with process physics. The soft sensor first constructs physical information based on causal relationships between variables using the conditional Granger causality test. Subsequently, it autonomously learns the unique sample information of each observation while employing a regularization loss to ensure the sparsity of the learned information. The model employs a two-stream structure for spatiotemporal encoding of both the physical and sample information. The modeling and prediction results on a penicillin fermentation process indicate that, using the proposed method, the knowledge gained from the data aligns with existing prior knowledge. This approach shows promise in filling the gap between data-driven and physics-based modeling in chemical processes.

Soft Sensor Modeling Method Considering Higher-Order Moments of Prediction Residuals

Traditional data-driven soft sensor methods can be regarded as an optimization process to minimize the predicted error. When applying the mean squared error as the objective function, the model tends to be trained to minimize the global errors of overall data samples. However, there are deviations in data from practical operation, in which the model performance in the estimation of the local variations in the target parameter worsens. This work presents a solution to this challenge by considering higher-order moments of prediction residuals, which enables the evaluation of deviations of the residual distribution from the normal distribution. By embedding constraints on the distribution of residuals into the objective function, the model tends to converge to the state where both stationary and deviation data can be accurately predicted. Data from the Tennessee Eastman process and an industrial cracking furnace are considered to validate the performance of the proposed modeling method.

Bringing a scalable adaptive hybrid modeling framework closer to industrial use: Application on a multiscale fungal fermentation.

Digitalization has paved the way for new paradigms such as digital shadows and digital twins for fermentation processes, opening the door for real-time process monitoring, control, and optimization. With a digital shadow, real-time model adaptation to accommodate complex metabolic phenomena such as metabolic shifts of a process can be monitored. Despite the many benefits of digitalization, the potential has not been fully reached in the industry. This study investigates the development of a digital shadow for a very complex fungal fermentation process in terms of microbial physiology and fermentation operation on pilot-scale at Novonesis and the challenges thereof. The process has historically been difficult to optimize and control due to a lack of offline measurements and an absence of biomass measurements. Pilot-scale and lab-scale fermentations were conducted for model development and validation. With all available pilot-scale data, a data-driven soft sensor was developed to estimate the main substrate concentration (glucose) with anormalized root mean squared error(N-RMSE)of 2%. This robust data-driven soft sensor was able to estimate accurately in lab-scale (volume < 20× pilot) with a N-RMSE of 7.8%. A hybrid soft sensor was developed by combining the data-driven soft sensor with a mass balance to estimate the glycerol and biomass concentrations on pilot-scale data with N-RMSEs of 11% and 21%, respectively. A digital shadow modeling framework was developed by coupling a mechanistic model (MM) with the hybrid soft sensor. The digital shadow modeling framework significantly improved the predictability compared with the MM. The contribution of this study brings the application of digital shadows closer to industrial implementation. It demonstrates the high potential of using this type of modeling framework for scale-up and leads the way to a new generation of in silico-based process development.

Integrating transfer learning within data-driven soft sensor design to accelerate product quality control

The measurement of batch quality indicators in real time operation is plagued with many challenges, hence soft sensing has become a promising solution within industrial research. However, small data has traditionally been a severe problem, hindering the ability to create accurate, reliable soft sensors, especially within industrial research and development for new product formulations. Nevertheless, it is often the case that modelling knowledge is available for a related system. In order to exploit this, we have developed a generalisable transfer learning methodology which takes advantage of previous modelling efforts to accelerate and improve the construction of models for new systems. Specifically, we adapted a recently developed advanced data-driven soft sensing methodology made for an existing process formulation and integrated a feature-based transfer learning approach to facilitate the modelling of two new industrial process systems, each of which containing notable differences to the original. The performance of the transfer soft sensors was tested rigorously and compared to a benchmark approach under different data availability conditions. It was shown that, the proposed transfer mechanism yielded high accuracy, and is robust to small data scenarios, indicating its potential for use in soft sensing of novel systems.

Soft sensor for in-line quality control of turning processes based on non-destructive testing techniques and advanced data fusion

AbstractThis study describes the systematic process of training, testing, and validating a soft sensor designed for quality control of a turning process on components made of AISI 4140 steel. The soft sensor allows product quality to be predicted and unfavorable surface conditions to be identified, in particular the appearance of a phenomenon known as “White Layer”, often characterized in the case of AISI 4140 steel by an ultra-fine-grained microstructure (UFG). Basis of the soft sensor is a data fusion supported by non-destructive testing techniques (NDT), particularly micromagnetic methods (3MA). A critical part of this work is to address challenges such as lift-off compensation and in-process detection using 3MA. The application of machine-learning techniques, including Principal Component Analysis (PCA) and regression analysis, is detailed. These techniques result in robust models capable of detecting the occurrence of the White Layer phenomenon while minimizing the influence of measurement setup variations and process disturbances. In addition, the study demonstrates the integration of NDT into the machining process which drives the soft sensor and allows suitable adjustments of the process parameters. The data-driven soft sensor approach demonstrates a possible In-Line control system and discusses different control theories and their respective advantages and disadvantages. This system can effectively set targeted surface conditions in real time during the turning process.

A Soft Sensor for Multirate Quality Variables Based on MC-CNN.

In recent years, data-driven soft sensor modeling methods have been widely used in industrial production, chemistry, and biochemical. In industrial processes, the sampling rates of quality variables are always lower than those of process variables. Meanwhile, the sampling rates among quality variables are also different. However, few multi-input multi-output (MIMO) sensors take this temporal factor into consideration. To solve this problem, a deep-learning (DL) model based on a multitemporal channels convolutional neural network (MC-CNN) is proposed. In the MC-CNN, the network consists of two parts: the shared network used to extract the temporal feature and the parallel prediction network used to predict each quality variable. The modified BP algorithm makes the blank values generated at unsampled moments not participate in the backpropagation (BP) process during training. By predicting multiple quality variables of two industrial cases, the effectiveness of the proposed method is verified.

Novel Deep Layers-Extended Autoencoder With Correlation and Its Industrial Soft Sensing

Due to the increasing complexity of industrial processes, accurately constructing soft sensors has become a daunting challenge. Recently, stacked autoencoder (SAE) has been utilized to develop data-driven soft sensors due to their superior capability in feature extraction. However, the conventional SAE lacks correlation in feature extraction for each autoencoder (AE) as it is independent. Additionally, the training process of AE is unsupervised, ignoring the correlation between input and output data. To address these issues, a novel SAE with a variant structure called deep layers-extended autoencoder (DLEAE) is proposed. In the DLEAE, an extended layer is introduced, and the original inputs and the information of the hidden layers of previous LEAE (Layers-extended autoencoder) units are added to the follow-up LEAE unit. This extends the width of DLEAE and strengthens the utilization of the features extracted from preceding layers. Furthermore, the expected outputs are introduced into the feature extraction process of each LEAE unit, which filters out output-irrelevant information. Finally, experimental verifications using two actual complex industrial processes are performed to validate the effectiveness and superiority of the proposed DLEAE.

Interpretable Prediction Modeling for Froth Flotation via Stacked Graph Convolutional Network

Real-time prediction of key quality variables based on data-driven soft sensor modeling is an important way to monitor flotation status and product quality in the froth flotation process. However, the existing data-driven methods have limitations in terms of nonlinear feature extraction and interpretability. In addition, the prevalent correlations between process variables can help improve the model interpretability and feature extraction of the model, but there are still challenges in exploring the potential correlations between process variables due to the complexity of the industrial process mechanism and the presence of high noise in process data. These relationships can be abstracted as edges between nodes in a graph representation. To fully explore the deep process-related structural features of data, a novel stacked graph convolutional network (S-GCN) is proposed. First, S-GCN aggregates the features of neighboring nodes by an adaptive adjacency matrix to achieve the extraction of structural features among process variables. Then, the labeled quality variable data detected by laboratory assay analysis are utilized to pre-train the network layer by layer to obtain better initial parameters and adjacency matrices for quality prediction. Finally, the prediction model is constructed based on the learned parameters to explore the interpretability of model and perform prediction tasks. Experimental results on real industrial potassium chloride froth flotation process data show a significant improvement in the prediction accuracy and interpretability of the proposed method.