Soft Sensor Modeling Method Research Articles

Just-in-time framework for robust soft sensing based on robust variational autoencoder

Modeling with high-dimensional data subject to abnormal observations have always been a practical interest. In this paper, under the just-in-time learning (JITL) framework, a robust soft sensor modeling approach is developed based on robust Variational Autoencoder (VAE). Unlike the vanilla VAE that extracts features from the given dataset under the Gaussian prior assumption, robust VAE employs Student’s t-distribution as prior distribution to handle abnormal data. Under assumption of the Student’s t-prior, the proposed robust VAE model is capable of describing collected data contaminated with outliers. Once the robust VAE model is trained, each robust feature variable in the latent space can be determined. Subsequently, similarity measure is calculated using robust Kullback-Leibler divergence between two Student’s t-distributions, that is, the distribution of a new data sample and that of each historical data sample. After completing similarity measurement for a query sample, the weights for input-output historical data can be determined. Based on these weighted historical data samples, a robust probabilistic principal component regression (PPCR) is utilized to perform local modeling for prediction. Numerical simulations, including the Tennessee Eastman and Penicillin fermentation benchmark processes, are utilized to validate the proposed JITL-based robust soft sensor modeling method.

Soft sensor modeling method and application based on TSECIT2FNN-LSTM

To address the issue of low accuracy in soft sensor modeling of key variables caused by multi-variable coupling and parameter sensitivity in complex processes, this paper introduces a TSK-type-based self-evolving compensatory interval type-2 fuzzy Long short-term memory (LSTM) neural network (TSECIT2FNN-LSTM) soft sensor model. The proposed TSECIT2FNN-LSTM integrates the LSTM neural network with the interval type-2 fuzzy inference system to address long-term dependencies in sequence data by utilizing the gate mechanism of the LSTM neural network. The TSECIT2FNN-LSTM structure learning algorithm uses the firing strength of the network rule antecedent to decide whether to generate new rules to improve the rationality of the network structure. TSECIT2FNN-LSTM parameter learning utilizes the gradient descent method to optimize network parameters. However, unlike other interval type-2 fuzzy neural network gradient calculation processes, the error term in the LSTM node parameter gradient of TSECIT2FNN-LSTM is propagated backwards in the time dimension. Additionally, the error term is simultaneously transferred to the upper layer network to enhance network prediction accuracy and memory capabilities. The TSECIT2FNN-LSTM soft sensor model is utilized to predict the alcohol concentration in wine and the nitrogen oxide emission in gas turbines. Experimental results demonstrate that the proposed TSECIT2FNN-LSTM soft sensing model achieves higher prediction accuracy compared to other models.

Soft sensor modeling based on self-organizing fuzzy neural network with clustering, merging, and splitting scheme

Abstract As a pivotal role in the control, optimization, and monitoring of contemporary industrial processes, soft sensors are frequently employed in the prediction of key quality variables. To achieve accurate prediction of key quality variables in industrial processes, a soft sensor modeling method based on the self-organizing fuzzy neural network with the clustering, merging, and splitting scheme (SOFNN-CMS) is proposed. First, the supervised fuzzy C-means clustering algorithm is proposed to identify the appropriate initial center and width of the fuzzy neural network, obtaining appropriate initial fuzzy rules. Then, a neuron merging and splitting strategy is designed to adjust the structure of the fuzzy neural network, by merging and splitting the hidden neurons according to the distance of clusters, increasing the adaptability of the fuzzy neural network. Besides, to accelerate the convergence of estimation errors, an improved Levenberg Marquardt algorithm is utilized to update neural network parameters in the training phase, realizing the soft sensor modeling of key quality variables. The effectiveness of the proposed SOFNN-CMS neural network is demonstrated on two benchmark problems and an industrial debutanizer column. Finally, the experiments showcase that the proposed SOFNN-CMS neural network can obtain better soft sensor modeling performance with a compact structure.

Graph semi-supervised soft sensor modeling based on label propagation algorithm

Abstract Data-driven soft sensor modeling methods have become prevalent in the industry. Nonetheless, the complexity of industrial processes often leads to the absence or difficulty in obtaining key labeled data, and existing methods frequently fail to fully utilize the inherent correlations between variables. This paper proposes a novel graph semi-supervised soft sensor modeling method using the label propagation algorithm to address these issues. This method utilizes correlations within the data to assign pseudo-labels to unlabeled data reasonably and employs Graph Convolutional Networks (GCN) to capture spatial relationships between nodes. Additionally, by embedding a Long Short-Term Memory (LSTM) structure, the model can capture temporal dependencies of the data while focusing on spatial structures. Furthermore, the introduction of a residual structure enables the model to directly learn the differences between inputs and outputs, facilitating information transmission, and improving the model's feature extraction ability. Experiments demonstrate the effectiveness of the method.

Research on a semi-supervised soft sensor modelling method for complex chemical processes based on INGO-VMD-ESN

Abstract The dynamic and non-linear nature of complex chemical processes often leads to low prediction accuracy of key quality variables by traditional soft sensors, thus affecting the overall system control accuracy and operational efficiency. Therefore, this paper proposes a semi-supervised soft sensor modelling method based on improved the northern goshawk optimization (INGO)-variable mode decomposition (VMD)-echo state network (ESN). Firstly, a new semi-supervised fusion method is proposed to address the problem of model training difficulty due to the scarcity of labelled samples and process dynamics, which reconstructs the sample dataset by fusing labelled and unlabelled samples into more representative new samples, improving the model’s generalization ability. Secondly, for the noise interference present in the reconstructed data, the input data is denoised using the VMD method to improve the quality of data. Then, a soft sensor model is built based on ESN. Additionally, the denoising and prediction performance of VMD and ESN is significantly affected by parameters, therefore the paper utilizes the INGO algorithm to achieve parameter rectification for VMD and ESN. Finally, the method is validated based on actual sulphur recovery data from a refinery. The results demonstrate that the method effectively mitigates the impact of dynamics and nonlinearity in the complex chemical process which enhances prediction accuracy.

Multi-Scale Temporal Convolutional Networks for Effluent COD Prediction in Industrial Wastewater

To identify the complex time patterns in the process data and monitor the effect of wastewater treatment by predicting effluent chemical oxygen demand more accurately, a soft-sensor modeling method based on the multi-scale temporal convolutional network (MSTCN) was proposed in this paper. Data at different time scales are reconstructed according to the main frequencies determined by the Fourier transform approach, and the correlations between variables during that period are calculated and stored in the corresponding adjacency matrix. The specific temporal convolutional network (TCN) is built to learn the temporal dependencies within each sequence at the current scale, while the graph convolutional layer (GCN) captures the relationships among variables. Finally, predictions with less error can be obtained by integrating output features from GCN and TCN layers. The proposed model is validated on an annual dataset collected from a wastewater treatment plant employing biological processes for organic matter removal. The experimental results indicate that the proposed MSTCN reduces RMSE by 35.71% and 22.56% compared with the convolutional neural network and TCN, respectively. Moreover, MSCTN shortens the training period by 6.3 s and improves RMSE by 30.41% when compared to the long short-term memory network, which is outperformed in extracting temporal dynamic characteristics.

Research on semi-supervised soft sensor modeling method for sulfur recovery unit based on ISSA-VMD-ESN

Sulfur recovery unit (SRU) in the refining industry can effectively reduce sulfide emissions and achieve efficient use of resources. Predicting sulfide emissions with soft sensors can provide the basis for process optimization and control, to improve SRU’s stability and efficiency. However, due to the dynamicity and nonlinearity of the sulfur recovery process, existing soft sensor models have poor prediction accuracy. Therefore, this paper proposes a semi-supervised soft sensor modeling method based on variational mode decomposition (VMD), echo state network (ESN) and improved sparrow search algorithm (ISSA). Firstly, Reconstructing SRU dataset using semi-supervised fusion method to reduce the impact of dynamicity on prediction results. Secondly, through the ISSA to optimize the VMD parameters and using the optimized VMD to decompose the output sequences into multiple components, to reduce the nonlinearity of sequences. Then, the soft sensor modeling of each component using ESN. Additionally, the ESN parameters are optimized by ISSA to prevent the effect of improper parameter settings on the model’s prediction performance. Afterwards, the predicted values of each component are superimposed to obtain the final prediction. Finally, simulation experiments are conducted based on the SRU dataset of an oil refinery to verify the effectiveness and generalization of the proposed method.

Laplacian regularization of linear regression model for semi-supervised industrial soft sensor development

With more and more data collected from modern industrial processes, data-based soft sensor modeling methods have become popular for online prediction of those difficult-to-measure key variables. As one of the most widely used methods, the linear regression model has been studied for many years. In this paper, a new semi-supervised form of data-based soft sensor is developed upon this simple but commonly used linear regression model, in order to make it more engineering applicable. Particularly, a semi-supervised model structure is formulated by introducing a Laplacian regularization term, through which a lot more unlabeled data samples can be well incorporated for modeling. With an improved estimation for the input data distribution, the real data features can be better captured and thus the model parameters are well restricted, in order to provide a better performance. In addition, the basic Laplacian regularized linear regression model is further extended to nonlinear counterparts by the introductions of basis function and kernel trick in the input data space. For performance evaluation, two industrial case studies are provided, based on which both feasibility and effectiveness of the developed method are confirmed.

Research on soft sensor modeling method for complex chemical processes based on local semi-supervised selective ensemble learning

To address the adverse effects of nonlinearity and dynamic time-varying in complex chemical processes on the accuracy of the soft sensor model, a local-semi-supervised ensemble learning for soft sensor modeling (local semi-supervised-selective ensemble learning-long short term memory, LS-SEL-LSTM) method is proposed in this article. Firstly, a hierarchical clustering method incorporating spatiotemporal criteria is proposed to reduce the influence of nonlinearity in global model prediction accuracy. The method considers the dynamic time-varying characteristics of temporal data and generates multiple local datasets. Then, to address the issue of multi-rate between auxiliary variables and dominant variables, a semi-supervised weight fusion mechanism based on temporal correlation is proposed, which effectively utilizes auxiliary variables to reconstruct local semi-supervised datasets and establishes local soft sensing models using LSTM. Concurrently, the parameters of the established model were optimized using the flower pollination algorithm. Subsequently, a selective ensemble learning method based on sub-model prediction accuracy and an adaptive combination weight calculation method for sub-models were proposed to improve the prediction accuracy. Finally, the effectiveness of the proposed method was verified through the actual dataset of the sulfur recovery process. The results indicate that LS-SEL-LSTM performs well in handling complex chemical processes with nonlinear and dynamic time-varying characteristics.

Transformer enhanced by local perception self-attention for dynamic soft sensor modeling of industrial processes

The use of dynamic soft sensor modeling methods to mine the time-varying and dynamic characteristics of industrial process data is of great significance for improving production efficiency and quality, given the rapid development of industrial processes and the increasing prominence of dynamic changes in the production process. However, existing dynamic soft sensor methods have limited long-term memory capacity, making it difficult to capture the long dynamic dependence, which can severely affect the results of the soft sensor model. To address this issue, we propose a dynamic soft sensor model based on local perception transformer, where the transformer is applied to achieve global perception of the variables. Through the application of the self-attention mechanism in the transformer encoder, the dynamic tracking and prediction of parameters can be realized by assigning different weights to the process variables and quality variables at different time steps, thereby adapting to the time-varying nature of the process. Additionally, convolution is used to generate a Query and Key in the self-attention mechanism, thereby enhancing local information learning. The proposed dot product self-attention calculation method effectively utilizes local information, thereby reducing the potential impact of abnormal data at a certain moment. Furthermore, by utilizing LSTM to extract time series information, the final prediction result was obtained. In soft sensor modeling experiments of the sulfur recovery unit and debutanizer tower, our proposed model demonstrated higher prediction accuracy compared to other methods, such as SVR, MLP, LSTM, CNN + LSTM, and vanilla transformer.

A Soft Sensor for Multirate Quality Variables Based on MC-CNN.

In recent years, data-driven soft sensor modeling methods have been widely used in industrial production, chemistry, and biochemical. In industrial processes, the sampling rates of quality variables are always lower than those of process variables. Meanwhile, the sampling rates among quality variables are also different. However, few multi-input multi-output (MIMO) sensors take this temporal factor into consideration. To solve this problem, a deep-learning (DL) model based on a multitemporal channels convolutional neural network (MC-CNN) is proposed. In the MC-CNN, the network consists of two parts: the shared network used to extract the temporal feature and the parallel prediction network used to predict each quality variable. The modified BP algorithm makes the blank values generated at unsampled moments not participate in the backpropagation (BP) process during training. By predicting multiple quality variables of two industrial cases, the effectiveness of the proposed method is verified.

Dynamic nonlinear soft sensor modelling method using linear slow feature analysis and least squares support vector regression for batch processes

AbstractData‐driven soft sensor models have been extensively utilized in industrial processes. Batch processes are usually employed to manufacture low‐volume and high value‐added products in chemical, materials, and pharmaceutical industries. The most distinctive features of batch process lie in nonlinear, repetition, and slow time varying characteristics. In this paper, a data‐driven soft sensor modelling method based on linear slow feature analysis (LSFA) and least squares support vector regression (LSSVR) is proposed. In this method, LSFA was used to effectively capture the driving force behind the data features that change as slowly as possible. Then, a LSSVR model was constructed between the extracted slow feature variables and quality variables. Finally, a numerical example, industrial penicillin fermentation processes, and cobalt oxalate synthesis process were utilized to confirm the prediction accuracy and model reliability of the proposed approach.

A modeling method of wide random forest multi-output soft sensor with attention mechanism for quality prediction of complex industrial processes

Complex industrial production processes often involve multiple product quality indicators that are interrelated. There exists a complex nonlinear mapping relationship between the operational input feature variables and multiple output target quality variables, making it difficult to accurately model through first-principle models. In order to fully capture the complex relationship between measurable variables and difficult-to-measure quality variables, and achieve accurate prediction of multiple output variables to meet the needs of practical industrial sites, this paper proposes a broad random forest-based multi-output soft sensor modeling method based on the idea of attention mechanism derived from the concept of broad learning systems. This method comprehensively considers the dynamic impact of different feature variables on the target quality indicators in actual production processes. The attention mechanism assists the soft sensor model in capturing contextual information better when dealing with long sequences, with a focus on the relevant parts related to the current task. Additionally, the interpretable random forest algorithm is employed as the weight estimator for the basic feature learning unit of Broad-based learning, enabling regression modeling of multiple target quality variables. The use of Broad-based random forest improves the model’s learning ability, interpretability, and generalization capability. To validate the reliability of the proposed method, it was applied to real industrial cases. The results demonstrated that the multi-output quality variable prediction performance of the proposed soft sensor outperforms existing soft sensors in terms of prediction accuracy. This indicates promising industrial application prospects for the proposed method.

Soft Sensor Modeling Method for the Marine Lysozyme Fermentation Process Based on ISOA-GPR Weighted Ensemble Learning.

Due to the highly nonlinear, multi-stage, and time-varying characteristics of the marine lysozyme fermentation process, the global soft sensor models established using traditional single modeling methods cannot describe the dynamic characteristics of the entire fermentation process. Therefore, this study proposes a weighted ensemble learning soft sensor modeling method based on an improved seagull optimization algorithm (ISOA) and Gaussian process regression (GPR). First, an improved density peak clustering algorithm (ADPC) was used to divide the sample dataset into multiple local sample subsets. Second, an improved seagull optimization algorithm was used to optimize and transform the Gaussian process regression model, and a sub-prediction model was established. Finally, the fusion strategy was determined according to the connectivity between the test samples and local sample subsets. The proposed soft sensor model was applied to the prediction of key biochemical parameters of the marine lysozyme fermentation process. The simulation results show that the proposed soft sensor model can effectively predict the key biochemical parameters with relatively small prediction errors in the case of limited training data. According to the results, this model can be expanded to the soft sensor prediction applications in general nonlinear systems.

An soft-sensor method for the biochemical reaction process based on LSTM and transfer learning

Due to significant differences in data distribution under different working conditions during Pichia pastoris biochemical reaction process, traditional soft-sensor model suffer from the model failure and deterioration, this paper propose a soft-sensor modeling method combing long short-term memory network (LSTM) and balanced distribution adaptation method (BDA). Firstly, the source domain data is used to establish an accurate source domain LSTM prediction model, and the structure and parameters of the first layer of LSTM are fixed to migrate to the target domain prediction model. Then use the balanced distribution adaptation method to shrink the distribution differences between different domains of data. Finally, data that has been modeled with balanced and adaptive distribution assist the real-time data to train the remaining layer of the network, and the accurate target domain prediction model is finally obtained. The simulation results show that the mentioned method has the preponderance of timely prediction and high prediction accuracy, which validates the effectiveness and practicality of the method. This method solves the problem of soft-sensor modeling under unknown modes of multiple operating conditions in Pichia pastoris biochemical reaction process, achieving the prediction of key parameters under different operating conditions, which can be widely applied, and also providing a new method for soft-sensor modeling of other non system systems.

Gaussian mixture model sample selection strategy–based active semi-supervised soft sensor for industrial processes

Soft sensors have become reliable tools for estimating difficult-to-measure target variables in modern industrial processes. In order to make full use of labeled and unlabeled samples, an active semi-supervised soft sensor modeling method is proposed, which combines active learning and semi-supervised learning to maximize model performance and minimize the laboratory analysis cost of expanding the labeled sample data set. First, manifold regularization is introduced into the deep extreme learning machine (DELM) algorithm to form a semi-supervised DELM that improves the performance of a model trained with unlabeled samples. Then, considering non-Gaussian processes and the error information between the predicted and true values, an active sample selection strategy based on error Gaussian mixture model is developed. Using this strategy, the most uncertain and representative unlabeled samples are selected for labeling, and thereby expanding the labeled sample data set. Finally, the effectiveness of the proposed method is verified using industrial debutanizer process data.

A two-phase soft sensor modeling framework for quality prediction in industrial processes with missing data

Quality prediction plays an essential role in modern process industries to improve product quality, ensure production stability, and increase economic efficiency. The high-dimensional, nonlinear, and dynamic characteristics of process variables make it hard for the traditional soft sensor modeling methods. Meanwhile, high-quality data is particularly scarce due to the economic and technological constraints, making it easy to overfit and crash of the model training. In order to solve those problems, a two-phase soft sensor modeling framework based on time series generative adversarial network (TimeGAN) and minimal gated unit (MGU) is proposed. The first phase is performing data augmentation for small samples by TimeGAN to improve the generalization performance of model training, and the second phase uses MGU for soft sensor modeling with the purpose of quality prediction. Finally, two datasets with different thicknesses for the hot rolling process (HRP) are used to verify the accuracy and validity of the proposed scheme.

Development and Optimization of a Novel Soft Sensor Modeling Method for Fermentation Process of Pichia pastoris.

This paper introduces a novel soft sensor modeling method based on BDA-IPSO-LSSVM designed to address the issue of model failure caused by varying fermentation data distributions resulting from different operating conditions during the fermentation of different batches of Pichia pastoris. First, the problem of significant differences in data distribution among different batches of the fermentation process is addressed by adopting the balanced distribution adaptation (BDA) method from transfer learning. This method reduces the data distribution differences among batches of the fermentation process, while the fuzzy set concept is employed to improve the BDA method by transforming the classification problem into a regression prediction problem for the fermentation process. Second, the soft sensor model for the fermentation process is developed using the least squares support vector machine (LSSVM). The model parameters are optimized by an improved particle swarm optimization (IPSO) algorithm based on individual differences. Finally, the data obtained from the Pichia pastoris fermentation experiment are used for simulation, and the developed soft sensor model is applied to predict the cell concentration and product concentration during the fermentation process of Pichia pastoris. Simulation results demonstrate that the IPSO algorithm has good convergence performance and optimization performance compared with other algorithms. The improved BDA algorithm can make the soft sensor model adapt to different operating conditions, and the proposed soft sensor method outperforms existing methods, exhibiting higher prediction accuracy and the ability to accurately predict the fermentation process of Pichia pastoris under different operating conditions.

A novel semi-supervised soft sensor modeling method based on deep dynamic and semantic information extraction for concentrate grade prediction in froth flotation

With the rapid development of machine vision technology, this technology is being widely used in grade prediction for the froth flotation process. Several previously proposed froth visual features based methods to predict grade have yielded promising results, but some problems still exist, i.e., limited labeled data and complex data description. This can make it difficult for models to learn valid data representation and thus reduce prediction accuracy. To address the above-mentioned problem, a deep learning based semi-supervised soft sensor model, integrating dynamic and semantic information, is proposed to predict the concentrate grade. More specifically, two autoencoders, Stacked autoencoder (SAE) and Long Short-term Memory autoencoder (LSTM-AE), are pre-trained without supervision to respectively extract the deep semantic and dynamic temporal behavior hidden in all available input data (labeled and unlabeled data). Then, the representation vectors obtained from the two autoencoders transferred are concatenated to train a specific-task model on top using the limited training data. In addition, a self-evaluation module is introduced for tracking the predictive performance of our proposed model. Finally, a real-world antimony concentrate flotation process is used to validate our proposed method. Computational results show that our model eliminates the need to get large amounts of labeled data, and leverages unlabeled data, which is superior to traditional statistical models.

Diversified Kernel Latent Variable Space and Multi-Objective Optimization for Selective Ensemble Learning-Based Soft Sensor

The improvement of data-driven soft sensor modeling methods and techniques for the industrial process has strongly promoted the development of the intelligent process industry. Among them, ensemble learning is an excellent modeling framework. Accuracy and diversity are two key factors that run through the entire stage of building an ensemble learning-based soft sensor. Existing base model generating methods or ensemble pruning methods always consider the two factors separately, which has limited the development of high-performance but low-complexity soft sensors. To work out this issue, a selective ensemble learning-based soft sensor modeling method based on multi-kernel latent variable space and evolutionary multi-objective optimization is proposed, referred to as MOSE-MLV-VSPLS. This method designs a multiple diversity enhancement mechanism in the base model generation stage. Diversified input variable subspaces are first constructed using the maximum information coefficient on the bootstrapping random resampling subset. Then a set of base models that combine accuracy and diversity are generated on supervised latent variable subspaces under multiple kernel function perturbations. Further, two quantifiable parameters are designed for accuracy and diversity, and the multi-objective gray wolf optimization algorithm is used to select the base models that maximize these two important parameters to achieve effective ensemble pruning at the model ensemble stage. The MOSE-MLV-VSPLS method is applied to two typical industry processes, and the experimental results show that the method is effective and superior in selective ensemble-based soft sensor modeling.