Just-in-time Learning Methods Research Articles

Just-in-time learning method based on two kinds of local samples combined with two-stage training parallel learner for nonlinear chemical process soft sensing

Just-in-time Learning (JITL) is a soft sensor method to develop the corresponding local model for each query sample, which has strong adaptability. Aiming at the lack of feature information of dynamic law of chemical processes in similar samples, a JITL method based on two kinds of local samples combined with two-stage training parallel learner (JITLTKLS-TSTPL) was proposed to address this issue. Firstly, similarity measurement and moving window are used to select local samples respectively. And the length of two kinds of samples is optimized by genetic algorithm. Then, the proposed two-stage training parallel learner is proposed to learn local nonlinear features of similar samples and dynamic trend information of dynamic samples by cross-freezing network weights. Finally, the JITL model was verified by three chemical processes. The results show that the total error and maximum error of JITLTKLS-TSTPL are the smallest compared with the JITL method using a single sample for modeling. In the public data set, compared with other research methods, the R2 of JITLTKLS-TSTPL was the highest, reaching 0.983.

Integration of just-in-time learning with variational autoencoder for cell culture process monitoring based on Raman spectroscopy.

Protein production in the biopharmaceutical industry necessitates the utilization of multiple analytical techniques and control methodologies to ensure both safety and consistency. To facilitate real-time monitoring and control of cell culture processes, Raman spectroscopy has emerged as a versatile analytical technology. This technique, categorized as a Process Analytical Technology, employs chemometric models to establish correlations between Raman signals and key variables of interest. One notable approach for achieving real-time monitoring is through the application of just-in-time learning (JITL), an industrial soft sensor modeling technique that utilizes Raman signals to estimate process variables promptly. The conventional Raman-based JITL method relies on the K-nearest neighbor (KNN) algorithm with Euclidean distance as the similarity measure. However, it falls short of addressing the impact of data uncertainties. To rectify this limitation, this study endeavors to integrate JITL with a variational autoencoder (VAE). This integration aims to extract dominant Raman features in a nonlinear fashion, which are expressed as multivariate Gaussian distributions. Three experimental runs using different cell lines were chosen to compare the performance of the proposed algorithm with commonly utilized methods in the literature. The findings indicate that the VAE-JITL approach consistently outperforms partial least squares, convolutional neural network, and JITL with KNN similarity measure in accurately predicting key process variables.

A unified just-in-time learning paradigm and its application to adaptive soft sensing for nonlinear and time-varying chemical process

In traditional just-in-time learning (JITL)-based adaptive soft sensors for complex chemical process, relevant sample selection and local model construction are carried out separately, resulting in that the developed predictive models are usually lack of optimality and validity. This has long been a tough issue for the JITL-based soft sensors. Therefore, this paper proposes a novel unified JITL paradigm based on representation learning and ridge regression. In the UniJITL, the tasks of relevant sample selection and local expert construction are integrated into a unified framework, where the two tasks are cooperatively considered and designed into one optimization problem. Moreover, an efficient solution to the optimization problem with guaranteed convergence is developed. The performance of the UniJITL is evaluated using two real-life chemical processes, and is compared with the state-of-the-art JITL methods, which effectively demonstrate both the predictive advantage and computational feasibility of the UniJITL.

Rethinking the Value of Just-in-Time Learning in the Era of Industrial Big Data

Just-in-time learning (JITL) has become a widely used industrial process modeling tool. With the advent of the industrial big data era, rich data information has brought new opportunities to JITL. Specifically, the completeness of data samples in the era of big data provides an important premise and support for the JITL method, prompting us to rethink the application value of JITL in the context of industrial big data. At the same time, the huge amount of data causes certain difficulties in data searching, which is a key issue for JITL. In this article, a parallel computing strategy is adopted to divide the entire computational searching task into several subtasks, and assign the subtasks to parallel computing nodes to complete parallel searching. In this way, not only the full advantage of big data information is effectively utilized, but also the search capability and efficiency under big data are improved. In addition, in order to improve the real-time nature of JITL, a model library management (MLM) strategy is adopted, and the query similar samples are used to operate with existing similar models. And by selectively adding new data, the database management (DBM) strategy is also developed, which not only alleviates the problem of information redundancy, but also reduces the search pressure caused by the increasingly large database. Obviously, MLM and DBM are particularly important as JITL auxiliary tools under industrial big data. Combining parallel computing, a parallel JITL (P-JITL) framework is proposed. As an example, the variational Bayesian factor regression model is transformed into the parallel Bayesian-JITL method for big process data modeling, which is further extended to a nonlinear form. To evaluate the feasibility and efficiency of the developed methods, a real industrial case is demonstrated.

Research on Prediction Method of Hydraulic Pump Remaining Useful Life Based on KPCA and JITL

Hydraulic pumps are commonly used; however, it is difficult to predict their remaining useful life (RUL) effectively. A new method based on kernel principal component analysis (KPCA) and the just in time learning (JITL) method was proposed to solve this problem. First, as the research object, the non-substitute time tac-tail life experiment pressure signals of gear pumps were collected. Following the removal and denoising of the DC component of the pressure signals by the wavelet packet method, multiple characteristic indices were extracted. Subsequently, the KPCA method was used to calculate the weighted fusion of the selected feature indices. Then the state evaluation indices were extracted to characterize the performance degradation of the gear pumps. Finally, an RUL prediction method based on the k-vector nearest neighbor (k-VNN) and JITL methods was proposed. The k-VNN method refers to both the Euclidean distance and angle relationship between two vectors as the basis for modeling. The prediction results verified the feasibility and effectiveness of the proposed method. Compared to the traditional JITL RUL prediction method based on the k-nearest neighbor algorithm, the proposed prediction model of the RUL of a gear pump presents a higher prediction accuracy. The method proposed in this paper is expected to be applied to the RUL prediction and condition monitoring and has broad application prospects and wide applicability.

An Improved JITL Method for Soft Sensing of Multimodal Industrial Processes for Search Efficiency

In the industrial process, due to product change, working condition switch, or controller adjustment, process data often presents multimodal characteristics. Data-driven approaches are often based on single-modal assumptions, which may fail to describe process characteristics. The traditional just-in-time learning (JITL) method can continuously update the model to describe the multimodal data, but it takes much time and cannot meet the real-time requirements. In this paper, an improved JITL method is proposed to find similar samples quickly. The new samples are divided into the main category first, and then find the similar samples to improve the search efficiency. The effectiveness of the method is proved by a case of an industrial soft sensor case combined with partial least squares (PLS). Compared with the basic JITL, the root mean square error (RMSE) of the proposed method is reduced by 0.09, and the running speed is increased by 8.8 times.

A Novel Just-in-Time Learning Strategy for Soft Sensing with Improved Similarity Measure Based on Mutual Information and PLS.

In modern industrial process control, just-in-time learning (JITL)-based soft sensors have been widely applied. An accurate similarity measure is crucial in JITL-based soft sensor modeling since it is not only the basis for selecting the nearest neighbor samples but also determines sample weights. In recent years, JITL similarity measure methods have been greatly enriched, including methods based on Euclidean distance, weighted Euclidean distance, correlation, etc. However, due to the different influence of input variables on output, the complex nonlinear relationship between input and output, the collinearity between input variables, and other complex factors, the above similarity measure methods may become inaccurate. In this paper, a new similarity measure method is proposed by combining mutual information (MI) and partial least squares (PLS). A two-stage calculation framework, including a training stage and a prediction stage, was designed in this study to reduce the online computational burden. In the prediction stage, to establish the local model, an improved locally weighted PLS (LWPLS) with variables and samples double-weighted was adopted. The above operations constitute a novel JITL modeling strategy, which is named MI-PLS-LWPLS. By comparison with other related JITL methods, the effectiveness of the MI-PLS-LWPLS method was verified through case studies on both a synthetic Friedman dataset and a real industrial dataset.

Soft sensor design using transductive moving window learner

Changes in characteristics of the industrial processes require implementation of adaptive mechanisms in soft sensors. In the current study, we propose combining two common adaptation methods, moving window (MW) and Just-In-Time-Learning (JITL), using transductive inference (MWtr). Transductive learning exploits the knowledge from the feature values of the query points in determining predictions of the response variable at these points. Here, we use JITL predictions for obtaining predictions for the query points, which are used in training the MW learner in a weighted Lasso regression setting. Tests of the proposed method on three publicly available industrial real datasets show that prediction accuracy of MWtr is higher than both MW, and JITL models, and MWtr is able to combat against various types of concept drifts via integrating the capabilities of MW and JITL methods. Ease of implementation, robustness and stability render MWtr a convenient adaptive method in industrial soft sensor design.

Soft Sensor Modelling based on Just-in-Time Learning and Bagging-PLS for Fermentation Processes

For process modelling of complex and dynamic industrial processes, it has been proven that just-in-time learning (JITL) method has some advantages. The key to success of JITL modelling is how to select appropriate relevant samples for modelling. However, the size of most relevant samples selected by using traditional similarity indexes is difficult to be determined and still needs further research. To handle the problem, a novel relevant sample selection method is proposed in this work by constructing a similarity index based on Gaussian kernel function. In the method, confidence value is used to replace traditional error limit as a threshold value for relevant sample selection. Meanwhile, the bagging strategy is used to resample samples from selected samples to avoid setting the size of selected samples. In this work, the common modelling method on industrial processes, partial least square (PLS), is used to build regression models. To verify the proposed method, a mathematical model of a nonlinear system and the benchmark of a Penicillin fermentation process simulation were both studied. Results show that compared to traditional JITL methods, the proposed method has significant advantages in terms of the robustness and prediction precision of the model.

An adaptive 2DoF P-PI controller based on an improved just-in-time learning technique for ultra-low-velocity linear stages driven by PMLSMs

In this paper, an adaptive two-degree-of-freedom (2DoF) proportional/proportional-integral (P-PI) controller is presented for ultra-precise linear motion stages driven by permanent magnet linear synchronous motors (PMLSMs) under ultra-low-velocity conditions. The adaptive 2DoF P-PI controller is designed based on reliable position output predictions obtained via just-in-time learning (JITL). To implement the JITL method, the current velocity commands, current position outputs and former position outputs are employed to formulate the JITL model. To guarantee high real-time performance and high prediction accuracy, the JITL method with an improved database (JITLIDB) is presented. Through comparative experiments, JITLIDB is shown to effectively reduce computation time by virtue of its use of a layered database structure to decrease the computational burden and a forgetting factor to prevent the JITL database from infinitely increasing in size. Based on the JITL predictions, a 2DoF P-PI parameter updating algorithm is designed, which is proven to guarantee that the system position errors converge to zero using the Lyapunov method. Through ultra-low-velocity simulations using the proposed adaptive 2DoF P-PI controller based on JITLIDB, it is shown that the proposed controller achieves satisfying performance. These simulations correspond to experiments performed on an ultra-precision air-bearing linear stage driven by a PMLSM. Finally, comparative experiments are presented, in which the proposed adaptive 2DoF P-PI controller based on JITLIDB achieves superior ultra-low-velocity error performance compared with a traditional 2DoF P-PI controller with fixed parameters.

Robust Just-in-time Learning Approach and Its Application on Fault Detection

In data-driven state estimation and process monitoring, the correctness of results mainly rely on the accuracy of measurement. Actually, noises, outliers and measured errors always exist in real industrial systems. Just-in-time learning (JITL) is an useful on-line learning method and can be applied for data-based state estimation. Due to the reality of inaccurate measurement, an improved JITL method with strengthen robustness is necessary to be studied. In this paper, a robust version of just-in-time learning strategy is proposed. It is inspired from the leverage weight. By calculating the leverage impact, the outliers in high leverage cases are treated to reduce their weight and affect less on output prediction. A typical nonlinear system experiment is employed to prove the robust and veracity of the proposed strategy. Finally, the robust JITL is implemented for fault detection on a three-tank system to verify its applicability.

A Probabilistic Just-in-Time Learning Framework for Soft Sensor Development With Missing Data

Just-in-time learning (JITL) is one of the most widely used strategies for soft sensor modeling in nonlinear processes. However, traditional JITL methods have difficulty in dealing with data samples that contain missing values. Meanwhile, data noises and uncertainties have not been taken into consideration for relevant sample selection in existing JITL approaches. To overcome these problems, a new probabilistic JITL (P-JITL) framework is proposed in this brief. In P-JITL, variational Bayesian principal component analysis is first utilized to handle missing values and extract Gaussian posterior distributions of latent variables. Then, symmetric Kullback-Leibler divergence is creatively employed to measure the dissimilarity of two distributions for relevant sample selection in the JITL framework. Finally, a nonlinear regression model, Gaussian process regression, is carried out to model the nonlinear relationship between the output and the extracted latent variables. In this way, the proposed probabilistic JITL (P-JITL) is able to deal with missing data and select relevant samples more accurately. To evaluate the effectiveness and flexibility of P-JITL, comparative studies between P-JITL and traditional deterministic JITL (D-JITL) are carried out on a numerical example and an industrial application example, in which missing data are simulated with percentages from 0% to 50%. The results show that P-JITL can provide more accurate prediction accuracy than D-JITL in each scenario considered.

Just-in-Time-Learning based Extended Prediction Self-Adaptive Control for batch processes

This article presents a new Extended Prediction Self-Adaptive Control (EPSAC) algorithm based on the Just-in-Time Learning (JITL) method. In the proposed JITL-based EPSAC design, linearization of the process model is achieved by a set of local state-space models, each of which can be independently and simultaneously identified by the JITL method along the base trajectory. For the end-product quality control for a simulated semi-batch pH-shift reactive crystallization process where shrinking prediction and control horizons are essential, the proposed EPSAC algorithm not only simplifies the control weight tuning but also provides better and more robust closed-loop control performance than its previous counterpart.

Adaptive Gaussian Mixture Model-Based Relevant Sample Selection for JITL Soft Sensor Development

Just-in-time learning (JITL) has recently been used for online soft sensor modeling. Unlike traditional global approaches, the JITL-based method employs a local model built from historical samples similar to a query sample so that both nonlinearities and changes in process characteristics can be handled well. A key issue in JITL is to establish a suitable similarity criterion for selecting relevant samples. Conventional JITL methods, which use distance-based similarity measures for local modeling, can be inappropriate for many industrial processes exhibiting time-varying and non-Gaussian behaviors. In this article, a GMM-based similarity measure is proposed to improve the prediction accuracy of a JITL soft sensor. By taking the non-Gaussianity of the process data and the characteristics of the query sample into account, a more suitable similarity criterion is defined for sample selection of a JITL soft sensor, and better modeling performance can be achieved. Case studies involving a numerical example and ...

Adaptive speed control based on just-in-time learning technique for permanent magnet synchronous linear motor

In this paper, an adaptive two degrees of freedom (2Dof) PI controller based on a just-in-time learning (JITL) method is proposed for predictive speed control of permanent magnet synchronous linear motor (PMSLM). Firstly, to guarantee the high identification accuracy and high real-time performance simultaneously, an improved JITL method is proposed to estimate the controlled model parameters of speed control system. Then, based on the dynamic controlled model, a simplified generalized predictive control (GPC) supplies a 2Dof proportional integral (PI) controller with suitable control parameters to follow a sinusoid-type speed command in operating conditions. The main motivation of this paper is the extension of the predictive controller to replace traditional PI controller in industrial applications. Finally, the efficacy and usefulness of the proposed controller are verified through the experimental results.