A Just-in-Time Fine-Tuning Framework for Deep Learning of SAE in Adaptive Data-Driven Modeling of Time-Varying Industrial Processes

Abstract

In modern industrial processes, soft sensors have played increasingly important roles for effective process monitoring, control and optimization. Deep learning has shown excellent ability for hierarchical nonlinear feature representation in soft sensors. However, the existing deep learning based soft sensors are mostly trained offline and applied online without updating mechanism. This may cause their performance degradation in time-varying processes. To deal with this problem, an adaptive updating framework is proposed for deep learning, which is based on just-in-time fine-tuning of stacked autoencoder (JIT-SAE). In JIT-SAE, an offline SAE model is first trained with layer-wise unsupervised pre-training and supervised fine tuning. For online prediction, the network is dynamically fine-tuned upon the query sample. For each query sample, the most relevant labeled samples are selected to form a fine-tuning dataset from the historical labeled database, which is regularly augmented once new labeled samples are available from laboratory analysis. Moreover, each relevant sample is assigned with a weight according to its similarity with the query sample. Then, the deep network is fine-tuned with these relevant labeled samples by designing a weighted loss function. Thus, JIT-SAE is able to track the newest process running state timely and match the data pattern accurately. Case study on an industrial hydrocracking process is provided to demonstrate the effectiveness of the JIT-SAE framework.