Semi-supervised echo state network with partial correlation pruning for time-series variables prediction in industrial processes

Abstract

For an ordinary echo state network (ESN), redundant information in the huge reservoir will lead to degradation of the prediction performance of the network, especially when the labels of the samples are limited. To solve this problem, a semi-supervised ESN with partial correlation pruning (PCP-S2ESN) is proposed in this paper to scientifically capture the essential association between two reservoir variables while controlling for the influence of other factors. In this way, redundant neurons and their connection weights in the reservoir are eliminated, so that the prediction accuracy is significantly enhanced by optimizing the network structure. Moreover, an unsupervised pre-training procedure is introduced to modify the input weight matrix and reservoir connection weight matrix of the ESN, which successfully achieves precise prediction of time-series variables with limited labels. The superiority of the PCP-S2ESN model is demonstrated through two benchmark prediction tasks and the fed-batch penicillin cultivation process.