A weighted heteroscedastic Gaussian Process Modelling via particle swarm optimization

Abstract

In many chemical engineering applications, it is often difficult to get accurate first-principle models because of complexity of modern processes. Even if it is possible to do so, it is often time consuming and computationally expensive. Hence, there is a growing need to develop data-driven models. Gaussian process regression (GPR) model has been extensively applied in data-based modelling due to its good adaptability to deal with high dimensional, small samples, and nonlinear problems. The standard GPR algorithm assumes constant noise power throughout the sampling process. However, in process systems, the observation noise often varies so that different sample points are corrupted by different degrees of noise. Under these circumstances, the standard GPR algorithm may not work properly. To model Gaussian process with heteroscedastic noise, this paper introduces a weighting strategy into the standard GPR algorithm, and proposes three weighted GPR algorithms: the clustered GPR (C-GPR) algorithm, the partial weighted GPR (PW-GPR) algorithm and the weighted GPR (W-GPR) algorithm. Different from the standard GPR algorithm, three weighted algorithms put the weight on sampled data by calculating the noise variance for each data point. In addition, in order to optimize the proposed algorithms, this paper utilizes the particle swarm optimization (PSO) algorithm to estimate hyper-parameters of the GPR model, instead of using the traditional conjugate gradient (CG) method. The effectiveness of the three weighted GPR algorithms is verified by means of two numerical examples and a wet spinning coagulation process. Extensive simulation results demonstrate that the proposed algorithms optimized by the PSO algorithm can improve prediction accuracy of the GPR model.