Robust Gaussian process modeling using EM algorithm

Abstract

Gaussian process (GP) regression is a fully probabilistic method for performing non-linear regression. In a Bayesian framework, regression models can be made robust by using heavy-tailed distributions instead of using normal distribution for modeling noise. This work focuses on estimation of parameters for robust GP regression. In literature, these are learned by maximizing the approximate marginal likelihood of data. However, gradient-based optimization algorithms which are used for this purpose can be unstable or may require tuning. In this work, an EM algorithm based approach is derived and implemented to infer the parameters. The pros and cons of the two approaches are analyzed. The advantage of EM algorithm lies in its ease of implementation and theoretical guarantees of numerical stability and convergence while its prediction performance is still comparable to gradient-based approaches. In some cases EM algorithm may be slow to converge. To circumvent this issue a faster EM based approach known as Expectation Conjugate Gradient (ECG) is implemented on robust GP regression. Finally, the proposed EM approach to robust GP regression is validated using an industrial data set.