Abstract
This paper proposes an improved most likely heteroscedastic Gaussian process (MLHGP) algorithm to handle a kind of nonlinear regression problems involving input-dependent noise. The improved MLHGP follows the same learning scheme as the current algorithm by use of two Gaussian processes (GPs), with the first GP for recovering the unknown function and the second GP for modeling the input-dependent noise. Unlike the current MLHGP pursuing an empirical estimate of the noise level which is provably biased in most of local noise cases, the improved algorithm gives rise to an approximately unbiased estimate of the input-dependent noise. The approximately unbiased noise estimate is elicited from Bayesian residuals by the method of moments. As a by-product of this improvement, the expectation maximization (EM)-like procedure in the current MLHGP is avoided such that the improved algorithm requires only standard GP learnings to be performed twice. Four benchmark experiments, consisting of two synthetic cases and two real-world datasets, demonstrate that the improved MLHGP algorithm outperforms the current version not only in accuracy and stability, but also in computational efficiency.
Highlights
G AUSSIAN process (GP) has been proven to be a powerful Bayesian nonparametric method for solving nonlinear regression or multi-class classification problems [1]
An improved most likely heteroscedastic Gaussian process (MLHGP) algorithm is proposed to deal with a kind of nonlinear regression problems with inputdependent noise
The improved model follows the same idea in the current MLHGP that adopts a point estimate to replace the full noise posterior distribution
Summary
G AUSSIAN process (GP) has been proven to be a powerful Bayesian nonparametric method for solving nonlinear regression or multi-class classification problems [1]. The most likely heteroscedastic Gaussian process (MLHGP) [14] as a typical representative of the most likely noise approaches, is not guaranteed to converge but rather might oscillate due to empirical estimation of the input-dependent noise. After attesting to the fact that the empirical estimate of the noise level in the current MLHGP is biased for most input-dependent noises, an approximately unbiased noise estimate is proposed based on the method of moments for Bayesian residuals. This refinement in the noise estimate can significantly benefit the most likely noise approaches in algorithmic accuracy and stability when dealing with regression problems with input-dependent noise.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.