6 - Gaussian process: The function space view

Abstract

Gaussian Processes (GPs) are non-parametric Bayesian methods that can be used for classification, regression, and optimization problems alike. They help to model an arbitrary number of random variables (potentially capable of modeling infinitely many random variables but this situation almost never arises, given the data observed and made available is always finite, albeit huge) by placing non-parametric functional distributions over them to select the best fit function that describes the data well. This chapter serves as an introduction to GPs whilst formalizing the computational form of GPs and highlighting a variety of jargons that exist in the literature associated with GPs. This chapter primarily focuses on the formalization of the regression capabilities of GPs, although they can be very well used for other problems like classification, etc., as mentioned earlier. The aim here is to describe both the upside and downside of GPs and discuss why the GPs are a common model of choice of complex regression problems.