Gaussian Process Regression for Transportation System Estimation and Prediction Problems: The Deformation and a Hat Kernel

Abstract

Gaussian process regression (GPR) is an emerging machine learning model with potential in a wide range of transportation system estimation and prediction problems, especially those where the uncertainty of estimation needs to be measured, for instance, traffic flow analysis, the transportation infrastructure performance estimation problems and transportation simulation-based optimization problems. The kernel function is the core component of GPR, and the radial basis function (RBF) kernel is the most commonly used one, suitable for tasks without special knowledge about the patterns of data, like trend and periodicity. However, an inappropriate hyperparameter of the kernel function may lead to over-fitting or under-fitting of GPR. During hyperparameter optimization, the usage of the RBF kernel often suffers from the issue of failing to find the optimal hyperparameter. This paper aims to address this problem by promoting the use of the hat kernel, which can reduce the risk of under-fitting. Moreover, we propose the notion of deformation, corresponding to severe over-fitting of a GPR. To further address this issue, we investigate the connection between deformation and the Bayesian generalization error of GPR. Two lower bounds for the hyperparameter of the hat kernel are also proposed to avoid deformation of GPR.