Gaussian Process Prediction Research Articles

An economic model predictive cruise controller for electric vehicles using Gaussian Process prediction

Recent political decisions in most developed countries urge car manufacturers to reduce vehicle emissions and fuel consumptions, essentially through development of hybrid and battery electric vehicles. Alongside with transfer to electro mobility, many OEMs promote research in car2x communication and cloud services as such connectivity features offer new opportunities to vehicular technology. This work presents a framework for an economic cruise controller for electric vehicles (EVs) utilizing traffic speed data obtained from a cloud server and aiming for real time applicability. Within the scope of this research, an economic model predictive cruise controller based on leading vehicle speed prediction is designed. Novelty of this approach is to utilize real time traffic speed data which significantly improves leading vehicle speed prediction. Error in speed prediction is compared to existing approaches and computed energy consumption using model predictive control is analyzed and discussed for different approaches of leading vehicle speed prediction.

A New Class of High-Order Methods for Fluid Dynamics Simulations Using Gaussian Process Modeling: One-Dimensional Case

We introduce an entirely new class of high-order methods for computational fluid dynamics based on the Gaussian process (GP) family of stochastic functions. Our approach is to use kernel-based GP prediction methods to interpolate/reconstruct high-order approximations for solving hyperbolic PDEs. We present a new high-order formulation to solve (magneto)hydrodynamic equations using the GP approach that furnishes an alternative to conventional polynomial-based approaches.

Respiratory motion compensation for the robot-guided laser osteotome.

The use of a robot-guided laser osteotome for median sternotomy is impeded by prohibiting cutting inaccuracies due to respiration-induced motions of the thorax. With this paper, we advance today's methodologies in sternotomy procedures by introducing the concept of novel 3D functional cuts and a respiratory motion compensation algorithm for the computer-assisted and robot-guided laser osteotome, CARLO®. We present a trajectory planning algorithm for performing 3D functional cuts at a constant cutting velocity. In addition, we propose the use of Gaussian process (GP) prediction in order to anticipate the sternum's pose providing enough time for the CARLO® device to adjust the position of the laser source. We analysed the performance of the proposed algorithms on a computer-based simulation framework of the CARLO® device. The median position error of the laser focal point has shown to be reduced from 0.22mm without GP prediction to 0.19mm with GP prediction. The encouraging simulation results support the proposed respiratory motion compensation algorithm for robot-guided laser osteotomy on the thorax. Successful compensation of the respiration-induced motion of the thorax opens doors for robot-guided laser sternotomy and the related novel cutting patterns. These functional cuts hold great potential to significantly improve postoperative sternal stability and therefore reduce pain and recovery time for the patient. By enabling functional cuts, we approach an important threshold moment in the history of osteotomy, creating innovative opportunities which reach far beyond the classic linear cutting patterns.

Uncertainty Bounds on Higher-Order FRFs from Gaussian Process NARX Models

One of the most versatile and powerful algorithms for the identification of nonlinear dynamical systems is the NARMAX (Nonlinear Auto-regressive Moving Average with eXogenous inputs) approach. The model represents the current output of a system by a nonlinear regression on past inputs and outputs and can also incorporate a nonlinear noise model in the most general case. In recent papers, one of the authors introduced a NARX (no noise model) formulation based on Gaussian Process (GP) regression and derived the corresponding expressions for Higher-order Frequency Response Functions (HFRFs). This paper extends the theory for the GP-NARX framework by providing a means of converting the GP prediction bounds in the time domain into bounds on the HFRFs. The approach is demonstrated on the Duffing oscillator.

A Trust Model based on Markov Model Driven Gaussian Process Prediction

A Trust Model based on Markov Model Driven Gaussian Process Prediction

Speeding Up Neighborhood Search in Local Gaussian Process Prediction

Recent implementations of local approximate Gaussian process models have pushed computational boundaries for nonlinear, nonparametric prediction problems, particularly when deployed as emulators for computer experiments. Their flavor of spatially independent computation accommodates massive parallelization, meaning that they can handle designs two or more orders of magnitude larger than previously. However, accomplishing that feat can still require massive computational horsepower. Here we aim to ease that burden. We study how predictive variance is reduced as local designs are built up for prediction. We then observe how the exhaustive and discrete nature of an important search subroutine involved in building such local designs may be overly conservative. Rather, we suggest that searching the space radially, that is, continuously along rays emanating from the predictive location of interest, is a far thriftier alternative. Our empirical work demonstrates that ray-based search yields predictors with accuracy comparable to exhaustive search, but in a fraction of the time—for many problems bringing a supercomputer implementation back onto the desktop. Supplementary materials for this article are available online.

Gaussian process prediction of the stress-free configuration of pre-deformed soft tissues: Application to the human cornea

Image-based modeling is a popular approach to perform patient-specific biomechanical simulations. One constraint of this technique is that the shape of soft tissues acquired in-vivo is deformed by the physiological loads. Accurate simulations require determining the existing stress in the tissues or their stress-free configurations. This process is time consuming, which is a limitation to the dissemination of numerical planning solutions to clinical practice. In this study, we propose a method to determine the stress-free configuration of soft tissues using a Gaussian process (GP) regression. The prediction relies on a database of pre-calculated results to enable real time predictions. The application of this technique to the human cornea showed a level of accuracy five to ten times higher than the accuracy of the topographic device used to obtain the patients’ anatomy; results showed that for almost all optical indices, the predicted curvature error did not exceed 0.025D, while the wavefront aberration percentage error did not overcome 5%. In this context, we believe that GP models are suitable for predicting the stress free configuration of the cornea and can be used in planning tools based on patient-specific finite element simulations. Due to the high level of accuracy required in ophthalmology, this approach is likely to be appropriate for other applications requiring the definition of the relaxed shape of soft tissues.

Fast Algorithm for Non-Stationary Gaussian Process Prediction

Algorithm's time complexity is an essential issue for time series prediction in numerous practices.A novel fast exact inference method for Gaussian process model is proposed in this paper to accelerate the task of non-stationary time series prediction. Experiment was done on the real world power load data.

PREDICTING PROJECT SUCCESS IN CONSTRUCTION USING AN EVOLUTIONARY GAUSSIAN PROCESS INFERENCE MODEL

There are many factors that affect the success of the implementation process of a project. The importance of each of these factors varies according to the different phases of the project lifecycle, which makes it very difficult to predict the final result of a project. In practice, foreseeing the result of a project is based on the judgment of those in management, which is grounded in their experience. This study aimed to build an Evolutionary Gaussian Process Inference Model (EGPIM), using a Gaussian process, along with Bayesian inference and particle swarm optimization, which helps to optimize the hyper-parameters required for making Gaussian process predictions. With this model at its core, this study can efficiently extract expert knowledge and experience from case studies and historical data to determine relationships between factors which significantly influence the outcome of a project so that its success may be predicted. Historical cases were ordered as a time series based on the Continuous Assessment of Project Performance (CAPP) research results. The model was trained using the EGPIM and these cases to predict the success of a project. This model proved quite accurate at predicting the success of a project and had outstanding performance in time-series applications.

Semisupervised multitask learning with Gaussian processes.

We present a probabilistic framework for transferring learning across tasks and between labeled and unlabeled data. The approach is based on Gaussian process (GP) prediction and incorporates both the geometry of the data and the similarity between tasks within a GP covariance, allowing Bayesian prediction in a natural way. We discuss the transfer of learning in a multitask scenario in the two cases where the underlying geometry is assumed to be the same across tasks and where different tasks are assumed to have independent geometric structures. We demonstrate the method on a number of real datasets, indicating that the semisupervised multitask approach can result in very significant improvements in performance when very few labeled training examples are available.

Mixtures of Gaussian process models for human pose estimation

Discriminative human pose estimation is the problem of inferring the 3D articulated pose of a human directly from an image feature. This is a challenging problem due to the highly non-linear and multi-modal mapping from the image feature space to the pose space. To address this problem, we propose a model employing a mixture of Gaussian processes where each Gaussian process models a local region of the pose space. By employing the models in this way we are able to overcome the limitations of Gaussian processes applied to human pose estimation — their O(N3) time complexity and their uni-modal predictive distribution. Our model is able to give a multi-modal predictive distribution where each mode is represented by a different Gaussian process prediction. A logistic regression model is used to give a prior over each expert prediction in a similar fashion to previous mixture of expert models. We show that this technique outperforms existing state of the art regression techniques on human pose estimation data sets for ballet dancing, sign language and the HumanEva data set.

Sparse inverse kernel Gaussian Process regression

AbstractRegression problems on massive data sets are ubiquitous in many application domains including the Internet, earth and space sciences, and finances. Gaussian Process regression (GPR) is a popular technique for modeling the input–output relations of a set of variables under the assumption that the weight vector has a Gaussian prior. However, it is challenging to apply GPR to large data sets since prediction based on the learned model requires inversion of an order n kernel matrix. Approximate solutions for sparse Gaussian Processes have been proposed for sparse problems. However, in almost all cases, these solution techniques are agnostic to the input domain and do not preserve the similarity structure in the data. As a result, although these solutions sometimes provide excellent accuracy, the models do not have interpretability. Such interpretable sparsity patterns are very important for many applications. We propose a new technique for sparse GPR that allows us to compute a parsimonious model while preserving the interpretability of the sparsity structure in the data. We discuss how the inverse kernel matrix used in Gaussian Process prediction gives valuable domain information and then adapt the inverse covariance estimation from Gaussian graphical models to estimate the Gaussian kernel. We solve the optimization problem using the alternating direction method of multipliers that is amenable to parallel computation. We compare the performance of this algorithm to different existing methods for sparse covariance regression in terms of both speed and accuracy. We demonstrate the performance of our method in terms of accuracy, scalability, and interpretability on two different satellite data sets from the climate domain. © 2013 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 6: 205–220, 2013

Gaussian process modeling with inequality constraints

La modélisation par processus Gaussiens est une des approches les plus utilisées pour construire un métamodèle dans le cas de simulateurs numériques coûteux. Souvent, les sorties du code correspondent à des quantités physiques dont le comportement est connu à l’avance : les concentrations chimiques sont comprises entre 0 et 1, la sortie est croissante par rapport à un des paramètres, etc. Plusieurs approches ont été proposées pour prendre en compte de telles informations. Dans cet article, nous introduisons un nouveau cadre théorique pour inclure des contraintes dans la modélisation par processus Gaussiens, qui englobe les contraintes de bornes, de monotonie et de convexité. Nous étendons également ce cadre à tous les types de contraintes linéaires. Cette nouvelle méthodologie fait appel aux moments conditionnels de lois normales multivariées tronquées. Nous proposons plusieurs approximations basées sur une hypothèse de décorrélation, des outils d’intégration numérique et des techniques d’échantillonnage. D’un point de vue pratique, nous illustrons l’amélioration des performances de prédiction par processus Gaussiens lorque l’on inclut des contraintes. Nous comparons finalement les différents prédicteurs approchés sur des exemples avec contraintes de bornes, monotonie et convexité.

The application and limitations of mathematical modelling in the prediction of permeability across mammalian skin and polydimethylsiloxane membranes

Predicting the rate of percutaneous absorption of a drug is an important issue with the increasing use of the skin as a means of moderating and controlling drug delivery. One key feature of this problem domain is that human skin permeability (as K(p)) has been shown to be inherently non-linear when mathematically related to the physicochemical parameters of penetrants. As such, the aims of this study were to apply and evaluate Gaussian process (GP) regression methods to datasets for membranes other than human skin, and to explore how the nature of the dataset may influence its analysis. Permeability data for absorption across rodent and pig skin, and artificial membranes (polydimethylsiloxane, PDMS, i.e. Silastic) membranes was collected from the literature. Two quantitative structure-permeability relationship (QSPR) models were used to compare with the GP models. Further performance metrics were computed in terms of all predictions, and a range of covariance functions were examined: the squared exponential (SE), neural network (NNone) and rational quadratic (QR) covariance functions, along with two simple cases of Matern covariance function (Matern3 and Matern5) where the polynomial order is set to 1 and 2, respectively. As measures of performance, the correlation coefficient (CORR), negative log estimated predictive density (NLL, or negative log loss) and mean squared error (MSE) were employed. The results demonstrated that GP models with different covariance functions outperform QSPR models for human, pig and rodent datasets. For the artificial membranes, GPs perform better in one instance, and give similar results in other experiments (where different covariance parameters produce similar results). In some cases, the GP predictions for some of the artificial membrane dataset are poorly correlated, suggesting that the physicochemical parameters employed in this study might not be appropriate for developing models that represent this membrane. While the results of this study indicate that permeation across rodent (mouse and rat) and pig skin is, in a statistical sense, similar, and that the artificial membranes are poor replacements of human or animal skin, the overriding issue raised in this study is the nature of the dataset and how it can influence the results, and subsequent interpretation, of any model produced for particular membranes. The size of the datasets, in both absolute and comparative senses, appears to influence model quality. Ideally, to generate viable cross-comparisons the datasets for different mammalian membranes should, wherever possible, exhibit as much commonality as possible.

Learning GP-BayesFilters via Gaussian process latent variable models

GP-BayesFilters are a general framework for integrating Gaussian process prediction and observation models into Bayesian filtering techniques, including particle filters and extended and unscented Kalman filters. GP-BayesFilters have been shown to be extremely well suited for systems for which accurate parametric models are difficult to obtain. GP-BayesFilters learn non-parametric models from training data containing sequences of control inputs, observations, and ground truth states. The need for ground truth states limits the applicability of GP-BayesFilters to systems for which the ground truth can be estimated without significant overhead. In this paper we introduce GPBF-Learn, a framework for training GP-BayesFilters without ground truth states. Our approach extends Gaussian Process Latent Variable Models to the setting of dynamical robotics systems. We show how weak labels for the ground truth states can be incorporated into the GPBF-Learn framework. The approach is evaluated using a difficult tracking task, namely tracking a slotcar based on inertial measurement unit (IMU) observations only. We also show some special features enabled by this framework, including time alignment, and control replay for both the slotcar, and a robotic arm.

Gaussian Processes for Machine Learning (GPML) Toolbox

The GPML toolbox provides a wide range of functionality for Gaussian process (GP) inference and prediction. GPs are specified by mean and covariance functions; we offer a library of simple mean and covariance functions and mechanisms to compose more complex ones. Several likelihood functions are supported including Gaussian and heavy-tailed for regression as well as others suitable for classification. Finally, a range of inference methods is provided, including exact and variational inference, Expectation Propagation, and Laplace's method dealing with non-Gaussian likelihoods and FITC for dealing with large regression tasks.

Sequential Bayesian Prediction in the Presence of Changepoints and Faults

We introduce a new sequential algorithm for making robust predictions in the presence of changepoints. Unlike previous approaches, which focus on the problem of detecting and locating changepoints, our algorithm focuses on the problem of making predictions even when such changes might be present. We introduce nonstationary covariance functions to be used in Gaussian process prediction that model such changes, and then proceed to demonstrate how to effectively manage the hyperparameters associated with those covariance functions. We further introduce covariance functions to be used in situations where our observation model undergoes changes, as is the case for sensor faults. By using Bayesian quadrature, we can integrate out the hyperparameters, allowing us to calculate the full marginal predictive distribution. Furthermore, if desired, the posterior distribution over putative changepoint locations can be calculated as a natural byproduct of our prediction algorithm.

Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes

Supervised machine learning (ML) algorithms are increasingly popular tools for fMRI decoding due to their predictive capability and their ability to capture information encoded by spatially correlated voxels. In addition, an important secondary outcome is a multivariate representation of the pattern underlying the prediction. Despite an impressive array of applications, most fMRI applications are framed as classification problems and predictions are limited to categorical class decisions. For many applications, quantitative predictions are desirable that more accurately represent variability within subject groups and that can be correlated with behavioural variables. We evaluate the predictive capability of Gaussian process (GP) models for two types of quantitative prediction (multivariate regression and probabilistic classification) using whole-brain fMRI volumes. As a proof of concept, we apply GP models to an fMRI experiment investigating subjective responses to thermal pain and show GP models predict subjective pain ratings without requiring anatomical hypotheses about functional localisation of relevant brain processes. Even in the case of pain perception, where strong hypotheses do exist, GP predictions were more accurate than any region previously demonstrated to encode pain intensity. We demonstrate two brain mapping methods suitable for GP models and we show that GP regression models outperform state of the art support vector- and relevance vector regression. For classification, GP models perform categorical prediction as accurately as a support vector machine classifier and furnish probabilistic class predictions.

GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models

Bayesian filtering is a general framework for recursively estimating the state of a dynamical system. Key components of each Bayes filter are probabilistic prediction and observation models. This paper shows how non-parametric Gaussian process (GP) regression can be used for learning such models from training data. We also show how Gaussian process models can be integrated into different versions of Bayes filters, namely particle filters and extended and unscented Kalman filters. The resulting GP-BayesFilters can have several advantages over standard (parametric) filters. Most importantly, GP-BayesFilters do not require an accurate, parametric model of the system. Given enough training data, they enable improved tracking accuracy compared to parametric models, and they degrade gracefully with increased model uncertainty. These advantages stem from the fact that GPs consider both the noise in the system and the uncertainty in the model. If an approximate parametric model is available, it can be incorporated into the GP, resulting in further performance improvements. In experiments, we show different properties of GP-BayesFilters using data collected with an autonomous micro-blimp as well as synthetic data.

Spatial sampling design for prediction with estimated parameters

We study spatial sampling design for prediction of stationary isotropic Gaussian processes with estimated parameters of the covariance function. The key issue is how to incorporate the parameter uncertainty into design criteria to correctly represent the uncertainty in prediction. Several possible design criteria are discussed that incorporate the parameter uncertainty. A simulated annealing algorithm is employed to search for the optimal design of small sample size and a two-step algorithm is proposed for moderately large sample sizes. Simulation results are presented for the Matérn class of covariance functions. An example of redesigning the air monitoring network in EPA Region 5 for monitoring sulfur dioxide is given to illustrate the possible differences our proposed design criterion can make in practice.